Welcome to Evaluation Connections, a quarterly e-newsletter designed to provide Making Connections grantees with easily accessible evaluation resources.

View this email in your browser
EC Issue 1 | June 2016

Focus on

Understanding Community
Strengths and Needs  


This issue of Evaluation Connections (EC) provides an overview of data analysis and discusses how this process can inform your planning and implementation efforts. Making Connections (MC) grantees are conducting community needs assessments and beginning to analyze their data. Data about community strengths and needs support the development of programs that better serve their community.

Methods of Data Collection

MC grantees have approached community needs assessment in a variety of ways. Click here to see a table of site-by-site data collection methods.

Purpose of Data Analysis

Data analysis makes your data usable. Instead of a stack of survey responses or a pile of focus group transcripts, analysis transforms these data into something that can be discussed by your MC team, partners, and community.  Results of your analysis can help you:
  • Assess community strengths and resources;
  • Respond directly to community concerns;
  • Identify community issues you were unaware of;
  • Prioritize action steps;
  • Develop prevention strategies; and 
  • Understand and measure progress.
Ultimately, the analysis of your strengths and needs assessment data will inform your MC theory of change and set the stage for action planning and implementation.

Click here for more about collecting and analyzing data

True or False? 

Data analysis and interpretation is the same thing.

Kinds of Data

Local evaluation efforts often make use of both qualitative and quantitative data. So far, MC grantees are relying more heavily on qualitative data than quantitative data. This makes sense because qualitative data are so useful for understanding community context.  

Qualitative Data = Data in the form of text, narrative, or images.  For MC grantees, data collection has included interviews, observations, focus groups, culture circles, visual methods (e.g., photo voice, mandalas), THRIVE scans, and documents such as meeting minutes and attendance lists.

Quantitative Data = Data in the form of numbers. For MC grantees, this has included surveys. It has also included tracking attendance at community meetings, conferences or trainings, and community demographics.

Advantages of Mixed Methods
Using qualitative or quantitative data is not an either-or decision. The kind of data you collect depends on what you want to learn. Keep in mind that there may be significant advantages to using both. For example, you may need to know more about the incidence of depression among young men in your community. A survey of youth would yield quantitative data about the prevalence of depression within this population of focus.  Interviewing a sample of youth could help you understand the experience of depression and the impact of depression on their lives. Mixed method evaluations seek to build understanding by integrating findings across qualitative and quantitative data.

Strategies for Data Analysis

Qualitative data are often used to explore or explain “how” and “why.”  Analysis of qualitative data involves identifying themes and patterns in the data. This is most often accomplished by classifying or coding data into similar categories. For example, interviews with youth, their families, and faith leaders in the community might be coded across respondents to identify risk factors for youth. The coded data can then be analyzed to compare perspectives across these different groups of respondents such as:
  • Do youth perceive risk factors similarly or differently than their families?
  • How do faith leaders feel in comparison? 
Qualitative analysis can be accomplished by hand coding, using pens or markers to highlight specific passages and make notes in the margins. You can even analyze your data with Microsoft Word by using the comment function to make notes of highlighted passages. Computer software such as ATLAS.ti and HyperRESEARCH can also be used in qualitative analysis. Some of the disadvantages of computer-assisted analysis can be the cost and time spent learning the new software.  Advantages include easier management of large quantities of data and improved validity and reliability of analyses. 
Quantitative data are analyzed using statistical procedures that let you explore relationships and/or differences across variables. Descriptive statistics are used to answer “what”, “how many”, “how much”, and “how often” questions. Results are represented as frequencies or percentages and also as mean (average), median (midpoint), or mode (most frequent). Much of the quantitative analyses done by MC grantees will focus on these kinds of descriptive statistics. For example: 
  • the number of attendees at a focus group, meeting, or training;
  • the range and median age of veterans in your community;
  • the percentage of youth graduating from high school; and
  • the frequency of drug use reported in a survey.
Program evaluation, particularly outcome evaluation, also makes use of inferential statistics. Inferential statistics can be used to compare outcomes between two groups (for example, comparing outcomes of those who did and did not participate in an intervention). To learn more examples of inferential statistics, click on T-tests and ANOVA. Although data analysis programs such as SPSS are often used for more advanced analyses, easily accessible programs like Microsoft Excel are quite useful and advantageous for data analysis.


Data Analysis

  • Organize your data- To avoid unintentional errors during analysis, review and organize your data. It is common to have missing sections that were accidentally left blank or duplicated information. Organizing your data by categories such as the questions asked in your focus groups will make it easier and more manageable to work with your data before analysis.
  • Keep your analysis focused – At this point in time, your efforts will be concentrated on identifying community needs and strengths. You may find, however that your data are rich in other information. To prevent getting bogged down, keep in mind that you can always go back to these data and analyze with different questions in mind.
  • Triangulate – This is a technique of comparing data across methods (e.g., what you learned from focus groups, surveys, and interviews) and across people (e.g., what you learned from veterans, their families, and their service providers). Triangulation is used to improve the validity of your analyses.
  • Member checking – This is a technique that involves getting participant validation of findings. This strategy improves accuracy of findings and can increase credibility of results.

Data Management

  • Identify who will analyze your data. Some MC grantees are doing their own data analysis; others have contracted with partners or outside consultants to conduct their analyses. If you are working with an outside organization, be clear about what you need to learn and your timetable for completion.
  • Determine how/where data will be stored. This should be in a secure location (locked filing cabinet or protected digital files).
  • Be clear about who can access your community’s data and for what purposes.

Making Meaning

Interpretation gives meaning to your data and helps set your course of action. Regardless of the type of data you collect or method selected for analysis, reflecting on your results and making meaning of the findings are fundamental components of your data analysis and plan development. This process can confirm assumptions you may have had prior to conducting a needs assessment, but it can also illuminate new/unknown information about your community that will inform the programs and prevention strategies you develop. Discussing results with your team, partners, and community members may yield varying interpretations, but engaging in a conversation can aid in building a shared vision that will ultimately help guide the work you do and programs you develop for the communities you serve.

Here’s what’s coming up next:

Evaluation Connections Issue 2 (August 2016) will focus on Using Data to Inspire Action.

The Making Connections Evaluation emphasizes theory-driven evaluation, an approach that allows planners and implementers to test their ideas (theories) about how to create program, community, and system change. Theory-driven evaluation promotes development of shared vision, clear linking of strategies to intended outcomes, and a commitment to using data to inform decision making at all stages and levels of implementation.
For more information about Evaluation Connections, please contact Melissa Tirotti
or your Making Connections Evaluation Liaison.

The Making Connections for Mental Health and Wellbeing Initiative is designed to improve the mental health and wellbeing of men and boys in the U.S. This initiative is funded by the Movember Foundation and implemented in collaboration with the Prevention Institute. USF provides independent evaluation of the Making Connections Initiative.

Copyright © 2016 University of South Florida, All rights reserved.

Want to change how you receive these emails?
You can update your preferences or unsubscribe from this list

Email Marketing Powered by MailChimp