Call Us MN: 24/7 bilingual crisis line (651) 772-1611
Building evidence
How do I evaluate a program?
As we mentioned in the previous section, evaluating a program is in many ways similar to cooking. You start by selecting a recipe, gathering all the ingredients, and then transforming them into a delicious meal. In evaluation, you start by deciding the type of evaluation that you will conduct, gather the information, and transform it. To be able to do this transformation you need to learn how to organize, analyze, and interpret data. In this section we will review this process.
Organizing, analyzing and interpreting data
You probably already collect a great deal of information about your program. For example, information about who is using your service and how many people you see every day. This information is called raw data. To make this raw data useful you need to transform it into meaningful, helpful information by organizing it, analyzing it and interpreting it. This raw data can be in the form of numbers (which we call quantitative data) or words (which we call qualitative data).
Quantitative data
The first step is to organize the data to make it easy to understand and easy to find.
We might ask our participants to fill out a survey and tell us how old they are, their gender and their place of origin. To organize this data so that we can understand it and find it easily, we enter the data (information) from each survey into a table in a Word document or in an Excel spreadsheet.
Here is an example of a table we made using the quantitative data (numbers) from our participants’ survey:
This table makes it much easier to see all the data we have collected. If we did not make that table, we would have to read all of the surveys every time we wanted to find out something about our participants/clients.
Analyzing quantitative data
The next step is to analyze the data so that we can get information from it that actually tells us something we want to know.
For example, we can analyze the data in the table that we made from our participant survey to tell us about the people who use our services. Common forms of data analysis include frequency counts and averages.
Sample analysis
- 3 participants (frequency count)
- 40 years is the average age (average)
- 1 female and 2 males (frequency count)
Interpreting quantitative data
The next step is to interpret the data so that it is useful to us.
Sample interpretation
It appears as though from the three participants that the average age is 40 years old, and that we have more male participants than female participants.
As you can see, we were able to transform the raw data that we have in the table into a succinct paragraph that give us meaningful information about the program participants. Now let’s see how we can transform raw qualitative data (words).
Qualitative data
Most of the qualitative data (words) that we collect usually come from recordings of interviews or focus group discussions. Sometimes we use data from notes either staff or participants collected about the program or specific program activities.
Organizing qualitative data
Similar to the quantitative data, the first step is to organize the data. For qualitative data, this might mean transcribing the information from recordings to a Word document or from hand-written notes to a Word document. This allows you to have all the information in the same format so you can analyze it.
Analyzing qualitative data
The next step is to analyze the data which consist of identifying the main themes. The following step-by- step process helps guide you through this process:
- First, you should write down your own observations of the activity, and any themes or topics that you remember.
- Next, review your notes and/or the transcriptions. Begin to highlight key statements and label these statements. For example, if you notice that several statements have to do with effective ways to engage the community, one label could be community engagement.
- Repeat the process with the rest of the information you collected.
- As you label the statements, you will notice that the themes will begin to repeat (this is what we mean by recurrent themes).
- Next, group together the themes based on those that are most similar. You may need to merge theme to narrow down your results to a manageable number.
- This list of themes is the results of your analysis.
Let’s look at an example. This sample statement was obtained from focus group activities with youth about the Deferred Action for Childhood Arrivals (DACA) program.
“DACA allowed me to attend a university in another state. Without DACA I would have never applied to a university far from home as I knew I could not safely travel across country because of my documentation status.”
From this sentence we were able to identify the following themes:
- THEME: Increased Educational Opportunities
- DACA widens the pool of potential universities youth can apply to
- DACA allows youth to consider more educational opportunities
- THEME: Safety
- The program protects youth regardless of documentation status
Interpreting qualitative data
The final step is interpreting the data (inferring meaning from the data).
Sample interpretation
From the testimony of one of our participants, we were able to learn that the implementation of Deferred Action for Childhood Arrivals also known as DACA has had a positive impact on the life of our youth participant by opening more doors to educational opportunities and increasing a sense of safety.
This task can be a great opportunity to build capacity among your staff, community leaders, and/or partner with other organizations by engaging them in the process.
If you do not have a person on your staff that can carry out the process, you should consider hiring an external evaluator.