Measuring results is important, but not every result can be counted, tracked and fit neatly into an M&E framework. Some results are intangible – things like “empowerment”, “confidence” or “improved capacity”.
Just because these things are difficult to count that doesn’t mean you can’t measure them. You just need to measure them in a different way, using qualitative methods or mixed methods (qualitative and quantitative combined).
Define success
Before you decide on a method you need to define what you are looking for. For example, if your program aims to empower women, what does “empowerment” actually look like? Does it mean that women have control over some household decisions? Does it mean they participate in community meetings? Or that they can leave abusive relationships?
When deciding your definition you should start by reviewing how other people have defined the same concept. For example, empowerment is often divided into ‘four powers’ – power within, power with, power to and power over.
Choose your method(s)
There are many different methods that can be used to measure qualitative results. Just a few are described here. It can be useful to combine different methods in order to get different perspectives. You should also adapt or modify methods to suit your program.
Interviews or focus groups
When people think of qualitative methods they usually jump to interviews and focus groups first. These methods can be used to meet directly with beneficiaries and stakeholders to discuss their experiences, and the results they have seen from the program.
Although interviews and focus groups are the most common methods, it doesn’t mean they are the best methods for every circumstance. Think carefully about what what you are trying to measure, and whether another method might give you more useful information.
Should I use interviews or focus groups?
How to do great semi-structured interviews
Diaries, logs or journals
Diaries, logs and journals can by used by people managing a program, by participants or by stakeholders. For example in a program I am currently working on, the Program Manager keeps a log of all meetings and interactions with government partners. Through this log it is possible to see improvements in government services that have come about through advocacy by the program staff.
An excerpt from an activity log
Similarly, participants could be provided with a journal or diary to record their experiences and thoughts. Reviewing the journal entries could show if they have changed their attitudes or practices as a result of the program.
Pictures, paintings or photos
Some people find it easier to express changes visually. Pictures, paintings and photos can all be used to measure qualitative results. For example, vulnerable children could be asked to draw a picture showing their life before and after they joined the program.
In photovoice you give each participant a camera and ask them to take photos of their community, focussing on particular themes. In many photovoice projects the best photos are included in an exhibition for community members and stakeholders.
A photovoice exhibition at the Delaware Center for Contemporary Arts where Wilmington teens exhibited their images to parents, community leaders and local policy makers. Photo by Christiana Care
Tiny Tools
Tiny Tools are a collection of quick and easy tools to help community members express how their lives have changed. They were developed by NGO-IDEA.
Each Tiny Tool is designed to be used by a facilitator with a group of community members during a short workshop or focus group. The purpose of the tools is to help the community members assess the impact that the project has had on their lives.
Simple, quick and easy – Tiny Tools for impact assessment
An example of a Lifeline from the Tiny Tools
Most Significant Change
The Most Significant Change technique works by collecting stories of change brought about by the program and systematically analysing them for their significance. Stories are collected from the field with the most significant stories being filtered up the organisation.
It is considered a participatory technique because many stakeholders are involved in collecting and analysing the data.
Using the ‘Most Significant Change’ (MSC) Technique to measure the intangible
An example of a story produced by the Most Significant Change technique:
| Title of story | Woman community facilitator voted as president of community network. |
| Who was involved and what took place? | A local woman who has received capacity building support and training on leadership skills, has been voted as president of the community facilitator network representing 5 villages. |
| What change took place? | A local woman is now representing her community as a leader in local development issues. She had the confidence to put herself forward and was voted by her community to play a leading role in the community’s development process. |
| Why is story significant? | The story shows that women in communities where the NGO works are playing an increased role in local politics and decision making as a result of capacity building and training from the NGO. |
| Lessons learned / recommendations for the wider program | Training and capacity building can help build women’s confidence to participate in local politics. If women are given the opportunity to become involved in local politics they will. The beneficiary can be a role model for other local women. |
Photo by Al Jazeera English
