Developing M&E is a process which is integrated with conceptualizing the project implementation itself. The M&E should ideally form a kind of critical dialogue, which upholds the reality of beneficiaries, and the complex context at the forefront.
Step 1: Needs Analysis
Where a project is not responding to a real, subjectively experienced need, with the right solutions to motivate beneficiary participation, it is unlikely that it will be effective. Although you may observe a need, a deep exploration of whether the same is perceived by beneficiary groups is an important piece of groundwork. Beneficiaries and their communities are always best placed to determine whether a need exists, and have frequently put a great deal of thought into the design of relevant and viable solutions. A thorough needs analysis will allow you to take your concept, and test is out for relevance and viability in the space where it is expected to create the change you hope see.
Step 2: Programme and M&E Design
Once you’re clear on the need, you can begin with the creative process of solution design, the work that will ultimately define your project, as well as your M&E. Once you’re clear on the need to be addressed, it is a good idea to conduct extensive research on similar projects addressing similar needs, exploring the lessons learnt and reporting. Although real solutions may lie in truly novel work, there is a lot to be said for starting from an understanding of what has and what has not worked in similar contexts.
Step 3: Stakeholder Mapping; beneficiary Identification
Although you will have already identified and engaged with beneficiary groups by the time you are mapping stakeholders, this part of the process will enable you to explore how the different stakeholders are linked in the context of the work you are aiming to do. This forms an integral part of the M&E process, as you aim to develop specific indicators, and important project milestones which frequently includes actions, or necessary participation of key people, or groups of people. This section of the work is not only about mapping individuals in their capacity as participants, but identifying important officials, and understanding how their role is integral to your project success.
Step 4: Defining the Theoretical Framework
In this step, you are ultimately defining the change you hope to create, within the situational, as well as the theoretical context. You are creating your Theory of Change. This is an important part of the process where implementers should work hand-in-hand with the M&E team to make sure that the results you hope to achieve, and the empirical evidence you are basing the expected causal pathways on resonate with everyone working on the project, particularly those who are most familiar with the particular circumstances of the implementation such as field workers who understand the communities. Workshopping a theory of change using the language of results, and then researching around the salient points that emerge is one way of doing this, and ensuring the breadth and depth of the work.
Step 5: Defining the Logic, Mapping the Indicators
Once you have the high-level theory; the story behind the change well-defined, then it is time to explore the technical aspects of the work you are doing, and to define incrementally how specific, observable occurrences, which form the basis of the evidence of the change you’re creating, can be counted and mapped to record the change taking place. In this step, it is important to research and identify these indicators, where possible adhering to global best practice, or sets of pre-defined indicators, particularly if you are working with global funders and aim to show how your work aligns with their objectives.
Step 6: Milestone Identification, planning and scheduling
Once you are clear on what the results should look like and how to intent to measure them, you need to consider timing really carefully. Consider your assumptions. What needs to be in place, before your activities will be able to achieve the change to hope to see? Do you need critical buy-in from key stakeholders? Or perhaps you are working on an agricultural project and certain programme activities are seasonal. Do you need buy-in from your beneficiaries and what might this look like? As you think about these things, you will realize whether you can map your activities within the calendar year, or whether the achievement of goal B might be expected no less than 6 months after securing milestone A! This type of thinking will help to define your funding cycles, fundraising, and the overall life span of the work you are doing. Think carefully, be conservative, make strong assumptions but revisit these often.
Step 7: Designing the Instruments; selecting the tools
Once you have your indicators identified and specified in time, and your programme plans are clear, then you can begin to design instruments and select tools which will work to collect the information against key indicators at specific points in time. These may be anything from paper-base to biometric attendance tools, to questionnaires about what benefit people perceive, to assessment to see whether skills have been successfully imparted. Spend time on this, and consider this very carefully in the context of the type of overall evaluation approach you are using. These instruments will form the very basis of the information you will have to describe the change you have created and ultimately to determine whether your programme was a success. It’s never too late to add an instrument, but you can never go back in time, and as your programme may change the very way people think and feel about the challenges they face, make sure you’ve considered everything you’re going to wish you’d asked or known when you first started the work.
Step 8: Implement and Monitor
Ensuring that quality is maintained as you implement is key. Monitoring and implementation should take place side-by side with monitoring teams continually ensuring that the programme work is consistent in quality such that the data gathered is reliable, with the implementation team providing feedback where certain things about how the work is done might need to change due to circumstances. This is really where the magic happens, and is the vast majority of the work that you will do. As the M&E team, be sure to listen, and respond. Facilitating implementation while giving sound strategic advice can add immense value, and boost the ability of the implementing team to create meaningful and lasting change.
Step 9: Analyze
Cleaning data is no one’s favorite job, but with well designed instruments, and good tools, the analysis can be the most fun. Explore different types of analysis; think about whether you’re looking for overall reach, or whether the analysis is looking to identify some kind of causation. Visualization software can be extremely helpful to provide graphical analysis of your data. Thematic analysis for qualitative survey feedback can be a helpful method. Statistical tools are most frequently used to try and show directions and intensity of causation (be cautious about applying these to complex problems though). Get to know your data. Keep a base set, and each time you revisit it, think of another way you might look at it. Let the answers emerge as you consider what the data is really saying. Never underestimate the outliers. Although you will have spend a great deal of time on the theory, remember that the data may tell an entirely different story.
Step 10: Report
Finally, write your report. In my work, it is always most useful to consider a range of approaches and apply different methods to different parts of the work. Impact really is a story. George Shackle, an obscure but brilliant radical economist once wrote that there is no such thing as repeatable experiment. What he meant by this is that even the same experiment at a different time is a different experiment. In development practice there are no closed systems, yet we seek to develop an understanding which can help ourselves and other elsewhere to do good, to make good, and to spend precious resources as wisely as possible. Reporting should aim to map such a path, between the immense complexity, and the ‘answers’. When writing your report, context is key, be sure that programme teams provide input. It is always better to understate rather than overstate success. Be true to the values, and true to the beneficiaries. How you ultimately compile the reporting will depend on the M&E methods you have chosen and on the project success. Whether highly quantitative and analytical, or rich in case studies around small, radically different and subjective change, this report is a record of the important work taking place.
