SMART indicators, and the theory around using the SMART criteria to select what you measure has had a significant positive impact on the way the implementers think about the projects which they are doing.
SMART indicators are:
- Specific
- Measurable
- Achievable
- Relevant and
- Time-Bound
This useful framework is an excellent way for implementers to view their work in the most constructive way, and draws out the clearest possible articulation of what the project hopes to achieve. A specific indicator will be narrowly defined, and will describe exactly what needs to be measured. A measurable indicator is one which can be aligned with a specific numeric or ranked value to show improvement over time. Defining your indicators for milestones which are realistically achievable is key to success, and also in ensuring that during the conceptualisation phases, the project exists within the realm of what is actually possible to achieve. Ensuring indicators are relevant provides for a consideration of the context in which the project is operating. Finally, time-bound indicators are those which include a date by which you expect to see the change, giving substance and life to the project as a whole.
By checking each of the your project indicators against these criteria, you can be sure that the M&E which you conduct can make clear statements about the impact which your programme is having and will allow for a rigorous and quantitative approach to measuring your work. If you work well with your M&E team, using this framework will also allow for the development of improvement feedback loops, where the project can continuously improve as implementers realise that perhaps certain things are not achievable, or at least not within a given time frame.
Adhering to this framework, and having knowledge of it is key in any M&E practice, particularly where the funding base requires a rigorous approach to development projects. Especially in the measurement of outputs and short- and medium-term impacts, these indicators are essential to good programming.
Taking your SMART indicators to the impact level involves more in-depth research, reaching into the literature around empirical relationships, proven in international contexts, as you design a project which aims to have broader change, change which is less in the direct sphere of control of the project activities. Even if you use SMART indicators, positioning this within development theory, and ensuring a rich contextual record of the circumstances in which the project operates is key. An excessive focus on SMART indicators being the only indicators tended to undermine the importance of theory and reasoning in M&E practice. Indicators, where these are used should be as SMART as possible, but findings based on SMART indicators can be given great value within a broader, more explanatory impact story. Also, although SMART indicators work well for outcomes-based M&E, other methods frequently require different considerations.
In my quest to measure and deeply understand things which don’t always fit neatly into the smart box have brought me to Dennis Bour’s article on the ‘cream of good indicators’ where CREAM stands for indicators which are Clear, Relevant, Economic, Adequate and Monitorable, a framework which works well for techniques such as SROI, or for outcomes monitoring focusing on results frameworks, where what is ultimately being measures is programme performance. Another variation, well suited to complex spaces is the SPICED principle. SPICED indicators are Subjective, Participatory, Interpreted and Communicable, Cross-checked and Compared, empowering and Diverse and Disaggregated (Roche 1999).
The great news is you don’t have to pick one. If you’re writing an impact story which has a clear programme set of outcomes, which are highly quantifiable, you can have your SMART outcomes framework, while further from the direct control of the programme, toward the fringes of your influence, you see the work changing individual mindsets, you might use some SPICED questionnaires to compile some case studies, a rich record of individual experience and what you might expect to see happen when the programme is over, and only changed skillsets and mindsets might carry the change into the future.
