Handbook on Data Collection / Phase Ten: Evaluate and Apply Lessons Learned
< Handbook on Data CollectionFor any project implementation, it’s important to consistently check progress during the implementation and evaluate where things can be improved to achieve more of the expected results, more inclusivity, more sustainability, and more efficiency. But how do you make an evaluation and draw lessons learned?
Practical suggestions on evaluation and applying lessons learned
|
Contents
What do we mean by evaluation?
In the management of programmes, we use the terms monitoring and evaluation (M&E) and planning as well as monitoring, evaluation and learning (PMEL) to describe the methodology used to improve the effectiveness and sustainability of development projects. An evaluation aims to capture what the real effects were and what could be improved next time. It is a way of identifying lessons that can be learned and used in similar up-scaled or the next phases of a project or programme.
Planning and using evaluations
When planning for evaluations there are a number of things to consider:
- Do you do the evaluation yourself or do you involve an external party to evaluate? The risk of bias is overcome by involving an external party. Most funders require the end evaluation of the project to be conducted by an external party.
- When should you evaluate? If you perform an evaluation before the end of a project, e.g. mid-term, it allows for not only drawing conclusions and lessons but also allows the immediate implementation in the project before it ends. This is referred to as a “formative evaluation” as opposed to a “summative evaluation” at the end of the project or programme.
- Will you share the results of the evaluation? With whom exactly? Usually the funder requests the evaluation, but you may decide to share evaluation reports more widely or even publicly. This can be a valuable resource for other programmes, while promoting transparency and accountability and instilling trust within the (funding) partners.
- There is a wide range of evaluation approaches, from appreciative inquiry to social return on investment. Find a comprehensive list including descriptions of each at BetterEvaluation.org.
Lessons learned
An (external) evaluation usually has a number of evaluation questions. These usually cover three main areas of a project:
- Appropriateness/relevance
- Effectiveness
- Efficiency
If the evaluation is aimed at finding what works, what doesn't, and why, then the evaluation findings will result in lessons learned: “what would we do differently next time? And what would we do the same?”
Key tips for formulating lessons learned
- Lessons learned should consist of a generalised principle that can be applied to, or could be relevant for, other situations.
- Do not write the lesson only as an observation, description, or recommendation that lacks justification. Justify the lesson with proof of why it is valid.
- Explain the lesson in the context of the project. For it to be useful to others, they need to understand the situation in which it occurred to know if it might be appropriate or useful for them.
If you are looking for a format for lessons learned, you can consider this template: If you do X and Z conditions are present, then Y will happen. This is further described and provided with examples from this guide or in Identifying and documenting “Lessons Learned”: A list of references.
Draw lessons from implemented activities at any time during the project. An approach that can be applied for that purpose is the After Action Review. Lessons learned can be converted into recommendations for a new project or the next phase of a project.
Where to start
As you may have noticed, BetterEvaluation.org is a great source of information, materials and examples on evaluations. A good place to start if you are not familiar with the evaluation process is the Manager’s guide to evaluation, which is also available in French. It’s really useful for people who are managing an evaluation or contracting a party to perform one, including examples of terms of references.
Conclusion
Having clarity on the purpose of your evaluation is key. Will it benefit your work and/or comply with funders’ requirements? For enhancing future work, drawing lessons learned can be very valuable.
Suggested reading
If you still feel very new to the subject, refer to Evaluation resources for newbies.
Acknowledgements
Author: Marten Schoonman (Akvo.org)
Contributors: Anita van der Laan (Akvo.org), Annabelle Poelert (Akvo.org), Arun Kumar Pratihast (Akvo.org), Harro Riedstra (Akvo.org), Natacha Amorsi (Office International de L'Eau, OIEau)
AfriAlliance
The Africa-EU Innovation Alliance for Water and Climate (AfriAlliance), is a 5-year project funded by the European Union’s H2020 Research and Innovation Programme. It aims to improve African preparedness for climate change challenges by stimulating knowledge sharing and collaboration between African and European stakeholders. Rather than creating new networks, the 16 EU and African partners in this project will consolidate existing ones, consisting of scientists, decision makers, practitioners, citizens and other key stakeholders, into an effective, problem-focused knowledge sharing mechanism. |
AfriAlliance is lead by the IHE Delft Institute for Water Education (Project Director: Dr. Uta Wehn) and runs from 2016 to 2021. The project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 689162. |