Handbook on Data Collection / Phase Two A: Design Project

< Handbook on Data Collection
English Français

Designing a project or programme in a constantly changing environment can be challenging. The context in which a project operates is often complex, with many different stakeholders and factors involved. For that reason, it’s important to base the design of a project on an understanding of the context and choose an approach that allows for flexibility instead of stasis. That way, adaptations can be made when needed. Even a relatively straightforward data collection project needs to keep in mind which stakeholders will be involved and which problems and opportunities exist, at the start and throughout. The methods explained in this article are not just about optimising the outputs of your project design and implementation, but also emphasise the importance of the process in itself, which helps stakeholders to align and feel ownership of the project. The process should be carefully documented in order to capture lessons learned that can be shared within the sector and can be used to feed into future projects. This approach to project design is Theory of Change (ToC)-based, a methodology which helps you to structure reality and understand how your project can start a process of change.1

This article gives suggestions on the three steps of designing a project: how to do an analysis of the context (step one), how to design a ToC (step two), and how to build a monitoring framework (step three). Before going into detail, this article gives a brief outline of the three steps and an introduction to the Theory of Change methodology. Consult the Handbook glossary for definitions of each level of development.


Designing a project in three steps

Step one: Conduct context analysis
Before designing the ToC, have a thorough and common understanding of the context in which your project is operating. Start with context analysis: a factor, issue and stakeholder analysis, and a map of the findings.

Step two: Design Theory of Change

Define an impact that you want to reach or contribute to with your project and think backwards. Which outcomes need to be realised to reach the impact, and how are they interconnected? Which strategies will help to achieve these outcomes? Underlying causal assumptions recorded can be made explicit.

Step three: Build monitoring framework

From the ToC, collectively agree on what key expected outcomes (and impact) all stakeholders want to monitor. For those, design a planning, monitoring, evaluation and learning (PMEL) framework, with indicators and means of verification. In addition, monitor the causal assumptions that you are unsure about. Based on your monitoring findings, the ToC should be revised on a yearly basis and adjusted accordingly.

NOTE: all three steps require a highly participatory approach, to ensure relevance and co-ownership from the start

Contents

Introduction to the Theory of Change based project design

Creating a Theory of Change (ToC) helps you to structure reality and understand how your project can start a process of change. If you have a certain impact in mind that you want to contribute to as a project, a ToC helps you to understand which different outcomes you need to achieve in order to reach your envisioned impact and how these outcomes are interrelated. While the word impact refers to the ultimate change that your project aims to contribute to, the outcomes are changes that need to happen beforehand. Designing a ToC together with all stakeholders will result in a common understanding and co-ownership of the project and will facilitate the planning of your activities in a participatory way. It will also help you to discover what you collectively want to learn, and therefore to decide what you want to monitor during the project.

Although in this article the steps are presented in a sequence, in reality, the three often overlap and are circular in nature. For example, in order to be able to map all relevant stakeholders, there needs to be an awareness of the context and of what the problems and opportunities are. You might realise after the context analysis that some important stakeholders were overlooked during the analysis. Each step in a ToC based project design can make you realise that something was missing or not clear enough in a previous step and may lead to revisions.

Step one: Performing a context analysis

Before designing a project, it is crucial to understand the context in which it is operating to ensure that everyone involved has a similar understanding of the situation and that the project is designed to address the relevant issues with the right stakeholders. Context analyses are often outsourced to external consultants. However, our experience shows that some of the knowledge, understanding, and connections that the external consultant acquires during the analysis may be lost in the transfer of information. Therefore, we suggest having the context analysis done in a participative way with skilful insiders. This way, the exercise can result in a deeper understanding and higher usefulness to the design of the project. The context analysis usually consists of two interlinked exercises: mapping and analysing the stakeholders and the factors. Naturally, existing analyses need to be made use of for this step to prevent repetition.

Stakeholder analysis and mapping

The participatory stakeholder analysis helps you to identify and map all relevant actors and their roles, responsibilities, relationships, interests, and relative influence/power. It is important to make sure all actors are taken into account, including vulnerable and underprivileged groups who may otherwise be overlooked. This exercise is of most value when performed in a group or workshop setting, and will help identify which strategic stakeholders need to be involved in your project. During the project, power relations may change and new stakeholders may appear or knowledge gaps regarding existing stakeholders may be filled. It is therefore advisable to review your stakeholder analysis on a regular basis.

Stakeholders2 can be categorised into three types, all of whom need to be considered in the stakeholder analysis and mapping:

  1. Communities: The people who experience the problem directly, and interact with problem solvers.
  2. Problem solvers: The civil servants, non-governmental organisation (NGO) staff, frontline responders, and others on the ground.
  3. Policy and decision makers: The people who have access to resources and control allocation, or can influence decision making.

Helpful methods in performing a stakeholder analysis are making a stakeholder map and an interest and influence matrix. In a stakeholder map, all actors who are relevant for the project’s success are noted down on colour-coded cards (different colours for public sector, civil society organisations, private sector, platforms/networks), and organised according to the level at which they are most active (international, national, regional, district, village). The nature and strength of relationships between actors can be indicated with lines of different colours and thickness. See the stakeholder map below (Figure 2), which is intentionally generic and simplified to be useful as an example for any programme or project.


Figure 2. Example of a stakeholder map


The influence-interest matrix builds on the stakeholder map, see the generic example below (Figure 3). All of the actor cards can be plotted on four quadrants of the y-axis (interest) and x-axis (influence), according to how interested they are in the success of the specific project, and how much influence they have in making it happen. Powerful stakeholders can also have a strong negative effect on the success of the project, and it is important to keep this in mind from the design phase onwards.


Figure 3. Example of an Influence - Interest matrix


While doing both exercises, keep in mind that there are formal and informal power structures to take into account. These exercises produce the richest insights when done in a participatory way. The discussions generated by the exercises help to bring out the different perspectives at a time when these can be taken on board, and helps to create co-ownership and a shared focus for all involved.

In the data research phase, which comes after design, the stakeholder analysis is mentioned once again. In practice, the stakeholder analysis will usually be done once, but will consist of two levels of reflection: first as a context analysis exercise to make explicit all stakeholders, and subsequently to zoom into the key stakeholders and discover more about their interest in and influence on the data collection and data-based decision making in the project. The following questions are asked to address the second reflection:

  • What is their influence on the problem?
  • How might this person benefit from the project?
  • What could this person do with better data on the problem?
  • How does data support this person’s decision making now?
  • What could they do to undermine the project?
  • What is the best way to keep them engaged?
  • How can they contribute to a solution?

Factor analysis and mapping

Apart from stakeholders, external factors also need to be taken into account when designing a project. Are there any environmental, historic, political, cultural or socio-economic factors that are likely to have an affect on the success of the project, or impact that which the project affects? Identifying these factors will help to determine the problems and opportunities that need to be addressed. Documenting all factors at play will help to justify decisions on the project’s scope and focus. Helpful methods in performing a factor analysis are interviews with key stakeholders and data research. Read more on data research in phase three of this Handbook.

After performing a stakeholder analysis and a factor analysis, the relevant stakeholders need to reach a common understanding of the problem that the project is trying to address. What are the issues that lead to the overall problem and how are they interrelated? It can help to visualise the issues and their causal relationships. Pay attention to issues that may be overlooked by identifying gaps in your map. The mapping of issues will help to align everyone involved in the project on the most important factors to focus on.

A helpful method for mapping out issues is creating a problem tree or a conceptual model (Figure 4). A conceptual model is made by writing down on cards, in a participatory way, all the factors (problems and opportunities) related to the desired impact: what is hampering the achievement of this impact, why is it not happening now? The next step is to cluster the cards according to topics on a wall, and then organise the cards in cause and effect relationships on a map. Such a map, or conceptual model, helps to create a common understanding of the problems, how they are interrelated, and what the root causes are. It can also help to distill what interventions should be used to address the problems that the project is trying to address, and what the scope of the project should be.


Figure 4. Example of a conceptual model


In the factor analysis part of the context analysis, there is a close link with the research phase of data collection, phase three of the Handbook, in which we zoom into the more specific data collection and data-based decision making questions:

  • Context: What is the key problem that you are trying to address and why is it important?
  • Environment: What factors contribute to this problem?
  • People: Who does it directly affect?
  • Current problem solvers: Who is already working on this issue?
  • Time: At what time intervals are decision being made about this problem?
  • Existing data: What data relevant to this problem already exists?

References / Footnotes

  1. The size or scope of a project influences the time required to conduct a context analysis, design a ToC, and build a monitoring framework.
  2. In the next phase, data research, we use the same categorisation of stakeholders.

Acknowledgements

Authors: Anita van der Laan (Akvo.org), Annabelle Poelert (Akvo.org)
Contributors: Marten Schoonman (Akvo.org), Karolina Sarna (Akvo.org), Tarryn Quayle (Local Governments for Sustainability Africa (ICLEI Africa)

AfriAlliance

The Africa-EU Innovation Alliance for Water and Climate (AfriAlliance), is a 5-year project funded by the European Union’s H2020 Research and Innovation Programme. It aims to improve African preparedness for climate change challenges by stimulating knowledge sharing and collaboration between African and European stakeholders. Rather than creating new networks, the 16 EU and African partners in this project will consolidate existing ones, consisting of scientists, decision makers, practitioners, citizens and other key stakeholders, into an effective, problem-focused knowledge sharing mechanism.
Afrialliance-logo-RGB.png
AfriAlliance is lead by the IHE Delft Institute for Water Education (Project Director: Dr. Uta Wehn) and runs from 2016 to 2021. The project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 689162.
EU flag RGB.jpg