Main knowledge bank page | Recent additions | Recent changes | What links here | Categories | Category cloud
How-to guides | Organisation profiles | Project profiles
 

edit this page

MandE Toolkit - Six steps to Results Based Monitoring (RBM) by Melanie Djédjé


Table of Contents

Boiling Point
Front cover of Boiling Point issue 55
Issue 55 (2008) Toolkit
ArticleToolkit
AuthorMelanie Djédjé
The following six steps should help you to set up your Results Based Monitoring (RBM) system. From identifying all stakeholders in the system, formulating the assumptions on which your strategy is based, analysing risks and side effects, choosing observation fields, specifying indicators for measurement up to their operationalisation to data collection – and there you go!

Monitoring is a systematic observation of a given situation and the changes that occur. It may focus on activities carried out by a project or on services being delivered. With the new responsibility given to development actors to achieve envisaged development results the attention has in recent times switched to the monitoring of results, not just activities.

Results are the changes occuring as an effect of a development intervention and they can be attributed to it. They may be positive or negative, intended or unintended. The project’s strategy aims to achieve positive results by carrying out activities that produce certain products or services (i.e. outputs) for specific user groups. If those users make use of the services they will change their way of doing a certain thing and experience a change or obtain a benefit (i.e. an outcome) as a result. These outcomes may contribute together with the results of other development interventions to higher development goals (these are impacts). Certainly, the project’s impacts cannot be attributed to just one single project or programme – they are beyond the “attribution gap”. The Results Model shown in Figure 1 helps to visualise the assumptions of a given strategy.

Figure 1: The Results Model as adapted by GTZ
Figure 1: The Results Model as adapted by GTZ


Two boundaries are of specific interest: the system boundary separating the project system which is under the control of project management from its environment; and the limit of responsibility indicating the level of the goal the project is meant to achieve. Other elements are risks that may have a negative influence on the results chain and side effects that might occur as unplanned results from the development intervention.

Preferably, the whole monitoring process, from its design to the active data collection and use should be done together with partners and – whenever possible – with target groups. Thus, different viewpoints on the occurring changes can be shared and valued at the same time. The joint development of the strategy contributes to increased ownership for the development action.

There are six steps to setting up a Results Based Monitoring system:

[top] [end]Step 1 – Identifying the system boundaries

A project can be perceived as a system which has control on its resources or inputs. With them, it generates services for users outside the system and beyond its control. It is helpful, as a first step, to identify all actors that participate together with the project team in service delivery and to distinguish them from the users of the outputs.

Staff of partner organisations, external experts, NGOs, government institutions, actors from the private sector or even representatives from the users’ groups may thus participate in the generation of outputs. They are therefore part of the project system and under its control, because they are paid for their work or because they are bound to a mutual agreement. On the other side of the system boundary are the potential users of the outputs. Whenever they have an interest in using the output or see an advantage in it, they will use it. However, the project cannot control them. They could be from the private sector, from government institutions, from NGOs or among the general population. Some actors might have a double function – contributors to the service delivery and users of the service.

Guiding questions:
  • Which actors participate in generating outputs?
  • Who are the users of the outputs? What interest do they have in these outputs? What advantage do they see in using them?

[top] [end]Step 2 – Formulating the results chains

The results chains of a project reflect the underlying hypothesis of its strategy and constitute the foundation of the whole RBM system. In an ongoing project one could start with the main products and services delivered. For example, in a typical stove project there might be main outputs for the stove producers (e.g. training), for the stove users (e.g. awareness campaigns), for other NGOs (e.g. training manuals) or for stove dealers associations (e.g. organisational development). For each main output the corresponding results chain would be formulated.

Next, the main activities necessary to achieve the output would be formulated. For example, the main activities for the output “training delivered to stove dealers” could be: a) to develop concepts for technical and business training; b) to conduct training courses for stove producers. It is not necessary to go into details of the sub-activities as this is not part of operational planning.

A useful third step in the process of developing Results Chains is to specify the supposed use of the outputs by the users. For example, the stove dealers may have received good training in new techniques for stove production, but if this knowledge is not applied, then no improved stoves would be produced and no change would occur. However if they implement the new techniques then good quality improved stoves should be produced, leading to them being available on the market. This direct result is at the same time a goal for the project.

Continuing along the results chain will generate further indirect results that might occur in the medium and long run and to which the direct result contributes. These impacts could be a) more improved stoves in use; b) less pressure on natural resources; c) more income for stove producers, and so on. The Millennium Development Goals (MDGs) are part of the impacts.

Guiding questions:
  • What are the main outputs?
  • Which main activities lead to these outputs?
  • What do the users do in a different way when using the output?
  • What direct benefit is resulting from this?
  • What are the indirect results, to which the direct result contributes?

Box 1: Key terms according to the OECD-DAC glossary

Inputs: are the financial, human, and material resources used for the development intervention.

Outputs: the products, capital goods and services, which result from a development intervention.

Outcome: the likely or achieved short-term and medium term effects of an intervention’s outputs.

Impacts: positive and negative, primary and secondary long-term effects produced by a development intervention, directly or indirectly, intended or unintended.

Results: the output, outcome, or impact (intended or unintended, positive and/or negative) of a development intervention.

[top] [end]Step 3 – Analysing risks and side effects

Risks are external factors that may have a significant negative influence on the results chain. They can be influenced (for example by the intervention of other donors, conflict of interests among actors, etc.) or not (e.g. natural catastrophes, global economic developments etc.). As they can hinder the achievement of the project goals, they usually have to be reported to donors or other stakeholders. Project strategy should be designed in such a way as to minimize the negative influence of risks when they occur and limit the unwanted results as far as possible.

Side effects are unplanned results of a development intervention, that might be positive or negative, expected or not. As in the case of risks, they should also be monitored to allow for an adjustment of the project strategy if necessary. An analysis of risks and potential side effects can help make the stakeholders aware of them and to define alternative strategies.

Guiding questions:
  • What are the main risks that might have a negative influence on the results chain? What is their cause? What alternative strategy could be used to minimize their influence?
  • What are the potential negative side effects? What is their cause? What alternative strategy could be used to minimize their influence?

[top] [end]Step 4 – Choosing observation fields

Observation fields are those parts of the results chains that need to be monitored regularly in order to know whether the project is on target to achieve its goal. It is important to choose which of the results chain hypothesis, risks and side effects we need to be informed about- it isn’t necessary to monitor every part of the results chain!

Guiding questions:
  • What results hypotheses are particularly important?
  • What results hypotheses are uncertain?
  • What parts of the results chains are under negative external influence?
  • What negative side effect might occur?

[top] [end]Step 5 – Examination of given, and formulation of new indicators

Indicators are yardsticks that are used to measure results. They indicate what makes a difference, to what degree and until when. They should be precise, specific, realistic and measure a specific aspect of the desired result. Reference values (baselines) are required for any indicator so that the initial situation can be compared to the expected change. Indicators may be given by the donor, particularly for the project goal (level of direct result). For the lower levels, milestones or process indicators are formulated by the project team for their internal monitoring.

The given indicators should be examined and a clear common understanding developed. For all other observation fields chosen, new indicators are formulated to measure the expected or unwanted change.

Guiding questions:
  • What indicators are given?
  • What exactly do they mean?
  • What should be further specified, changed, and agreed upon?
  • Where in the results chains do they have an effect?
  • What additional indicators are necessary for the selected observation fields?

[top] [end]Step 6 – Operationalisation of indicators, data collection and use

Now the indicators have to be made feasible. It is necessary to decide who will collect what data, how it will be collected, at what frequency and, eventually, at what additional cost.

Also, the users of the data have to be identified to assure future distribution of monitoring results. Indicator sheets may be created with the specifications of each given and chosen indicator.

Guiding questions:
  • What initial value is available for a particular indicator? Where can information be found about it?
  • What data / information is needed to measure the indicator?
  • How will the data be collected? What method will be used to collect it?
  • How often and when will the data be collected?
  • Who will be responsible for data collection and processing?
  • Who needs the information, when, in which form and what for?
  • What additional resources are needed?

Now your RBM is ready to start! You just need to go to the field and collect the data for your observation fields chosen on the basis of your results chains. Then make use of the monitoring results for your project steering, learning processes and reporting. Have fun!

[top] [end]@HEDON


[top] [end]Download the original article

pdf file link Toolkit Six steps to Results Based Monitoring (RBM) including Hedon news (361 KB)

[top] [end]Contents: Boiling Point 55 - Monitoring and Evaluation

.
.
BP55: Theme Editorial - BP55: MandE of health and socio-economic impacts - Key lessons learnt - BP55: Results based monitoring in GTZ cooking energy interventions - BP55: MandE Experiences from the Field - BP55: Stakeholders have different interests - BP55: Low-cost temperature loggers as stove use monitors - BP55: So you finally bought a Combustion Analyser! - BP55: GTZ News - BP55: Practical Action News - BP55: GVEP International News - BP55: Monitoring and Evaluation Case Study - BP55: Monitoring and Evaluation Toolkit - BP55: HEDON News



edit this page

Page created: 23 June 2008; Last edited: 17 July 2008; Version: 4
Knowledge Bank text is available under the terms of the GNU Free Documentation License.

Pagename: BP55:MonitoringAndEvaluationToolkit @HEDON: RUJA