| Main knowledge bank page |
Recent additions |
Recent changes |
What links here |
Categories |
Category cloud How-to guides | Organisation profiles | Project profiles | |||||||||||||
Results based monitoring in GTZ cooking energy interventions - A burden or a benefit? by V.Brinkmann
Results based Monitoring (RBM) is an international monitoring method developed and agreed upon by the OECD DAC [1] to monitor development results [2]. GTZ has applied RBM in all its development interventions since 2003 [3]. A key element of RBM is the results chain. It describes the causal sequence for a development intervention with the necessary steps to achieve the desired objectives – beginning with inputs to implement activities, to generate outputs, the use of outputs by target groups, leading to the outcomes (the objective of development intervention), and finally contributing to impacts. The results chains of the Energy Advisory Project (EAP) in Uganda are shown in Figure 1 and illustrate the scaling up of improved biomass stove programmes. Two main chains are described here: for stove supply targeting producers and traders; and for stove demand targeting users and the public sector. A long these results chains the main fields of observation are identified and indicators developed for RBM [4]. [top] [end]Outcome based approach and EnDev monitoring
Until December 2007, the EnDev cooking energy interventions were implemented in 16 different countries, 11 of them in Africa, providing more than 2.5 million people with access to efficient and clean cooking technologies. Every six months the projects report the achievement of the outputs of their activities, the use of outputs by target groups and outcomes, according to their project-specific results chain. This requires a solid monitoring system for each project. For the outputs of the EAP Uganda work, monitoring was undertaken of a number of trained producers, the quality control system, marketing events, as well as the provision of information to and the knowledge of consumers. With regard to the use of outputs, the production and sales figures, quality and the purchasing awareness of households (HH), social institutions (SI) and small and medium enterprises (SME) were monitored. Finally, in order to assess the outcomes, the EnDev monitoring system measured the number of people with access to modern and clean cooking energy by checking sales figures and construction, as well as correct usage by HH, SI and SME. Some key outcomes for any intervention are reduced levels of biomass energy consumption and indoor air pollution. While technologies can be selected according to their performance, by testing for efficiency and emissions reductions in both the laboratory and at the project site, a more critical factor is the performance when used in the “real life” conditions of the HH, SI and SME. This aspect is strongly related to the users’ ability to achieve the desired operation and so they are given kitchen and firewood management training as part of each cooking energy intervention. GTZ projects also carry out testing in order to monitor performance at the household level. Further outcomes are not monitored as part of biannual reporting, but are nonetheless important for EnDev donors and so procedures for Impact Assessments have been developed by the EnDev monitoring team, including GTZ HERA [6]. [top] [end]MDGs & Impact assessmentThe EnDev cooking energy programmes stimulate impacts that contribute to the Millennium Development Goals (MDGs). Even though these impacts cannot be attributed directly to a GTZ intervention, and thus monitored, a plausible hypotheses on the projects’ contributions to impacts and MDGs needs to be provided.At a suitable period after implementation, when outputs have been provided and used, GTZ carries out Impact Assessments (IA) on the EnDev projects. In these IAs, cooking-specific outcomes such as generated income, women engaged, time and expenditure saved, indoor air pollution, and accident reduction are monitored and compared with the baseline. These outcomes can then be directly linked to a project intervention and their contribution to the achievement of impacts such as increased business development, women’s empowerment, decreased deforestation, decreased respiratory and eye diseases, and finally to the MDGs, can be assessed and plausibly demonstrated. Impact Assessments have been carried out in EnDev interventions in Uganda, Malawi and Ethiopia. The findings are impressive. About 300,000 Rocket Lorena stoves are in use in Ugandan households. Families save 3.1 kg of firewood per day or 1.1 tonnes per year, which translates into annual savings of 26 EUR (euros), equal to an extra month’s pay. The IA in Malawi demonstrated that households are using the saved money mainly for household matters (50%) and food (30%) and that they use the saved time mainly for housekeeping (54%) and farming (31%). In Ethiopia 170,000 households are using Mirt stoves for Injera baking. According to consumer ranking, fuel economy is the most important impact, followed by protection from fire/heat, reduced smoke, speed in cooking and better quality Injera. The households reported a saving of 30 EUR per year, with the money being used for food, electricity and education [7]. Further studies are being implemented in EnDev interventions in Kenya and Bolivia. For these IAs different tools and procedures were developed. The following procedure reflects the current IA in Kenya, which started with the formulation of the project’s results chains (Figure 2).
For the final analysis and assessment, a consultant team was employed. Where feasible, consultants can support the whole process of IA, beginning with definition of crucial indicators and questionnaire development. For the data analysis and assessment a set of guiding questions was made available by GTZ HERA. A high degree of participation of project staff, counterparts and local NGOs, students etc. is helpful and a matter for both capacity development as well as for the creation of ownership amongst all stakeholders. In Kenya, agricultural officers were excited about the enthusiasm of the target beneficiaries with one student who conducted the interviews even starting his own stove business. Such examples show the secondary effect of these monitoring instruments, which might not be expected, but are ultimately beneficial. [top] [end]Participatory Impact Assessment – an experiment?In some of the countries where ProBEC operates, IA interviews have been conducted by local stove artisans. In 2004, the stove promoters and producers from Malawi, Mozambique, Zimbabwe, Zambia, Tanzania and Kenya were invited to a ProBEC Workshop on ‘Experience Exchange on low-cost Clay and Ceramic Stoves’. For workshop preparation and a better understanding of impacts, these participants were asked to carry out IAs themselves in the form of a short impact questionnaire provided by ProBEC. During the workshop, when discussing the results of the IA with the producers and promoters, some very interesting feedback was given. For many it was the first time that they had talked to their customers about stove usage, difficulties, problems and demands.The stove producers, builders and promoters considered this assessment as so beneficial that they passed a resolution asking for training in monitoring and IA as part of their regular training programme. By asking the questions themselves they had realised that this increased their awareness of stove quality and improved their marketing skills, and thus their access to customers. In this sense participatory IA can increase the ownership of stakeholders and be complementary to IAs, but one of its limitations is information bias. [top] [end]Cost-Benefit-Analysis on the basis of IA dataIn two other cases IA results provided the starting point for a Cost Benefit Analysis (CBA) of cooking energy interventions. A CBA of the EAP in Uganda demonstrated the economic value of using the Lorena Rocket stove for individual households as well as for the public sector. Similarly for ProBEC Malawi, where more than 4200 institutional Rocket stoves are installed in social institutions like schools, hospitals and prisons, savings of between 12-38% of total catering budgets were made due to avoided firewood costs.[top] [end]Experience - pros and consExperience shows that the establishment of a solid RBM system is helpful for project management as it makes it possible to assess if the outputs offered are really used, and if this use really leads to an outcome relevant for development. If necessary, the project strategy can be adjusted, additional activities included or further key stakeholders involved. Additional RBM is of use for evaluation purposes and, as has been shown with some examples, for the creation of ownership. The major purposes of RBM are therefore the steering of interventions, accountability and contributions to internal learning and knowledge management.However, the remaining challenges should also be mentioned. It is important to realise that RBM is not an exercise that can be fully delegated to a consultant who occasionally visits the project. As a management tool RBM is the responsibility of the project manager as well as the whole project team, and as such needs to be included in project as well as budget planning. RBM is based on a complex model and so creates a lot of discussion among M&E experts. This means that capacity is required for its implementation and all relevant personnel should be well trained and skilled in the subject. In certain situations it makes sense to involve external consultants in RBM, especially when an independent view is required from a third party. This person should have expertise in qualitative and quantitative M&E methodologies, a minimum knowledge about cooking energy, experiences with statistical and analytical tools, and finally, adequate time. Experienced enumerators are helpful for larger samples. The interpretation of analysed data should be a participatory step and include both consultants and the project team, so that members are able to comment on analysed data and have a greater ownership. Following data interpretation, recommendations should be developed, discussed and agreed upon. A final celebratory step can be the presentation of results and recommendations to stakeholders but care should be taken to prepare this properly by inviting relevant stakeholders to celebrate their own results. This can be a good motivational basis for further activities. [top] [end]Way forward for GTZ HERARBM is a management strategy focusing on performance and the achievement of outputs, outcomes and impacts. The existing RBM system for cooking energy interventions will be further applied, findings will be made available and tools developmed. HERA will continue to contribute to international working groups like the GVEP M&EED, so that we can learn from each other’s experiences.For more information please contact GTZ HERA: hera@... or the author directly: Verena.Brinkmann@... [top] [end]Notes and References
[top] [end]@HEDON
[top] [end]Download the original article Results based monitoring in GTZ cooking energy interventions: A burden or a benefit? (365 KB) [top] [end]Contents: Boiling Point 55 - Monitoring and Evaluation
Categories: Boiling Point 55| Monitoring and Evaluation | |||||||||||||
Page created:
23 June 2008; Last edited:
17 July 2008; Version: 5 | |||||||||||||
Pagename: BP55:ResultsBasedMonitoringInGTZCookingEnergyInterventions @HEDON: AUJA | |||||||||||||
