Introduction

Frequent and intense storms, widespread and destructive fires, shrinking water supplies, desertification, and changes to ocean environments increasingly evidence climate change. The impacts of climate change—which include slow or poor crop production, higher food and fuel prices, drought and famine, higher inflation, and slower economic growth—are especially severe for the poor, who often lack the resources to cope and adapt. As the Global Environment Facility (GEF, n.d.) explained,

Taking action on climate change means adopting and implementing ambitious programs to limit emissions of greenhouse gases to levels compatible with the well-being of the ecosphere, while supporting communities around the world to adapt to the unavoidable impacts of the climatic changes that are already being observed. It also means embracing the potential of the green economy—a more sustainable way of life that balances economic, social, and environmental priorities. (para. 2)

One such ambitious program that aimed to balance economic, social, and environmental priorities was a project to promote energy-efficient cook stoves in micro- and small-scale food-processing industries in Chad. Given its location and as a Sahelian country, Chad has a great solar potential of about 4.5 billion MWh/year and thus ranked 20 worldwide for its solar potential in 2008 (Price & Margolis, 2010). Despite this, biomass accounted for 94% of the primary energy supply in 2008, and only 2.2% of Chadian households had access to electricity; of these, only 1% were outside of the capital city of N’Djamena.Footnote 1 About 79% of the energy supply in urban and 90% in peri-urban and rural areas in Chad derive from ligneous sources. Burning firewood for fuel produces greenhouse gases directly, which has negative health effects on human health, and reduces the capacity of forests to act as carbon sinks. Moreover, the imbalance between the firewood supply and demand accelerates desertification and poses concerns for rural, peri-urban, and urban development. To curb high deforestation rates, the Government of Chad passed an act in 2009 that prohibited cutting green wood (Republic of Chad, 2017; United Nations Development Programme [UNDP], 2017).

However, an outcome of the act has been to triple the price of wood in the market, which has had a negative effect on micro and small entrepreneurs in Chad, especially beer brewers and meat grillers in N’Djamena. A 2010 study on firewood consumption estimated that the beer brewing and meat grilling sectors in N’Djamena alone consume around 14,000 tons of firewood per year, with more than 3300 cabarets (the equivalent of bars) and 2300 meat-grilling stands (Vaccari et al., 2012).

The traditional breweries produce a local beer called bili bili, a sorghum-based alcoholic drink that requires intensive cooking for 10–16 h per batch. The breweries are cottage industries that are exclusively operated by women entrepreneurs in the back yards of their houses. Each brewery employs four to 15 women and produces two to three 150- to 200-l batches of beer per week. Each batch of beer requires at least 54 kg of wood. The stoves used are outdated and highly inefficient, further increasing the amount of wood needed per batch.

The tchélé, in which men smoke or grill various types of meat, represents another sector that consumes a lot of firewood. Run by men, the tchélé is either a simple street booth or a small shop that includes an extension with benches where patrons can consume the meat on the spot. A tchélé employs between two and five people, depending on its size and on whether they also do butchery-related work.

To take advantage of the opportunity these traditional industries presented to improve energy efficiency and reduce demand for firewood, the Global Environment Facility (GEF) financed a project implemented by the United Nations Industrial Development Organization (UNIDO) in 2014, titled “Promoting Energy-Efficient Cook Stoves in Micro- and Small-Scale Food Processing Industries.” The project drew on lessons learned from a similar intervention in Burkina Faso (UNIDO Office of Independent Evaluation, 2015). Project components included improving the design of cook stoves to achieve optimum fuel efficiency, creating sustainable financial programs to finance acquisition of the energy-efficient cook stoves, and improving the business performance of micro and small entrepreneurs. This project also included a monitoring and evaluation component with the express intent of facilitating smooth and successful project implementation and sound impact. This chapter explores the role of evaluation in a development project, with discussion of the project background; the evaluation findings including project performance, coordination and management, gender mainstreaming, cross-cutting issues, and conclusions; and a summary of the importance of evaluation in such a project.

Project Background

High firewood consumption exerts a negative impact on the environment and on the livelihoods of micro and small entrepreneurs in Chad. Traditional food processing, such as brewing beer and grilling meat using large commercial cook stoves with low energy efficiency due to incomplete combustion of firewood, requires long cooking times and high consumption of firewood. The use of energy-efficient cook stoves could reduce firewood consumption by at least 50% and considerably decrease cooking times. As a corollary of the shift to energy-efficient cook stoves, entrepreneurs would also be able to increase their income and profit margins by decreasing their fuel costs and optimizing the production process. The shift would further contribute to decreasing greenhouse gas emissions and reducing deforestation rates and negative impacts on health.

Prior to this project, two local cook stove manufacturers had developed prototypes for energy-efficient stoves with support of the Association pour le Développement de Micro-Crédit and the Association of Appropriate Technology for the Protection of Environment (ATAPED). However, these prototypes had not yet been commercialized and disseminated into the Chadian market because the manufacturers faced constraints related to limited financial resources, high costs of raw materials, absence of relevant technical training and support, and weak market demand. These barriers prevented the local manufacturers from producing and selling their energy-efficient stoves at a market scale. The following were the main barriers to the introduction of energy-efficient stoves in the micro-scale food-processing sectors in Chad that the project aimed to overcome:

  • Technology: Improved cook stove technology should not only be appropriate to the needs of each sector/usage, but also affordable, easy to use, durable, widely available, and socially acceptable or desirable.

  • Financing: Producing and selling improved cook stoves would require significant investments to adapt the technology to users’ needs and further improve the energy efficiency and performance of the stoves for market.

  • Dissemination: Shifting from traditional, energy inefficient stoves to improved models would require adoption at a larger scale (demand vs. supply sides).

  • Resistance: The introduction and dissemination of new energy-efficient technologies often faces reluctance to change; thus, a strategy was needed to overcome resistance and enhance acceptance among users and consumers.

  • Awareness: Lack of awareness about the benefits of a new technology can stall buy-in and uptake. Benefits of potential new energy-efficient technologies are reduced fuel consumption and costs, and improved production processes, health conditions, and livelihoods.

The GEF-funded project sought to address some of these key barriers to the introduction of energy-efficient stoves among beer brewers and meat grillers in Chad. Therefore, the primary targets of the project were beer-brewing and meat-grilling micro enterprises, cook stove suppliers and local project developers, and experts from relevant policymaking and implementing institutions. The project aimed to create a market for energy-efficient stoves in three ways:

  • Promoting energy-efficient stoves that consume 50–80% less firewood for processing food.

  • Developing clustersFootnote 2 within the beer brewing and meat grilling sectors to support demand for improved cook stoves, generate collective gains, and empower female and male entrepreneurs.

  • Facilitating access to finance to aid in acquiring improved cook stoves by implementing a credit and savings program and linking to voluntary carbon markets to generate additional revenues for end users switching to improved cook stoves.

Four intervention zones were identified as the focus areas for project activities based on two criteria: (a) high concentration of the selected types of enterprises and (b) proximity to N’Djamena, which is one of the main intervention zones, employing more than 40,000 beer brewers and 2300 tchélé workers. Table 1 lists the project’s stakeholders and their roles in the project.

Table 1 Project stakeholders (UNIDO, 2013)
Full size table

The initial project budget was $2,600,000 in both cash and in-kind support over 2 years (2015–2017), with a final deadline in 2018 to accommodate pending activities and outputs. A significant budget shortfall from the national government of Chad and the National Agency for Domestic and Energy and Environment Development (AEDE) reduced the project budget by more than half. This was due to increased engagement of the national government under the Special Fund for Environment (FSE) in supporting civil society organizations (CSOs) with small-grant interventions related to environmental preservation.Footnote 3

Evaluation Findings

To enhance future projects, identify lessons learned, and provide recommendations, the evaluation team implemented evaluation activities (see the Appendix for the evaluation methodology) and prepared a final report. The team met with a sample of 18 key informants, four groups/cooperatives of blacksmiths, 14 groups/cooperatives of meat grillers, and 25 groups/cooperatives of beer brewers. The evaluation report was organized around project performance with targeted evaluation criteria and discussed project coordination and management, gender mainstreaming, and cross-cutting themes. It concluded with recommendations and lessons learned.

Project Performance

The project performance component of the evaluation targeted four criteria: project relevance, effectiveness, efficiency, and sustainability.

Relevance

The evaluation team found the project design to be relevant and aligned to several Chadian national policies. First, the project fit into the goals of the National Development Plan 2017–2021 (Republic of Chad, 2017) and the national poverty reduction plan (Republic of Chad, 2010) with respect to its five strategic axes:

  1. 1.

    Strengthening the resilience of agricultural, forestry, and fisheries systems

  2. 2.

    Promoting actions to mitigate greenhouse gas emissions

  3. 3.

    Improving sustainable access to diversified energy sources

  4. 4.

    Preventing risks and managing extreme weather events

  5. 5.

    Strengthening the capacity of institutions and actors in the fight against climate change and enhancing instruments and capacities for mobilizing climate finance

Moreover, as Chad is a Sahelian country that is exposed to advancing desertification, climate change, and environmental degradation, the project also supported the energy sector’s policy framework (Ministry of Economy and Planning, Republic of Chad, 2008). The framework was established to promote technical and economic support for optimum energy development to reduce national dependency on wood fuel, which represents about 96.5% of energy consumption, thereby supporting the climate change strategy (Republic of Chad, 2017). Several of its priority actions aim to promote alternative energy sources and energy efficiency, such as extending the prohibition on cutting wood for fuel, using butane gas, and developing efficient domestic energy—actions that need to be strengthened to find substitutes for wood and charcoal, which are generally used to cook food and bake bricks for building houses.

Key informant assessments indicated that the project was unique in focusing on productive activities such as cook stoves for brewing beer and grilling meat. The project met one of the key priorities of both UNIDO and GEF strategic policy for renewable energy and energy efficiency (UNIDO, 2009b), which aims to help pro-poor actors in the Global South enhance their access to modern services and increase the viability of their small-scale industries by augmenting the availability of renewable energy for productive uses (UNIDO, 2009a). Thus, GEF resources allocated to support the dissemination of energy-efficient cook stoves in the traditional food-processing industry met one of the government’s priorities to fight desertification and climate change (UNDP, 2017).

The project was relevant despite its limited scale; it targeted 15–30% of existing beer brewers and meat grillers in the selected project zones, a figure limited by the reduced financial resources of the project. The project linked to six of the Sustainable Development Goals (SDGs): 1 – poverty reduction; 3 – good health and well-being; 5 – gender equality; 7 – affordable energy and clean energy; 9 – industry, innovation, and infrastructure; and 13 – climate action. The project was also in line with Chad’s Poverty Reduction Strategy Paper (Republic of Chad, 2010), reflecting the national development policy that prioritizes the food-processing sector while focusing on protecting the environment and promoting the private sector.

Effectiveness

Evaluation of the project’s effectiveness included whether the project had achieved its expected outputs and outcomes. As a pilot program, the evaluation team considered the objective of improving the adoption of cook stoves to be about 70% achieved given the targeted number of beer brewers and meat grillers in the selected clusters. Three models of cook stoves were tested with these two cluster sectors. Beneficiaries rejected the first as too expensive and poorly adapted to local needs. The second model had limited capacity and was therefore also rejected, especially by women beer brewers. The third model, which the beneficiaries ultimately adopted, provides some protection from fire and harms, is cost efficient, and consumes about 35% less firewood than previous cook stoves.

The second project output was to create a sustainable financial mechanism for beer brewers and meat grillers to purchase the new cook stoves. Although awareness meetings were conducted in the cluster zones, no loan had been granted to any cooperative at the time of the evaluation. This was due to delay on the part of UNIDO in signing the contract and providing the guarantee fund to FINADEV, attributable in part to UNIDO’s not having a physical presence in Chad. Despite the lack of an agreement with the Union for Credit and Loan (UCEC) in Mandélia, 25 cooperatives opened and were operating accounts without loans. The project succeeded in linking cluster groups to the formal financial system by providing all members with an individual savings account at the FINADEV and at the UCEC. However, the project budget shortfall compromised several activities planned for Output 2, including lack of training for project developers on project identification and development.

The third and final output was to improve the business performance of micro and small enterprises through clustering. Gathered in cooperatives, beneficiaries were trained on several themes such as enterprise management, cooperative governance and financial literacy, and financial mechanisms using microfinance tools and marketing techniques. The objective was for cluster members to improve their business performance and profits by at least 40%. However, evidence indicated that the project focused primarily on cook stoves for energy efficiency to the exclusion of other training. Only about ten cluster associations were organized in all of the cluster zones, but all of these associations were formalized with official cooperative authorization. The positive feature of this output was the linkage of cooperative units to energy-efficient cook stove distributors.

The project completed about 75% of its objectives under the expected budget shortfall; the financial constraints did not permit completion of the overall planned activities. Evaluators did observe several unplanned changes among the meat grillers of N’Djamena and the beer brewers of Mandélia—the clusters with the greatest level of adoption of the energy-efficient stoves.

Efficiency

The evaluation team found the project to be very cost-efficient. Despite the budget constraint, the project management unit was able to ensure a good value-for-money management process based mostly on signed memorandums of agreement or financial conventions, with payment based on delivery of technical reports and site visits when necessary, and/or on evidence from previous financial segment and reporting. Completion of project activities/outputs was high, considering that the project budget was 40% of the amount originally planned. Given this working environment, the evaluation team was impressed with the good quality of expertise provided.

The achieved project outputs were completed in a timely manner despite the lengthy administrative process for compliance with the UNIDO legal system and procurement obligations, exacerbated by the organization’s lack of a physical presence in Chad. The evaluation did not find any deviation between the disbursements and project expenditures, and the level of achieved outputs affirmed satisfactory project cost-effectiveness.

Although the evaluation determined that the project was cost effective and project outputs were timely, monitoring of the production of improved cook stoves and distribution was less effective. The evaluation team found inconsistency in the new cook stoves distributed to cluster cooperatives even from the same production line, either in terms of size or the quality of iron used, which compromised the functionality and affected adoption of the new stoves. This situation had potential to significantly affect adoption of the technology and its dissemination. A final concern was that the evaluation found no established partnership between the project and other donors’ projects; however, synergies were developed internally with sectoral ministries working in environment and energy such as the Ministry of Environment, the Ministry of Oil and Petroleum of Chad, and AEDE.

Sustainability

The evaluation team found thematic training and awareness meetings to be assets for the sustainability of the project. Moreover, training provided to the existing cooperative of blacksmiths was also helpful. Other assets could support the sustainability of the achieved outputs such as creating a community-based forest and providing support for existing traditional credit mechanisms, namely tontines, as informal opportunities for private ownership of cook stoves. However, the lack of knowledge among beneficiaries about the credit system approach and accountability and the delay in establishing the financial mechanism were negative factors that would likely affect replication of project results. Another factor was illiteracy on the part of most beneficiaries.

Implementing the planned exit strategy could sustain this pilot project’s outcomes in the middle term, including an improved financial program coupled with a peer-to-peer learning strategy to support synergies among cluster groups. Training on the development and operationalization of future cluster teams and a coaching approach by cluster development officers would serve as key drivers for this skills-transfer strategy. At the time of this writing, the team expected the post-project sustainability plan of large-scale dissemination of energy-efficient cook stoves to have a significant impact on greenhouse gas reduction and improvement of beneficiaries’ living conditions. These measures, coupled with the knowledge transfer, could support synergies within individual and cooperative unions at all levels. Cluster members are expected to fund the platform through a fee of $0.50 paid to the cluster cooperative on each energy-efficient cook stove sold. The collection of this symbolic fee will be encouraged to improve the platform continually and to meet user needs while protecting the environment. However, the evaluation did not find any practical measure related to establishing the fee, such as a signed agreement.

The evaluation team found that sustainability of the energy-efficient cook stoves was somewhat influenced by the operational costs for raw material. An in-depth assessment of whether new cook stoves built using local materials could be more efficient than the energy-efficient cook stoves made with iron could provide a useful further measure.

The project’s visibility was relatively weak. For instance, limited attention was paid to the involvement of media, use of flyers, and development of publicity spots and/or brochures, which could have improved buy-in from beer brewers and meat grillers not targeted by the project. Another shortcoming was that the project was not able to schedule peer-to-peer learning missions with other countries’ beneficiaries for knowledge sharing, such as in Burkina Faso, Sierra Leone, or Kenya, where similar projects have been tested.

The evaluation considered the project’s original aim of linking the voluntary carbon markets to generate additional revenues for businesses that switched to the energy-efficient cook stoves. Evaluators found that because the stoves for the beer-brewing and meat-grilling sector consumed less firewood than bakers and brick construction, sectors that have previously benefitted from voluntary carbon markets, the demand was not great enough to pursue this approach.

Project Coordination and Management

The project adopted in-depth consultation with stakeholders. The evaluation team found that gathering key actors in a steering committee was an asset to managing the project activities and to achieving the project outputs given the ongoing budget shortfall. The project’s approach incorporated a signed financial convention with service providers such as Verichad and FINADEV, and the overall procurement and provision of funds were managed by UNIDO with the local assistance of AEDE. Final energy-efficient cook stoves were distributed to the five cluster groups later than originally planned due to delays in signing these contracts. The varied availability of the technical assistant from UNIDO headquarters also led to irregular scheduling of meetings of the steering committee; regular meetings to monitor and report on project activities would likely have reduced or eliminated delays.

Under the leadership of the project assistant, cluster agents ensured permanent monitoring of field activities by collecting progress information, including feedback from partners and beneficiaries. This information resulted in ten timely progress reports on the project. The technical assistant from UNIDO carried out five field missions, each of which provided opportunity for performance review and generating corrective measures or action plans. The evaluation team appreciated the adaptive exit strategy, which was based on performance review of achieved project outputs to support the financial program.

Gender Mainstreaming

Project design and implementation phases ensured a gender balance of women and men in project activities, including the project’s management and participants. Strategically, the project was aligned with the National Gender Policy (NGP), which aims to serve as a guiding instrument for integrating the concerns and specific needs of women and men, especially for pro-poor targets (United Nations High Commissioner for Refugees [UNHCR], 2012). The project targets one of the most vulnerable small-scale industrial business groups in Chad, and the evaluation team found that the baseline study did include gender analysis with planned gender-related project indicators. About 65% of project beneficiaries were women and 35% were men. On the project management team, the project manager and the technical assistant were women, with three women among the eight people in the project management unit (38%), and two among the eight (25%) on the steering committee.

The evaluation found no negative factors that might affect the gender mainstreaming within the course of the project. The precariousness and inaccessibility of energy sources affect women more, but this factor decreases dramatically the longer women spend on production- and capacity-building activities. The training modules provided to women entrepreneurs helped to empower and integrate them into the energy-efficient stove value chain, especially in the business management, marketing, saving, and credit mechanism. The project’s comparative advantage has been having a balanced gender equity focus on female and male business groups.

Cross-Cutting Issues

By considering both the context of and cultural practices around the beer-brewing and meat-grilling businesses in Chad, the project covered cross-cutting thematic issues. The participatory approach, adopted with in-depth consultative meetings, was a key asset in this regard.

First, all cluster agents were recruited locally and each was assigned to their zone of greatest concern given their long-term knowledge and working experience in the project zones. This approach enabled the project to work well with local and traditional authorities and with respect to local practices and concerns. The choice of business model was aligned with traditional food-processing practices with traditional stakeholders. For decades, non-Muslim women have managed beer brewing industrial processing, while Muslim men have largely dominated meat grilling industrial processing. Chad is primarily a Muslim country, with more than 75% of the total population identifying as Muslim, thus requiring that all animals be slaughtered as required by Muslim practice either for household consumption or for sale in a tchélé.

Finally, the project management, implementation, and evaluation fieldwork followed cultural and contextual practices. For instance, the last 3 days of the evaluation fieldwork coincided with the beginning of Ramadan. Therefore, meetings were scheduled within nonprayer times.

Conclusions

The evaluation team found that the project linked to the SDGs and aligned with Chad’s Poverty Reduction and Growth Strategy (Republic of Chad, 2010), reflecting the national development policy that prioritizes the food-processing sector while focusing on environmental protection and promotion of the private sector. In terms of performance and results, the project design was relevant and aligned with several Chadian national policies, such as the national development plan (Republic of Chad, 2017); the policy framework of the energy sector, which considers measures as gathered by the energy sector master plan (Minister of Economy and Planning, Republic of Chad, 2008); the NGP, which considers gender mainstreaming to provide gender equity-focused interventions for both men and women (UNHCR, 2012); and the national strategy to combat climate change and support environmental preservation in Chad (UNDP, 2017). The financial support has been helpful to the beer-brewing and meat-grilling sectors.

The project was effective in achieving 75% of its goals. It included individual business processor actors in cooperatives and provided timely training on themes such as enterprise management, cooperative governance and financial literacy. Individual entrepreneurs appreciated the financial mechanism available, but establishing a sustainable financial mechanism remains an ongoing challenge.

Project management adopted in-depth consultation with stakeholders and engaged participation from female and male food processors. This equitable development of the gender dimension succeeded with consideration of cross-cutting themes such as working within the local context and cultural practices of the targeted zones.

The project has been cost-efficient, providing a good value for money, and cost-effective despite the budget shortfall of about 45%. However, the project failed to initiate partnership with other donor projects. The ongoing exit strategy could sustain the project outputs for a middle-term duration; this strategy focuses on an improved financial mechanism coupled with peer-to-peer learning to support synergies among cluster groups.

The evaluation team provided several recommendations to support continued progress on the project’s objectives and for UNIDO’s potential future collaboration in Chad. For the project management team, evaluators recommended continuing to monitor and coordinate efforts and increase attention to awareness and close collaboration with cluster groups, including the blacksmith cooperative to complete the cycle of the value chain. Second, the management team should continue to pursue the mobilization of resources committed by AEDE FSE to support the project.

The evaluators recommended that UNIDO consider signing tripartite agreements with AEDE and local contractors and promoting nonfinancial collaboration with other microfinance institutions, such as UCEC in Mandelia, to cover other clusters. A second recommendation was developing an evidence-based operational exit strategy and enhancing visibility of project outputs. Specifically, UNIDO should activate the financial program as soon as possible by coupling it with peer-to-peer learning to increase synergies among cluster groups. An audience-tailored approach to communication would further promote the project and improve awareness. Another recommended step for UNIDO was assessing other prototypes of energy-efficient cook stoves built using local material such as mot or bricks. Finally, the evaluators suggested that UNIDO consider sustaining the energy-efficient technology in future projects by including a field technical advisor and promoting partnership by aligning and enforcing the exit strategy with a framework on capacity development with other donors’ projects.

Four overall lessons shared by the evaluation team offered insight for future projects.

  1. 1.

    In a sustainable development project dealing with energy-efficient cook stoves, consulting in depth with targeted beneficiaries and testing various prototypes are key steps toward the adoption of a technology by cluster groups.

  2. 2.

    In a cultural and contextual development environment with high female illiteracy, like Chad, having a woman cluster agent can generate buy-in from female participants.

  3. 3.

    Considering the size of this project and the huge amount of groundwork required at the community level, an important step is ensuring a manageable number of targeted clusters and beneficiaries to demonstrate success to scale up in future programs.

  4. 4.

    Considering local context is important when selecting energy-efficient technology to ensure suitable infrastructure and affordability for targeted beneficiaries.

Value of Evaluation in Development Projects

The energy-efficient cook stove project in Chad illustrates the critical importance of evaluation in development projects. Clearly, one important aspect is the ability to demonstrate effectiveness and efficiency in using donor funds, but perhaps more important to the long-term success of a project is ensuring its relevance and sustainability for the project participants and communities. In this case, the project was found to be relevant to several goals related to both the environment and to private sector business development. Evaluation identifies areas for improvement to enhance performance of a project while it is underway and for future similar efforts, and also recognizes aspects that work well. With this specific project, an identified difficulty was the delayed or reduced funding. Features of the project that were helpful and instructive for future undertakings were extensive stakeholder engagement and attention to the local context and customs.

As for sustainability, the evaluation hinted at a mutually reinforcing ecosystem in which beer brewers and meat grillers gain the financial benefits of reduced fuel cost and come together in clusters to improve their business skills and for mutual support for financing further purchases of energy-efficient equipment. This, in turn, supports the growth and development of a local industry for producing energy-efficient cook stoves.

Evaluation has an essential role in movement toward transformational change, as this project demonstrates. By identifying specifics of a project’s successes and shortcomings and pinpointing the project aspects that directly affect its outcomes, evaluation highlights opportunities for scaling up an effort from a pilot to a broader initiative, which can deliver more successful outcomes for people and environmental measures. Also aiding in transformation is the cumulative effect of evaluation from one project to another. Just as this cook stove project in Chad built upon success identified in evaluation of similar projects in other nations, future efforts to develop environmentally beneficial opportunities for small entrepreneurs can build on this project’s evaluation.