Action on Climate Change: What Does It Mean and Where Does It Lead To?

Abstract

In 2014, the second conference on evaluating climate change and development offered the opportunity to take stock of evaluative evidence of the challenges, failures and success of climate change action. In 2011 one of the authors raised the possibility of a micro-macro paradox of climate change action (van den Berg, Evaluation 17:405, 2011): in his view evaluations of climate change action provided evidence that climate action works and achieves direct impact – yet climate change seems unstoppable. Several major, comprehensive evaluations were presented at the 2014 conference and provided an overview of actions taken and their successes and failures, as well as obstacles on the way to global impact. This chapter presents an overview of issues, evidence and the way forward for evaluators tackling climate action and sustainable development. The evidence provides support for the micro-macro paradox of 2011 and indicates that the global community has the technology and knowledge on how to stop climate change. However, actions that promote climate change still outweigh remedial climate action with at least a factor of 100. Thus current successes of programs and projects will not impact global trends, unless at the same time the non-sustainable subsidies and actions are stopped.

2.1 Introducing the Micro-Macro Paradox: Success at the Micro-level Does Not Lead to Success at the Macro-level?

In development economics the question whether aid contributed to economic growth was hotly debated after Mosley (1987, 139ff) identified this as the “micro-macro” paradox. He could not find any statistically significant correlation between development aid and the economic growth rate of recipient countries, taking into account other factors that cause growth. Mosley defended aid nonetheless, as benefits at the micro level were often shown to be substantial and essential. Nevertheless, economic growth was supposed to be the engine of future development that would make aid unnecessary, and if aid would not contribute to economic development, it could turn out to be ineffective in the longer run and not have meaning beyond just the benefits of a specific and localized project or intervention. Even if a project has significant short term outcomes, but it did not contribute to economic growth, it could be argued that the sustainability of its benefits are questionable.

A second milestone in this discussion was reached in 1998 with the publication of the World Bank report on “Assessing aid: what works, what doesn’t and why?” (Dollar and Prichett 1998), which focused on the role of aid in reducing poverty , and rekindled the micro-macro paradox discussion, as it better identified when aid could potentially contribute to economic growth: when countries had good policies, good governance and management and well-functioning institutions. The ensuing debate in development economics revived the micro-macro paradox, until in (2010) Arndt, Jones and Tarp aimed to close the arguments by demonstrating a positive and statistically significant causal effect of aid on growth in poor countries over the long run. “There is no micro-macro paradox”, they conclude (p. 27). What is interesting in their analysis is that they attribute their success in demonstrating evidence for growth to “methodological advances in the programme evaluation literature”, which have “improved the profession’s capacity to identify causal effects in economic phenomena” (p. 26).

Evaluation methodology thus has helped to solve the micro-macro paradox in development economics, according to Arndt, Jones and Tarp. If we accept that, let us explore whether evaluation methodology is also able to help us in solving the paradox of successful climate change interventions, versus a devastating trend of global warming and associated climate variability that does not appear to be influenced by climate change interventions.

The opportunity for a broad perspective on this issue presented itself at the 2nd International Conference on Evaluating Climate Change and Development, where several comprehensive evaluations of Climate Change aid were presented. They offered an opportunity for a meta-analysis of the results of some of the largest public sector efforts to address climate change in developing countries. Of special interest is whether these evaluations offer any hope regarding the micro-macro paradox of climate interventions that make a local difference but do not seem to impact at the global level.

Seven comprehensive evaluations will be assessed and the evidence they present of the discrepancy between micro and macro impact will be judged. The first four were presented at the 2nd International Conference on Evaluating Climate Change and Development; the last three were added as they emerged in the same year and complement the picture:

• 1. The Fifth Overall Performance Study of the Global Environment Facility, undertaken by the Independent Evaluation Office of the GEF;

• 2. The Independent Evaluation of the Climate Investment Funds, undertaken by ICF International on behalf of the five independent evaluation departments of the multilateral development banks;

• 3. The evaluation of climate change support in the Inter-American Development Bank, conducted by the Office of Evaluation and Oversight of the IDB;

• 4. The evaluation of the effectiveness of Swiss International Cooperation in climate change, conducted by a consortium led by Gaia Consulting Oy for the Swiss Agency for Development and the State Secretariat for Economic Affairs;

• 5. The real-time evaluation of initiatives of the Asian Development Bank to support access to climate finance , implemented by the Independent Evaluation Department of the ADB;

• 6. The real-time evaluation of Norway’s International Climate and Forest Initiative (NICFI), undertaken by a consortium led by LTS International;

• 7. The external evaluation of the UN-REDD programme (Reducing Emissions from Deforestation and Forest Degradation in Developing Countries).

Furthermore, we will also include the older but still highly relevant climate change evaluations of the Independent Evaluation Group of the World Bank (World Bank/IEG 2009, 2010, 2012), as well as the Fourth Overall Performance Study of the Global Environment Facility (GEF/EO 2010), the precursor to OPS5.

2.2 The Micro-Macro Paradox: Successful Climate Action But No Global Impact ?

The micro-macro paradox of successful environmental interventions was raised by one of the authors in a keynote address at the Second Global Assembly of the International Development Evaluation Association in Amman, Jordan, on 14 April 2011 (van den Berg 2011). The argument is that a sizeable proportion of interventions were demonstrated to have direct and long-term impact in the sense of achieving lasting success in for example reducing greenhouse gas emissions from a specific source, but they have made no impact on global environmental trends, that have continued their downward slide. This is the case for climate change, for the historical loss of biodiversity that is now increasingly seen as a human caused mass extinction to be compared with the extinction of dinosaurs 65 million years ago, and for the increasing pollution of our environment with chemical substances, which endanger human health and the health of our habitat.¹

A first indication of the paradox and its solution emerged in the comprehensive evaluations of the Global Environment Facility (GEF). The GEF was established as (interim) financial instrument of the main environmental conventions resulting from the 1992 Earth Summit, on climate change (UNFCCC), biodiversity (CBD) and some of the various conventions on chemicals (most notably Stockholm). For more than two decades it had been the core organization for support to developing countries and countries with economies in transition, raising a considerable amount of funding itself, and as a co-funding agency, an even larger amount from other sources. The GEF is replenished every 4 years by its donors. One of the important documents of this replenishment is an independent comprehensive evaluation of the performance of the institution up to that time. In the fourth Overall Performance Study (OPS4) of the GEF some elements of the micro-macro paradox were first explored (GEF/EO 2010). Interventions financed by the GEF had started in 1992 and the 2010 Overall Performance Study was the first to be able to report on the longer term impact of these interventions. OPS4 concluded that the processes set in motion by GEF co-funded projects were progressing toward longer term impact, provided follow-up actions were taken by countries and stakeholders. Nevertheless, global environmental trends continued to “spiral downward” (conclusion 1, p. 15). A first indication of why this was the case was provided in a calculation of the purchasing power of GEF funds over time: the fourth replenishment of the GEF, while nominally higher than the first replenishment, represented 83 % of the value of the first replenishment, while at the same time funding needs had increased dramatically (p. 16–18).

The Fifth Overall Performance Study of the GEF (GEF/IEO 2014) provides more details to the same arguments. It concludes again that environmental trends “continue to decline” (p. 10), whereas the “intervention logic of the GEF is catalytic and successful in achieving impact over time” (p. 13). Like OPS4, the evaluation focuses on funding levels to explain the paradox between evidence of impact and declining global trends. This time the context is broadened and includes public funding that leads to environmental decline. At the time of OPS5, annual commitments of the GEF had reached the level of US$ 1 billion. Overall public funding for environmental support to developing countries had reached the level of US$ 10 billion annually. However, funding needs for action on global environmental issues “are conservatively assessed as at least US$ 100 billion annually” (p. 17). Thus a funding gap emerges that in itself provides an explanation of the paradox. However, “global public funding of at least US$ 1 trillion annually is available for (…) unsustainable environmental practices, such as subsidies for fossil fuels” (p. 17). In other words, the GEF is ten times out-funded by others, whereas ten times more than overall public funding for global environmental public goods is required, and ten times more than that is actually spent through public funding on subsidies that destroy our environment . The paradox is thus revealed as a veridical paradox²: a seeming conflict between impact of GEF versus impact on global environmental trends, that is resolved if competing funding channels are taking into account.³

The questions we pose in this chapter are whether the other comprehensive evaluations provide further evidence for this; whether the historical path that the GEF has followed to arrive at this situation has been matched by others, or whether they have been able to tackle the barriers to impact; and whether their specific routes offer insights into what may be done to increase the chances of success at systems level – i.e. whether they provide insights into what works, when and where, for whom and under what circumstances to achieve success in humanity’s efforts to address the potentially disastrous consequences of climate change over time.

2.3 From Early Results to the Slow Materialization of Impact

In the years leading up to OPS5 the project portfolio of interventions supported by the GEF has matured over time, since its inception in 1992, to enable a judgment on the effectiveness and impact of these interventions. The First and Second Overall Performance Studies of the GEF were not able to provide comprehensive assessments of the results and impact of the GEF, due to the fact that many interventions had just been completed or were still on-going at the time of the evaluations⁴ (this paragraph based on ICF 2005, 21–22). The Third Overall Performance Study was expected to be the first to report on results and impact, and it had to disappoint its readers on impact. It was able to report on results, as these were mostly at the outcome level. On longer term impact the OPS3 team was confronted with “general unavailability of impact-level results data” (ICF 2005, 21). Several reasons were identified why these data were unavailable: lack of an overall results measurement framework including baselines, indicators and targets; lack of efforts at the project level to generate data; lack of systematic efforts to conduct “end-of-project” evaluations and perhaps most importantly: the time horizon. Whereas GEF projects on average take no longer than 5 years, environmental change may take decades before it becomes measurable (ICF 2005, 22). However, OPS3 noted with muted optimism that monitoring and evaluation had become more important in the GEF and there was evidence of growing harmonization of goals and processes across the GEF (ICF 2005, 12). We will see these themes return in other organizations and their evaluations.

The Climate Investment Funds (CIF), initiated in 2008, were set up to overcome two obstacles that the GEF had to face: slow procedures and fragmentation of funding. The GEF had to spread its contributions over a large group of countries (more than 150) and not just in climate change, but in other priority areas such as biodiversity, international waters and persistent organic pollutants. The slow implementation of GEF interventions, also led to time delays in achieving impact, while time is of the essence in the fight against climate change. The CIF would focus on a relatively small number of countries, to enable it to provide higher levels of funding, “potentially allowing greater impact” (ICF 2014, viii) and it would apply a “light touch” approach to ensure quick decision making – relying on the multilateral development banks to provide the technical expertise to design, review and implement projects. However, up to May 2014 only a small proportion – about 9 % – of the approved funding had been disbursed to action on the ground (ICF, vii). The evaluation notes in 2014 that “most CIF projects are still on the drawing board or in early execution” (ICF, viii) and thus the effort to speed up procedures in comparison to the GEF largely failed. Failure to overcome the second barrier of insufficient funding to achieve longer term impact cannot yet be ascertained: the question cannot yet be answered.

Yet “transformative impact is a major goal of the CIF, and a justifiable one” (ICF, x). The evaluation notes that CIF resources, even though more focused and considerably higher than the GEF’s in its partner countries, “are small relative to global needs”, so they need to be focused on countries and on activities where they will be able to support transformative change. However, the evaluation also notes that many of the CIFs activities lack a convincing theory of change that provides a clear picture of how broader adoption would be achieved. On the positive side the evaluation commends the CIF for its learning and piloting objectives, and notes the “vast potential” for providing knowledge on how countries can respond to the challenge of climate change (ICF, xii).

The evaluations of climate change efforts of the World Bank Group go back in time from 2009 (when the first study was published) to 2012 (when the third report was published on the IEG website). They refer to a much broader and older portfolio of activities that the Bank implemented, many of which were undertaken with co-funding from the GEF. The longer term impact on several areas of work could be evaluated. However, the primary focus of many interventions was often on aspects such as support for energy policies, deforestation , low carbon technologies, and adaptation , and differed in how they related to climate change. The emerging picture is thus less straight-forward than the GEF assessment. Nevertheless, the three World Bank evaluations provide indirect support for the paradox and some hopeful signs of where the paradox may be solved.

First and foremost, the evaluations identify energy efficiency as a crucial pathway towards climate action that potentially funds itself.⁵ Well guided efforts toward energy efficiency tend to have economic returns that dwarf those of most other development projects, while at the same time resulting in lower greenhouse gas emissions. Especially the second evaluation (World Bank 2010, p. 32) identifies several promising avenues: efficient lighting that offers very high economic returns and significant emission reductions; reducing losses in the transmission and distribution of energy; large-scale efforts in energy efficiency may reduce the need for power plants (World Bank 2010, p. xv). The 2010 evaluation was one of the first to provide evaluative evidence that energy subsidies are “expensive, damage the climate and benefit the rich” (World Bank 2010, p. 119).

These findings in the World Bank/IEG evaluations (most notably the second evaluation) were further supported by evaluative evidence from the Asian Development Bank, the European Bank for Reconstruction and Development and the GEF. In a briefing note the Evaluation Cooperation Group of the multilateral banks noted strong evidence from independent evaluations that⁶:

• Energy efficiency investments are highly cost-effective;

• Fossil fuel subsidies discourage energy efficiency;

• The financial sector can be persuaded to provide energy efficiency loans;

• Genuine demonstration projects can transform markets;

• Biases against energy efficiency projects can be overcome.

However, presenting this evidence to the climate change negotiators could to some extent be characterized as “preaching to the converted” and the evidence for these points still needs to sway governments to reduce fossil fuel subsidies and promote energy efficiency.

The Inter-American Development Bank’s 2014 evaluation of its climate change strategy notes that the IDB has seen its largest contribution to greenhouse gas emission reductions from its support for renewable energy investments (mainly hydropower – IDB 2014, p. 34), rather than energy efficiency in which the Bank has not been as active. The 2014 evaluation aligns the IDB with the earlier ECG briefing note in suggesting that “improvements in energy efficiency have perhaps the greatest potential impact in reducing GHG emissions at the lowest costs”, for which energy subsidies “remain a key barrier” (IDB 2014, p. x). A second sector that turned out to be highly relevant for climate change was transportation: bus rapid transport systems and road projects, where especially the former have contributed to greenhouse gas emissions.

However, a lack of clear classification of climate change related activities and investments, as well as the lack of a transparent measurement system for greenhouse gas emission reductions, caused considerable difficulties in identifying which projects and loans of the Bank were of relevance for its climate change strategy and how much they contributed. The IDB climate change strategy was approved in 2011 to bring activities together that are relevant for climate change and to approach them in a more systematic way, to enable mainstreaming and upscaling. While the activities themselves have considerable history in the IDB, their relationship to climate change goals has been relatively recent. The benefit of an older portfolio is that it enables a look at finished projects, even if this means extra efforts to reconstruct what its specific contribution to climate change mitigation was. Compared to the CIF evaluation, the IDB evaluation is able to provide evaluative evidence on effectiveness, though longer term impact remains elusive due to measurement problems (IDB 2014, p. x). The Independent Evaluation Department of the Asian Development Bank in its real-time evaluation of the ADB’s initiatives to support access to climate finance also noted the difficulty of assessing the climate impact of activities that may have other primary objectives and the lack of a consistent framework for measuring greenhouse gas emission reductions (ADB/IED 2014, p. xi).

The evaluation of the Swiss International Cooperation in Climate Change from 2000 to 2012 develops the same argument for the portfolio of interventions it looked at. The focus on climate change is relatively new in Swiss cooperation, and many of the older projects were formulated and implemented from development and poverty alleviation perspectives. As a result, no consistent data sets are available to measure the impact of especially the earlier interventions on climate change (Gaia Consulting 2014, p. 9). Yet the portfolio scores high on effectiveness, showing “moderate to strong” effectiveness in reducing greenhouse gas emissions and increasing people’s abilities to cope with the impacts of climate change (Ibidem, p. 5). Due to the methodological challenges in evaluating a portfolio that emerged from other objectives, but is now seen as central to climate change efforts, the evaluation is reduced to noting that there are “numerous examples of successful emission reductions” but no overall picture emerging.

Norway’s support to Reducing Emissions from Deforestation and Forest Degradation (REDD+) through Norway’s International Climate and Forest Initiative (NICFI) has been evaluated in 2013 in a summative evaluation, looking at what had been achieved so far. This support has similarities with the CIFs in that funding is aimed to achieve impact through focusing on a few countries, so that the amounts of funding become catalytic. The evaluation concludes that the portfolio is “providing a substantial, direct contribution towards the conservation of natural forests ” (LTS International et al. 2014, p. xxii), and that it is “likely” that this will lead to higher level and long term impact, as the supported activities contribute to “sustainable development” (LTS, p. xxiv). Yet this would be dependent on future funding, which is uncertain – it is this lack of certainty that the evaluation proclaims to be the “single greatest risk” to the sustainability of the REDD+ initiatives. Important to note is that greater coherence and consistency has been achieved in measuring greenhouse gas emission reductions – the deciding factor no doubt was that this support was set up as climate change support from the beginning (see LTS, p. xxx).

The picture emerging from the Norwegian evaluation is complemented by the independent evaluation of the UN-REDD+ programme, undertaken in 2014. This evaluation concludes that the programme has been moderately successful in delivering outputs, whereas its overall (programme) effectiveness is rated as moderately unsatisfactory (Frechette 2014, p. iv). Its efficiency is rated as unsatisfactory: the three UN partners in UN-REDD+ continue to have their separate procedures, which leads to inefficiency in the management of the programme (Frechette, p. 30).

We may draw the following conclusions from this overview of the findings of the seven comprehensive evaluations, which are presented in Table 2.1 First of all, three conditions at the portfolio level emerge for an evaluation to be able to provide evidence of direct impact and of impact at the global level:

Table 2.1 Evaluative evidence on progress towards global impact in seven comprehensive evaluations

• 1. Only funding agencies that have steadily built a coherent portfolio focused on climate change can expect evaluative evidence on the impact of this portfolio; portfolio’s that are gathered from interventions with other aims as primary objective tend to show a lack of data related to climate change, different interpretations of what should be done and a wider range of activities to achieve outputs.

• 2. The portfolio needs to be coherent and mature to find solid evidence of direct impact; this is the case for the GEF only. The UN-REDD+ evaluation managed to gather evidence on the “likelihood” of impact and sustainability .

• 3. Only the GEF and UN-REDD+ have a consistent set of instructions for measuring greenhouse gas emission reductions. These instructions are still under development and will no doubt further improve over time; but they make it possible to aggregate GHG reductions at the portfolio level. The IDB, ADB and the Swiss Cooperation evaluations faced difficulties for using GHG reduction data because of the lack of coherence in the portfolio, with interventions now counted as important for climate change which were not set up for this purpose originally. Even though their portfolios are mature, they do not lend themselves to providing evidence at the impact level, as the lack of comparable data leads to problems of aggregation that cannot be overcome, at least not until the portfolios have matured further and measurement norms and standards are agreed.

The first important element of the micro-macro paradox is evident in the judgments on efficiency and effectiveness. Where these were rated, efficiency was deemed to be low or unsatisfactory. Where effectiveness was rated, evidence pointed in the direction of moderately satisfactory to fully satisfactory outputs. On the direct impact level, of amounts of GHG emission reductions in the new situation, only the GEF provided evidence at the portfolio level, but other evaluations certainly provided evidence at the intervention level, such as the IDB, ADB, and Swiss Cooperation. The only discrepancy in findings emerged between the NICFI and UN-REDD+ evaluations, where the Norwegian evaluation found a “likelihood of higher level and longer term impact”, but the UN-REDD+ evaluation rated the same as “moderately unsatisfactory”. Although the NICFI and UN-REDD+ evaluations overlap to a large extent (NICFI being the biggest donor to REDD+ initiatives), the difference may be due to a willingness or reluctance to look into the future. On global level impact the evaluations that were willing to enter into a somewhat reflective and speculative mode – i.e. the CIF and NICFI evaluations – came to similar conclusions as the GEF, that funding in these climate change initiatives remains relatively small to global needs and may also be unpredictable – thus putting a huge question mark on the global level impact of climate change interventions.

The main thesis of the Fifth Overall Performance Study of the GEF, that success at the micro level is not leading to a change in trends at the macro level because of funding issues, is thus supported in the CIF and NICFI evaluations . Various elements are mentioned in the funding gap: the subsidies for non-sustainable use of natural resources which are substantially higher than funding of international action against climate change; the relative low amount of funding versus the identified global needs; and the unpredictability of funding in the coming years. The core of the micro-macro paradox is further substantiated in the older World Bank evaluations and the briefing note that the Evaluation Cooperation Group of the multilateral banks developed for COP17 in Durban in 2010.

2.4 Surviving the Negative Effects of Climate Change

While the onslaught of climate change continues unabated, the relevance and urgency of adaptation to changing circumstances has been increasing. While this is still questioned in developed countries where climate change deniers hold office, many if not all developing countries are working on national priority and action plans for adaptation to climate change. While support for adaptation did not figure prominently in the early years of climate action after the Earth Summit in 1992, it has come to the foreground and is now seen as of equal importance as mitigation in guidance of the UN Framework Convention for Climate Change. The international portfolio for support to countries on adaptation is not as mature as the portfolio on mitigation. Furthermore, international agreement on a comprehensive framework for adaptation – what adaptation is, what it would be composed of and how it should be measured – is still developing. While UNFCCC and the Intergovernmental Panel on Climate Change have done important work in providing a first understanding of adaptation and how countries can develop national priority plans for adaptation action, evaluations have not yet delivered a critical mass of evaluative evidence .

It could be argued that it is not necessary to look at adaptation from a global perspective. To adapt or not adapt is not something that happens on a global scale. While greenhouse gas emissions lead to climate change for the globe, adaptation is by definition more local – if one country is well adapted, it does not lead to better adaptation in its neighboring countries. Furthermore, while greenhouse gas emissions can be measured through technical processes, there is no similar measurement for adaptation. No single indicator for adaptation will suffice. We argue that adaptation has at least three distinct dimensions: (1) changes in social and economic development that ensure that the outputs and outcomes are sustainable from a climate change perspective; (2) preparedness for and dealing with natural disasters that may increase in intensity due to climate change; (3) resilience of populations and societies to tackle unexpected changes in the natural environment that they are living in. The first dimension increasingly overlaps with mitigation action. While mitigation may be primarily directed at reducing greenhouse gas emissions, the micro-macro paradox establishes clearly that for these actions to be ultimately successful, systems need to change and become environmentally sustainable. This is the route towards durable emission reductions, and it is also the route towards increased adaptive capacity. For this reason we see an increasing use of the same transformative mechanisms for adaptation as for mitigation.

Adaptation and mitigation are two different but linked dimensions in social, economic and environmental sustainability . Adaptation concerns the ways in which the social and economic domains are “ready” for change in the environmental domain, and includes resulting actions. Mitigation focuses on one particular way society and the economy use natural resources and aims to make this use environmentally sustainable. Adaptation perspectives in mitigation often are termed “climate proofing” of actions; ensuring that the mitigation interventions will be resilient against climate change. Both adaptation and mitigation ultimately require action that transforms the interaction between the social, economic and environmental domains. One of us argued that sustainability is fundamentally an adaptation issue (van den Berg 2014, p. 34–35): “achieving a sustainable balance among civil society, the economy and the environment will require constant adaptation”. In this light we include some of the evaluative evidence on adaptation in our discussion of transformative action.

2.5 Three Priority Areas for Transformative Action

The seven evaluations and their predecessors also provide much information and evaluative evidence on how transformative processes can be set in motion and what is essential for these processes. A coherent picture emerges of action at the country level, from civil society, the private sector and the government; action which requires legal and regulatory amendments and changes in markets and behaviour in society; of engaging with civil society which collaborates or is the main actor for behaviour change; of engaging with the private sector which introduces new solutions and technologies that could together with changed behaviour lead to market change and transformation. A crucial cross-cutting issue is whether activities take gender, equity and inclusiveness into account, as they are essential to ensure the transformation will not just have an economic and environmental, but also a social impact . For this reason the next section of this chapter discusses briefly the interaction with civil society through the example of small grants provided to local communities, the introduction of new technology in collaboration with the private sector and the gender, equity and inclusiveness dimensions.

2.6 Civil Society Action Supported Through Small Grants

While civil society is active at all levels of governance, including the global level, it tends to be rooted in local organizations and action. Bottom-up action and representation are considered essential by many civil society organisations. Change in behaviour initiated by and in civil society often follows its own dynamics – some of it top-down, where governments impose rules and regulations, or behavior is modelled on the example of popular characters or opinion leaders, but often durable change is initiated at local levels and gradually (or quickly) spreading to the general population. Of the organisations evaluated the Global Environment Facility has supported local civil society initiatives through its Small Grant Programme (SGP). We turn now to the evaluations of this programme to look at whether this provides a promising avenue for civil society engagement in climate action .

The SGP was established in 1992 and implemented by the United Nations Development Programme (UNDP) on behalf of the Global Environment Facility (GEF). SGP provides small grants of up to US$50,000 to local communities as they take action on sustainable use and conservation of biodiversity, climate change, land degradation, international waters, sustainable forest - and chemical management. SGP has provided over 18,000 grants to communities in more than 125 countries. In its fifth operational phase (2011–2014), the Programme’s aim included expanding its coverage to 136 countries. US$288.28 million was allocated to the SGP and total co-financing mobilized from diverse sources amounts to US$345.24 million (GEF/IEO and UNDP/IEO 2015, p. 1).

From 2013 to 2015, the GEF and UNDP Independent Evaluation Offices jointly evaluated SGP. One of the conclusions of the evaluation states that ‘SGP continues to support communities with projects that are effective, efficient, and relevant in achieving global environmental benefits while addressing livelihoods as well as promoting gender equality and empowering women’ (GEF/IEO and UNDP/IEO 2015, p. xiii). The evaluation further notes that SGP’s system ensures global policies are translated into action at the local level. The results at local level were impressive, with high percentages of projects that contributed to livelihoods, poverty reduction and gender.

In many countries, SGP achievements were replicated, upscaled and mainstreamed, sometimes to the extent of policy influence, into local and sometimes national development processes. Replication often takes place on a local scale only: other villages and communities copying what had been achieved in a specific SGP supported activity. Thus, successful introduction of conservation of mangrove forests in Senegal at the local community level was replicated in other villages. Mainstreaming happened less often, but an interesting example was an SGP grant in Uganda to initiate behaviour change in local communities on the waste they produced and to take responsibility for this waste, rather than to expect the government to remove it. This behaviour change was then promoted throughout Uganda, supported amongst others by the World Bank (Examples from GEF/IEO 2015, p. 18). However, broader adoption at the national and regional levels tends to run into obstacles – success is most prevalent at local levels (GEF/IEO 2015, p. 19). The micro-macro paradox is therefore also visible in civil society involvement and action. Successes at the local level do not necessarily translate to national and regional levels, even though these successes evidently “extend beyond the project level” (GEF/IEO 2015, p. 19).

One can also see the SGP success with a bottom up approach as it contributes to numerous institutional and policy changes at the local, provincial, and national levels, and to building capacities within civil society and academic organizations to address global environmental concerns. Its success has resulted in a high demand for support (GEF/EO 2010, p. 18).

This is further demonstrated in a regional GEF project in the Pacific on Biodiversity Conservation⁷ which aimed to introduce community based conservation approaches throughout the Pacific Islands. This approach, focusing on solving land-use problems between villages, while integrating livelihood issues in local conservation planning, is now in use throughout the Pacific and has been successfully adapted to local circumstances. However, evaluative evidence in Vanuatu shows that success in communities does not (yet) equal success at the national level, as the government has not been able to dedicate resources to institutionalize the new approach (and with the devastation caused by Cyclone Pam in 2015, it may take extra time before the approach can be integrated in its national policies).

Evaluative evidence thus shows that the bottom-up activism of civil society organisations and local communities, when supported with focused funding as provided by the Small Grants Programme of the GEF, can be successful and provide solutions that can be incorporated at national and even regional scale. However, the micro-macro paradox is also evident at this level and additional action is required to achieve broader adoption and systems change .

2.7 Introducing New Technologies Through the Private Sector

There is wide-spread agreement that climate action involves a substantial and transformative technological overhaul of production processes in the private sector. Innovation, together with the promotion, development and transfer of environmentally sound technologies, and uptake of these in the private sector is critical in enabling countries to combat climate change and to pursue their sustainable development objectives. This can mean using renewable energy or transforming current equipment or technologies into something that is cleaner and more climate-resilient. This is reflected in the comprehensive evaluations : the introduction of technology scores high in the support provided by the organisations evaluated.

Since 1991, the GEF has been facilitating technology transfer to support developing countries through know-how, goods and services, equipment, as well as organizational and managerial procedures. The GEF has invested around US$250 million annually⁸ in, among others, energy efficiency, renewable energy , emerging, low carbon and energy generating technologies and sustainable urban transport.

In July 2008, World Bank Executive Directors approved the establishment of the Clean Technology Fund (CTF), under the Climate Investment Funds (CIF). CTF is a US$5.6 billion fund that empowers the transformation in middle income and developing countries by providing resources to scale up the demonstration, deployment and transfer of low carbon technologies with significant potential for long-term greenhouse gas (GHG) emission savings.⁹ Although implementation is still at its early stages, CTF investment plans, if successful, would boost renewable energy generation capacity or reduce national power consumption by 1–8 %. CTF funding for concentrated solar power, if successful, could boost total global capacity by more than 40 %.

In 2012, with financing from the GEF and in collaboration with the United Nations Environment Programme (UNEP) , the Asian Development Bank established the Climate Technology Finance Center (CTFC). The Center is designed to promote transfer of and investment in climate technologies and to help mainstream climate technology considerations in development planning. Since inception, the Center has provided lessons on climate change initiatives to other multilateral development banks (MDBs) (ADB 2014).

While other organisations may not have dedicated instruments for technology transfer , they show a similar emphasis on innovation and introduction of technologies, especially in collaboration with the private sector. Evaluative evidence in the seven comprehensive evaluations focuses on the following issues.

Technologies That Work Best Tend to Be Already Tested Elsewhere

A lot of technology transfer has been successful precisely because it was focused on well proven technologies. Replication was typically taken on by the private sector as a result of evidence showing that a technology was both cost-effective and profitable. Sound monitoring that demonstrates the benefits of a technology becomes even more important to its broader adoption (GEF/IEO 2014, p. 54). CTFC experience showed that many country governments do not give high priority to the introduction of relatively high-cost climate change risk reduction technologies. As a result, CTFC undertook a phased approach whereby it is required to first demonstrate benefits to countries (ADB/IED 2014, p. 21). The Report on Effectiveness of the Swiss International Cooperation in Climate Change (2014, IV) highlighted that groups of projects with strong scores for mitigation effectiveness were found to include projects that targeted the rehabilitation of hydropower systems and power systems with direct energy efficiency benefits and enabling impacts for renewable energy promotion, the strengthening of measuring, reporting and verification capacity and carbon market readiness, the use of knowledge sharing among cities and companies, and the rehabilitation and re-deployment of used Swiss trams to other countries.

A Fully Supportive Enabling Environment Is Necessary

Wörlen’s (2014) meta-analysis of mitigation interventions led to a systematic overview of all the barriers to change – providing a “theory of no change” – an explanation of why market change or transformation was not happening. The theory of no change demonstrated that introduction of technology will only be successful if all potential barriers for change have been tackled. The ICF evaluation showed that in more than half of CTF countries, policy , regulatory, and macroeconomic situations have the potential to slow down or limit transformation and replication. These countries have supportive policies in place that provide building blocks, but lack implementing regulations specifying key details of the regulatory environment, weakening the potential for immediate replication. Non-investment-grade credit ratings are also a limiting factor in some countries (ICF 2014, X). ADB’s Climate Technology Finance Center (CTFC) also encountered difficulties during its design and launch. Barriers include financial constraints, insufficient knowledge base and expertise, and inadequacies of public policies, regulations, and enforcement (ADB/IED 2014, p. 21).

A Crucial Supporting Factor Is the Availability of Financing

If loans for investment in new technology are unavailable, then this technology will not be widely adopted. The Fifth Overall Performance Study of the GEF (2014) showed that mainstreaming typically took place because of financial incentives provided by the national government to adopt the technologies (p. 54). The IDB/OVE evaluation notes that promoting the development of small-scale energy efficiency projects has proven to be more difficult, as small firms face high transaction costs and low financial returns from these investments (partly because of energy subsidies), and they require access to long-term financing (2014, p. 67).

The CIF evaluation could not see a clear path towards broader adoption of many technologies tested and demonstrated in CIF support, because these projects and programmes lacked a convincing theory of change that would explain how replication and market change and transformation would take place. This seems at least partly due to investment criteria, for example in CTF, that focus on quantifying GHG emission reductions rather than causal pathways to transformative change (ICF 2014, x). The focus on GHG emission reductions is visible in other evaluations as well – it points to the possibility that technology is easier judged on its contribution to climate change mitigation , without full recognition that any technology will only perform if the social, economic and environmental prerequisites are in place over time.

2.8 Gender, Equity and Inclusiveness

The Rio Declaration on Environment and Development,¹⁰ adopted at the Earth Summit in 1992, introduced principles (10, 20, 21, 22) on participation and importance of specific groups (civil society, women, youth and indigenous groups) for sustainable development . The Sustainable Development Goals recognize that society and the economy need to ensure an equitable distribution of wealth, attention for gender perspectives and ensure inclusiveness of both civil society and the private sector, as well as government. Achieving this will ensure sustainable development in the social and economic domains. Without this balance, the balance of social and economic needs with the environmental domain will be meaningless.

Climate change impacts affect men and women, with the poorest being the most vulnerable. Seventy percent of the world’s poor are women,¹¹ making them extremely affected. On the other hand, they also play a large and important role in tackling climate change. As impacts of climate change increase, work predominantly undertaken by women (i.e. food production, supplying household water, ensuring fuel for heating and cooking) is becoming increasingly more difficult. Coping strategies and their resilience give them a practical understanding of innovation and skills to adapt to changing realities, as well as contribute to finding solutions.

The GEF has recognized gender as highly important to achieve behavioural change that will lead to broader adoption of sustainable solutions to global environmental problems. In 2010, OPS4 highlighted that ‘social and gender issues in GEF strategies and projects are not addressed systematically, and the GEF cannot rely completely on the social and gender policies of its Agencies.’ (GEF/EO 2010, p. 30). As a response, the GEF developed its policy on gender mainstreaming and adopted it in May 2011. There has also been an increase in the proportion of projects that aim to mainstream gender. These improvements may be attributed to adoption of gender mainstreaming by several GEF agencies, of which the best international practices come from IFAD, UNDP and the World Bank. Despite the adoption and review of a gender policy and designation of a focal point OPS5 provides evaluative evidence that attention for gender in projects is often lacking. No less than 43 projects evaluated qualified themselves as “gender not relevant”, but provided evidence that gender turned out to be relevant after all. OPS5 concluded that “omitting attention for gender where it is needed may have led to unintended negative gender-related consequences”. A baseline study undertaken by OPS5 revealed that many climate action projects were formulated by experts insufficiently aware of gender issues. On the good side it should be noted that the same study also revealed projects that tackle gender issues adequately (GEF/IEO 2014, p. 61).

When the Climate Investment Funds (CIF) started in 2008, they did not have an explicit gender focus – most countries did not include women’s organisations in investment plan consultations. However, in 2009 and 2010, 15 % of the plans started declaring gender considerations. Some works remains to ensure that gender considerations are mainstreamed in CIF planning and carried through to investment projects in the field. In a positive step forward, the CIF hired a gender specialist to develop and implement an action plan to support collaboration among MDBs.

Attempts have been made throughout the NICFI portfolio to address gender issues in REDD+. However, it is stated by the evaluation that among partners, there is a lack of understanding of, and low general capacity to address gender. The strongest contribution has been through the UN-REDD programme, whereby numerous publications on REDD+ and gender have been produced.

For UN-REDD, the importance and need for gender mainstreaming is reflected in most of its policy and programmatic documents and guidelines. However, the implementation of gender mainstreaming activities at the country level is not taking place in a cohesive and systematic way throughout the programme. The evaluation (2014) stated that drivers of deforestation will be better addressed if gender considerations are integrated especially at the local level.

The track record on equity and inclusiveness is even less impressive. While equity and inclusiveness are essential dimensions of social, economic and environmental sustainability , they are perhaps too far removed from the often technical nature of the climate actions reviewed in the seven comprehensive evaluations. The Fifth Overall Performance Study of the GEF does not mention equity or inclusiveness, while the CIF evaluation only mentions equity in relation to investments and inclusiveness of stakeholders in consultations. There is indirect attention to the issues – for example in the attention for local livelihoods, involvement of indigenous peoples and civil society organisations. An example is to be found in the NICFI evaluation: since 2008, NICFI provided a total of NOK 1 billion or 9 % of its funding to civil society to generate needed knowledge, for advocacy (international and political), piloting and facilitating implementation (Frechette etc. 2014, xix). UN-REDD’s evaluation stated that ‘The Programme provides an enabling platform for Indigenous Peoples and civil society organisations to influence global discussions on REDD+. The ability of forest -dependent populations to influence REDD+ processes has so far proven to be more limited at the country level, and non-indigenous communities are not well represented in the programme, overall’ (Frechette etc. 2014, vi).

While attention for gender, equity and inclusiveness is on the rise, the evaluative evidence is overwhelming that these dimensions have not yet been fully included in climate action . With the adoption of the Sustainable Development Goals, this has become more important and we hope to see evaluative evidence emerging the coming decade.

2.9 When Will We Achieve Systems Change?

The comparison of the seven comprehensive evaluations published in 2014, and some influential evaluations from the years before, leads to the conclusion that if we want to achieve transformational change, we need to ensure that the impact drivers working towards such a change are stronger than the impact drivers that cause climate change. An important part of the fight to mitigate climate change is therefore outside of climate change action: continue the fight against public subsidies for non-sustainable use of natural resources; take action to ensure that the costs of climate change are paid for by the “polluters”, by industries and people who are causing climate change to happen. Until that time climate change action will consist of beautiful flowers in a walled garden: just a demonstration that we can have a beautiful planet, if only the winds blowing against us would not destroy these beautiful flowers if they emerge from the walled garden.

The conclusions we draw from the seven comprehensive evaluations are as follows.

• 1. The OPS5 conclusion that a high percentage of climate action is effective is supported by all other comprehensive evaluations that have been able to look at effectiveness;

• 2. For reduction of greenhouse gas emissions, the energy sector and energy policies hold greatest promise, and tackling energy efficiency issues is more effective than support for renewable energy , but the latter is effective as well, even if costlier.

• 3. Subsidies for non-sustainable use of natural resources (fossil fuels, agricultural practices, overuse of water resources, etc.) prevent the reductions in greenhouse gas emissions to have more than a marginal impact on climate change: i.e. the pace of climate change is slowed almost imperceptibly.

• 4. The micro-macro paradox is thus shown to exist: anything the international organizations and the bilateral donors do to prevent climate change continues to be effective in its own right, but powerless against the enormous spending power and damage done by subsidies for non-sustainable use of natural resources, with fossil fuel subsidies as the largest barrier to change.

• 5. To change the system, action from many partners, bottom-up and top-down, with full recognition of cross-cutting issues such as gender, equity, inclusiveness is needed, and evaluative evidence shows that pieces of the puzzle are known and can be effectively set up and used.

• 6. To change the system, an important input of technology is needed – the shift from fossil fuels to a low-carbon economy needs technological innovation and change. The technologies to do this exist and need to be fostered and promoted: evaluative evidence shows that when they are introduced properly and supported by all actors, they can be effective fertilizers of change.

• 7. On adaptation to climate change many promising evaluative findings are emerging; what is lacking is a concentrated effort to gather the evaluative evidence and interpret it, learn at higher levels of aggregation and integrate adaptation into social, economic and environmental development as the essential ingredient that will ensure sustainability .

2.10 Recommendations for Future Evaluations

As argued in van den Berg (2013, p. 47), evaluations of climate change interventions, especially if they aim for a higher more comprehensive level of understanding what the interventions mean and what they achieve in the longer run, need to evaluate in the context of the continuing societal and economic winds that are causing climate change. Evaluators need to point out to policy makers and decision makers that what they promote with one hand, is more than sufficiently undone with a very active and much bigger other hand.

If the forces of destruction can be reduced or even halted, climate change action will become successful. How successful can currently not be established fully for all actions – there is some international agreement on measurement of reductions of greenhouse gas emissions, but this agreement needs to be further developed. Many countries, multilateral and bilateral organizations are currently using various measurement systems at the same time in different projects – this needs to be improved.

Evaluations, not just the seven comprehensive ones, but the many evaluations at the intervention level as well (many of which are highlighted in this book), provide an increasing body of knowledge that has been insufficiently explored for policy makers and decision makers on what works, where and when and for whom under what circumstances. This should focus on:

• How systems change can be effected through activities on key issues that will “tip” or “tilt” the system in the right direction;

• Identify the top-down actions that can and should be taken as they have proven to be effective; similarly look at bottom-up actions that can and should be supported;

• Present evidence on the difference in effectiveness between inclusive, gender sensitive, equity based approaches versus approaches that lack these perspectives;

• Contribute to a repository of evidence on which technologies under which circumstances, for whom, have proven to be effective in supporting more sustainable and low-carbon growth;

• Contribute to a repository of evidence on what works for whom, when, where, and under which circumstances on adaptation to climate change.

The evaluation departments of countries, bilaterals and international organizations, as well as philanthropic foundations and private and social enterprises should increase their collaboration, as the war on climate change requires a concentrated effort, rather than everybody focusing on their own constituency and their own accountability. Your children and your children’s children will one day ask you, as an evaluator, what you have done to stop climate change. Your response should not be that you have provided evidence to your government or Board on how the money was spent, but on what is useful and a potential winner in our battle to keep the planet habitable for humankind.