Evaluating Climate Change Action for Sustainable Development: Introduction

Abstract

This chapter considers evaluation as essential for learning and for reflecting on whether actions to address the complex challenges pertaining to climate change are on track to producing the desired outcomes. The Paris Agreement of 2015 was an important milestone on the road towards a zero-carbon, resilient, prosperous and fair future. However, while the world has agreed on the need to tackle climate change for sustainable development, it is critical to provide evidence-based analysis of past experiences and ongoing innovations to shed light on how we might enhance the effectiveness and efficiency of actions at various levels. Thorough and credible evaluations help us identify what works, for whom, when and where and under what circumstances in order to mitigate climate change, achieve win-win situations for the society, the economy and the environment, reduce risk and increase resilience in the face of changing climate conditions. This chapter serves as an introduction to the book on Evaluating Climate Change Action for Sustainable Development that sets the scene on the current state of climate change evaluation and brings together experiences on evaluating climate change policy, mitigation and adaptation.

Climate change has emerged as one of the preeminent challenges facing humankind in the twenty first century. The Intergovernmental Panel on Climate Change states unequivocally that there has been an unprecedented warming of the global climate system since the 1950s and that this warming has been influenced by human actions (IPCC 2015). The anthropogenic emissions of greenhouse gases have increased constantly since the pre-industrial level, driven largely by economic and population growth, and are now at the highest historical peak. The impacts of the climate change will affect – and are already affecting – all people and parts of the world often in negative and sometimes unexpected ways. Urgent and concerted action is required to address climate challenges through mitigation efforts as well as through improving the ways in which societies and the global economic system adapt to the effects of climate change. Actions have been initiated on multiple fronts. What is needed is evidence-informed understanding of the effectiveness and efficiency of such actions. Therefore, robust evaluation is a must. That is what this book focuses on.

The year 2015 was a historic turning point for global action on climate change. The Paris Agreement under the UN Framework Convention on Climate Change (UNFCCC) was adopted by the Conference of Parties in its 21st Session.¹ The Paris Agreement is a binding commitment intended to set the world on a path towards a zero-carbon, resilient, prosperous and fair future. In 2015, the plan of action called the 2030 Agenda for Sustainable Development and its associated Sustainable Development Goals (SDGs) was also adopted by member States of the United Nations.² The seventeen SDGs are universal and share a common global vision of progress towards a safe, just and sustainable space for all human beings. They reflect the moral principles that no one and no country should be left behind and that everyone and every country should share a common responsibility for delivering the global vision. Specifically goal 13 calls for urgent action to combat climate change and its impacts, recognizing the key linkages of climate change to development and human wellbeing. Goal 13 also refers to the UNFCCC as the global forum to tackle climate change. Also in 2015, the Third UN Conference on Disaster Reduction in the Japanese city of Sendai adopted the Sendai Framework for Disaster Reduction 2015–2030 which also identifies climate change as one of the drivers of increased disaster risk.³ All these political commitments at the global level demonstrate the urgent concern of the international community and individual governments for climate change and its direct impacts on sustainable development.

Impacts of changing climate express themselves in a multitude of ways.⁴ Already now melting snow and ice, and changing precipitation patterns are altering hydrological systems affecting water resources quantity, quality and continuity, as streams of water from the glaciers and rainfall patterns become more erratic. Terrestrial, marine and freshwater species have started to alter their geographical range and migration patterns, and their abundance has started to be affected. IPCC projections indicate that climate change will in the future undermine food production through changed weather patterns and ecosystem impacts. Notably, production of three main crops that sustain humanity – wheat, rice and maize – is projected to be negatively affected. Similarly, fisheries productivity will likely be challenged, adding to the problems caused by overfishing. A large proportion of both animal and plant species will face extinction thus exacerbating the loss of biological diversity. Human health may also be affected negatively, as a warmer climate will facilitate the spread of vector borne and tropical diseases to higher latitudes. Extreme weather and climate events are on the rise. These include increased frequency and intensity of storms, as well as climatic variability. While rainfall will increase in some areas, others will face more frequent and prolonged droughts.

Climate risk and vulnerability vary considerably between different regions and groups. Coastal areas are generally the most vulnerable due to storms and sea level rise and associated saline intrusions to coastal ecosystems and aquifers. More and more people are concentrated in coastal areas: it is estimated that more than 40 % of the world’s people live within 100 km from the coast and over the past decade more than 60 % of disaster losses have occurred in coastal areas (DasGupta and Shaw 2016). Despite these losses, concentration of the world population on coasts continues. The worst affected are low-lying coastal countries, which are exposed to rising seas and increasing storms. Small island developing states and poor countries, such as Bangladesh, face challenges of survival, but also rich countries like the Netherlands must invest increasing resources to deal with coastal hazards. A large proportion of major cities are located in the coastal areas and, thus, exposed to climate related hazards. Cities, such as London, New York and Tokyo are all coastal, but so are megacities in the poorer regions of the world: Lagos, Kolkata, Dhaka, Jakarta and others. Their ability to cope with and adapt to climate related disasters and rising sea levels is much lower. Similarly, mountain and highland areas experience the risk of climate change acutely. They are the water towers of the world and home to some of the poorest people in the world (FAO 2015). They are doubly vulnerable in terms of global freshwater availability and local food security. Adaptive capacity and vulnerability have deep social, economic and political determinants (Pelling 2011). Risk is defined as a function of hazard exposure and vulnerability to it (Wisner et al. 2004). Apart from direct physical factors, vulnerability has a strong social dimension: people with fewer economic means and political power have less ability to cope with and recover from disasters. In addition, they are often confined to living in the most hazardous places, such as informal settlements on denuded slopes (Surjan et al. 2016).

It is incumbent upon us to deal with climate change in a comprehensive manner. There is a need to address the root causes of climate change to mitigate it. IPCC links anthropogenic greenhouse gas emissions to key drivers that include: population size, economic activity, lifestyle, energy use, land use patterns, technology and climate policy. All are directly related to virtually all aspects of human activity and aspirations. As societies get richer, their energy use and emissions tend to increase. There is therefore a compelling need for decoupling economic growth from increases in energy use and emissions (Mulder and Groot, 2004). Development and spread of energy efficient technologies and renewable energy will play a key role in the process (Yang and Yu 2015; Edenhofer et al. 2011). The adoption of low-carbon transport systems is also high on the agenda (Dalkmann and Huizenga 2010). While such mitigation measures are needed, their impacts will be long-term and dependent on widespread societal adoption. Even in best scenarios, it will take many decades before they take effect. According to IPCC (2015), emissions scenarios that keep warming below 2°C over the twenty-first century relative to pre-industrial levels will involve 40–70 % reductions in global anthropogenic emissions by 2050 and near-zero emission levels by 2100. Although this is consistent with the Paris Agreement targets, the current voluntary mitigation efforts by signatory countries fall well short of this.

It is consequently necessary to invest in adaptation to climate change and to enhance societal resilience to climate change impacts. Adaptation refers to reducing the adverse effects of climate change on human and natural systems. At the 2010 UNFCCC conference in Cancun, Mexico, the parties adopted the Cancun Adaptation Framework⁵ affirming that adaptation must be addressed with the same level of priority as mitigation. They further agreed that adaptation is a challenge faced by all parties, and that enhanced action and international cooperation is urgently required to enable and support the implementation of adaptation actions aimed at reducing vulnerability and building resilience in developing countries (para 11). IPCC (2015) recognizes that adaptation options exist in all sectors but their context and potential differ between sectors and regions. Furthermore, adaptation and mitigation responses are underpinned by common factors, including effectiveness of institutions and governance, innovation and investments in environmentally sound technologies and infrastructure, sustainable livelihoods and behavioural and lifestyle choices (SPM 4.1).

Several international financial and technical facilities have been set up to help countries address climate change challenges. The Global Environment Facility⁶ (GEF) has already been in existence for a quarter century as the financial mechanism to the UNFCCC. It finances projects in developing countries that focus on mitigation efforts. The GEF recognizes the multidisciplinary nature of mitigation. While greenhouse gas emission reductions through promotion of sustainable transport, energy efficiency and renewable energy are important, emissions reductions from sectors, such as land use and forestry are also important, as is protecting global carbon sinks like the oceans. The World Bank manages the Climate Investment Funds⁷ that operate through four key programmes that help developing countries pilot low-emissions and climate resilient development: the Clean Technology Fund, Forest Investment Programme, Pilot Programme for Climate Resilience, and Scaling up Renewable Energy Programme. The Adaptation Fund⁸ helps developing countries to build resilience and adapt to climate change through financing projects and programmes that focus on vulnerable communities. The Green Climate Fund⁹ was set up in anticipation of the Paris Agreement to mobilize funding and invest in low-emission and climate-resilient development. It intends to address both mitigation and adaptation equally. Importantly all these funds and facilities involve concrete strategies and action by all governments, the private sector, civil society, as well as individual citizens. But clearly global climate action is not limited to these funds.

The Paris Agreement fully recognizes what also emerged from the UN Conference on Financing for Development¹⁰: public funds will not be sufficient to tackle climate change, not to prevent it nor to adapt to its effects. In this regard public-private partnerships and private initiatives are envisaged to play a key role. Governments are invited to ensure an enabling environment and level playing field for initiatives of the private sector and civil society. These have emerged over the past decade and include social and environmental impact investing, corporate responsibility to contribute to sustainable development, market oriented social initiatives and so on. The G8 has published several documents regarding the promise of impact investing¹¹ and the World Business Council for Sustainable Development has been instrumental in developing the inclusion of natural resources accounting in business practices.¹² Civil society initiatives range from fair trade to climate-smart agricultural practices (the range of initiatives is staggering), as is demonstrated in the Social Enterprises World Forum.¹³

Climate change is a complex issue encompassing physical, technological, institutional, economic, social and political spheres. For a sustainable future for all of us, it is essential that we identify the best and most suitable measures and make the right choices for mitigation and adaptation to climate change. This is where evaluation comes in.

1.1 Critical Role of Evaluation

Evaluation is essential for learning and for reflecting on whether we are taking the right actions for the right things for current and future generations. Evaluation of climate change policies, mitigation and adaptation actions helps us assess progress on the complex challenges we are facing. Evaluation also helps us identify what works, under what circumstances and for whom. Evidence-based analysis of past experiences and ongoing innovations is likely to shed light on how we might enhance the effectiveness and efficiency of actions at various levels and achieve win-win situations and multiple benefits, such as reducing risk and increasing resilience.

Evaluation makes a judgement of the value or worth of the evaluand – the subject of evaluation – be it a policy, strategy, programme, project or any other type of intervention. It can take several forms. It can be formative, looking into the ways an intervention is implemented in order to identify ways in which the intervention and its performance could be improved. It can be summative to determine the extent to which the intervention has achieved its anticipated desired results. An evaluation can also be prospective, assessing the likely outcomes of proposed interventions a priori (Morra Imas and Rist 2009). A category of summative evaluations is impact evaluation that looks into whether the programme or intervention has contributed in a measurable way to a larger longer term goal (such as transforming national policy or the market towards a more climate friendly directions) than just the direct outputs and outcome of the intervention itself. An impact evaluation can use a range of approaches and methodologies that are rigorous (Stern et al. 2012). It has been argued that it is important to distinguish between the ‘direct’ and ‘final or ultimate’ impact of interventions (van den Berg 2013). As we address issues critical to climate change, we must ensure that interventions make a difference and help to significantly increase mitigation or adaptation or both while also ensuring sustainable development, or be able to identify and measure trade-offs.

In this context, a special challenge is posed by the private sector and civil society initiatives on impact investing, corporate responsibility and sustainable development, as well as civic initiatives and social enterprise. The role of evaluation in these relatively new areas of work is not yet established, which is why they have been identified as the ‘New Frontiers for Evaluation’, an initiative of the Centre for Development Impact in the UK.¹⁴ A Wilton Park conference in July 2015 discussed the potential role of evaluation in various initiatives, calling for a gap analysis of what has been evaluated and where methods and capacity need to be developed.¹⁵ Much of this is relevant for climate change to inform investments in green technologies and transitions towards sustainable resource use in business practices. We also need to take stock of what we know to start, operationalize and manage climate smart enterprises. Evaluation can play a very important role in measuring effectiveness, cost-effectiveness and longer term impact.

Evaluating climate change can be challenging primarily because climate change is a global good (Puri and Dhody 2016). Other challenges include the fact that climate change programmes are frequently multi-sector, multi-objective complex programmes that aim to affect not just environment but also poverty, livelihoods, health, income and food security. Additionally climate change programmes aim to affect not just immediate outcomes but outcomes over generations. Last but not least, is the absence of data and capacity in this area – most evaluators are trained in more traditional sectors and hence think about evaluations in traditional ways. Indeed Picciotto (2007) identifies climate change as a significant challenge for development evaluation. Evaluating climate change action is a relatively new frontier for the field that has only emerged in the first decade of the 2000s (van den Berg and Feinstein, 2009). We maintain that evaluation in the field of environment and sustainable development has the opportunity to leapfrog, while borrowing from other disciplines, but also innovating to generate high quality and relevant evidence to inform national and international efforts directed at environment and sustainable development (Rowe 2012; Uitto 2014). This book has its origins in the Climate-Eval community of practice started and hosted by the Independent Evaluation Office of the GEF. The book brings together state-of-the-art contributions of evaluations pertaining to climate change policy, mitigation and adaptation.

1.2 Book Structure

The book contains 18 chapters in which leading authors examine innovative and emerging evaluation knowledge and practice of climate change and its link to sustainable development. The authors discuss methodologies and approaches to better understand, learn from and assess interventions, strategies and policies. The contributions also discuss evaluation challenges encountered and lessons learned to better understand and tackle difficult areas of evaluation.

Chapter 2 or overview chapter by Rob D. van den Berg and Lee Cando-Noordhuizen, ‘Action on climate change: What does it mean and where does it lead to?’ discusses the micro-macro paradox of climate change action. There is evidence that climate action works and achieves direct impact – yet climate change seems unstoppable. An analysis of multiple comprehensive evaluations indicates that technology and knowledge are available to fight climate change. However, economic development and subsidies harmful to the climate still outweigh remedial climate action with at least a factor of one hundred. Current successes of programmes and projects will not impact global trends unless unsustainable subsidies and actions are stopped.

Chapter 3 written by Rob D. van den Berg, ‘Mainstreaming impact evidence in climate change and sustainable development’ examines the demand for impact evidence and concludes that this demand goes beyond the experimental evidence that is produced during the lifetime of an intervention. Van den Berg argues for impact considerations to be mainstreamed throughout interventions, programmes and policies and for evaluations to gather evidence where available, rather than focusing the search for impact and its measurements on one or two causal mechanisms that are chosen for verification through experimentation.

Chapter 4 by Tonya Schuetz, Wiebke Förch, Philip Thornton and Ioannis Vasileiou from the CGIAR Research Program on Climate Change, Agriculture and Food Security (CCAFS) describes the design of an impact pathway-based Monitoring, Evaluation and Learning (MEL) system that combines classic indicators of process in research with innovative indicators of change. The chapter highlights the importance of engaging users of research in the development of impact pathways and continuously throughout the life of the program. Results show that partnerships with diverse actors such as the private sector and policy makers are key to achieving change. The chapter concludes that research alone is insufficient to bring about change. However, research does generate knowledge that stakeholders can put to use to generate development outcomes.

Chapter 5 by Monika Egger Kissling and Roman Windisch, ‘Lessons from taking stock on 12 years of Swiss international cooperation on climate change’ highlights the challenges encountered and lessons learned from this assessment where a bilateral donor puts climate change lens on a longstanding development cooperation portfolio. The chapter discusses the need (1) for evaluators to put more effort in identifying best methodological practices amidst a large volume of information, diverse portfolio and absence of reliable data; (2) for practitioners to invest more in strategic project design and monitoring to provide accurate data; and (3) for policy makers to be cognizant of the value that evaluation brings, as it is an important tool that contributes to accountability.

Chapter 6 by Michael Carbon discusses the approach, process and lessons from the evaluation of UNEP’s Climate Change Sub-programme. It shows the importance of developing an appropriate analytical framework that is well-suited for the scope and complexity of the object of evaluation, and how the Theory of Change approach helped make a credible assessment of UNEP’s contribution towards impact, sustainability and upscaling.

Chapter 7 written by Aryanie Amellina focuses on an assessment of the initial phases of the Joint Crediting Mechanism (JCM) in Indonesia. It highlights JCM governance and ease of use of methodologies related to measurement, reporting and verification (MRV). The author concludes with recommendations to strengthen methods to determine reference emissions and for clarifying ways to allocate credit among countries to define a pathway to a tradeable credit mechanism.

Chapter 8 by Jyotsna Puri, ‘Using mixed methods to assessing trade-offs between agricultural decisions and deforestation’, demonstrates the importance of using qualitative and quantitative methods to assess and measure win-win development policies that also help mitigate climate change. The author’s study explores the poverty and environment nexus using historical data on land rights and panel data on land use in Thailand. The chapter concludes that it is important to measure the differential effects of policies on different crops, agricultural intensity and agricultural frontier. In the case examined by the author, she advises that policies that encourage cultivation may not be detrimental to forest cover after all.

Chapter 9 written by Aaron Zazueta and Neeraj Negi presents the methodological approach adopted in the evaluation of climate change mitigation projects supported by the Global Environment Facility in four emerging markets, namely China, India, Mexico and Russia. The authors demonstrate the use of the Theory of Change approach to carry out a comparative analysis across projects seeking to bring about changes across diverse markets or market segments in different countries. Zazueta and Negi highlight how the evaluation focused on understanding the extent and forms by which GEF projects are contributing to long-term market changes, leading to reduction in GHG emissions, and on assessing the added value of GEF support in the context of multiple factors affecting market change.

Chapter 10 written by Yann François and Marina Gavaldão explore how climate change mitigation projects can reduce greenhouse gas emissions, potentially have adaptation benefits, and achieve sustainable development objectives. ‘Integrating avoided emissions in climate change evaluation policies for LDCs’ provides an example of socio-economic benefits gained if accounting for avoided emissions are incorporated in projects, in this case, passive solar housing technology.

Chapter 11 by Debora Ley, ‘Sustainable development, climate change, and renewable energy in rural Central America’, demonstrates the potential and multiple benefits of decentralized renewable energy. The author also demonstrates how specific drivers can facilitate or hinder projects in achieving multiple objectives using on the ground, qualitative methods.

Chapter 12 by Jasmine Hyman, ‘Unpacking the black box of technology distribution, development potential and carbon markets benefits’ explores whether and how carbon markets can support a pro-poor development agenda. The author introduces a ‘Livelihood Index’ to understand the employment impact of a carbon intervention. Their study finds that variations in the distribution framework means that development outcomes may compete rather than complement one another. Methods used include value chain analysis and a qualitative analysis to understand how carbon finance recipients access the mechanism, perceive the project and conceptualise its benefits.

Chapter 13 by Takaaki Miyaguchi and Juha Uitto presents the methodology of a meta-analysis of ex-post evaluations of climate change adaptation (CCA) programmes in nine countries using a realist approach. The authors conclude that adopting a realist approach to evaluating complex development projects is a useful way of providing relevant explanations, instead of judgments, about what type of intervention may work for whom, how and under what circumstances for future programming.

Chapter 14 written by Jacques Somda, Robert Zougmoré, Tougiani Abasse, Babou André Bationo, Saaka Buah and Issa Sawadogo, ‘Adaptation processes in agriculture and food security: Insights from evaluating behavioural changes in West Africa’ focuses on the evaluation of adaptive capacities of community-level human systems related to agriculture and food security. The study highlights findings regarding approaches and domains to monitor and evaluate behavioural changes from CGIAR’s research program on climate change, agriculture and food security (CCAFS). Results suggest that application of behavioural change theories can facilitate the development of climate change adaptation indicators that are complementary to indicators of development outcomes. The authors conclude that collecting stories on behavioural changes can contribute to biophysical adaptation and monitoring and evaluation.

Chapter 15, written by Irene Karani and Nyachomba Kariuki, ‘Using participatory approaches in measuring resilience and development in Isiolo County, Kenya’ highlights the use of participatory approaches through a Tracking Adaptation and Measuring Development (TAMD) Framework to measure resilience in Kenya. The authors outline the process of developing subjective indicators and demonstrate the advantage of empowering the local community in collection of baseline, monitoring and early outcome data as they develop Theories of Change. The article concludes by sharing lessons and policy implications.

Chapter 16 by Joanne Chong, Anna Gero and Pia Treichel, ‘Evaluating climate change adaptation in practice: a child-centred, community-based project in the Philippines’ documents a research and evaluation approach applied in a child-centred and community-based CCA project implemented across four provinces in the Philippines. The authors emphasise the success of the methodology due to its participatory foundations – local voices and perspectives matter in understanding the impact of the project.

Chapter 17 written by Emilia Bretan and Nathan L. Engle focuses on real time milestones and outcomes from Brazil’s Drought Preparedness and Climate Resilience Programme (Drought NLTA). Evidence gathered through the participatory monitoring and evaluation (PM&E) approach showed that the programme was able to convene key-regional and federal level multi-sector stakeholders, resulting in a bottom-up and regionally-led collaboration. Through engagement and commitment of the partners, the programme illustrates good practice for coordination and continuous sharing of knowledge and data between service providers, secretariats, municipalities and other stakeholders from distinct sectors, states, and governmental levels.

Chapter 18 by Timo Leiter presents a decision-support tool developed by the German International Cooperation (GIZ GmbH), the Adaptation M&E Navigator. The author explains the rationale, structure and how this tool can help policy- and decision-makers select a suitable M&E approach by providing a list of specific M&E paradigms and matching them with relevant approaches.

Evaluation plays an ever crucial role in learning: why are things happening or not happening? Are we doing the right thing or not? Why and why not? Are there better ways? The evaluation profession has become more adept at introducing scientific tools and the link between science and evaluation is becoming stronger. Evaluation is helping bridge the science-policy divide.

The contributions included in this book demonstrate a good understanding not only of assumptions and outcomes, but also of context as they attempt to explain how and for whom interventions may work. Methodologies used are varied and may sometimes be sophisticated. However, they all answer operational and practical questions.

We are in a world with changing boundaries. Our boundaries have changed in terms of what we want from our programs and strategies, what we want from evaluations and what types of tools we have access to. We are now witnessing the surge and availability of big and open data and a variety of innovative techniques that will also enable this sector to leapfrog and push the frontiers of learning and evaluation. It is our hope that this book will contribute to this push.