18.1 Introduction

The use of Earth observation (EO) information and geospatial information technologies (GIT) for evidence-based decision making is a growing opportunity because of open access and increased availability of data. The services from EO have a significant impact on all aspects of everyday life and were recognized at the 2018 United Nations World Geospatial Information Congress for their utility in social, economic, and environmental development (Sheldon et al. 2018). It is a cost-effective solution for advanced understanding of both natural and human-induced global or regional changes through real-time monitoring data. The use of EO and GIT aims at improving understanding about complex environmental and social interactions and enables decision making based on scientific evidence, thus helping in enhancing the well-being and livelihood of the people (Leibrand 2019).

The relevance of science and technology to the HKH region is even higher because of the difficult terrain; the region is also undergoing rapid changes driven by stressors such as climate change and human conflicts, and factors like globalization, infrastructure development, migration, tourism, and urbanization (Sharma et al. 2019). It also faces a data gap that hinders the overall assessment of the region (IPCC 2007; Singh et al. 2011). ICIMOD, therefore, has been working as a SERVIR hub to leverage EO science and GIT toward the region’s development since 2010 (Chap. 1). The SERVIR-HKH program aims to strengthen the capacity of the member countries to incorporate EO information and GIT into development decision making in order to address complex problems. The SERVIR-HKH hub works with diverse partners from local, regional and global levels, and each of them have their own priorities and interests. More than science, the issues are geopolitical at the HKH front. Further, socioeconomic issues strongly prevail in the region, influencing lives and livelihoods. Understanding these complexities is essential in order to answer some important questions regarding the successful implementation of the SERVIR program and achieving its objective. These questions are as follows:

  • What motivates the stakeholders to use EO and GIT in decision-making?

  • What hinders them to promote these applications?

  • Who should be engaged and why in the process of creating positive change?

Monitoring and evaluation (M&E) can, but does not always answer these questions. Traditional M&E generally uses the logical framework approach which describes the relationship between different components of a program but does not take into account the emerging needs, processes, and results when the context is uncertain to deal with (James 2011). To answer difficult questions, there should be in-depth understanding of complexities; the problem with traditional M&E in dealing with a complex program is that it does not pay enough attention to the change process that unfolds in the midterm; this is an obstacle to achieving long-term goals (Weiss 1995).

On the other hand, modern M&E approaches such as theory of change (ToC) and participatory impact pathway analysis (PIPA) are aware of the complexities, bring stakeholders together, and address the difficult questions that manage large, complex programs. Harnessing complexity requires close observations and interactions at different levels (Douthwaite et al. 2020). The approach promotes collective understanding, reviews, reflections, and adaptive management practices in the SERVIR program.

This chapter carries out an after-action review of the deployment and implementation experiences of SERVIR-HKH and its M&E system that was aware of all the complexities involved; so that many others can learn and adopt such approaches in implementing their programs.

18.2 The Planning, Monitoring, and Evaluation Approach

18.2.1 Theory of Change and Participatory Impact Pathway Analysis

ToC and PIPA are innovative approaches that are being increasingly used to understand the complexity of a program where the desired changes are uncertain (Isabel 2012; Alvarez et al. 2010). According to Davies (2012), ToC is the description of a sequence of events that is expected to lead to a particular desired outcome. From that perspective, ToC helps the relevant practitioners to gain a better understanding about the pathways of change that lead to outcomes and impacts which are often nonlinear and complex. ToC is about the central processes or drivers—such as psychological, social, physical, or economic—through which change comes about in individuals, groups, or communities; this could derive from a formal research-based theory or an unstated, tacit understanding about how things work (Funnell and Rogers 2011).

PIPA in SERVIR-HKH involves an actor-based approach bringing together different stakeholders who are also users of EO products and services; they then develop a collective understanding about the project-impact pathways, and the underlying assumptions of change. ToC adds to an impact pathway by describing the causal assumptions behind the links in the pathway—meaning, what has to happen for the causal linkages to be realized (Mayne and Johnson 2015).

By applying these approaches in planning, monitoring, and evaluation (PM&E), SERVIR-HKH aims for a better understanding of and harnessing of the complexity of the program for navigating change toward outcomes and impacts. Ensuring the adoption and use of EO services for societal benefit is the major objective; societal benefit is obtained when any support has a meaningful impact on the well-being of society (Giovannini et al. 2011), and it can be measured (Bornmann 2012). Result-oriented PM&E focuses on achievements and guarantees that resource allocation and planning are closely tied up with the outcomes and impacts rather than inputs and activities. Comparing what we achieve with what we wanted to achieve provides an opportunity to reflect and learn. It also helps policy and decision makers to demonstrate the impact of a given intervention or policy (Kusek and Rist 2004).

18.2.2 Implementation Framework

The implementation of a framework walks along the project cycle, starting at the design of ToC (Fig. 18.1, left). The use of ToC and PIPA in PM&E along the project cycle aims to facilitate three key functions: accountability, learning, and adaptive management (Fig. 18.1, right). These three functions are interlinked with each other for navigating change in any program or project. The framework shows how the PM&E system (left, outer circle) is based on ToC and how PIPA supports the program cycle (left, inner circle) in terms of the three key functions (right). The implementation of the framework aims at navigating change beyond EO products toward impact for adoption and use of services for societal benefit.

Fig. 18.1
figure 1

Framework for planning, monitoring, and evaluation

Full size image

Accountability in development may refer to the obligations of the partners to act according to clearly defined responsibilities, roles, and performance expectations, often with respect to the prudent use of resources (OECD 2002). Result-oriented M&E helps to collect evidence and demonstrates whether results have been achieved and also justifies value for money, while performance data can be collected at different levels of a program or project’s result hierarchy.

Learning as a function of result-oriented M&E is drawn especially through the assessment of the response to a particular policy, program, or project. M&E also involves the wider context of knowledge management as an element of organizational learning and one that strengthens performance (Adrien et al. 2008).

In this regard, adaptive management helps decision makers or managers to respond to an emerging situation, especially in a complex program, to achieve the desired impacts. M&E provides an opportunity to compare the emerging changes with the desired ones and to reflect on learnings and make adjustment in plans. Thus, adaptive management helps to choose the right path, but the goal remains the same. Knowing the path and the destination also guide us in following the more important path (Örtengren 2016).

18.3 Implementation Process

The implementation of the framework in our model followed eight steps in a broader three-phase project cycle (Fig. 18.2). The planning phase began with PIPA for developing the program’s ToC and the plan for a monitoring, evaluation, and learning (MEL). The implementation of the plan involved the capacity building of the staff on MEL, based on ToC and process documentation. After a while of the implementation process, the remaining steps began. Reality checks, assessments, and evaluations were carried out, along with documentation of the evidence of change. Then, the progress was periodically reported in order to address the information needs of the stakeholders. Later, learning syntheses, reviews, and reflections were carried out to facilitate learning among the stakeholders. Finally, based on the lessons learnt from M&E, the plans were adjusted or revised according to ToC.

Fig. 18.2
figure 2

Implementation process of PM&E framework

Full size image

The aspect of gender has been considered across the PM&E framework as a part of the attempt to mainstream gender and social inclusion in EO and GIT applications and services. We adopted a seven-step framework (Kadel et al. 2017) in the process of project cycle management and encouraged for gender focus right from the beginning at design and planning.

18.3.1 Participatory Impact Pathway Analysis

PIPA was the first step in the PM&E cycle which began with country consultations workshop attended by diverse stakeholders. PIPA is a forward-looking approach built on the past experiences of the stakeholders and provided an opportunity to develop clarity and collective understanding about the program in terms of ToC and performance indicators. The stakeholders systematically followed the seven steps (Fig. 18.3) to come up with a gender-responsive ToC and performance indicators which provided a good basis for program implementation, monitoring, and evaluation.

Fig. 18.3
figure 3

PIPA process followed in SERVIR-HKH program

Full size image

Both the process and products are equally important in PIPA. The process provides the opportunity for different stakeholders which are also users of EO products and services to come together and develop a collective understanding about the project’s impact pathways and the underlying assumptions of change. These assumptions are made explicit through network maps and outcome and impact logic models. The definition of success criteria is based on the indicators that have been identified for monitoring. As for the products, they provide a good basis for program implementation, monitoring, evaluation, and learning. The ToC that was initially developed was not expected to be perfect, but it gradually improved as the understanding evolved. Therefore, once ToC is developed, it is considered as a living document. And the PIPA and ToC concepts were also operationalized at the service level in order to dive deep into the thematic context and develop a collective understanding about the service among the stakeholders.

SERVIR-HKH Theory of Change

In 2014, SERVIR-HKH developed a ToC for the first time for an ongoing project in order to revisit its outcome trajectories. The project team came together at PIPA workshop, reflected on their tasks and achievements about what had worked and what had not and discussed revised outcome trajectories based on the learnings. The ToC was then revised several times in order to simplify it. Figure 18.4 is the simple version of SERVIR-HKH’s ToC.

Fig. 18.4
figure 4

Theory of change of SERVIR-HKH

Full size image

SERVIR-HKH has adopted three interconnected pathways to create an impact: awareness and access, capacity building in cutting-edge EO and GIT applications and provisioning of data tools, applications, and services. The awareness and access pathway helped the country partners to increase their access to EO data, applications, and tools; this was enabled by awareness about open access to already available data, developing data portals, improved networks, and sharing mechanisms. The capacity pathway enabled in updating a particular hub’s knowledge on cutting-edge EO and GIT applications, thereby transferring the same to the regional member countries. Both women and men were part of this process. These two pathways also built synergy with the third pathway to co-develop user-tailored and gender-responsive EO services. The immediate results of the three pathways led to the applications and services that were developed and tailored to the needs of users which increased adoption and use of EO and GIT applications for evidence-based decision making. Ultimately, the adoption and use at scale would lead to improved environmental management and resilience to climate change.

Actor analysis plays a big role in creating a better understanding about the pathways and makes them explicit by striking out all the assumptions. Understanding how this logic works is crucial in order to stabilize or amplify the beneficial outcome trajectories for achieving the expected change, this was a continuous process during the implementation of the program. When clarity was achieved about the ToC, the stakeholders were in a better position to discuss how the program would be monitored or measured. These discussions led to identifying the performance indicators and due consideration was given to make them gender responsive.

18.3.2 Developing a Plan for Monitoring and Evaluation

M&E depends on efficient planning and clarity about what is being monitored (Simister and Smith 2010). The indicators identified in the first step were further explained or defined for everyone’s understanding about what kind of data would be collected and why. Agreement was also reached on M&E activities. Thus, the development of the MEL plan was completed in two steps: preparation of indicator reference sheet and developing the M&E activity plan.

The indicator reference sheet covered details about indicator data, so as to ensure quality and consistency in data collection, compilation, analysis, and use. To guide this process, a standard format was developed which covered important elements like definition, data disaggregation, baseline, target, source of information, the data collection method, frequency of data collection, responsibilities, analysis, and use. The aim of tracking results and measuring change was that the managers would be held more accountable and responsible for achieving results beyond the application or service development. Besides, wherever possible, the setting of gender-disaggregated target was encouraged.

The timely planning of MEL activities was another important part, which is often neglected in practice. This not only ensured budgetary control but also provided enough time to prepare for the implementation of important activities. Systematically planned M&E functions are more effective as they help in timely execution and retain focus. Apart from the factor of indicator data, periodic reviews, assessments, evaluations, learning synthesis, and revision of ToC are important for accountability and adaptive management.

18.3.3 Orientation and Building Staff Capacity

To ensure the adoption and use of EO-based solutions for societal benefit, the managers and data scientists in SERVIR-HKH are expected to think beyond the EO applications. While the staff involved in this program are experts in their subject, they had not been trained in approaches like results-based management (RBM). Thus, the participatory process used for PM&E and M&E trainings, refresher workshops, reviews, reflections, and guidelines helped to build their capacity in RBM. At the monthly staff meetings, M&E was part of the discussion agenda which helped to maintain a continuous focus on this function. The M&E guidelines were also simplified for the benefit of non-M&E professionals.

18.3.4 Process Documentation

While moving to the implementation phase, process documentation became a priority. Process data were important for monitoring the targets and tracking the results. An online event database system was set up for systematic collection, compilation, and analysis of progress data. A standard data collection format was established which could also support data disaggregation. This system has also helped in addressing the problem of duplication.

Besides, pre- and post-training assessment tools were used which helped in making the necessary adjustments in content and method, so as to best fit the participants. While the pre-training assessment mechanism solved for the management the prickly issue of selecting the right candidate, the post-training assessment mechanism paved the way for follow-up actions that needed to be taken to achieve the outcomes envisaged by the ToC.

18.3.5 Reality Checks and Gathering Evidence

After a while into the implementation stage, the partners and stakeholders were expected to respond to the project activities. Both formal and informal processes were used to assess the emerging situation and measure the changes. Knowing whether the targeted stakeholders were adopting and using EO and GIT in decision making or not, and why, helped the managers and the data scientists to adapt to the emerging situation.

Formal tools were used to validate the information collected through the informal process and also to address the information gaps. Tracer studies were conducted to know whether the trainees were using the knowledge and skills acquired at the trainings. This helped in assessing the relevance and effectiveness of the training programs. Besides, periodic assessments were carried out on the organizational capacity levels of the different countries. A post approach to revisiting impact pathways was also adopted, so that the stakeholders could collectively assess as to what was working and what was not. Further, using the ToC, evaluations were carried out to measure the success or failure rate of each EO service. And be it success or failure, all of it has been documented.

The informal process was adopted more at the individual level of EO professionals. After a certain duration of implementation, they were able to closely observe the emergent and expected changes. This also helped them in reflecting on their work to see whether it had made any difference to the overall scheme of things. As for the gender aspect, both formal and informal processes addressed the area.

Box 1. FFWC Bangladesh taking ownership of strengthening FEWS

As a result of a series of trainings, the engineers at the flood forecasting and warning center (FFWC) of Bangladesh have developed a better understanding of the streamflow of upstream rivers, thereby significantly improving their capacity to deliver better and more effective flood warnings. They are now in a position to calibrate and validate model outputs with the data collected during the 2018 floods.

In order to stabilize and amplify this outcome trajectory of strengthened flood early warning system, SERVIR-HKH has been working closely with the FFWC engineers; this has also helped to strengthen these engineers’ capacity. Besides, enough care is being taken to build a sense of ownership over the process and achievements so as to bolster the flood early warning system (FEWS) in Bangladesh.

18.3.6 Reporting and Quality Control

Reports were prepared to fulfill the information needs of the different stakeholders to whom the program is accountable. It was therefore important in the M&E process to ensure that the reports were of a high quality. In this regard, factors such as data consistency and evidence were of prime concern. These were assessed periodically by a dedicated team and by the donors. This helped in improving the overall system of data collection and validation. And all along, gender was made a mandatory part of the reporting process

18.3.7 Reviews, Reflections, and Learnings

Different mechanisms such as program advisory committee meetings, biannual reviews, and monthly staff meetings provided opportunities for collective understanding, reviews, reflections, and learnings. In all of this, monitoring and evaluation played a big part, and evidence seeking was accorded high priority. It was also important to ensure maximum participation at these gatherings, both of women and men. The meetings were also avenues for individuals to reflect upon their work and conclude whether it had made any difference to the entire program.

18.3.8 Revisiting ToC and Adjusting Plans

One of the important steps in the MEL framework is to help managers and decision makers adjust their plans and adapt to any emerging situation. Comparing what is happening with what was expected and planned for helps determine how a program can adjust its plans and responses (Douthwaite et al. 2018). In this regard, biannual reviews and planning workshops and learning synthesis were duly conducted. These reviews shed light on what was working and what was not and created a better understanding about the emerging situation, thereby helping populate the original ToC with greater detail and paving the way for necessary amendments, whether minor or major (Douthwaite et al. 2018). The ToC for this project was first developed in 2014 was later revised in 2016 in order to bring about a stronger focus on user engagement and to sustain the impact of EO services (see Chap. 2 for more details). As things stand, the plans are usually adjusted biannually if the need arises.

18.4 Results and Discussions

The implementation of the framework based on ToC and PIPA has produced some results; some of them are briefly described below.

18.4.1 The Ability of the Implementing Staff to Change Direction

Since its inception in 2010, SERVIR-HKH has carried out decadal mapping of land cover to record changes and to support monitoring system in the region. In 2015, the Development Alternatives Incorporated (DAI) evaluated this system. Its report says that the country partners were reluctant to use this system, and the accuracy of the products was their major concern.

But the situation has changed by 2020. The scientists, together with the country stakeholders, have developed a ToC for the system. This has helped the stakeholders and the experts to not only develop clarity on the objectives, but has also provided them the opportunity to tailor the system to the needs and priorities of the users. The system has now been upgraded with capability to generate annual land cover map. The Forest Research and Training Center (FRTC) of Nepal which is the key partner and user has now pre launched the Nepal land cover monitoring system (NLCMS) and has allocated fund for the field-based validation process. This shows a sense of full ownership and a buy-in of the system. Now Bangladesh’s Department of Forest is planning to adopt this at the national scale after it successfully launched a pilot. ICIMOD is providing support for the system in both the countries.

This example demonstrates the fact that the EO data scientists and the stakeholders are now giving more importance to adoption and use rather than merely to the application. With this shift in focus, the scientists have proactively started to engage the users in the process of formal and informal service development. As a result and because of the increased trust, the service is now more tailored to the needs of the users. The individuals concerned have also deepened the evidence-based reflection process, whereby they evaluate as to whether their work choices have made any changes as explained by the ToC.

The implementation of this ToC and PIPA approach in planning, monitoring, and evaluation has made the managers and decision makers to change the direction of their work toward one that makes impact. ToC and PIPA continuously focus on the outcome right from the design stage. The PIPA process through which the ToC and performance indicators were developed has brought in clarity, and a collective understanding about how EO and GIT applications should be used for addressing complex problems. Once the managers understood the pathways to ensure the adoption and use of EO products and services, their management priority changed to one of appropriately adapting to any emergent situation. The focus is now more on outcomes and impacts rather than inputs and activities.

18.4.2 Staff Buy-in to the Approach

The staff at SERVIR-HKH has also been slowly able to develop an interest in this approach; this came about after they understood the facet and value of accountability, learning, and adaptive management. Several things have helped in instilling this interest. For example, PIPA has provided an opportunity to the stakeholders to have a better grasp on the pathways of change. Besides, they have found that the ToC which was developed at the PIPA workshop has been extremely useful in strengthening their communication network. This has also helped them to systematically plan, monitor, and evaluate the program, so as to achieve the stated objectives. Now, in discussions and management decisions, the ToC has become a permanent feature.

Once the managers and decision makers understood the value of this approach as well as of monitoring and evaluation, there has been a key change in the management culture. At the individual level, the people concerned are reflecting on their own actions, so as to engage in a process of continuous learning and adaptation to change (Schon 1983). The staff and the stakeholders have now started to ask critical questions of each other at meetings and discussions, and all of these are based on the program’s ToC. This has also ushered in a culture of learning and sharing. At staff meetings, M&E is a prime agenda, and the focus is on regular reflections on actions to achieve the best possible results. As there is now a systematic approach to result tracking and measurement, evidence seeking and trust have become an intrinsic part of the work culture. These reflections have also made individuals break away from the traditional patterns of thinking and acting (Klerkx et al. 2012). Thus, there has been a palpable change in the management approach of the scientists, whereby they place the welfare of the stakeholders at the forefront. And, they are reflecting on their work to see if their work choices have brought any changes, so as to make right choices from best possible way.

18.4.3 Partners Able to Bring Multiple Perspectives

The earlier flood early warning system (FEWS) of Bangladesh, with a four-day lead time, had been unable to reduce the loss and damage from floods. So, the country’s flood forecasting and warning center (FFWC) decided to increase the lead time with support from ICIMOD. While the discussions focused on the impact pathways to change, the stakeholders realized that merely increasing the lead time of the warning would not be as effective if it was not combined with information on the flood-risk level and its impact on the ground. Therefore, FEWS was strengthened by increasing the lead time to 15 days, and a provision was also set up to provide information on the potential impact of the flood. Further, there was the realization that the field staff played an important role and that there should be community access to such information.

This case demonstrates that the data scientists and the stakeholders were able to think differently by bringing in multidimensional perspectives on service design. There was also an increased focus on outcomes, impacts, reviews, and reflections, all guided by ToC and PIPA. This is a continuous process facilitated by PIPA that allows different stakeholders to come up with diverse ideas, so as to tailor a project according to the practical context on the ground. Moreover, this flexible process helps in adapting to any emerging situation and paves way for out-of-the-box solutions. Open dialog among all the stakeholders also helps them to analyze and reflect on their actions to determine what is working and what is not (Earth Village 2020). This improves the mechanism of informed decision making to achieve the intended results.

18.4.4 Stakeholders Able to Develop a Sense of Ownership and Trust

A sense of ownership and trust among stakeholders is important for the adoption and use of EO and GIT services. The implementation of this framework based on ToC and PIPA has helped the stakeholders to feel a better sense of ownership and trust mainly in three ways. The first has to do with the overall PIPA process. It has provided an opportunity for the stakeholders to be engaged right from the beginning of the program, at the design stage itself. This has helped them to have a deeper understanding about the project or service and its relevance to their needs. This made for a good beginning in terms of building trust. The collective process of PIPA has also helped the staff and stakeholders to be properly motivated about implementing their respective projects (Douthwaite et al. 2020). Secondly, being able to co-create and co-develop solutions have also fostered a heightened sense of ownership. The third aspect has to do with improved relations among all the relevant parties. The continual formal and informal process of user engagement has improved the relationship between the data scientists and the country stakeholders which has also helped to increase trust among them. The stakeholders, by being increasingly engaged, feel more responsible toward the products and services, and along the way, interpersonal trust is also built, thereby leading to greater adoption and internalization of technology (Lippert and David 2016).

18.4.5 Breaking the Gender Silence in Data Service

Mainstreaming gender tends to be context-specific as opposed to an “off-the-shelf” process (Kadel et al. 2017). The field of EO–GIT is considered especially challenging in terms of mainstreaming gender. So, the aspect of gender was given prime priority when it came to developing the ToC for SERVIR-HKH. PIPA organized with multidisciplinary team became a good enabler to bring exclusive gender lens at design stage itself. This process initially triggered heated discussions between the data scientists and the social scientists, with the former opposing the idea and the latter supporting it. They had an argument that mainstreaming gender would not always be possible and EO and GIT field could be the one. Finally, they agreed to two key mainstreaming approaches: consider gender while developing the EO products, so as to reduce gender gaps and promote gender balance in participation in project events and in staffing. This step helped managers to develop a gender mainstreaming framework for the entire SERVIR-HKH program (Chap. 15). But while the silence on gender has been broken in the EO and GIT field, much more remains to be done.

18.5 Challenges

18.5.1 Iterative Learning Process

The staff involved in this project are experts in EO and GIT but not trained in approaches such as RBM. In several cases, they had to teach themselves about these approaches. As mentioned earlier, the use of ToC and PIPA are aimed at creating a better understanding about all the complexities involved in a program. This is more about “learning-by-doing” for which the staff and the stakeholders have to possess the requisite patience. However, at the same time, the managers are under pressure to produce results. They cannot keep trying out things for too long. They should be able to make the best choice by keenly observing and monitoring the emerging situation. But all said, it is always a challenge to strike a balance between the learning process and achieving results.

18.5.2 Balancing Simple and Complex Theories

The implementation of the ToC and PIPA approach demands the engagement of diverse groups; so, to keep their interest alive, it is essential to simplify the whole process. But it is also equally important not to ignore the complex part and oversimplify matters. Striking a balance between these two is challenging. The idea is to keep things as simple as possible but not at the cost of losing important information.

In SERVIR-HKH, we have localized the ToC and PIPA approach at the service level to facilitate deeper analysis. A tabular format of the ToC with all the necessary elements has further simplified the process. This tabular format has helped diverse stakeholders to engage in the PIPA process, especially at the early stage of developing the EO service. Once clarity came about, the staff and stakeholders again referred to the ToC diagram with its feedback loops as it nicely summarizes how and why change happens. We have always been for simplifying the ToC diagram as far as possible but without losing important information. This is because such simplification will make it easier for the non-M&E professionals to understand the pathways better and use them frequently for communication, reflection, and learning. But the accompanying narrative is still important, particularly to explain the feedback loops and assumptions that can be subsequently tested (Barnett and Gregorowski 2013). Besides, it is important that M&E professionals work closely with other staff and stakeholders to strengthen this process.

18.5.3 Impact Assessment

EO and GIT aims at supporting evidence-based decision making for addressing the problems that people face. The larger society will benefit only if the decisions are implemented properly; the primary consideration is that these applications and products ought to be widely adopted and used at different levels. It has also got to be accepted that the impact on the beneficiaries or the wider civil society may not be seen until well after the time frame of a particular project or a program (Simister and Smith 2010). Therefore, the purpose of capacity building of the country stakeholders to generate and use EO data and GIT services in most cases is less discussed and loosely defined. Impact assessment in such a situation becomes difficult. There are also many people working in this field who argue that measuring the effect of adoption and use of EO applications and services is beyond the scope of this entire program.

18.6 Lessons Learnt

18.6.1 Transformation Requires Support from Top Management

In our decade-long experience, we have observed a big behavioral change in the SERVIR staff involved in project management. The availability and access to EO data and products were the primary goals in the early days of SERVIR-HKH. Now, the focus has shifted to the adoption and use of EO services for societal benefit.

This change would not have been possible without support from the senior management. Trust and support from the senior management in terms of these approaches have encouraged the staff to embrace the new ways. Learning from both success and failure is important, as is also understanding the complexity of a program and the pathways of change. In this context, incentives need to be in place to regularly collect evidence around a theory, test it periodically, and then reflect and reconsider its relevance and assumptions (Barnett and Gregorowski 2013). This requires consistent commitment and support from the top management. Buy-in from the top (Schuetz et al. 2017) and acceptance by the managers and decision makers enable effective implementation of such approaches and strategies.

18.6.2 Localized Approach Simplifies Operation

While there is a general understanding that a ToC has to be placed at the higher level of a program on the complexity front, the SERVIR-HKH experience shows that equal importance ought to be accorded to a localized approach at the service level. This localized approach helps to contextualize the broader concept and renders the operations more effective. In fact, each EO service is a unique case in terms of issues, partnerships, and opportunities. The PIPA process adopted to develop EO services in SERVIR-HKH has provided the opportunity to understand the context better and align the service with the needs of the stakeholders and the users. And the ToC keeps the common goals intact and helps the stakeholders in nurturing them during the implementation phase.

18.6.3 Building Individuals’ “MEL Value Perspective”

Value perspective plays a big role in the RBM and learning process. “MEL value perspective” refers to a different way of looking at and working with MEL values where individual work choices are driven by continuous learning through evidence seeking, monitoring, and evaluation (Hyatt and Ciantis 2014). Our decade-long experience with this program shows that developing an MEL value perspective in scientists is a slow and complex process, but it is important if they are also managers of a complex program or project. Institutionalizing collective planning and learning as a part of M&E requires the capacity building of staff, and they also have to be properly motivated (Douthwaite et al. 2018). The staff may be experts in their subject but they may not have had training in the ToC and PIPA approach and RBM. This was reflected in the early days of the SERVIR-HKH program, wherein the EO applications could only be put to limited use (DAI 2015; Morrison et al. 2017). Ensuring that EO applications benefit people in need is more about management rather than science. The effective management of a complex program for sustainable impact is a continuous learning process. In this regard, a multidisciplinary work culture where different views are valued requires to be promoted for collective understanding and learning.

At SERVIR-HKH, we place more value on the participatory process and continued reflections, in line with the ToC, at both individual and group levels. Breaking away from the earlier practice, M&E has become a regular feature at staff meetings. Besides, practical trainings and refresher workshops have been held, focusing on the individual’s M&E roles and responsibilities, thereby enabling them to contribute toward achieving the bigger objective. As a result, the scientists are now more open to different views and learnings in the overall process of delivering EO services. The user has also been placed at the forefront of this process—recognizing that it is the needs and interests of the user that matter the most, and user engagement has been accorded the topmost priority in the SERVIR-HKH program.

18.6.4 Flexibility Matters

The ToC and PIPA approach is used to build better understanding about a complex program or change process. This aspect of complexity may not always be fully understood, at least in the beginning (Barnett and Gregorowski 2013). Therefore, the flexibility of accepting imperfection in the ToC at the beginning is important. It will be improved upon as the understanding develops over time. Flexibility may also be required in the steps and overall process since a diverse set of stakeholders may be involved in a particular program.

18.7 Conclusion

In the context of SERVIR-HKH, the ToC and PIPA used in planning, monitoring, and evaluation has helped managers and decision makers to push the boundary beyond EO applications and toward creating meaningful impact. The successful adoption of this approach has helped managers, data scientists, and country stakeholders to change direction; wherein the focus now is on results, developing a sense of ownership, and building trust in the EO applications, this has led to a spurt in the use and adoption of these applications. The PIPA approach has allowed the stakeholders and users to come together in order to arrive at a common understanding about how EO and GIT applications would help in addressing the problems of the HKH region. And the ToC produced through PIPA has provided a good basis for program implementation, monitoring, and evaluation. The participatory approach used in the implementation of the program has not only helped in harnessing the complexity of the program, but has also tailored the services to suit the needs of the users; the inclusion of the aspect of gender is a good example here.

Nevertheless, several challenges are yet to be overcome. It is important here to emphasize upon the iterative nature of the learning process; so, the staff and stakeholders need to be patient all through the whole process. In this, support from the top management also has a vital role to play. Another challenge is in simplifying the whole process while there are several complex elements to it. It has also got to be borne in mind that programs like SERVIR-HKH follows a long pathway by its nature to create impact. But in the first place, the impact part has to be clearly defined, so as to avoid ambiguity about the bigger picture and to accurately measure the impact.

The focus of SERVIR-HKH has been on adopting a localized approach to operationalize the bigger picture, so that there is complete understanding about the context in which the program is being run. While having a ToC in place is a good beginning, it is as important to make adjustments as the project cycle wheels along and the strategy being adopted matters for the best results.