Introduction: need for actionable knowledge to improve programs for children

Significant progress in maternal and child health has been achieved over recent decades. Global under-five mortality dropped by more than half since 1990 [1]. Global maternal mortality fell 38% since 2000 [2]. Despite these achievements, unacceptable inequities in intervention coverage and child mortality remain, both among and within countries. Attention is needed to improve the quality of health and nutrition services and address systems challenges. Also, the contexts in which children live are changing [3, 4]. In 2030, children will live in a world that is more urban, mobile, interconnected, and with an aging population. Income growth will shift some children into wealthier, but not necessarily healthier environments. Fragility is also expected to persist in countries struggling with extreme poverty, conflict, and weak governance. Emergencies, including public health emergencies and those stemming from environmental causes and climate change, are expected to increase in frequency [5].

Leroy et al. [6] noted that research on development of new interventions in child health and nutrition could potentially reduce under-five child mortality by 22%, whereas if existing proven interventions were fully implemented, these programs could reduce under-five mortality by 63%. They note the paradox that the majority of research funding focuses on new interventions (97%), rather than addressing implementation challenges (3%). This paradox demonstrates the urgent need to focus on implementation research (IR) to identify barriers and effective strategies to implementation of existing proven interventions. Evidence of the effect of long-term consistent investments in embedded IR on improved service coverage and efficient use of routine health system resources has recently emerged from Ghana [7, 8] and Latin America and the Caribbean [9].

This paper presents UNICEF’s embedded IR approach, its application to maternal, child and nutrition programs, and present experiences to date; describes challenges and lessons learned; and considers implications for future work. The authors are all UNICEF staff who have developed and implemented the approach across the organization.

Embedded implementation research

Implementation research is part of the broader field of implementation science. Rapport et al. define Implementation Science as “the scientific study of methods translating research findings into practical, useful outcomes,” but also note that the science is currently “contested and complex” [10]. Eccles and Mittman in the launch of the journal Implementation Science defined implementation research as “the scientific study of methods to promote the systematic uptake of research findings and other evidence-based practices into routine practice, and, hence, to improve the quality and effectiveness of health services and care” [11], while Peters et al. [12, 13] note that “The basic intent of implementation research is to understand not only what is and isn’t working, but how and why implementation is going right or wrong, and testing approaches to improve it.” This form of research addresses implementation bottlenecks, identifies optimal approaches for a particular setting, and promotes the uptake of research findings. Further, Ghaffar and colleagues [14] argued that IR should be ‘embedded’ in programming in partnership with policymakers and implementers, integrated in different settings and take into account context-specific factors to ensure relevance in policy priority-setting and decision-making. This view is further supported by Langlois and colleagues [9]. Churuca et al. note that “Embedded implementation research involves a knowledgeable researcher working with, or within, the team responsible for change, adoption, or take-up” and go on to describe four approaches of “embedded” research: dichotomized research-practice, collaborative linking-up, partially embedded, and deep immersion, describing the researcher-implementer relationship [15].

Varying definitions of operational, implementation and health systems research often cause confusion for both researchers and program managers. For example, the distinction between operations research and IR has been debated [16] as the two types of research are often similar in intent and scope. Definitions are often progressive without clear delineations, so many times operational, implementation, and health system research overlap [17]. Similar issues exist across other terms, such as formative research, process evaluation, and translational research.

UNICEF’s approach

Working in 190 countries and territories, UNICEF advocates for the protection of children’s rights, meets children’s basic needs, and expands their opportunities to reach their full potential. UNICEF’s comparative advantage is to work across sectors and across the life-cycle to protect these rights, focusing particularly on protecting the most disadvantaged and vulnerable children. UNICEF’s strategic plan 2018-20219 describes research as a “how” strategy to achieve targets [18]. UNICEF notes that in some situations while substantial evidence exists on what needs to be done, there are evidence gaps when it comes to identifying viable approaches for sustainable, full implementation and scale-up [19] of responses to improve programs for children.

As a large multi-lateral organization with a primary mandate of improving programs for children, UNICEF approached implementation research based on practical approaches and frameworks from the literature, with a focus on ‘how’ as defined in the strategic plan. UNICEF uses definitions outlined by Remme et al. [17], where operational research is a subset of IR, which is a subset of broader health systems research. In addition, UNICEF has primarily adapted taxonomy as described by Peters et al. and others [12,13,14] and has further defined embedded IR as “the integration of research within existing program implementation and policymaking to improve outcomes and overcome implementation bottlenecks,” and primarily uses either the” collaborative linking-up” or “partially-embedded” approaches for establishing the relationship of researcher and implementer as defined by Churucca et al. [15] UNICEF is adapting these innovative approaches for health systems strengthening in which research is made integral to decision-making. It includes (a) positioning research within existing programs and systems, building a new evidence ecosystem, and drawing siloed sectors together; (b) meaningful engagement and leadership roles for decision-makers and implementers within the research team; and (c) when possible, aligning research activities with program implementation cycles. Embedding increases the likelihood of evidence-informed policies and programs. Embedding IR into the policymaking and systems strengthening process amplifies ownership of evidence, recognizes decisions are not made on evidence alone, and also takes societal context and values into account.

Peters et al. note that research on innovations encompasses basic science through translation to sustainable implementation of new or existing evidence-based programs [12]. IR in UNICEF focuses on the latter stages of this continuum, i.e., issues ranging from how a program works in real-world settings to systems integration and scale-up, including decision-making, policy development, and creating an enabling environment for implementation.

IR to support UNICEF programs is generally located within monitoring, evaluation, and knowledge translation or learning, as a basis for program innovation, systems strengthening and implementation. IR does not take the place of routine data collection, monitoring and evaluation, but is a complement to it. Figure 1 provides a conceptual model for how IR fits into the overall evidence and learning cycle for program management. Implementation research can also be associated quality improvement programming as part of learning and evaluation [20,21,22]. UNICEF uses IR at various stages of program implementation, including formative and initial implementation stages and throughout the program cycle. We see IR as particularly useful for understanding challenges or bottlenecks which might be found while monitoring implementation of the program. This is a practical applied adaptation of IR suited for the needs of UNICEF. Recognizing that implementation science and implementation research has a variety of approaches and methods [12,13,14,15,16], UNICEF sought to apply IR within our program support cycles in collaboration with governments and implementing partners. While the broader field of implementation science is critical, we needed to adapt the approach to fit the targeted and very focused needs of our country programs.

Fig. 1
figure 1

Where implementation research fits in program knowledge and learning cycle

Full size image

Embedded implementation research at UNICEF: experiences 2015–2019

UNICEF, in partnership with the Alliance for Health Policy and Systems Research (The Alliance) and the Special Program for Research and Training in Tropical Diseases (TDR), has adapted an embedded IR approach for systems strengthening for country-based programs for children. As part of the UNICEF Health Systems Strengthening Approach [23], the research seeks to catalyze a shift in the way evidence is generated and used within countries to inform policy and decision-making. By bringing together (a) in-country decision-makers at national, sub-national, and local levels; (b) country-based researchers; and (c) global development partners, it puts local decision-makers and implementers in the driving seat in the research process, while identifying clear roles for different stakeholders.

The UNICEF approach aims to enhance ownership of the research among local implementers, similar to the co-production and collaborative approach to health systems research recently highlighted by Redman et al. on behalf of the “Co-production of Knowledge Collection Steering Committee” [24]. Although it may influence local or higher-level policy, it is primarily designed to prioritize research on questions of local relevance, build capacity to conduct local IR to generate feasible recommendations in “real-time” and underwrite policy and system strengthening. IR can also be considered during program initiation to answer questions on the acceptability, appropriateness and feasibility of alternative delivery strategies, as well as blending with evaluation, using “effectiveness-implementation hybrid designs” [12, 25], to address program effectiveness.

UNICEF works with both global and local partners to identify priorities on implementation barriers needing resolution through systematic and inclusive IR processes. Programs, through IR, can learn why implementation barriers and contextual variances mean that interventions work well in one context, but not in another. In addition, it can be used to test new approaches or innovations from pilot through scale-up across different contexts (e.g., geographic, political, physical environment, social, economic). Implementation research can also document failures, where an intervention success could not be replicated given local context, which is equally valuable to prevent wastage of funds before investing in scale-up.

UNICEF’s embedded IR approach generally starts with sensitization of national stakeholders including Ministry of Health and policymakers to what IR is and what the potential benefits are (Table 1). Implementation barriers, often previously identified through national, sub-national, or local program reviews, or monitoring and evaluation, are reviewed, summarized, and prioritized. In collaboration with national stakeholders and policymakers, implementation barriers are then transformed into priority research questions and potential related IR studies are identified. A research team comprising a partnership between national policy-makers, local decision-makers, and implementers (e.g. program managers, district managers, front-line health workers), and in-country researchers is convened. Through this process, UNICEF staff, in partnership with implementation researchers or research institutions (global or local), provide technical support and training to develop protocols, ensure ethical research standards are maintained, conduct studies, and support communication of results, recommendations, and use of the findings for policy and program changes. Figure 2 provides an example of the UNICEF embedded IR process.

Table 1 Key characteristics of UNICEF’s embedded implementation research approach
Full size table
Fig. 2
figure 2

UNICEF embedded implementation research process

Full size image

Figure 3 shows countries where UNICEF has worked with partners to embed IR within existing programming activities. In collaboration with in-country researchers, policy-makers and program implementers, UNICEF has been supporting IR projects globally since 2015. The research has varied from formative early stages of programming, through initial implementation, to full implementation. Methodological approaches varied by study and included quantitative, qualitative, and mixed-methods. Study questions have addressed a wide variety of multi-sectoral topics (e.g., immunization, child health days, birth registration, newborn and child health in humanitarian settings, prevention of mother-to-child transmission of HIV) and health system challenges (e.g., information systems, human resources, supply chain, demand for services, community engagement, integration).

Fig. 3
figure 3

Countries participating in UNICEF-supported embedded implementation research since 2015

Full size image

Measuring embedded implementation research success

UNICEF’s selected measure of success for embedded IR is that the research findings are used for policy and/or program changes. This near-real-time use of findings is key. Dissemination and publication of the results alone does not count as “use,” and policy or program changes should be documented, no matter how small or local. Selected examples can be found in Table 2 Case studies. Informal feedback suggests that from two-thirds to three-quarters of the IR studies successfully resulted in a documented policy or program change. Reasons for lack of use of findings suggested were limited time and resources and lack of formal follow-up after the research.

Table 2 Case study examples of UNICEF implementation research
Full size table

In addition to monitoring program results, we have also documented government implementers’ and local research partners’ [26] experiences in the program. Results suggest positive responses to participation in the process. Table 3 cites two quotes from a project in Pakistan, one from before the IR and one after, which exemplify the common responses seen from participants, and in particular highlight the co-production of knowledge.

Table 3 Quotes from participants in embedded implementation research in Pakistan [27]
Full size table

Funding embedded implementation research at UNICEF

To date, UNICEF-supported IR studies have been funded almost exclusively through projects, as part of the program monitoring or learning agenda. Studies are typically short-term and require limited funding. For example, five HIV-related IR studies in 2017 cost US$15–35,000 each for data collection and analysis and were completed within 5 months. Projects to date range from $10,000 to $70,000 (usually $20,000-$40,000) and 5 to 18 months (usually 12 months). This near real-time aspect of embedded IR is recognized as one of the advantages as research results can be available within planning cycles for decision making and program adaptation. While the research projects themselves generally require limited funding, building local capacity to run the studies, both for the implementers and local researchers, often requires additional resources beyond the actual research costs for training and technical assistance during the studies, consistent with building capacity of research co-production as discussed by Agyebong and colleagues [8].

Key challenges and considerations in developing UNICEF’s embedded IR approach

IR has been recognized as critical to strengthen health systems [14, 27, 28]. However, the concept of embedding research into real-world policy, practice, and implementation is somewhat new in the field of global children’s programming, and uptake of the approach has challenges. For example, we have found that in-country partners, including local and national-level government counterparts, and some donors, need to be convinced about the value of IR. By engaging stakeholders in this approach, we have seen a recognition of the use of IR for program planning and enthusiasm for continuation of the implementer-researcher partnerships and research co-production after completion of the initial IR project. Also, for IR to be truly country driven, donors and partners have to trust that countries can identify the most relevant implementation barriers, transforming them into questions to investigate. This will require adaptations for review of research proposals, which may be more undefined regarding objectives and methods, given that these will be defined as a first step of the research process. Greater emphasis on domestic resource mobilization for embedding IR into the decision-making process and into routine program funding is needed. This problem can be overcome by advocacy, showcasing the value IR brings, building the capacity of partners on IR, and engaging them early and throughout the research activities so as to address their stated priorities, gaps in knowledge, and improve policy and ownership. Weighing opportunity costs between investing in service delivery and/or implementation research requires continued focus, as turnover of leadership and staff can reverse gains made in many contexts.

Another challenge is that in some cases, our partner implementers and researchers have been overly ambitious or wanted to pursue larger scale research. However, our experiences show that time-limited, small-scale and relatively inexpensive IR studies can lead to important learnings that have translated into changes in policies or approaches. We saw that the IR brought these two communities closer together with the benefits of greater relevance of research to programming, introduction of new methods, and faster implementations of specific solutions. To build incentives for researchers to do IR, showing how their research led to policy and practice change and not just traditional peer reviewed publications could be valued by universities as a component for promotion or research funding. Recent programs that promote and fund partnerships for implementation and health systems research, such as those by the Australian and UK Medical Research Councils and the Doris Duke Foundation African Health Initiative, have been a welcome contribution to the funding landscape [29, 30].

Assessment of research quality and how the results are contributing to the existing evidence base also needs to be addressed. Many IR studies while being used locally for in country program improvements may not be published in peer-review literature, but nevertheless could contribute to expanding the evidence base on implementation strategies. Therefore, publishing in on-line platforms, such as the TDR Gateway (https://www.who.int/tdr/publications/tdr-gateway/en/) or similar sites, will allow for quality-assurance and rapid wider dissemination.

Conclusion

UNICEF has built on the work of The Alliance, TDR, and others, to adopt an innovative embedded IR approach to meet country program needs to assure the right to health and well-being “For Every Child.” Implementation research at UNICEF is now supported across several sectors and by the Office of Research, suggesting a sustainable future for the approach. In addition, program staff from more than 25 countries have received training on this approach and how to support it with their partners during program implementation. IR has also been added to the UNICEF-University of Melbourne-Nossal Institute Health Systems Research Massive Open Online Course (https://www.futurelearn.com/courses/health-systems-strengthening). We have also seen an expansion of global partners and universities supporting this research, such as the Implementation Research and Delivery Science Coalition (https://www.harpnet.org/wp-content/uploads/2018/10/Coalition-Statement.pdf) and several TDR Postgraduate programs (https://www.who.int/tdr/capacity/strengthening/postgraduate/en/) in Bangladesh, Zambia, and Ethiopia, have developed partnerships with local UNICEF country programs.

Embedding research into local systems and service delivery can address concerns of implementers and support selection of effective implementation strategies, taking into account local context and systems complexity to address implementation barriers. UNICEF embedded IR seeks to understand how to overcome these barriers within maternal, newborn, child, and adolescent programs—in and beyond the health sector. In addition, it can be used to test applicability of approaches in different contexts (e.g., geographic, political, physical environment, social, economic). Ultimately, the aim of these activities is to build embedded IR capacity and accelerate large-scale adoption, effective implementation, and dissemination of successful approaches that generate results for women and children.