Background

The push to achieve Millennium Development Goal (MDG) 4 —the reduction by two-thirds of the under-five mortality rate (U5MR) by 2015—has drawn greater attention to child survival. Over 44% of all deaths among children under five are neonatal deaths (0–28 days), and this proportion is increasing as deaths decline more rapidly amongst children after the first month of life [1, 2]. In 2013, nearly three million newborns died, with almost all (99%) occurring in low- and middle-income countries [2]. Approximately three-quarters of these deaths take place in the first week of life with more than a third dying within 24 hours of birth [3–5]. The majority of these deaths are due to preventable and treatable causes. Direct preterm birth complications are the leading cause of newborn deaths and the second leading cause of all under-five mortality. Preterm birth complications, intrapartum-related deaths, and infections account for more than 80% of deaths globally [6]. If mothers and newborns receive quality care over two-thirds of newborn deaths could be averted worldwide [7]. Newborn survival and health in resource-poor settings has recently emerged on the global health agenda with a resulting push to define the problem and generate evidence on best practices for care [8] but this agenda has not translated to emergency settings.

While under-five mortality rate (U5MR) has been decreasing globally [9–11], progress is slowest in sub-Saharan African countries. Many of the countries with the highest neonatal mortality rates are currently or have recently been affected by complex humanitarian emergencies (see Table 1) [12][13–15]. Complex humanitarian emergencies are due to conflict, war or civil disturbance, natural disasters, food insecurity or other crises that affect large civilian populations and result in significant excess mortality. These crises are often characterized by population displacement as well as the collapse of basic health services and local and national infrastructure that results in the need for international assistance as the emergency exceeds the capacity of the local government [16].

Table 1 Conflict and refugee status in the 25 countries with the highest neonatal mortality rate (NMR)[2, 13, 14, 17]
Full size table

In 2011 it was estimated that 42.5 million people were displaced by events such as war and natural disasters [17]. Women and children often make up a majority of the displaced population and account for a large proportion of disability, morbidity and mortality among this population [18]. The main adverse health consequences within this group are due to poor pregnancy outcomes that affect both the mother and newborn, and to the increased risk of sexual violence and exposure to HIV/AIDS [16, 19, 20].

Each humanitarian response situation varies enormously in how efficiently and equitably health services are provided. Many areas of humanitarian response are undergoing a process of reform based on operations research and information exchange on minimum standards of care and consistent principles for providing health services. For example, as a result of increasing knowledge on best practices for neonatal survival, particularly in developing countries, this issue has recently been emphasized in research and policy statements as an essential component of humanitarian response [21–23]. While guidelines exist on what to do to improve maternal and newborn health [24, 25], it is less clear on how to implement those practices systematically across settings [15].

Neonatal health in complex humanitarian emergencies

Few published studies exist on the burden of neonatal health in complex humanitarian emergencies and even fewer provide information on improving service delivery in these settings. Therefore, a literature review was undertaken to understand the epidemiological burden of neonatal mortality in complex humanitarian settings [Adler A, Amsalu R, Kerber K, Lawn J, Neonatal Mortality Literature Review unpublished]. Using 24 specific search terms, researchers identified 34 reports from 17 different sources including published papers, gray literature, unpublished work by epidemiologists, and digital and hard copy review of documents, Demographic Health Surveys (DHS) [26], Multiple Indicator Cluster Surveys (MICS 3) [27], and the International Disaster Databases [28]. Key findings from this review showed most mortality surveys included crude morality rate (CMR) and U5MR, with very few surveys identifying neonatal mortality rates or causes specific to newborns. Additionally, DHS and MICS surveys from conflict-affected countries were reviewed to determine if data were disaggregated for the portion of the population that was internally displaced and found that few countries had that information available. The lack of epidemiologic information on the causes of neonatal disability, morbidity, and mortality and effective interventions in complex humanitarian emergencies prompted a discussion on the need for more data on neonatal health in these settings.

Child Health and nutrition research Initiative method

Funding for public health research in complex humanitarian settings, much like many other settings, is limited [29]. Given the scarcity of resources, the Child Health and Nutrition Research Initiative (CHNRI) developed a systematic method to align health research investments with the burden of global child health and the potential impact of the research by having key stakeholders identify priority research questions [30–32]. The CHNRI method is a flexible process and includes up to 15 steps that enables prioritizing health research investments at the institutional, regional, national, international, or global level. The CHNRI methodology has been used to rank research priorities for preventing stillbirths [33] and for many other subject areas such as intrapartum-related neonatal deaths, newborn infection, preterm and low birth rate, zinc deficiency in children, as well as mental health and disability [29, 34–45]. Using the CHNRI method has the additional potential to engage researchers, donors, humanitarian agencies and health professionals to improve dialogue and sustain and expand beyond the research selection process to form part of a viable public health strategy [29, 37, 39–41, 46].

The purpose of this study was to use an adapted CHNRI methodology to prioritize neonatal health research most likely to reduce newborn related mortality in the unique setting of complex humanitarian emergencies. The purpose was also to facilitate greater openness, communication and collaboration among partners and improve the allocation of research funds for neonatal health research in complex humanitarian emergencies [40].

Methods

Save the Children and the US Centers for Disease Control and Prevention (CDC) invited technical experts to apply the CHNRI methods to systematically list research questions and use standard methods to score them (see Appendix 1 and Additional file 1). We applied the following processes: (i) Defining the research context, questions, and criteria for priority setting; (ii) Enlisting experts to systematically score the research questions, and; (iii) Computing and writing up the results.

Defining the research context, questions, and criteria for priority setting

Research context

The research context was defined by space, time, population, and burden of disease. The space we used was complex humanitarian emergencies, defined as acute or chronic situations of conflict, war or civil disturbance, natural disasters, food insecurity or other crises that affect large civilian populations, result in significant excess mortality, and are beyond the capacity of the local government to cope [16]. Time for the context of the research was defined as present day to the year 2020. The burden of disease was all cause mortality and disability during the neonatal period. The population of interest was internally displaced persons (IDP), refugees, and communities affected by complex humanitarian emergencies in low and middle income countries.

Research questions

Global experts in neonatal health working on Save the Children’s Saving Newborn Lives program compiled a list of 97 research questions, some of which were used in previous research prioritization exercises for non-emergency settings. Using a research pipeline model that promotes a global perspective and intent to scale-up in both low and high-income settings [47], the research questions were categorized into descriptive epidemiology (describing the situation and understanding the determinants), leading to development (improving existing interventions by increasing deliverability or reducing cost), delivery (health policy and implementation research and scale up), and discovery (new and basic science concepts or technologies). The 97 neonatal research questions were reviewed by the experts to remove duplication as well as ensure appropriate applicability and relevance to complex humanitarian settings. A final list of 35 questions was used for the prioritization exercise. The final questions were categorized into descriptive (n = 9), development (n = 8), and delivery (n = 18). Based on expert selection, there were no discovery questions included in the final list.

Criterion selection

Criteria were agreed upon for use in prioritizing the research questions. These criteria were chosen based on a standard methodology used in previous prioritization exercises developed by the Child Health and Nutrition Research Initiative (CHNRI) and used for multiple exercises in research priority setting for varied subjects [37]. Each of the 4 criterion had three subcomponents (see Additional file 2).

  1. 1.

    Answerability: likelihood that research would lead to new knowledge in an ethical way;

  2. 2.

    Feasibility: likelihood that research can be conducted cost-efficiently and leads to generalizable new knowledge;

  3. 3.

    Relevance: likelihood that research would address important condition and critical gap in knowledge and could be readily translated to inform policies and programs

  4. 4.

    Equity: likelihood that the proposed epidemiological research would have positive impact on equity and local ownership.

Enlisting experts to systematically score the research

An inclusive approach was taken and participation was widely solicited from persons with subject matter expertise (neonatal, child health, reproductive health and complex humanitarian emergencies), including both field staff and those at head offices. Initially, an email was sent to members of the Inter-agency Working Group (IAWG) on Reproductive Health in Crises sub-working group on Research, Data and Heath Information Systems and the general IAWG list serve [48]. In addition, researchers were advised to extend the invitation to others who may have appropriate technical expertise. Lastly, information about the study was presented at the IAWG 13th annual workshop where all attendees who had the appropriate expertise were encouraged to participate and they were followed up with an email invitation [49]. This activity took place between October 14 and December 10, 2011.

Participants were asked to complete the ranking by completing either a web-based survey or spreadsheet received through e-mail from research coordinator. Within the email and web link, participants were provided with the definitions regarding the space, time and disease burden of interest. The instructions asked participants to prioritize epidemiological methods for measurement and implementation research questions to achieve a reduction in the number of newborn deaths in complex humanitarian emergencies.

For each of the 35 research questions, participants were asked to review all four criteria. Each criterion had three subcomponents (see Additional file 2). Participants were asked to select “YES” or “NO” to the best of their knowledge. The written instructions in the spreadsheet or survey stated that if they felt they understood the question well and possessed knowledge to answer it, but if they felt the answer wasn’t a clear “YES” or “NO” to enter the option “YES or NO”. In the cases where participants felt they did not have enough knowledge or information to answer a question, they were instructed to select “DON’T KNOW” rather than guess.

Data analysis

Responses were scored: “YES” (1 point); “NO” (0 points); “YES or NO” (0.5 points); and “DON’T KNOW” (missing data). If the survey was not done via the web, the completed worksheets were returned to the research coordinator. Using SAS 9.2 (SAS Institute, Cary, NC). An overall mean research prioritization score (RPS) was calculated by taking the average of the four criteria [38]. For each research criterion, the average score was computed by taking the average of the subcomponent scores. Questions were ranked in order of the highest RPS.

The overall research priority score (RPS) was computed as the mean of the scores for the four criteria, according to the input from the experts, according to the formula:

RPS = mean C 1 + meanC 2 + meanC 3 + meanC 4 4

C designates the scores for relevant criteria.

As a measure of agreement among scorers, the Average Expert Agreement (AEA) [37, 38] scores were computed for each research question. The AEA was defined as the average of the highest proportion of the 12 subcomponents among scorers for each of the 28 questions asked. This gave us an indication of the most frequent response among scorers, for example an AEA measure of 0.70 would say that on average 70% agreed on the most frequent response. The AEA was computed for each scored research question as:

AEA = 1 12 × ∑ sc = 1 12 N of scorers who provided most frequent response N of all scorers

We conducted a sensitivity analysis to determine if the priorities changed when information was included from participants that completed the survey and those that were incomplete. We also reviewed all questions to determine the proportion of those who answered “yes/no” or “don’t know” for all subcomponents of an individual question. Additional information on methods is available in web appendices (A-D).

The Center for Global Health at the US Centers for Disease Control and Prevention reviewed the protocol and determined that the activity was not human subject’s research and the primary intent was public health response.

Funding

No specific funding was received for this study and the time of the experts was covered by their respective institutions.

Results

Thirty-eight experts participated and 28 experts completed the survey. Surveys were completed from October 17 until December 8, 2011. Experts were from UNHCR, UNICEF, UNFPA, Save the Children, the US government - CDC, the European Union Humanitarian Aid and Civil Protection- (ECHO), three academic institutions and nine non-governmental international organizations.

Research prioritization scores (RPS) ranged from 0.679 to 0.846; scores closer to 1.0 were considered a higher priority (see Table 2). Average expert agreement (AEA) ranged from 0.411to 0.667. Table 2 presents the priority questions ranked by RPS and the correlating AEA for those who completed the survey. The mean score for each criterion is also presented.

Table 2 Survey completers’ priority ranked questions (n = 28), research priority scores, average expert agreement and mean criterion scores
Full size table

The majority of questions had only one or two respondents that selected “don’t know” for all components of a question. There was one question where eight respondents (26%) selected “don’t know” for all subcomponents; “What is the attribute of clean delivery kit distribution in decision making process for home or facility based delivery?”

The results were similar for those who completed the survey and those that did not finish the entire survey (n = 10). The ranking, based on RPS, changed for only a few of the questions in the mean score. The overall correlation was very high. The top 5 questions remained in the 5 priority questions for both those that completed the survey and those that did not.

The top 10 research questions covered a range of issues along the research continuum. Reflecting the lack of basic data to describe the extend of the problem of neonatal morbidity and mortality in these settings, four questions dealt with measurement –namely, determination of additional burden of neonatal mortality in emergency situations, pregnancy surveillance to measure neonatal mortality, verbal autopsy to capture causes of neonatal mortality and risk factors for neonatal sepsis. The programmatic priorities included a focus on safe birth, such as increasing demand for skilled attendance, coverage of clean birth practices, and facility delivery uptake. Additionally, two of the programmatic questions focused on the specific components rather than general care–specifically, hand washing and prevention of congenital syphilis. Only one development question was ranked in the top 10–develop strategies to identify preterm babies at the community level.

Discussion

To our knowledge, this is the first systematic ranking of research priorities for neonatal survival in emergency contexts. We focused on research priorities for neonatal mortality reduction because newborns are the most vulnerable members of any population, particularly in populations affected in emergencies. The identified priorities are also of relevance for women and many also relate to older children.

Four of the top ten priority questions were identified under the “description” category, highlighting the lack of knowledge about the burden of neonatal mortality and morbidity in emergency settings. The need for defining the scope of the public health problem prior to implementing interventions is outlined in the CDC Public Health Model, which begins by describing the problem, then identifying risk and protective factors, followed by developing and testing prevention strategies, and finally disseminating prevention strategies and assurance of widespread adoption [50]. Description of the problem was prioritized here possibly because the burden of the problem needs to be better identified before opportunities for “delivery” and “development” research questions can be put forth.

The highest ranked question was, “What strategies are effective in increasing demand for and use of skilled attendance?” highlighting the importance of the time of birth and the issue of access to providers who have an appropriate level of skill. In a recent research priority ranking on reduction of stillbirths and preterm deliveries at the community level, the highest ranked question was: “Evaluate the financial barriers to facility births at the community level (user fee exemption, emergency loans, conditional cash transfers, transportation vouchers, etc.)” [32]. In both the development and humanitarian settings, the importance of increasing demand for a safe birth with trained providers is evident, but is especially challenging in emergency contexts. Two other questions in the top ten priorities relate to care at birth, but with a focus on clean birth (number 2 on clean birth kits, and number 9 on handwashing). Prioritizing hygienic care may be the first and most feasible steps towards ensuring clean and safe births in these challenging settings with limited skilled personnel and many home births.

The overall high RPS range (0.679 to 0.846) which is closer to 1.0 suggests that the majority of questions were considered high priority. It is possible that questions were reviewed and deemed high priority in part because of the initial review of the original list of questions reduced the total number to those only applicable to this setting. This range was similar to only one study thus far (range 0.56 to 0.86) [32] whereas another study showed a broader range (0.25 to 0.90) [34, 36, 42–44]. Additionally, similar to other research prioritization exercises, the AEA showed a direct positive association with the RPS, indicating that the agreement among experts was greater for the top ranking questions than for the lower ranking questions. High and low RPS scores represent high levels of agreement whereas a AEA closer to the mean value represents more disagreement [32, 34, 36, 42–44]. The one question, where eight respondents selected “don’t know” for all subcomponents, “What is the attribute of clean delivery kit distribution in decision making process for home or facility based delivery?” may have not been clear to some respondents. Review of question wording with a broader audience to ensure clarity of meaning should be conducted in the future.

The CHNRI methodology provides a systematic way to evaluate and prioritize research questions, although some limitations exist. This process, while valuable, was time consuming for participants. For each of the 35 questions, 12 subcomponents needed to be assessed. This burden may be reflected in the lack of survey completion by ten of the invited participants; however, the sensitivity analysis revealed the highest ranked priorities were consistent among those who finished the survey and those who did not. It is also possible that the respondents most interested in this topic would have completed the survey and that respondent bias may exist. This exercise was conducted in English only. In the future, multiple languages and additional perspectives from those currently operating within complex emergency settings would be valuable to include in the prioritization. Although affiliations were provided for each expert, data on location of work (field-based vs. headquarters) and educational background were not obtained. All attempts were made to engage field-based staff with a broad range of educational backgrounds and expertise; however, the success of these efforts may have been limited given the challenge of accessing field-based staff in acute emergency situations in particular. Location of the participant’s primary work environment, educational background and type of occupation should be determined in the future.

The priority research questions identified in this review are timely, given the increasing focus in global health on newborn survival and health, within the context of an integrated continuum of care from pregnancy, childbirth, the postnatal period and beyond, across all levels of service delivery. The global Every Newborn Action Plan has identified that in order to accelerate progress for newborn health different contexts require tailored approaches, with specific attention to preparedness for, and rapid response to, complex humanitarian emergencies [51]. Filling the knowledge gaps for these settings is an overdue need which will be tracked alongside specific milestones and indicators through the Every Newborn process and inter-agency collaboration.

Conclusions

Meeting the needs of pregnant women and newborns in complex humanitarian emergencies is challenging but not impossible. This ranking of priority research questions provides a clear opportunity and need to fill knowledge gaps to meet this challenge. However, the acute nature of many emergencies requires systematic capacity building before emergencies occur to ensure proven solutions can be delivered and monitored for impact in these high-risk settings. The research priorities identified here emphasize an immediate need to focus on both descriptive epidemiology as well as operations research to improve outcomes, especially around the time of birth. This exercise has already stimulated discussion and action to address gaps in research by donors, international non-governmental organizations (NGOs), national stakeholders and researchers. Additional financial and human resources dedicated to conducting research in these settings may bridge the gap and answer critical questions. Engaging partners that have expertise in neonatal health and those with expertise in emergencies to work together is critical to reducing neonatal mortality. A wide network of partners in both the development and humanitarian sphere will be needed to address these knowledge and action gaps in complex humanitarian fields and together to improve maternal and neonatal health and survival, especially amongst the women and newborns who are most at risk.

Appendix 1

The CHNRI methodology for setting priorities in health research investments

STAGE 1: Defining the context and criteria for priority setting

Research priority scores for many research investment options may change based on different contexts so specifying the context a priori is a critical part of the CHNRI process.

The research context was specified by space, time, population, and burden of disease context by Save the Children and the US CDC as follows:

  • The space: complex humanitarian emergencies, defined as acute or chronic situations of conflict, war or civil disturbance, natural disasters, food insecurity or other crises that affect large civilian populations result in significant excess mortality, and are beyond the capacity of the local government to cope;

  • Time: from present day to the year 2020;

  • The population of interest: internally displaced persons (IDP), refugees, and communities affected by complex humanitarian emergencies in low and middle income countries;

  • The burden of disease was all cause mortality and disability during the neonatal period.

STAGE 2: Choice of technical experts

The research coordinator used an inclusive approach to solicit participation from persons with subject matter expertise (neonatal, child health, reproductive health and complex humanitarian emergencies), including both field staff and those at head offices. To appropriately contact those with subject matter expertise the following steps were taken

  1. 1.

    Initially, an email was sent to members of the Inter-agency Working Group (IAWG) on Reproductive Health in Crises sub-working group on Research, Data and Heath Information Systems and the general IAWG list serve.

  2. 2.

    In addition, researchers were advised to extend the invitation to others who may have appropriate technical expertise.

  3. 3.

    Lastly, information about the study was presented at the IAWG 13th annual workshop where all attendees who had the appropriate expertise were encouraged to participate and they were followed up with an email invitation.

Every effort was made to invite a mix of people with different backgrounds (clinicians, epidemiologists, public health experts, program leaders and donors) and from different countries (both developed and developing ones) and representing headquarters and field staff so that the mix contains a diversity of views from the wider research community.

STAGE 3: Scoring of research investment options

Experts were then asked was to score all research questions independently, according to the four agreed criteria. For each of the 35 research questions and each criterion, each expert answered three questions targeted to assess the likelihood of the proposed research to comply with the priority setting criterion (see Additional file 2). This task was completed by all experts. The entire process was conducted and completed via e-mail and web survey between October 14 and December 10, 2011.

Further information on methods related to this part of the priority-setting process are presented elsewhere in greater details.

STAGE 4: Computations of “research priority scores”

All the experts answered the questions listed in Additional file 2 by “Yes” (1 point) or “No” (0 points). They were also allowed to declare an informed but undecided answer “Yes or No” (0.5 points) or declare themselves insufficiently informed to answer the question “Don’t Know” (missing input).

An overall mean research prioritization score (RPS) was calculated by taking the average of the four criteria. For each research criteria, the average score was computed by taking the average of the subcomponent scores. Questions were ranked in order with the highest RPS. They represent a direct measure of collective optimism of the scorers. Each of the 35 listed research questions received four intermediate scores (each ranging between 0–1.00).

Assessment of agreement between scorers

As a measure of agreement among scorers, the Average Expert Agreement (AEA) was defined as the average of the highest proportion of the 12 subcomponents among scorers for each of the 28 questions asked. The AEA is informing us, for an average question, what proportion of scorers gave the same most frequent answer.

“CHNRI methodology has the ability to expose the issues of greatest agreement and controversy. This allows more focused discussion among experts following this exercise, and informs the investors and policy makers about the amount of controversy that surrounds each research question. The datasets that CHNRI methodology produces are not appropriate for application of the usual Kappa agreement statistics” [37].

Advantages and limitations of the CHNRI methodology

The applied CHNRI methodology proved to be helpful to systematically list and score a very large number of specific research questions, as shown recently in exercises on research prioritization for preventing stillbirths, intrapartum-related neonatal deaths, newborn infection, and preterm and low birth rate, zinc deficiency in children, as well as mental health and disability [29, 33–45].

Additionally, the CHNRI process is systematic, transparent, that uses an a priori, well-defined context and criteria. This process engages multiple stakeholders and allows for scoring that limits influence of strong-minded individuals on the rest of the scorers and quantitatively determines research priorities and degree of agreement or disagreement between experts.

Although the CHNRI methodology offers many advantages, there are still limitations. Though initial list of questions evaluated by the experts represents a systematic determination of potential priorities, there may be additional questions not included on this list that warrant consideration. Additionally a concern that has been raised previously, is that he result of the CHNRI process could represent a biased opinion of a very limited group of involved experts [33]. The number of people globally who possess enough experience, expertise and knowledge on this neonatal health and complex humanitarian emergencies to be able to judge a very diverse spectrum of research questions is rather limited. As this is a limited pool of people, we feel reassured that a response from 28 participants falls within the range of other exercises [37, 38] and represented a diverse group of people.

Validation of CHNRI methodology

The CHNRI methodology has been used in many previous studies with success. Additional information on the validity of the methods can be found in the web appendices of “Setting Research Priorities to Reduce Almost One Million Deaths from Birth Asphyxia by 2015” [37].