Background

Accurate information on the causes of neonatal and child mortality is needed to help prioritize health expenditures and shape effective health policies and programs in developing countries. It is a long-held assumption in the international public health literature that the causes of child deaths in health facilities, mainly hospital, differ from those that occur in the community [1, 2]. These differences are likely to be even more exaggerated at secondary and especially tertiary care hospitals that provide specialized services including neonatal intensive care to which severely ill patients are referred from lower level facilities. At the same time, evidence shows that half or more of child deaths in most developing countries occur at home [2,3,4], with many of these children having received no formal health care during the course of their fatal illness [4,5,6,7,8]. Civil registration and vital statistics (CRVS) systems with medical certification of the causes of death in such settings are often weak [9, 10]. Thus, arises the need for verbal autopsy (VA), the widely used and generally accepted best available method for estimating cause of death at the population level in settings with inadequate medical certification, until such time as functioning CRVS systems are put in place.

The VA method involves a structured interview of the child’s main caregiver to identify the signs and symptoms (hereafter referred to as “illness signs” or “signs”) of the fatal illness, such as fever, rash, difficulty breathing, and loose or liquid stools, from which the cause of death is estimated. There are several methods available to assess cause of death from VA data, the most widely used including physician coding (PCVA) [11], expert algorithms (EAVA) [12], Tariff [13], InterVA [14], and InSilicoVA [15]. The validation of verbal autopsy methods has been by comparison to causes of hospital-based deaths, leaving the question of how well verbal autopsy can ascertain causes of death in the community.

Even if the causes of death in hospital and the community were identical, some concerns would remain. Limited access to health care, especially at the secondary and tertiary levels, is a serious problem in many developing countries, and persons of higher socioeconomic and educational status may be overly represented among those who receive formal health care. These factors, and the hospital experience itself, might affect recognition, recall, and reporting of illness signs. In addition, because hospitalization might alter illness course and so the illness signs present for observation, respondents for community and hospital deaths might witness and report somewhat different signs in association with the same cause of death.

Therein lies the VA paradox, that its validation must rely on hospital deaths for the very reason that VA is needed, because cause of death in the community cannot be established with sufficient accuracy to serve as a standard for testing VA methods. Yet the reliance on hospital data for its validation calls into question VA’s accuracy in assessing cause of death in the community.

A well-selected sample of health facilities, representative of the deaths occurring at all health facilities in a given geographic area, with proper medical diagnosis and certification, could represent the causes of death not only in facilities but also in the community at-large [16]. Moreover, in the case that facility deaths differed from community deaths, they could potentially be modeled using periodically collected census or survey data to predict the causes in community and thus at the population level [17, 18].

But how to test this methodology, when the medical causes of community deaths cannot be determined with sufficient accuracy to even support the validation of verbal autopsy? Again, we must turn to VA for a feasible approach to the problem. If the same VA methodology was used to determine the causes of death both in health facilities and the community, then the causes in these two spheres should be reasonably comparable, short of the potential effect of demographic factors that could be adjusted for.

From 2012 to 2016, we conducted verbal/social autopsy (VASA) studies of neonatal and child deaths in four African countries. We undertook the current analysis to determine how similar or dissimilar were the causes of death that occurred in health facilities and the community; to assess the implications of these findings on the use of VA for the identification of neonatal and child causes of death at the population level; and if causes of community and facility deaths were dissimilar, to determine whether community causes could be estimated from the facility causes with adjustment using a small set of demographic factors.

Methods

We conducted VASA studies of representative samples of neonatal (0–27 days old) and child (1–59 months old) deaths in Cameroon, Malawi, Niger, and Nigeria. Several publications describe the full verbal autopsy methods and findings of two of these studies [19, 20]. In brief, deaths in each country were identified by a full birth history of all women participating in a household survey of a representative population sample: in eastern Cameroon, the 2010 baseline census of 16,954 households for the Population Services International Community Case Management study in Doume, Nguelemendouka, and Abong-Mbang districts [21]; in central and southern Malawi, the 2011–2012 midline survey of 24,000 households for the Real-time Mortality Monitoring project in Balaka and Salima districts [22]; in Niger, the 2011 National Mortality Survey of 25,024 households [23]; and in Nigeria, the 2013 national Demographic and Health Survey of 38,522 households [24]. The VASA studies sampled one under-5 death from each surveyed household with one or more such deaths, and conducted a VASA interview with each child’s main caregiver using the Population Health Metrics Research Consortium VA-Johns Hopkins University/Institute for International Programs SA questionnaire [25]. The final sample sizes and recall periods from death to interview, respectively, of neonatal and 1–59-month-old deaths with completed VASA interviews were 164 (mean 3.6 years, range 2–6 years) and 635 (3.0, 2–6) in Cameroon, 320 (2.3, 0–4) and 691 (2.5, 0–4) in Malawi, 453 (3.5, 2–5) and 619 (2.7, 2–5) in Niger, and 722 (3.6, 1–6) and 2055 (3.8, 0–6) in Nigeria.

The EAVA and PCVA analysis methods were utilized in each country to identify the proportions of neonatal and 1–59-month deaths from, respectively, 10 and 12 causes. The PCVA analysis was conducted by one well-trained local physician (FN, GK, A-MR, and WAA, respectively, in Cameroon, Malawi, Niger, and Nigeria) employing pre-defined required minimal diagnostic criteria combined with their clinical judgment. The physician completed a death certificate for each case, and the underlying cause appearing on the lowest line of section 1 of the certificate was selected for analysis. The EAVA method utilizes pre-defined expert algorithms based on illness signs and symptoms reported during VA interviews, arranged in a hierarchy that follows ICD-10 rules to the extent possible to select the underlying cause of death [12]. To estimate cause distributions in Malawi, Niger, and Nigeria, survey weights were applied and primary sampling units handled such that all analyses accounted for the multi-stage sampling designs of the platform surveys. Survey weights were not required in Cameroon, where the deaths were identified through a population census of the three study districts.

Statistical tests

We conducted three analyses. First, for each country, we compared the proportions of deaths from each neonatal and 1–59-month EAVA and PCVA cause of death that occurred in the community and in health facilities. The VASA questionnaire categorized the place of death as “hospital,” “other health provider or facility,” “on route to a health provider or facility,” “home,” or “other.” Health facility deaths included deaths from “hospital” or “other health provider or facility” and were further categorized as “health center,” “health post,” or “private doctor/clinic.” Deaths that occurred at home, on route to a health provider of facility, or other place were grouped as community deaths. “Other health providers” were separately determined to be trained community health workers, nurses, or midwives, and were included in the “other health provider or facility” group only if they were seen at a health facility. A Rao-Scott chi-square or Fisher exact test was used to assess the significance of any differences between the community and health facility cause-specific proportions [26].

Next, we estimated the CSMF accuracy [27] of the overall community/facility cause distributions of neonatal and 1–59-month deaths in each country accounting for the survey design. CSMF accuracy was originally designed to compare a non-reference to a reference standard CSMF, where 0 represents extreme differences and 1 represents no difference [27]. Here, we used this statistic to assess the level of correspondence between two non-reference CSMFs. We approximated 95% confidence intervals for this statistic by bootstrapping based on resampling primary survey sampling units [28].

In addition, we compared the observed population (facility plus community) distribution of the EAVA and PCVA causes of death for neonates and children to their corresponding predicted distributions, using several different methods to predict for causes of community death in a hypothetical scenario where community causes had not been measured and only the causes of facility deaths were available. First, in a “naïve” prediction (projection A), we used the observed cause distribution of facility deaths to represent population distributions, effectively assuming that causes in communities were the same as in facilities. We compared the resulting prediction of population-level cause distributions to the observed population distribution using CSMF accuracy. We also modeled neonatal and 1–59-month causes of death using multinomial logistic regression (MLR) [29] and random forests (RF) [30] to adjust for the potential influence of several factors on the community and health facility causes. We selected adjustment factors based on their availability in common population surveys such as the Demographic and Health Survey (DHS) conducted in many developing countries, to facilitate possible future use of the prediction models by public health officials.

We adjusted for age at death and mother’s education, as well as whether the mother received any antenatal care (for neonates). We estimated these models among deaths occurring in facilities and used the results to predict the causes of death for those occurring in the community, which proportionally combined with the facility causes of death provided a population-level prediction (MLR projection B). Additional factors were considered but not included in this MLR projection due to model instability. We used random forests to allow additional factors to influence projected causes. We built the random forest classification with deaths occurring in facilities based on age at death, mother’s education, child sex, birthplace, and wealth quintile, as well as, for neonates, whether the mother received antenatal care (RF projection C). We predicted the causes of death in the community with this model and combined these with the facility deaths to estimate the population-based cause of death distribution, which we compared to the observed population-based distribution using CSMF accuracy. Lastly, we conducted the random forest analysis utilizing only the predictors included in the multinomial regressions (projection D) and using the same predictors included in the multinomial regressions, plus birthplace (projection E).

We also examined CSMF accuracy by the percent of deaths occurring in facility. Within each study, we assumed a fixed cause distribution among the community and facility deaths, and then varied their weights to determine the population-based CSMF. All analyses were conducted in R version 3.5.0.

Results

Comparison of specific causes

Table 1 shows the EAVA and PCVA population cause-specific proportions and 95% confidence intervals of the neonatal and 1–59-month-old deaths in the four countries. Across the four study countries, the percent of neonatal and 1–59-month deaths that occurred in facilities ranged, respectively, from 18.8 to 55.0% and 18.6 to 50.5%. Only in Malawi did facility deaths exceed 50%.

Table 1 Cause-specific proportions of neonatal (0–27 days) and child (1–59 months) deaths in the study countries
Full size table

Figures 1a, b and 2a, b show the EAVA and PCVA neonatal and 1–59-month community and health facility cause distributions for the four countries, and Tables 2 and 3 show the causes of death for which there were significant differences between the community and health facility proportions.

Fig. 1
figure 1

Verbal autopsy causes of neonatal deaths in communities and health facilities, Cameroon, Malawi, Niger, and Nigeria. a Expert algorithm neonatal causes of death. b Physician-coded neonatal causes of death

Full size image
Fig. 2
figure 2

Verbal autopsy causes of child deaths in communities and health facilities, Cameroon, Malawi, Niger, and Nigeria. a Expert algorithm 1–59-month causes of death. b Physician-coded 1–59-month causes of death

Full size image
Table 2 Significantly different community and health facility causes of neonatal deaths in the study countries
Full size table
Table 3 Significantly different community and health facility causes of 1–59-month deaths in the study countries
Full size table

For neonatal deaths, there were significant differences between the community and health facility proportions of birth injury/asphyxia by the EAVA or PCVA analysis method in three countries, of sepsis by one or both VA methods in two countries, and for preterm delivery, pneumonia, and diarrhea, each by one VA method in one country. Examining the findings by country, Malawi had the most causes with differences between the community and health facility proportions, with two of the four such causes being “other” and “unspecified.” Niger and Nigeria each had significant differences between the community and health facility proportions for three neonatal causes (one of these, in Nigeria, being “other”), and Cameroon had such for just one cause.

AIDS was the only 1–59-month cause of death for which there was a significant difference between the community and health facility proportions in more than one country (Nigeria and Malawi), and Malawi was the only country for which this was true for more than one cause. For all the neonatal and 1–59-month causes not shown in Tables 2 and 3, the community and health facility proportions were similar, with p values above 0.05 (Additional files 1a and b).

Comparison of cause distributions

Table 4 displays the CSMF accuracy for the comparison of the overall EAVA and PCVA distributions of neonatal and 1–59-month causes of death that occurred in the community and in health facilities in each country. CSMF accuracy was generally high in these settings, indicating similarity between causes of community and health facility deaths. The largest differences between community and facility deaths were among neonates in Niger (0.74 for EAVA) and Malawi (0.76 for EAVA). The most similarity was among children in Nigeria (0.93 for EAVA and 0.95 for PCVA). CSMF accuracy also tended to be higher for deaths among children (average 0.90, range 0.85–0.95) than among neonates (average 0.80, range 0.74–0.87), indicating more similar cause distributions for children versus neonates.

Table 4 CSMF accuracy comparing the cause distributions of community and health facility deaths in the study countries
Full size table

CSMF accuracy also varied across countries. For neonates, accuracy was highest for EAVA in Nigeria (0.87) and for PCVA in Malawi (0.87), and for both EAVA and PCVA was lowest in Niger (0.74 and 0.79), while for children, CSMF accuracy both by EAVA and PCVA was highest in Nigeria (respectively, 0.93 and 0.95) and lowest in Malawi (0.87 and 0.85).

Comparison of populations

Table 5 compares the distributions of demographic factors among the community and facility deaths. There were many significant differences in these factors, despite the similarities in their cause of death distributions. Among neonates, facility deaths were younger and more likely to have been born in a facility. Mothers of facility deaths were also wealthier and more educated than mothers of community deaths in all countries but Cameroon. Use of any antenatal care during the index child’s pregnancy, as an independent measure of access to health care, was greater for facility deaths in all countries but Niger.

Table 5 Demographics among community and health facility deaths in the study countries
Full size table

The balance in most demographic factors between community and facility deaths was similar for 1–59-month-olds to that for neonates. Child facility deaths in all four countries were more likely than were community deaths to have been born in a health facility. Likewise, mothers of child facility deaths in Malawi and Nigeria, but not in Cameroon and Niger, were less likely to have had no formal education, and in all countries but Niger, facility deaths were better off economically. However, in contrast to the age distributions for neonates in all four countries, in Malawi, child facility deaths were older than community deaths.

Predicting population causes with facility deaths

Table 6 shows the CSMF accuracies for projections A, B, and E of the population distributions of neonatal and child causes of death compared to the observed population distributions shown in Table 1; projections C and D are included as well in Additional file 2. The simplest method to implement, substituting the observed causes of facility deaths for those in communities (projection A), had a CSMF accuracy for EAVA causes in neonates of 0.86 on average across the four countries (range 0.79–0.91), and for children an average of 0.93 (range 0.90–0.95). Average CSMF accuracies for neonatal (0.88) and child (0.93) PCVA causes were similar to those for EAVA causes, but with some variations across countries, the largest among neonates being for Malawi (EAVA 0.89 vs. PCVA 0.94) and Nigeria (EAVA 0.91 vs. PCVA 0.86) and among children for Niger (EAVA 0.90 vs. PCVA 0.94).

Table 6 Comparing population cause distributions estimated from facility deaths to observed population cause distributions in the study countries
Full size table

In general, adjusting for demographic factors did not yield substantial improvement over the simple unadjusted projection A. Average CSMF accuracy for the multinomial logistic regression (projection B) for EAVA causes was identical to that for projection A in neonates (0.86) and children (0.93), and for PCVA causes was identical in children (0.93) and nearly so in neonates (0.89 vs. 0.88). CSMF accuracies for the two projections within each country were also very similar. Average and within-country CSMF accuracies for RF projection E of EAVA and PCVA causes in children and PCVA causes in neonates also were very similar to those for projection A. Only the projection E average and within-country CSMF accuracies for EAVA causes in neonates showed consistent increases over those for projection A. CSMF accuracies for RF projection C (shown in Additional file 2) were similar to those for RF projection E.

Figure 3 depicts the hypothetical relationship between the percent of deaths occurring in facilities and the CSMF accuracy of facility causes for the population cause distribution, for neonatal and 1–59-month-old deaths in each of the study countries. It can be seen that CSMF accuracy increases as the proportion of all deaths that occur in facilities increases.

Fig. 3
figure 3

Relationship between percent of deaths in health facilities and predicted EAVA population cause-specific mortality fraction accuracy. Dots indicate the measured CSMF accuracy at the level of health facility deaths in each country

Full size image

Discussion

Verbal autopsy is the main source of data on causes of neonatal and child death in countries with weak civil registration and vital statistics systems, including over 80 countries with the highest burden of under-five mortality [31]. The critical nature of the accuracy and representativeness of VA data, whether for local or high level modeling purposes, is apparent. Our analysis found few differences in the fraction of specific causes between health facility and community, with variations from one to four of 10 examined neonatal causes and from one to three of 12 child causes by either VA analysis method across the four VASA studies. While there were large within-country differences between the EAVA and PCVA population proportions for some causes, this was not unexpected, as it is common for different VA analysis methods to yield different cause-specific proportions. What matters is that the within-country, within-method facility and community cause proportions were similar to each other. Since for each method, all the deaths were analyzed in the same way, this should not have been a factor in the comparison of facility and community causes. We also found high levels of CSMF accuracy between health facility and community deaths, generally exceeding those of the InSilicoVA method comparing estimated versus high-quality causes of death across (0.70) and within (0.85) study sites [15], as well as the Tariff 2.0 VA method comparing estimated versus high-quality causes of death for neonates (0.83) and 1–59-month-olds (0.78) [13].

This means that a health program manager utilizing neonatal and child medically determined causes of death in health facilities to estimate the population neonatal and child CSMFs could expect to achieve reasonable accuracy, given the following caveats: first, that the sample of facilities is reasonably representative of those in the population. Prior studies that found or presumed that facility deaths would find non-representative causes have examined or referenced hospital deaths, without including lower level facilities [1, 2]. The higher the facility level, especially secondary and tertiary hospitals with greater selectivity of patients, the less might one expect the causes of death to be representative of all deaths. Surveillance at 16 tertiary care hospitals throughout India found that infection was the third leading cause of neonatal death, after prematurity and birth asphyxia [32], whereas at a sub-district hospital in Haryana during the same time frame, there were as many neonatal deaths due to septicemia as from prematurity and birth asphyxia combined [33], similar to the mortality pattern found in a community study in rural Maharashtra [34]. The deaths in our study were identified by representative household surveys and so included deaths from wherever they occurred, be that in the community or a health facility of any level. We were not able to identify other studies that have compared head-to-head the causes of under-5 mortality in representative samples of community and facility deaths. It would be worthwhile to conduct analyses like those reported here in additional countries and regions to further examine this issue. There are also several published studies of child mortality from sample registration systems that could be re-explored to determine if they recorded place of death.

Second, the EAVA and PCVA methods utilized by our study assessed only the most prevalent causes of neonatal and 1–59-month-old deaths in low-income countries. Further work would be needed to determine if there is a similar correlation of hospital vs. community deaths for less frequent causes.

We attempted to improve on crude CSMF accuracy by adjusting for factors expected to influence health care access and utilization and therefore place of death, although how factors would be related to cause and place of death together is less intuitive. None of the adjusted projections of child causes of death increased CSMF accuracy substantially over the unadjusted projection.

Prior work suggests there are few causes of death for which health care is expected to be utilized more or less than for others. Illnesses characterized by convulsions, such as tetanus, meningitis and, in Tanzania, cerebral malaria [35], may be interpreted as having a spiritual cause and so not amenable to health care. However, this belief can vary, so is not easily predictable; in Mandiana, Guinea, mothers were more likely to seek health care for their children with convulsions [36]. Among the four VASA studies, only in Malawi were meningitis and malaria deaths proportionally higher in facilities than the community (Table 3). Meningitis proportions were much greater in Niger and Cameroon, and malaria somewhat higher, but both were equally distributed between facilities and community (Additional file 1b). Various explanations might be posited, such as past experience with the health system and setting-specific responses to particular symptoms. Access to child health care is an obvious possibility, but the DHS does not include a good measure of access for severe illness.

AIDS deaths in Malawi were more frequent in the community than facilities. Several HIV/AIDS program factors might explain this discrepancy. During the period examined by the VASA study, antiretroviral therapy (ART) of HIV in Malawi was based on WHO guidelines requiring a clinical stage 3 or 4 condition or a low CD4 count [37]; few HIV-positive women were receiving ART, leading to high levels of mother-to-child transmission; and there were program implementation challenges including long delays before children’s PCR results confirming HIV infection became available, a lack of pediatric ARV formulations, and unsynchronized mother-child clinic visits. In this environment, many HIV-infected patients were developing incurable complications that were managed in the community with palliative care. Careseeking may also have been affected by stigma and discrimination against persons with HIV/AIDS, issues being tackled with strengthened interventions since the years of the VASA study [38]. In Nigeria, AIDS deaths were higher in facilities, but the proportions were extremely low. Severe malnutrition deaths in Malawi also were higher in the community than in facilities. Ministry of Health guidelines emphasize community management of acute malnutrition [39], and mothers of children with complications may resist multiple health facility admissions due to the economic impact.

One might hypothesize that deaths from perinatal conditions that kill quickly, such as birth asphyxia and prematurity, are more likely to occur in health facilities in settings where many births occur in facilities and/or there is good health careseeking for maternal delivery complications. And, indeed, projections including birthplace outperformed those that did not, suggesting that birthplace may be related to place of neonatal death and cause of death. This was evident in our VASA data, with significantly more neonatal facility decedents born in facilities than in the community in all four countries and more facility than community deaths from birth asphyxia and prematurity in, respectively, three and one countries. In contrast, sepsis caused more community than facility deaths in two countries. These findings paralleled those of the India National Neonatal Perinatal Database, which found that prematurity and birth asphyxia predominated as causes of neonatal death in cases born in a tertiary hospital, whereas septicemia was the most common cause in neonates admitted from nursing homes and small hospitals [40].

While we were not able to identify any studies that compared facility and community causes of death directly, there is prior research relating to the prediction of community or population causes from facility deaths. Whiting et al. applied CSMFs estimated from facility deaths to counts of nearby sentinel vital registration (SVR) deaths, similar to our unadjusted projection [16], but tested their method comparing facility causes with medical certification to all VA-assessed SVR deaths. Other methods have required external sources of cause-specific mortality in communities. Murray et al. estimated the proportion of cause-specific mortality occurring in facilities from deaths reported in a national CRVS system to predict the population CSMF, specific to age and sex [17], a method extended by Williams et al., who used logistic regression to estimate the probability by cause of dying in a health facility based on a broad range of covariates including age, sex, and extensive demographic information, and reweighted the CSMFs with predicted probabilities to estimate the population cause distribution [18]. We have extended this work to examine how representative facility data could be used in a practical situation without relying on cause of death information from other sources. Figure 3 shows to what extent our unadjusted method might be influenced by varying levels of facility death, suggesting its superior performance over community VA assessment of child deaths in all settings and of neonatal deaths in settings with 40% or more facility deaths. We have also examined the utility of more sophisticated cause projections based on demographic factors, which we found only marginally improved on the simpler estimate.

The implication of our findings is that just as we have done with VA causes, the medical causes of neonatal and child deaths from a representative sample of health facilities can be used to estimate the population distribution of the causes. Further research on how to select a representative sample of health facilities, improve medical certification, ICD coding, and reporting of causes of death is needed to support this effort.

Limitations

We did not have data on medical causes of death from health facilities and communities to test our method on the type of data that we anticipate health program managers using the method would have access to. However, it would be impractical to have medical data on the causes of community deaths to test the method, for the very reason that VA is the main method used to estimate causes of community deaths in developing countries. This may become more practical in the near future, with the advent of new diagnostic technologies such as minimally invasive tissue sampling (MITS) being conducted on community deaths, although these are unlikely to ever be nationally representative.

The VASA studies were context-specific in that causes of death and their distributions across health facilities, communities, countries, and regions may vary with time. This concern is lessened by our having examined deaths that occurred over several years both in West and East Africa, but still remains.

Conclusions

The verbal autopsy-based causes of facility and community deaths of neonates and 1–59-month-olds were found to be similar to each other in the four African countries. These findings suggest that population causes of neonatal and child deaths in developing country settings can be projected from a representative sample of facilities where children die. This method should be tested with more recent deaths in additional countries and regions. If good agreement is found between facility and community deaths generally or in some settings, research should be conducted in selecting a representative sample of health facilities and strengthening medical certification so that VA will no longer be necessary.