Background

Healthcare providers in resource-limited countries use practical, standardized case management guidelines to diagnose pneumonia [1]. Although mortality reductions of 36 % have been attributed to standardized management, pneumonia remains the second leading cause of childhood death globally with nearly one million deaths annually [2]. Three-quarters of all deaths occur in Africa and Asia, and it is estimated that a majority of all childhood pneumonia deaths occur outside of the hospital [3, 4]. Incomplete antibiotic coverage and care-seeking delays negatively influence pneumonia outcomes in low-resource settings and improvements have been inconsistent and difficult to sustain [4].

The World Health Organization (WHO) recommends expansion of basic healthcare beyond traditional facilities to community health workers (CHWs). This strategy, called ‘integrated Community Case Management’ (iCCM), includes childhood pneumonia care [5]. iCCM can address gaps in antibiotic access and care seeking delays by providing antibiotics earlier in a pneumonia illness when the disease is more responsive to treatment [5, 6]. Likewise, iCCM can permit earlier referral if treatment fails - defined as disease persistence or progression despite antibiotics. Evidence from Zambia and Pakistan has demonstrated improved childhood pneumonia antibiotic coverage and reduced treatment failure with iCCM [7–9]. As a result, iCCM expansion is a key strategy against childhood pneumonia [5, 6].

Despite iCCM’s potential there are challenges, such as sustaining quality of care. CHWs receive limited training and are difficult to supervise in remote areas [5, 6]; consistent supervision and retraining is likely important for successful iCCM pneumonia management to ensure standardized recording of subjective clinical danger signs [10]. Strengthening CHW decision-making and identification of higher-risk patients, including those that do not meet current referral criteria, could reduce incorrect pneumonia diagnosis, antibiotic prescription, and referrals. Mobile health, or mHealth, may help to address these weaknesses [11]. mHealth utilizes user-friendly software compatible with mobile phones for healthcare delivery and capitalizes on mobile networks with wide coverage and usage in developing countries [11]. New mHealth applications that provide an array of performance support for CHWs are in development [12, 13].

An iCCM mHealth mobile phone application could assist CHWs to identify two groups of children with pneumonia; (1) those who meet current referral criteria but are mis-classified, and (2) those who do not meet current referral criteria but are still at greater risk for treatment failure based upon their initial clinical assessment. Both groups of children could then be referred for more appropriate treatment or be flagged for closer community-based monitoring. An improved referral mHealth tool could provide the result as a simple audio recommendation translated into the preferred local language, allowing both the CHW and caregiver to hear the recommendation. Facilitating objective decision-making by CHWs with a mHealth tool could complement existing guidelines to expand appropriate, timely antibiotic coverage and further reduce life-threatening treatment delays.

This systematic review and expert opinion survey aimed to identify predictors of WHO non-severe pneumonia treatment failure that are not currently used in referral algorithms and are also realistic for CHWs. The ultimate aim of a prediction algorithm would be to facilitate improved referral pathways originating from the community level, leading both to improved antibiotic stewardship and decreased morbidity and mortality.

Methods

Systematic review

We conducted a systematic review in accordance with PRIMSA (see Additional file 1) [14]. All studies from January 1, 1997 - June 25, 2014 listed in PubMed, Cochrane Central Register of Controlled Trials, and Embase were eligible. The following search terms were used: pneumonia; predictors or risk factors or prognostic; treatment failure or death; and Africa or Asia or developing country. These were combined with the following search filters: clinical trial, humans, and infant or preschool ages. No language restrictions were applied. We conducted the search on June 25, 2014. We also conducted a supplementary, non-systematic search that did not filter for geographical region. Please see Additional file 2 for these findings.

Titles and abstracts were reviewed twice and those which both reviewers agreed were relevant underwent full text review. Reference lists of reviewed manuscripts were screened for additional studies. Eligibility criteria included: 1) Study design: Prospective observational studies and clinical trials. 2) Participants: Aged 2–59 months treated with oral antibiotics for WHO-defined non-severe pneumonia per the 2013 revised WHO guidelines (Fig. 1) [15]. 3) Data collection dates: 1997 or more recent. The current WHO guidelines were formally disseminated in 1997 [1]. 4) Geographic regions: Africa and/or Asia, representing the highest burden of disease. 5) Outcome: A priori defined treatment failure, with statistical analysis for independent baseline clinical predictors of treatment failure.

Fig. 1
figure 1

2013 World Health Organization non-severe pneumonia case definition

Full size image

We used a structured data extraction tool to describe key data from the eligible studies including: region, study design, primary outcome, data collection dates, enrollment criteria, treatment failure definition, sample size, age range of participants, treatment failure rate, and independent predictors of treatment failure. The Cochrane Risk of Bias tool was used to qualitatively assess the risk of bias in individual studies [16].

Expert opinion

We complemented the systematic literature review by surveying childhood pneumonia experts. The experts were all healthcare professionals experienced in childhood pneumonia in the African or Asian region across hospital and community care settings, many with research experience [17]. We used a modified Delphi consensus process, administering two separate structured phone interviews and a final electronic survey to 13 experts, concluding when a general consensus was reached (see Additional file 3 for interview materials). Participants were asked to assess the potential role for a pneumonia treatment prognostic, including where in the health system (the home, a community health clinic, or the hospital) a prognostic would be best utilized.

The literature search provided the initial prognostic indicators to test with interviewees, using WHO definitions [15]. The panel was allowed to adjust these indicators, as well as the definitions of them and we encouraged the panel to use their own knowledge and experience. Fever was modified to an axillary temperature >38.0 °C, high blood lactate levels to >2 mmol/l, and low weight for age as a standardized z score < −2. The definition of ‘human immunodeficiency virus (HIV)-affected’ included HIV-exposed children (i.e. born to an HIV-infected mother but not HIV tested or previously tested HIV-negative with ongoing breastfeeding) or HIV-infected per standard guidelines [18].

The experts qualitatively and quantitatively estimated each predictor’s potential value. Qualitative estimates of each prognostic indicator’s value were allocated using an ordinal scale (0 = unfavorable, 1 = minimally favorable, 2 = favorable, 3 = highly favorable), along with any comments regarding each indicator’s likely performance and feasibility. Panel members provided quantitative estimates by assigning a childhood pneumonia mortality rate to community oral or parenteral hospital antibiotic treatment.

An anonymous summary of panel answers were provided to interviewees after the first interview and each expert was encouraged to revise their answers after considering the panel’s overall results. For the final survey, the expert opinion results were rephrased into binary yes/no questions to determine a final consensus. This research was approved by the Johns Hopkins Institutional Review Board (IRB00047406) and written consent was obtained from survey participants.

Results

Systematic review

Our search yielded 7,649 studies; of these, 59 manuscripts were retrieved for full text review. Nine studies met all eligibility criteria, including quality control, and were qualitatively analyzed (Fig. 2). We found no publications that included children with fast breathing and lower chest indrawing (LCI)-defined pneumonia together, probably because the WHO guidelines were only recently revised and disseminated in 2013.

Fig. 2
figure 2

Flow diagram of study identification and selection

Full size image

Table 1 summarizes the six studies reporting predictors of oral antibiotic failure in fast breathing non-severe pneumonia [19–24]. All studies were from developing countries in south Asia. Five of the six studies included amoxicillin treatment and only one included community clinics. Five studies defined treatment failure as either a persistence of fast breathing after 3–5 days or disease progression (development of LCI or danger signs) between 1–5 days after antibiotic initiation. One study included peripheral oxygen saturation (SpO2) less than 90 % in the definition of treatment failure. The proportion of children that failed oral antibiotic treatment ranged from 7.8 % to 22.9 %. Three studies identified baseline excess age-specific respiratory rate, and two studies found history of difficulty breathing, age, and illness duration of ≥3 days as independent predictors of oral antibiotic failure.

Table 1 Independent predictors of oral antibiotic failure in fast breathing WHO non-severe childhood pneumoniaa
Full size table

Table 2 summarizes the three studies reporting independent predictors of oral antibiotic failure in LCI non-severe pneumonia [25–27]. Two studies were multi-country with sites in Africa. In two of the three studies, treatment failure was assigned on day six after antibiotic initiation. Treatment failure rates for the three studies ranged from 8.1 % to 19.3 %. All studies found young age and very fast breathing at baseline to be independent predictors of treatment failure. One study collected SpO2 measurements and found low baseline SpO2 to predictive; another study found malnutrition to be predictive of treatment failure.

Table 2 Independent predictors of oral antibiotic failure in chest indrawing WHO non-severe childhood pneumonia
Full size table

Expert opinion

Of the 13 experts interviewed, eight reported experience in Africa, four in Asia, and one in both. All interviewees agreed that the development of a pneumonia treatment prognostic could significantly advance childhood pneumonia care and that the community-level was the most attractive setting for implementation.

The panel qualitatively evaluated seven prognostic indicators for performance and feasibility at the community level (Table 3). Abnormal SpO2 and malnutrition were ranked highest while fever and high blood lactate levels were lowest. Abnormal SpO2 was judged to be objective, although the panel identified the need for a robust, precise, low-cost pulse oximeter for young children. Malnutrition was also valued by the experts as it is likely to have a high positive predictive value and the feasibility of reliable malnutrition screening was attractive, with tools such as tapes and scales being simple, accurate, and relatively inexpensive.

Table 3 Expert panel qualitative assessments of potential prognostic indicators
Full size table

The panel’s quantitative performance estimates for each prognostic indicator are presented in Table 4. Regardless of setting, children that are malnourished, HIV-affected or have an abnormal SpO2, were estimated to be at the highest mortality risk by the panel. Estimated median mortality rates of community treatment with amoxicillin for children with abnormal SpO2, malnutrition, or an HIV-affected status ranged from 15.5–20 %.

Table 4 Expert panel quantitative assessments of potential prognostic indicators
Full size table

Discussion

WHO guidelines standardize pneumonia management in low-resource settings and stratify pneumonia severity according to the presence of severe clinical features [1, 15]. Currently, these guidelines do not further categorize non-severe pneumonia patients according to their treatment failure risk even though some children with non-severe disease have failure rates as high as 23 % [20]. Given the increased decentralization of pneumonia care to CHWs, improving baseline risk stratification with objective clinical measures may be an important next step for reducing pneumonia morbidity. To assist in algorithm development, we performed a systematic review and surveyed experts to identify predictors of oral antibiotic failure in children that fit the WHO case definition for non-severe pneumonia (2013 guidelines) [15]. We placed the greatest emphasis on predictors not currently incorporated into referral algorithms and those that were also measureable by CHWs. From the literature and expert review, we found some agreement for excess age-specific respiratory rate, abnormal baseline SpO2 and moderate malnutrition as important predictors of treatment failure. However, the literature was limited and highlighted several key evidence gaps, and there were notable differences between those predictors supported by the literature and those favored by the experts.

Our systematic review and expert survey yielded several interesting comparisons. The panel strongly favored abnormal SpO2 as a predictor, however, SpO2 was evaluated in only two studies [21, 25], none of which examined SpO2 at community clinics or assessed less severe, but still abnormal SpO2 ranges (90–94 %). Similarly, malnutrition was highlighted by the panel but was only found to be predictive for oral antibiotic failure in one study conducted at tertiary hospital clinics [26]. Milder degrees of malnutrition were not evaluated, and the experts defined malnutrition in the moderate range. Although young age was found to be predictive in multiple studies [20, 25–27], the experts did not recommend young age as a candidate predictor. Haemophilus influenzae type B (HiB) vaccine or pneumococcal conjugate vaccine (PCV) status were not examined as predictors in our systematic review since most studies were performed before introduction of these vaccines, but it was also not recommended by the panel. These differences between the published literature and expert panel may in part reflect the panel’s emphasis on personal experience to guide their answers and the overall lack of publications meeting eligibility criteria. In addition, a majority of the panel worked in Africa while in contrast the published literature was heavily represented by Asia.

The systematic review suggested that fast breathing and LCI pneumonia may be distinct groups with different drivers and predictors of failure. Our review of the literature consistently found excess age-specific respiratory rate as an important predictor in both LCI- and fast breathing non-severe pneumonia [20–22, 25–27], but abnormal baseline SpO2 and young age were clearer predictors of treatment failure in children solely with LCI [25–27]. In those studies looking at fast-breathing defined pneumonia, the independent predictors were less consistent and included persistent vomiting, diarrhea, wheeze, human respiratory syncytial virus infection and history of difficult breathing; these may suggest a primarily viral cause of illness. Other studies support the notion that a low proportion of fast breathing pneumonia cases may be due to bacterial pneumonia. Hazir and colleagues reported that only 1.7 % of Pakistani children with fast breathing pneumonia had a radiographic consolidation consistent with a bacterial etiology [28], and the same group also published findings of equivalency when fast breathing cases were treated with either placebo or amoxicillin [23]. At this time listening for wheeze with a stethoscope or testing for respiratory syncytial virus are not feasible for routine CHW care. The results from this systematic review should be interpreted within this context; LCI and fast breathing pneumonia may differ etiologically and the published literature on this subject has primarily focused on fast breathing pneumonia (only three studies meeting eligibility criteria for this work included LCI pneumonia).

Our systematic review found an overall paucity of community-level data, potentially limited by publication bias. However, we also found other important evidence gaps. Firstly, it is likely that predictors will differ between African and Asian populations, in part because HIV is endemic in southern Africa and a key driver of treatment unresponsive pneumonia [29]. However, our systematic review demonstrated that African data is limited, with only two studies including data from African countries. These studies did not explore region as a predictor of treatment failure, and did not report adjusting for region in their multivariate models. The experts also did not address this evidence gap since they did not explicitly factor regional differences into their responses, although the majority of experts worked in Africa and this is likely to have influenced their answers. The development of a sound prognostic algorithm for the southern African region that accounts for HIV status may be challenging since HIV stigma persists, particularly in more rural, intimate community settings where anonymous testing is difficult to achieve [30]. Second, it is likely that pneumonia epidemiology, and thus treatment failure predictors, will shift after HiB vaccine and PCV introduction. However, data from developing countries after full vaccine introduction is only more recently becoming available [31]. Due to these limitations in the literature, the findings from the literature review need to be interpreted cautiously.

Although validated prognostic models that identify higher risk pneumonia patients exist, only two studies (both secondary analyses from the Amoxicillin Penicillin Pneumonia International Study (APPIS) [25]) present predictive models for children who fulfill the LCI non-severe pneumonia case definition [32, 33]. Mamtani et al. validated a clinical tool for predicting oral amoxicillin failure in children with LCI pneumonia and no danger signs, using the child’s age and excess age-specific respiratory rate after 24 h of treatment to stratify patients’ risk [32]. Fu et al. predicted the risk of oral amoxicillin treatment failure after 48 h, with the best performing model including a second clinical evaluation after 12 h of treatment and SpO2 [33]. However both studies likely reflect different pneumonia epidemiology due to increasing HiB and PCV availability and current treatment failure definitions have been simplified [26, 27]. The feasibility of using models that rely on a single patient observation for an extended period or multiple clinical observations will need careful assessment at the CHW level.

At this time more evidence is needed to demonstrate effective use of pulse oximetry by CHWs, whether milder degrees of altitude-adjusted hypoxemia (90–94 %) are predictive of treatment failure, and if pulse oximetry can be cost-effective at the community level in low-resource countries where equipment maintenance can be challenging. Several low-cost respiratory rate counters, pulse oximeters and combination devices have recently been developed or are in development for low-resource settings [34–36]; there is an on-going evaluation of the clinical performance, usability, and acceptability of these devices currently underway by Malaria Consortium (D. Hay Burgess, personal communication).

Conclusion

In conclusion, we found that young age, excess age-specific respiratory rate, abnormal baseline SpO2 and moderate malnutrition should be carefully considered for an automated mHealth iCCM tool for predicting oral antibiotic failure in children with pneumonia. However, the evidence is currently limited. Additional formative research to identify antibiotic failure predictors is needed at the community-level, in HIV and non-HIV endemic regions, with and without malaria, and that have introduced the HiB vaccine and PCV. Modeling patient and health system-level consequences of a prognostic, along with a cost analysis of device scale-up, are important next steps.