Background

Previous studies have shown that medical interventions based on scientific evidence range from 10 to 80% [1–4]. These studies were performed in a broad spectrum of patients and settings in developed countries.

In paediatric practice, the proportion of evidence based interventions reported ranges from 75 to 91% [5–8]. These studies are clearly relevant to the quality of care in developed countries, with strong health systems, widely available high technology and qualified human resources. However, health services in developing countries are frequently weak, and they face too often severe lack of expensive technology. In addition, the level of qualification of health personnel may vary greatly within health facilities, even at referral level and in urban areas. Moreover, the prevalent childhood illnesses in developed countries are not necessarily those prevalent in developing countries.

The assessment of quality of care at referral level for severely ill children is an important component of the efforts for reducing the child mortality rate in poor countries, for reducing the burden on health systems and for investing money in favour of high priority health interventions [9, 10]. Thus we were prompted to assess the proportion of interventions based on sound scientific base in a paediatric referral setting of a developing country.

Methods

Referral care provided to children hospitalized in the paediatric department of the Instituto Especializado de Salud del Niño (IESN) was assessed for evidence base. IESN is a national paediatric hospital with more than 500 inpatient beds, serving mostly patients from deprived socioeconomic areas of Lima and inner cities of the country.

The clinical records of 195 children aged 1 month through 16 years old and hospitalized during January 2003 were initially revised.

One of the investigators (NYC) assessed the clinical records of children and assigned to each one a primary diagnosis and one or more primary therapeutic interventions, on the basis of the main clinical features and/or definitive diagnostic laboratory aids. Patients in whom a primary diagnosis was not possible to determine or those without a clear diagnosis were excluded.

The primary intervention was defined as the treatment or other manoeuvre that represented the most important attempt to cure, alleviate, or care for the patient in respect of his or her primary diagnosis [3].

To determine the level of evidence for each primary intervention, Cochrane reviews were searched. If there was not such a systematic review, a search through PubMed (MEDLINE) was performed by one of the investigators (NYC). All Cochrane reviews were searched through their own search tools. For PubMed, the period of search was 1966 through 2002. The key words used included those related to the primary diagnosis (e.g., pneumonia). Limits: "Title", "All child: 0–18 years". Publication type was sequentially searched for "Practice Guideline", "Meta-Analysis", "Randomized Controlled Trial" and "Review". Articles in English or Spanish were included. The National Guideline Clearinghouse was additionally visited for additional references. Published recommendations for judging the quality of guidelines were used for deciding the selection of the guidelines [11]. In addition, Clinical Evidence was used whenever deemed pertinent.

The level of evidence assigned to interventions was based on Ellis score (levels I, II and III) and the Oxford Centre for Evidence Based Medicine Levels of Evidence (grades A, B, C and D) [3, 12]. Ellis score considers one or more interventions for a given diagnosis as one primary intervention. For comparison purposes we ranked each individual intervention through Oxford classification.

As a next step to our study, we planned the dissemination of the results among the hospital policy makers and the suggestion of corrective courses of action for those interventions needing improvement.

Results

Results of the search strategies for PubMed are included as an appendix [See Additional File 1]. A guideline on management of pain in sickle cell disease was found in the National Guideline Clearinghouse website and the results of the search are also shown at the end of the appendix [See Additional File 1].

Overall, one hundred and ninety five clinical records were revised. Twenty eight clinical records were excluded because of undefined primary diagnosis (Table 1). One hundred and sixty seven remaining clinical records were further assessed. The most frequent primary diagnoses are shown in Table 2, being diarrhoeal dehydration and pneumonia the main causes for hospitalization. The childhood prevalent diseases are quite constant throughout the year at our hospital and thus it is unlikely that the results would have been different if we had chosen another study period.

Table 1 Patients excluded because of undefined diagnoses
Full size table
Table 2 Definitive diagnoses
Full size table

Out of 167 primary interventions, 21% were supported by level I evidence, 73% were level II, and 6% were ranked as level III, according to Ellis classification [3]. Table 3 shows that most interventions classified as level I are referred to acute asthma exacerbations. Nebulized beta-agonists and systemic corticosteroids in bronchiolitis, and antibiotics for acute otitis media were considered level I according to Ellis. They were classified as D{5} according to Oxford Centre for Evidence Based Medicine Levels of Evidence (Table 3) [12].

Table 3 Level I primary interventions (Ellis score) ranked by Oxford classification
Full size table

Most assessed interventions were considered as level II (Table 4). Diarrhoeal dehydration and community-acquired pneumonia were the predominant diagnoses. Considering each prescription separately (fluid restriction, furosemide, spironolactone and captopril in heart failure, for example) we obtained 146 interventions. When we assessed them through the Oxford classification, 11% interventions were classified as grade B, 1% as grade C, and 88% as grade D. Appropriate interventions for the same diagnoses presented in Table 4 and ranked by Oxford classification are shown in Table 5.

Table 4 Level II primary interventions (Ellis score) ranked by Oxford classification
Full size table
Table 5 Level II primary interventions (Ellis score) and their corresponding appropriate interventions ranked by Oxford classification*
Full size table

Overtly unsubstantiated therapy according to Ellis classification is shown in Table 6. Considering levels I and II evidence-based therapy, 94% of therapeutic interventions were evidence based through Ellis classification. Using the Oxford classification, we obtained 193 individualized therapeutic interventions, that were classified as Grades A (16%), B (8%), C (1%), and D (75%).

Table 6 Level III primary interventions (Ellis score) ranked by Oxford classification
Full size table

Comparison of grade of recommendation of the prescribed intervention with the appropriate one is shown in Table 7. It will be used as summary evidence documenting our current hospital health care quality standard.

Table 7 Therapeutic interventions graded according to Oxford classification
Full size table

Discussion

In this study 94% of therapeutic interventions were evidence-based by Ellis score. It may seem encouraging that more than 90% of therapeutic decisions in a referral paediatric hospital of a developing country are evidence based. However, the level II of evidence from Ellis includes interventions based in cohort studies, case-control studies, case series, expert's opinion, and even those based in basic sciences. We attempted therefore to classify the primary interventions according to more specific criteria. Using the Oxford classification, 75% of therapeutic interventions were based in expert opinions or in basic sciences (Grade of Recommendation D).

Some limitations of our study include the possible author' bias when assigning the primary diagnosis and primary intervention. The assignment of diagnosis by the clinician may have been influenced by both the choice of treatment and the available evidence. Only one of us classified the primary intervention. In addition, we evaluated only a primary intervention for a single primary diagnosis. Actually, many patients had more than one diagnosis and obviously more than one therapeutic intervention.

We used as evidence-base for rating the interventions assessed in our study, guidelines and evidence-based resources published in the developed world. This raises the issue of whether they are fully applicable to our setting. The most prevalent conditions found were diarrhoeal dehydration and community acquired pneumonia. For both of them we used British produced guidelines [20, 21] because there were not Cochrane reviews on them and because the guidelines fulfilled recommended criteria for methodological quality of published guidelines [11]. The main recommendation of the guidelines on diarrhoea favours rapid oral rehydration over intravenous rehydration for children with mild to moderate dehydration [20]. This recommendation is based on several studies performed in both developed and developing countries and thus it can be applicable to both settings. The only concern on the applicability from setting to setting is that related to the osmolarity of the oral rehydration solution (ORS). The guidelines recommend a solution with 60 mmol/l of sodium, whereas a recent expert consensus found sufficient evidence to recommend the universal use of an ORS containing 75 mmol/l of sodium [49].

Regarding community acquired pneumonia, the British guidelines recommend antibiotic treatment for all children with pneumonia, due to the difficulties in identifying the aetiology, and they also specify criteria for hospitalization [21]. These recommendations are in agreement with the World Health Organization published guidelines [50]. The main difference is that the WHO guidelines rest on fast breathing and chest retraction for the diagnosis of pneumonia, whereas the British guidelines emphasize the role of chest x-rays. Chest x-rays are widely available in referral hospitals in developing countries and thus they should be used in addition to the clinical findings.

We acknowledge that the evidence derived from studies performed in developed countries should be translated with caution to developing settings. However, when the native research is scarce or of low quality, we think that the transfer of knowledge from the developed countries is an acceptable approach, as far as the particular characteristics of patients in developing countries are considered on an individual basis.

Dehydration due to diarrhoea and pneumonia were the most frequent diagnoses. Oral rehydration for diarrhoeal dehydration and antibiotics for pneumonia are considered as interventions with sufficient evidence for implementing them widely [9, 10]. In our study, all children with mild to moderate dehydration were treated with slow intravenous infusion, and most children with uncomplicated community acquired pneumonia received intravenous antibiotics.

In addition to their enormous potential for saving lives, outpatient antibiotic therapy for pneumonia and outpatient oral rehydration can drastically reduce the rate of hospitalizations, the hospital stay, the hospital mortality rate, and the costs incurred. At our hospital, the mean stay time for hospitalized children is 4.7 days, and the mean crude mortality rate is 3.6%. We estimated the cost of managing hospitalized children with pneumonia and diarrhoeal dehydration as US$ 10.6/day and US$ 8.6/day, respectively. These costs are referred only to hospital bed and laboratory tests. A substantial amount of money could be saved treating these conditions on an outpatient basis.

We planned the dissemination of our results among the hospital policy makers. The tools that will be suggested for improving the standards include the development and systematic application of locally produced guidelines and/or the adaptation of published guidelines. A useful alternative that has been experienced for several years at our inpatient ward unit is to make available personal computers connected to Internet for attending physicians, residents and interns, and to encourage the use of online evidence-based resources. This last alternative may work better, particularly where there are motivated physicians who are able to lead the efforts for improving the health care standards. However, the ultimate decision to systematically introduce and monitor the suggested changes will rest on hospital managers. Such changes should also depend on taking into account the role of several other determinants of the clinical decision making by individual practitioners, such as continuous training, motivation, time, availability of drugs, equipments and supplies, supervision, and long-term health system strengthening strategies.

Conclusions

Caution must be taken when assigning the level of evidence that supports therapeutic interventions, as commonly used classifications may be misleading. Existing effective interventions for dehydration and pneumonia are not being implemented in developing countries, even at referral level, and thus there is the need to change the current medical interventional behaviours. For the remaining diagnoses, the majority of assessed interventions were based on weak or non-existent evidence, highlighting the need to conduct appropriate clinical studies.