Springer Nature is making SARS-CoV-2 and COVID-19 research free. View research | View latest news | Sign up for updates

Evaluation of severity scoring systems in ICUs—translation, conversion and definition ambiguities as a source of inter-observer variability in Apache II, SAPS and OSF

Abstract

Objective

To explore translation, conversion and definition ambiguities, when using severity scoring systems in patients admitted to intensive care units (ICUs).

Design

A prospective study of the prognosis of acute renal failure in ICUs.

Setting

The study was conducted in 20 French ICUs.

Patients

360 patients presenting with severe acute renal failure were studied during their ICU stay.

Measurements and results

The inter-observer variability of Apache II (acute physiology and chronic health evaluation), SAPS (simplified acute physiology score), and OSF (organ-system failure) was considered. For Apache II, we explored the uncertainty of measurements arising from conversion into SI units, the rounding procedures used for the non-inclusive intervals defined for quatitative parameters such as age, mean arterial pressure (MAP) or serum creatinite, the absence of definition of acute renal failure (ARF) and its consequence on doubling serum creatinine values, and the absence of guidelines in the case of spontaneous ventilation when arterial blood gases (ABG) and forced inspiratory oxygen (FIO2) were not measured. The resulting variability was evaluated, calculating the lowest and the highest value of the scoring system for each patient. The mean difference by patient was greater than 1.5 (p<0.0001). Other examples were presented and discussed for SAPS and OSF.

Conclusions

Translation, conversion and definition ambiguities are a source of inter-observer variability and increase the risk of classification and/or selection biases. This gives rise to particular concern in the design and analysis of multicenter trials or meta-analysis, and improvement of these scoring systems should be envisaged in the future.

This is a preview of subscription content, log in to check access.

References

  1. 1.

    Le Gall JR, Loirat P (1990) Evaluation en réanimation. (Collection d'anesthésiologie et de réanimation). Masson, Paris

  2. 2.

    Chang RWS, Jacobs S (1986) Use of Apache II severity of disease classification to identify intensive-care-unit patients who would not benefit from total parenteral nutrition. Lancet II: 1483–1487

  3. 3.

    Wagner DP, Knaus WA, Draper EA, Zimmerman JE (1983) Identification of low-risk monitor patients within a medical-surgical intensive care unit. Med Care 21:425–434

  4. 4.

    Knaus WA, Draper EA, Wagner DP, Zimmerman JE (1985) Prognosis in acute organ-system failure. Ann Surg 202:685–693

  5. 5.

    Chang RWS (1989) Individual outcome prediction models for intensive care units. Lancet II:143–146

  6. 6.

    Le Gall JR (1986) The intensive care unit: definitions and managerial differences. A French multicentric study on 38 units. Elsevier, Amsterdam, pp 39–54

  7. 7.

    Le Gall JR, Brun-Buisson C, Trunet P, Latournerie J, Chantereau S, Rapin M (1982) Influence of age, previous health status, and severity of acute illness on outcome from intensive care. Crit Care Med 10:575–577

  8. 8.

    French multicenter group of ICU research; INSERM Unit 169 of statistical and epidemiological studies (1989) Factors related to outcome in intensive care: French multicenter study. Crit Care Med 17:305–308

  9. 9.

    Knaus WA, Wagner DP, Draper EA et al (1991) The Apache III prognostic system. Chest 100:1619–1636

  10. 10.

    The French Study Group on ARF, Agostini MM, Mezzarobba P, Kleinknecht D, Loirat P, Brivet F, Landais P (1992) Prognosis of acute renal failure in intensive care units. A prospective multicenter study. Intensive Care Med 18 [Suppl 2]:S65

  11. 11.

    French Multicenter Group, Mezzarobba P, Agostini MM, Loirat P, Kleinknecht D, Brivet F, Landais P (1992) Comparison of SAPS, Apache II and OSF in the evaluation of severe acute renal failure. Intensive Care Med 18 [Suppl 2]:S65

  12. 12.

    Knaus WA, Draper EA, Wagner DP, Zimmerman JE (1985) Apache II: a severity of disease classification system. Crit Care Med 13:818–829

  13. 13.

    Le Gall JR, Loirat P, Alperovitch A, Glaser P, Granthil C, Mathieu D, Mercier P, Thomas R, Villers D (1984) A simplified acute physiology score for ICU patients. Crit Care Med 12: 975–977

  14. 14.

    Le Gall JR, Loirat P, Alperovitch A (1989) The simplified acute physiology score (SAPS). Probl Crit Care 3: 578–584

  15. 15.

    Rauss A, Knaus WA, Patois E, Le Gall JR, Loirat P (1990) Prognosis from multiple organ system failure: the reliability of objective estimates for survival. Med Decis Making 10:155–162

Download references

Author information

Correspondence to E. Féry-Lemonnier.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Féry-Lemonnier, E., Landais, P., Loirat, P. et al. Evaluation of severity scoring systems in ICUs—translation, conversion and definition ambiguities as a source of inter-observer variability in Apache II, SAPS and OSF. Intensive Care Med 21, 356–360 (1995). https://doi.org/10.1007/BF01705416

Download citation

Key words

  • Evaluation
  • Scoring systems
  • Inter-observer variability
  • Acute renal failure