Skip to main content

Advertisement

SpringerLink
Log in

The influence of missing components of the Acute Physiology Score of APACHE III on the measurement of ICU performance

  • Original
  • Published:
Intensive Care Medicine Aims and scope Submit manuscript

Abstract

Objective

To determine the impact of missing Acute Physiology Score (APS) values on risk-adjusted mortality.

Design

Retrospective review of prospectively collected Acute Physiology and Chronic Health Evaluation (APACHE) III database.

Setting

The intensive care units (ICUs) of an academic medical center.

Patients

38,411 patients admitted to ICU between October 1994 and December 2003.

Measurements and results

Data were collected on ICU type, missing first ICU day APS values, predicted and observed hospital mortality, standardized mortality ratio (SMR), 95% confidence interval (CI), odds ratio (OR). The overall observed and predicted hospital mortality rates were 8.7% and 10.8%, respectively, with SMR of 0.806 (95% CI 0.779–0.834). Complete data were available in 829 (2.2%). Vital signs were missing in almost none and serum albumin and bilirubin in over 80% of the patients. The number of missing variables was higher in less sick and surgical ICU patients. Logistic regression analysis showed that the risk of dying in the hospital was significantly associated with the number of missing APS variables (OR 1.058, 95% CI 1.027–1.090) when adjusted for the severity of illness. The risk of death was also associated with the type of missing variables.

Conclusions

Since missing APS values may lead to underestimation of the predicted mortality rates, the number and type of missing variables should be taken into consideration when assessing the performance of an ICU. Unless data collection is standardized, future prognostic models should use variables that are routinely measured in most critically ill patients without sacrificing statistical precision.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Knaus WA, Draper EA, Wagner DP, Zimmerman JE (1985) APACHE II: a severity of disease classification system. Crit Care Med 13:818–829

    PubMed  Google Scholar 

  2. Knaus WA, Wagner DP, Draper EA, Zimmerman JE, Bergner M, Bastos PG, Sirio CA, Murphy DJ, Lotring T, Damiano A (1991) The APACHE III prognostic system. Risk prediction of hospital mortality for critically ill hospitalized adults. Chest 100:1619–1636

    PubMed  Google Scholar 

  3. Knaus WA, Zimmerman JE, Wagner DP, Draper EA, Lawrence DE (1981) APACHE-acute physiology and chronic health evaluation: a physiologically based classification system. Crit Care Med 9:591–597

    PubMed  Google Scholar 

  4. Le Gall JR, Lemeshow S, Saulnier F (1993) A new Simplified Acute Physiology Score (SAPS II) based on a European/North American multicenter study. JAMA 270:2957–2963

    Article  PubMed  Google Scholar 

  5. Le Gall JR, Loirat P, Alperovitch A, Glaser P, Granthil C, Mathieu D, Mercier P, Thomas R, Villers D (1984) A simplified Acute Physiology Score for ICU patients. Crit Care Med 12:975–977

    PubMed  Google Scholar 

  6. Lemeshow S, Teres D, Klar J, Avrunin JS, Gehlbach SH, Rapoport J (1993) Mortality Probability Models (MPM II) based on an international cohort of intensive care unit patients. JAMA 270:2478–2486

    Article  PubMed  Google Scholar 

  7. Lemeshow S, Klar J, Teres D, Avrunin JS, Gehlbach SH, Rapoport J, Rue M (1994) Mortality probability models for patients in the intensive care unit for 48 or 72 hours: a prospective, multicenter study. Crit Care Med 22:1351–1358

    PubMed  Google Scholar 

  8. Lemeshow S, Teres D, Pastides H, Avrunin JS, Steingrub JS (1985) A method for predicting survival and mortality of ICU patients using objectively derived weights. Crit Care Med 13:519–525

    PubMed  Google Scholar 

  9. Lemeshow S, Le Gall JR (1994) Modeling the severity of illness of ICU patients. A systems update. JAMA 272:1049–1055

    Article  PubMed  Google Scholar 

  10. Glance LG, Osler TM, Dick A (2002) Rating the quality of intensive care units: is it a function of the intensive care unit scoring system? Crit Care Med 30:1976–1982

    Article  PubMed  Google Scholar 

  11. Knaus WA (2002) APACHE 1978–2001: the development of a quality assurance system based on prognosis: milestones and personal reflections. Arch Surg 137:37–41

    Article  PubMed  Google Scholar 

  12. Rapoport J, Teres D, Lemeshow S, Gehlbach S (1994) A method for assessing the clinical performance and cost-effectiveness of intensive care units: a multicenter inception cohort study. Crit Care Med 22:1385–1391

    PubMed  Google Scholar 

  13. Zimmerman JE, Alzola C, Von Rueden KT (2003) The use of benchmarking to identify top performing critical care units: a preliminary assessment of their policies and practices. J Crit Care 18:76–86

    Article  PubMed  Google Scholar 

  14. Zimmerman JE, Wagner DP, Draper EA, Wright L, Alzola C, Knaus WA (1998) Evaluation of acute physiology and chronic health evaluation III predictions of hospital mortality in an independent database. Crit Care Med 26:1317–1326

    Article  PubMed  Google Scholar 

  15. Metnitz PG, Vesely H, Valentin A, Popow C, Hiesmayr M, Lenz K, Krenn CG, Steltzer H (1999) Evaluation of an interdisciplinary data set for national intensive care unit assessment. Crit Care Med 27:1486–1491

    Article  PubMed  Google Scholar 

  16. Engel JM, Junger A, Bottger S, Benson M, Michel A, Rohrig R, Jost A, Hempelmann G (2003) Outcome prediction in a surgical ICU using automatically calculated SAPS II scores. Anaesth Intensive Care 31:548–554

    PubMed  Google Scholar 

  17. Perez A, Dennis RJ, Gil JF, Rondon MA, Lopez A (2002) Use of the mean, hot deck and multiple imputation techniques to predict outcome in intensive care unit patients in Colombia. Stat Med 21:3885–3896

    Article  PubMed  Google Scholar 

  18. Li TC, Phillips MC, Shaw L, Cook EF, Natanson C, Goldman L (1984) On-site physician staffing in a community hospital intensive care unit. Impact on test and procedure use and on patient outcome. JAMA 252:2023–2027

    Article  PubMed  Google Scholar 

  19. Afessa B, Keegan MT, Gajic O, Hubmayr RD, Peters SG (2005) The impact of missing components of the Acute Physiology Score on the standardized mortality ratio calculated by the APACHE III prognostic model. Crit Care 9:S96–S97

    Article  Google Scholar 

  20. Afessa B, Keegan MT, Hubmayr RD, Naessens JM, Gajic O, Long KH, Peters SG (2005) Evaluating the performance of an institution using an intensive care unit benchmark. Mayo Clin Proc 80:174–180

    PubMed  Google Scholar 

  21. Apolone G (2000) The state of research on multipurpose severity of illness scoring systems: are we on target? Intensive Care Med 26:1727–1729

    Article  PubMed  Google Scholar 

  22. Bosman RJ, Oudemane van Straaten HM, Zandstra DF (1998) The use of intensive care information systems alters outcome prediction. Intensive Care Med 24:953–958

    Article  PubMed  Google Scholar 

  23. Chen LM, Martin CM, Morrison TL, Sibbald WJ (1999) Interobserver variability in data collection of the APACHE II score in teaching and community hospitals. Crit Care Med 27:1999–2004

    Article  PubMed  Google Scholar 

  24. Glance LG, Osler TM, Papadakos P (2000) Effect of mortality rate on the performance of the Acute Physiology and Chronic Health Evaluation II: a simulation study. Crit Care Med 28:3424–3428

    Article  PubMed  Google Scholar 

  25. Goldhill DR, Sumner A (1998) APACHE II, data accuracy and outcome prediction. Anaesthesia 53:937–943

    Article  PubMed  Google Scholar 

  26. Polderman KH, Thijs LG, Girbes AR (1999) Interobserver variability in the use of APACHE II scores. Lancet 353:380

    Article  Google Scholar 

  27. Polderman KH, Girbes AR, Thijs LG, Strack van Schijndel RJ (2001) Accuracy and reliability of APACHE II scoring in two intensive care units Problems and pitfalls in the use of APACHE II and suggestions for improvement. Anaesthesia 56:47–50

    Article  PubMed  Google Scholar 

  28. Polderman KH, Jorna EM, Girbes AR (2001) Inter-observer variability in APACHE II scoring: effect of strict guidelines and training. Intensive Care Med 27:1365–1369

    Article  PubMed  Google Scholar 

  29. Rosenberg AL, Hofer TP, Strachan C, Watts CM, Hayward RA (2003) Accepting critically ill transfer patients: adverse effect on a referral center’s outcome and benchmark measures. Ann Intern Med 138:882–890

    PubMed  Google Scholar 

  30. Rosenberg AL (2002) Recent innovations in intensive care unit risk-prediction models. Curr Opin Crit Care 8:321–330

    Article  PubMed  Google Scholar 

  31. Suistomaa M, Kari A, Ruokonen E, Takala J (2000) Sampling rate causes bias in APACHE II and SAPS II scores. Intensive Care Med 26:1773–1778

    Article  PubMed  Google Scholar 

  32. Keulen JG van, Polderman KH, Gemke RJ (2005) Reliability of PRISM and PIM scores in paediatric intensive care. Arch Dis Child 90:211–214

    Article  PubMed  Google Scholar 

  33. Zanten AR van, Polderman KH (2004) Organizational changes in a single intensive care unit affect benchmarking. Ann Intern Med 140:674–675

    PubMed  Google Scholar 

  34. Champion HR, Sacco WJ (1982) Measurement of patient illness severity. Crit Care Med 10:552–553

    PubMed  Google Scholar 

  35. Fery-Lemonnier E, Landais P, Loirat P, Kleinknecht D, Brivet F (1995) Evaluation of severity scoring systems in ICUs-translation, conversion and definition ambiguities as a source of inter-observer variability in APACHE II, SAPS and OSF. Intensive Care Med 21:356–360

    Article  PubMed  Google Scholar 

  36. Zimmerman JE (2002) Measuring intensive care unit performance: a way to move forward. Crit Care Med 30:2149–2150

    Article  PubMed  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Division of Pulmonary and Critical Care, Department of Medicine, Mayo Clinic, 200 First Street SW, Rochester, MN, 55905, USA

    Bekele Afessa, Ognjen Gajic, Rolf D. Hubmayr & Steve G. Peters

  2. Division of Critical Care, Department of Anesthesia, Mayo Clinic, 200 First Street SW, Rochester, MN, 55905, USA

    Mark T. Keegan

Authors
  1. Bekele Afessa
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mark T. Keegan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ognjen Gajic
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Rolf D. Hubmayr
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Steve G. Peters
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bekele Afessa.

Additional information

This article refers to the editorial: http://dx.doi.org/10.1007/s00134-005-2750-x

Rights and permissions

Reprints and permissions

About this article

Cite this article

Afessa, B., Keegan, M.T., Gajic, O. et al. The influence of missing components of the Acute Physiology Score of APACHE III on the measurement of ICU performance. Intensive Care Med 31, 1537–1543 (2005). https://doi.org/10.1007/s00134-005-2751-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00134-005-2751-9

Keywords

Search

Navigation