Skip to main content

Advertisement

Log in

Evaluation of the uniformity of fit of general outcome prediction models

  • ORIGINAL
  • Published:
Intensive Care Medicine Aims and scope Submit manuscript

Abstract

Objective: To compare the performance of the New Simplified Acute Physiology Score (SAPS II) and the New Admission Mortality Probability Model (MPM II0) within relevant subgroups using formal statistical assessment (uniformity of fit). Design: Analysis of the database of a multi-centre, multi-national and prospective cohort study, involving 89 ICUs from 12 European Countries. Setting: Database of EURICUS-I. Patients: Data of 16,060 patients consecutively admitted to the ICUs were collected during a period of 4 months. Following the original SAPS II and MPM II0 criteria, the following patients were excluded from the analysis: younger than 18 years of age; readmissions; acute myocardial infarction; burn cases; patients in the post-operative period after coronary artery bypass surgery and patients with a length of stay in the ICU shorter than 8 h, resulting in a total of 10,027 cases. Interventions: Data necessary for the calculation of SAPS II and MPM II0, basic demographic statistics and vital status on hospital discharge were recorded. Formal evaluation of the performance of the models, comprising discrimination (area under ROC curve), calibration (Hosmer-Lemeshow goodness-of-fit H^ and C^ tests) and observed/expected mortality ratios within relevant subgroups. Main results: Better predictive accuracy was achieved in elective surgery patients admitted from the operative room/post-anaesthesia room with gastrointestinal, neurological or trauma diagnoses, and younger patients with non-operative neurological, septic or trauma diagnoses. All these characteristics appear to be linked to a lower severity of illness, with both models overestimating mortality in the more severely ill patients. Conclusions: Concerning the performance of the models, very large differences were apparent in relevant subgroups, varying from excellent to almost random predictive accuracy. These differences can explain some of the difficulties of the models to accurately predict mortality when applied to different populations with distinct patient baseline characteristics. This study stresses the importance of evaluating multiple diverse populations (to generate the design set) and of methods to improve the validation set before extrapolations can be made from the validation setting to new independent populations. It also underlines the necessity of a better definition of the patient baseline characteristics in the samples under analysis and the formal statistical evaluation of the application of the models to specific subgroups.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

Author information

Authors and Affiliations

Authors

Additional information

Received: 12 August 1996 Accepted: 22 October 1997

Rights and permissions

Reprints and permissions

About this article

Cite this article

Moreno, R., Apolone, G. & Reis Miranda, D. Evaluation of the uniformity of fit of general outcome prediction models. Intensive Care Med 24, 40–47 (1998). https://doi.org/10.1007/s001340050513

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s001340050513

Navigation