Skip to main content

Comparison between SAPS II and SAPS 3 in predicting hospital mortality in a cohort of 103 Italian ICUs. Is new always better?

Intensive Care Medicine volume 38, pages 1280–1288 (2012)Cite this article

Abstract

Purpose

More recent severity scores should be more reliable than older ones because they account for the improvement in medical care over time. To provide more insight into this issue, we compared the predictive ability of the Simplified Acute Physiology Score (SAPS) II and SAPS 3 (originally developed from data collected in 1991–1992 and 2002, respectively) on a sample of critically ill patients.

Methods

This was a prospective observational study on 3,661 patients from 103 Italian intensive care units. Standardized mortality ratios (SMRs) were calculated. Assessment of calibration across risk classes was performed using the GiViTI calibration belt. Discrimination was evaluated by means of the area under the receiver operating characteristic analysis.

Results

Both scores were shown to discriminate fairly. SAPS 3 largely overpredicted mortality, more than SAPS II (SMR 0.63, 95 % CI 0.60–0.66 vs. 0.87, 95 % CI 0.83–0.91). This result was consistent and statistically significant across all risk classes for SAPS 3. SAPS II did not show relevant deviations from ideal calibration in the first two deciles of risk, whereas in higher-risk classes it overpredicted mortality.

Conclusions

Both scores provided unreliable predictions, but unexpectedly the newer SAPS 3 turned out to overpredict mortality more than the older SAPS II.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

43,34 €

Price includes VAT (Finland)
Tax calculation will be finalised during checkout.

Fig. 1
Fig. 2
Fig. 3

References

  1. Knaus WA, Wagner DP, Draper EA, Zimmerman JE, Bergner M, Bastos PG, Sirio CA, Murphy DJ, Lotring T, Damiano A et al (1991) The APACHE III prognostic system. Risk prediction of hospital mortality for critically ill hospitalized adults. Chest 100:1619–1636

    PubMed  Article  CAS  Google Scholar 

  2. Zimmerman JE, Kramer AA, McNair DS, Malila FM (2006) Acute Physiology and Chronic Health Evaluation (APACHE) IV: hospital mortality assessment for today’s critically ill patients. Crit Care Med 34:1297–1310

    PubMed  Article  Google Scholar 

  3. Sirio CA, Shepardson LB, Rotondi AJ, Cooper GS, Angus DC, Harper DL, Rosenthal GE (1999) Community-wide assessment of intensive care outcomes using a physiologically based prognostic measure: implications for critical care delivery from Cleveland health quality choice. Chest 115:793–801

    PubMed  Article  CAS  Google Scholar 

  4. Minne L, Eslami S, de Keizer N, de Jonge E, de Rooij SE, Abu-Hanna A (2012) Effect of changes over time in the performance of a customized SAPS-II model on the quality of care assessment. Intensive Care Med 38:40–46

    PubMed  Article  CAS  Google Scholar 

  5. Le Gall JR, Lemeshow S, Saulnier F (1993) A new Simplified Acute Physiology Score (SAPS II) based on a European/North American multicenter study. JAMA 270:2957–2963

    PubMed  Article  Google Scholar 

  6. Metnitz PG, Moreno RP, Almeida E, Jordan B, Bauer P, Campos RA, Iapichino G, Edbrooke D, Capuzzo M, Le Gall JR (2005) SAPS 3–From evaluation of the patient to evaluation of the intensive care unit. Part 1: objectives, methods and cohort description. Intensive Care Med 31:1336–1344

    PubMed  Article  Google Scholar 

  7. Capuzzo M, Moreno RP, Le Gall JR (2008) Outcome prediction in critical care: the simplified acute physiology score models. Curr Opin Crit Care 14:485–490

    PubMed  Article  Google Scholar 

  8. Boffelli S, Rossi C, Anghileri A, Giardino M, Carnevale L, Messina M, Neri M, Langer M, Bertolini G; Italian Group for the Evaluation of Interventions in Intensive Care Medicine (2006) Continuous quality improvement in intensive care medicine. The GiViTI Margherita Project—Report 2005. Minerva Anestesiol 72:419–432

    Google Scholar 

  9. Poole D, Rossi C, Anghileri A, Giardino M, Latronico N, Radrizzani D, Langer M, Bertolini G (2009) External validation of the Simplified Acute Physiology Score (SAPS) 3 in a cohort of 28,357 patients from 147 Italian intensive care units. Intensive Care Med 35:1916–1924

    PubMed  Article  Google Scholar 

  10. Moreno RP, Metnitz PG, Almeida E, Jordan B, Bauer P, Campos RA, Iapichino G, Edbrooke D, Capuzzo M, Le Gall JR (2005) SAPS 3–From evaluation of the patient to evaluation of the intensive care unit. Part 2: Development of a prognostic model for hospital mortality at ICU admission. Intensive Care Med 31:1345–1355

    PubMed  Article  Google Scholar 

  11. Campbell G (1994) Advances in statistical methodology for the evaluation of diagnostic and laboratory tests. Stat Med 13:499–508

    PubMed  Article  CAS  Google Scholar 

  12. Lemeshow S, Hosmer DW Jr (1982) A review of goodness of fit statistics for use in the development of logistic regression models. Am J Epidemiol 115:92–106

    PubMed  CAS  Google Scholar 

  13. Bertolini G, D’Amico R, Nardi D, Tinazzi A, Apolone G (2000) One model, several results: the paradox of the Hosmer–Lemeshow goodness-of-fit test for the logistic regression model. J Epidemiol Biostat 5:251–253

    PubMed  CAS  Google Scholar 

  14. Kramer AA, Zimmerman JE (2007) Assessing the calibration of mortality benchmarks in critical care: the Hosmer-Lemeshow test revisited. Crit Care Med 35:2052–2056

    PubMed  Article  Google Scholar 

  15. Cox D (1958) Two further applications of a model for a method of binary regression. Biometrika 45:562–565

    Google Scholar 

  16. Finazzi S, Poole D, Luciani D, Cogo PE, Bertolini G (2011) Calibration belt for quality-of-care assessment based on dichotomous outcomes. PLoS ONE 6:e16110

    PubMed  Article  CAS  Google Scholar 

  17. Moreno R, Apolone G, Miranda DR (1998) Evaluation of the uniformity of fit of general outcome prediction models. Intensive Care Med 24:40–47

    PubMed  Article  CAS  Google Scholar 

  18. Strand K, Soreide E, Aardal S, Flaatten H (2009) A comparison of SAPS II and SAPS 3 in a Norwegian intensive care unit population. Acta Anaesthesiol Scand 53:595–600

    PubMed  Article  CAS  Google Scholar 

  19. Capuzzo M, Scaramuzza A, Vaccarini B, Gilli G, Zannoli S, Farabegoli L, Felisatti G, Davanzo E, Alvisi R (2009) Validation of SAPS 3 Admission Score and comparison with SAPS II. Acta Anaesthesiol Scand 53:589–594

    PubMed  Article  CAS  Google Scholar 

  20. Soares M, Silva UV, Teles JM, Silva E, Caruso P, Lobo SM, Dal Pizzol F, Azevedo LP, de Carvalho FB, Salluh JI (2010) Validation of four prognostic scores in patients with cancer admitted to Brazilian intensive care units: results from a prospective multicenter study. Intensive Care Med 36:1188–1195

    PubMed  Article  Google Scholar 

  21. Ledoux D, Canivet JL, Preiser JC, Lefrancq J, Damas P (2008) SAPS 3 admission score: an external validation in a general intensive care population. Intensive Care Med 34:1873–1877

    PubMed  Article  Google Scholar 

  22. Murphy-Filkins R, Teres D, Lemeshow S, Hosmer DW (1996) Effect of changing patient mix on the performance of an intensive care unit severity-of-illness model: how to distinguish a general from a specialty intensive care unit. Crit Care Med 24:1968–1973

    PubMed  Article  CAS  Google Scholar 

  23. Miller ME, Hui SL, Tierney WM (1991) Validation techniques for logistic regression models. Stat Med 10:1213–1226

    PubMed  Article  CAS  Google Scholar 

  24. Metnitz PG, Lang T, Vesely H, Valentin A, Le Gall JR (2000) Ratios of observed to expected mortality are affected by differences in case mix and quality of care. Intensive Care Med 26:1466–1472

    PubMed  Article  CAS  Google Scholar 

  25. Nimgaonkar A, Karnad DR, Sudarshan S, Ohno-Machado L, Kohane I (2004) Prediction of mortality in an Indian intensive care unit. Comparison between APACHE II and artificial neural networks. Intensive Care Med 30:248–253

    PubMed  Article  Google Scholar 

  26. Zimmerman JE, Wagner DP, Draper EA, Wright L, Alzola C, Knaus WA (1998) Evaluation of acute physiology and chronic health evaluation III predictions of hospital mortality in an independent database. Crit Care Med 26:1317–1326

    PubMed  Article  CAS  Google Scholar 

  27. Babyak MA (2004) What you see may not be what you get: a brief, nontechnical introduction to overfitting in regression-type models. Psychosom Med 66:411–421

    PubMed  Article  Google Scholar 

Download references

Acknowledgments

The study was wholly funded by GiViTI-Istituto di Ricerche Farmacologiche Mario Negri, Bergamo (Italy). GiViTI is the recipient of unconditioned grants from Bellco, Brahms, and Astellas, which did not, however, have any role in this study. The authors would like to thank Abramo Anghileri and Michele Giardino (Istituto di Ricerche Farmacologiche Mario Negri) for their contribution in developing and maintaining the data collection software.

Conflicts of interest

The authors declare they have no conflict of interest regarding the content of the manuscript.

Author information

Affiliations

  1. Servizio Anestesia e Rianimazione, Ospedale Civile San Martino, Belluno, Italy

    Daniele Poole

  2. Istituto di Ricerche Farmacologiche “Mario Negri”: Centro di Ricerche Cliniche per le Malattie Rare Aldo e Cele Daccò, Ranica, Bergamo, Italy

    Carlotta Rossi & Guido Bertolini

  3. SSVD di Neuroanestesia e Neurorianimazione, Spedali Civili, Universita’ di Brescia, Brescia, Italy

    Nicola Latronico

  4. I Servizio di Anestesia e Rianimazione, Spedali Riuniti, Livorno, Italy

    Giancarlo Rossi

  5. INO-CNR BEC Center and Dipartimento di Fisica, Universita’ di Trento, Trento, Italy

    Stefano Finazzi

  6. Department of Anesthesia and Intensive Care, S. Martino Hospital, Viale Europa, 22, 32100, Belluno, Italy

    Daniele Poole

Authors
  1. Daniele Poole
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Carlotta Rossi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Nicola Latronico
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Giancarlo Rossi
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Stefano Finazzi
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Guido Bertolini
    View author publications

    You can also search for this author in PubMed Google Scholar

Consortia

GiViTI

Corresponding author

Correspondence to Daniele Poole.

Additional information

This article is discussed in the editorial available at:doi: 10.1007/s00134-012-2579-z.

The complete list of study participants appears in the Appendix.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (DOC 917 kb)

Appendix

Appendix

GiViTI steering committee members

Guido Bertolini (Ranica, BG); Daniela Boccalatte (Lucca); Arturo Chieregato (Cesena); Daniela Codazzi (Bari); Roberto Fumagalli (Monza); Giorgio Gambale (Forlì); Martin Langer (Milano); Sergio Livigni (Torino); Paolo Malacarne (Pisa); Giuseppe Nardi (Roma); Daniele Poole (Belluno); Danilo Radrizzani (Legnano, MI); Mario Tavola (Lecco).

List of study participants

Marco Abastanotti (Manerbio—BS), Claudia Acciari (Ascoli Piceno—AP), Annamaria Acquarolo (Brescia—BS), Arnaldo Alberti (Porto Viro—RO), Armando Alborghetti (Ponte San Pietro—BG), Paolo Antolini (Fermo—AP), Enrico Arditi (Genova—GE), Massimo Barattini (Firenze—FI), Arcangelo Bartoccini (Borgomanero—NO), Teresa Bartoli (Antella—FI), Francesco Bassi (Cinisello Balsamo—MI), Eduardo Beck (Desio—MI), Uberto Belloni (Ivrea—TO), Mario Bellorini (Tradate—VA), Cesare Benanti (Pescia—PT), Mara Olga Bernasconi (Rovigo—RO), Andrea Bianchin (Montebelluna—TV), Angelo Blasetti (Avezzano—AQ), Daniela Boccalatte (Lucca—LU), Andrea Bonfà (Prato—PO), Monica Bonfiglio (Lavagna—GE), Cesare Breschi (Pesaro—PU), Emanuela Brunori (Macerata—MC), Roberto Buonanno (Lacco Ameno—NA), Virginio Buzzetti (Orbetello Scalo—GR), Gian Virgilio Cabano (Busto Arsizio—VA), Giuseppe Calicchio (Salerno—SA), Livio Carnevale (Pavia—PV), Lucia Casagrande (Treviso—TV), Pierpaolo Casalini (Faenza—RA), Giacomo Castiglione (Catania—CT), Elena Chinelli (Bologna—BO), Maddalena Coaloa (Savigliano—CN), Riccardo Colombo (Milano—MI), Sergio Colombo (Milano—MI), Walter Corsini (Carrara—MS), Eleonora Costanzo (Asti—AT), Tito Cottignoli (Lugo di Romagna—RA), Luciano Crema (Cremona—CR), Dolores Da Re (Este—PD), Roberto Alberto De Blasi (Roma—RM), Elvio De Blasio (Benevento—BN), Marino Del Ferro (Oderzo—TV), Pierfrancesco Di Masi (Castellana Grotte—BA), Dino Di Pasquale (Pontedera—PI), Ivano Donato (Cantù—CO), Lea Paola Fabbri (Firenze—FI), Emilio Fabbri (Forlì—FC), Maria Cristina Fabi (Fano—PU), Luciana Faccio (Torino—TO), Paolo Febbrari (Brescia—BS), Francesco Ferla (Cefalù—PA), Gilberto Fiore (Moncalieri—TO), Claudio Fogliati (Orbassano—TO), Emiliano Gamberini (Cesena—FC), Alberto Garelli (Ravenna—RA), Massimo Gianni (Aosta—AO), Stefano Giannoni (Empoli—FI), Daniela Giudici (Milano—MI), Pier Mario Giugiaro (Ciriè—TO), Maurizio Greco (Ariano Irpino—AV), Michele Isetta (Genova—GE), Luigi Lacquaniti (Rossano—CS), Sergio Livigni (Torino—TO), Roberto Madonna (Grosseto—GR), Maria Federica Magatti (Como—CO), Paolo Malacarne (Pisa—PI), Alba Malinconi (Este—PD), Stefano Mancosu (Cagliari—CA), Valerio Mangani (Firenze—FI), Giorgio Mantovani (Ferrara—FE), Alessandro Mastroianni (Chieri—TO), Marianna Messina (Olbia—OT), Renzo Miglioranzi (Pieve di Coriano—MN), Salvatore Murru (Cagliari—CA), Giuseppe Nardi (Roma—RM), Giuseppe Natalini (Brescia—BS), Lorenzo Odetto (Orbassano—TO), Carlo Oliviero (Novara—NO), Daniela Maria Pacini (Genova—GE), Giorgio Paganini (Sanremo—IM), Maurizio Palmer (Aversa—CE), Luigi Panella (L’Aquila—AQ), Silvano Papiri (San Benedetto del Tronto—AP), Mauro Pastorelli (Pinerolo—TO), Maurizio Pegoraro (Castelfranco Veneto—TV), Angelo Pezzi (Milano—MI), Donatella Pinciroli (Legnano—MI), Marialuisa Pizzaballa (Zingonia—BG), Maurizio Postiglione (Napoli—NA), Massimo Raffaeli (Milano—MI), Paolo Riverso (Sanremo—IM), Mario Romitti (Piacenza—PC), Nicola Rossi (Monza—MI), Maurizio Rossi (Menaggio—CO), Federica Rottoli (Bergamo—BG), Rosa Salcuni (Ivrea—TO), Monica Savioli (Milano—MI), Silvia Scarrone (Alessandria—AL), Vincenzo Segala (Torino—TO), Pieraurelio Segalini (Piacenza—PC), Alberto Sicignano (Milano—MI), Maria Josè Sucre (Castellammare di Stabia—NA), Claudio Tagliaferri (Piacenza—PC), Romano Tetamo (Palermo—PA), Giuseppe Tibaldi (Brescia—BS), Silvia Tinacci (Fucecchio—FI), Livio Todesco (Cittadella—PD), Gabriele Tomasoni (Manerbio—BS), Mauro Torta (Torino—TO), Carlo Tripepi (Monselice—PD), Pietro Vecchiarelli (Viterbo—VT), Maria Grazia Visconti (Cernusco sul Naviglio—MI), Nicoletta Vivaldi (Alessandria—AL), Giuseppe Angelo Vulcano (Rossano—CS), Sergio Zappa (Brescia—BS), Francesco Zuccaro (Matera—MT).

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Poole, D., Rossi, C., Latronico, N. et al. Comparison between SAPS II and SAPS 3 in predicting hospital mortality in a cohort of 103 Italian ICUs. Is new always better?. Intensive Care Med 38, 1280–1288 (2012). https://doi.org/10.1007/s00134-012-2578-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00134-012-2578-0

Keywords

  • Severity of illness index
  • APACHE
  • SAPS
  • Critical care
  • Prognosis
  • Logistic models