Skip to main content

Advertisement

SpringerLink
Log in

External validation of the Simplified Acute Physiology Score (SAPS) 3 in a cohort of 28,357 patients from 147 Italian intensive care units

  • Original
  • Published:
Intensive Care Medicine Aims and scope Submit manuscript

Abstract

Objective

To evaluate the SAPS 3 score predictive ability of hospital mortality in a large external validation cohort.

Design

Prospective observational study.

Setting and patients

A total of 28,357 patients from 147 Italian ICUs joining the Project Margherita national database of the Gruppo italiano per la Valutazione degli interventi in Terapia Intensiva (GiViTI).

Interventions

None.

Measurement

Evaluation of discrimination through ROC analysis and of overall goodness-of-fit through the Cox calibration test.

Main results

Although discrimination was good, calibration turned out to be poor. The general and the South-Europe Mediterranean countries equations overestimated hospital mortality overall (SMR values 0.73 with 95% CI 0.72–0.75 for both equations) and homogeneously across risk classes. Overprediction was confirmed among important subgroups, with SMR values ranging between 0.47 and 0.82.

Conclusions

The result strictly supported by our data is that the SAPS 3 score calibrates inadequately in a large sample of Italian ICU patients and thus should not be used for benchmarking, at least in Italian settings.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Altman DG, Royston P (2000) What do we mean by validating a prognostic model? Stat Med 19:453–473

    Article  CAS  PubMed  Google Scholar 

  2. Hadorn D, Keeler EB, Rogers WH, Brook HB (1993) Assessing the Performance of Mortality Prediction Models. Final report for HCFA severity project. RAND corporation 2008. http://www.rand.org/pubs/monograph_reports/MR2181/. Accessed on June 2009

  3. Metnitz PG, Moreno RP, Almeida E, Jordan B, Bauer P, Campos RA, Iapichino G, Edbrooke D, Capuzzo M, Le Gall JR (2005) SAPS 3–from evaluation of the patient to evaluation of the intensive care unit. Part 1: Objectives, methods and cohort description. Intensive Care Med 31:1336–1344

    Article  PubMed  Google Scholar 

  4. Ledoux D, Canivet JL, Preiser JC, Lefrancq J, Damas P (2008) SAPS 3 admission score: an external validation in a general intensive care population. Intensive Care Med 34:1873–1877

    Article  PubMed  Google Scholar 

  5. Soares M, Salluh JI (2006) Validation of the SAPS 3 admission prognostic model in patients with cancer in need of intensive care. Intensive Care Med 32:1839–1844

    Article  PubMed  Google Scholar 

  6. Krauss C, Sakr Y, Amaral A, Rea-Neto A, Specht M, Reinhart K, Marx G (2008) Assessment of the performance of the SAPS 3, SAPS II, and APACHE II prognostic models in a surgical ICU. Crit Care 12(Suppl 2):P502

    Google Scholar 

  7. Moreno RP, Metnitz PG, Almeida E, Jordan B, Bauer P, Campos RA, Iapichino G, Edbrooke D, Capuzzo M, Le Gall JR (2005) SAPS 3–from evaluation of the patient to evaluation of the intensive care unit. Part 2: Development of a prognostic model for hospital mortality at ICU admission. Intensive Care Med 31:1345–1355

    Article  PubMed  Google Scholar 

  8. Moreno R, Apolone G, Miranda DR (1998) Evaluation of the uniformity of fit of general outcome prediction models. Intensive Care Med 24:40–47

    Article  CAS  PubMed  Google Scholar 

  9. Hosmer DW, Lemeshow S (1995) Confidence interval estimates of an index of quality performance based on logistic regression models. Stat Med 14:2161–2172

    Article  CAS  PubMed  Google Scholar 

  10. Kramer AA, Zimmerman JE (2007) Assessing the calibration of mortality benchmarks in critical care: the Hosmer–Lemeshow test revisited. Crit Care Med 35:2052–2056

    Article  PubMed  Google Scholar 

  11. Bertolini G, D’Amico R, Nardi D, Tinazzi A, Apolone G (2000) One model, several results: the paradox of the Hosmer-Lemeshow goodness-of-fit test for the logistic regression model. J Epidemiol Biostat 5:251–253

    CAS  PubMed  Google Scholar 

  12. Harrison DA, Brady AR, Parry GJ, Carpenter JR, Rowan K (2006) Recalibration of risk prediction models in a large multicenter cohort of admissions to adult, general critical care units in the United Kingdom. Crit Care Med 34:1378–1388

    Article  PubMed  Google Scholar 

  13. Hyam JA, Welch CA, Harrison DA, Menon DK (2006) Case mix, outcomes and comparison of risk prediction models for admissions to adult, general and specialist critical care units for head injury: a secondary analysis of the ICNARC Case Mix Programme Database. Crit Care 10(Suppl 2):S2

    Article  PubMed  Google Scholar 

  14. Seillier-Moiseiwitsch F (1996) Predictive diagnostics for logistic models. Stat Med 15:2149–2160

    Article  CAS  PubMed  Google Scholar 

  15. Cox D (1958) Two further applications of a model for a method of binary regression. Biometrika 45:562–565

    Google Scholar 

  16. Miller ME, Hui SL, Tierney WM (1991) Validation techniques for logistic regression models. Stat Med 10:1213–1226

    Article  CAS  PubMed  Google Scholar 

  17. Boffelli S, Rossi C, Bertolini G (2008) Progetto Margherita—Promuovere la ricerca e la valutazione in Terapia Intensiva. Rapporto 2007. Edizioni Sestante, Bergamo

  18. Zimmerman JE, Kramer AA, McNair DS, Malila FM (2006) Acute Physiology and Chronic Health Evaluation (APACHE) IV: hospital mortality assessment for today’s critically ill patients. Crit Care Med 34:1297–1310

    Article  PubMed  Google Scholar 

  19. Black N (1997) Developing high quality clinical databases. BMJ 315:381–382

    CAS  PubMed  Google Scholar 

  20. Beck DH, Smith GB, Pappachan JV, Millar B (2003) External validation of the SAPS II, APACHE II and APACHE III prognostic models in South England: a multicentre study. Intensive Care Med 29:249–256

    PubMed  Google Scholar 

  21. Aegerter P, Boumendil A, Retbi A, Minvielle E, Dervaux B, Guidet B (2005) SAPS II revisited. Intensive Care Med 31:416–423

    Article  PubMed  Google Scholar 

  22. Metnitz PG, Lang T, Vesely H, Valentin A, Le Gall JR (2000) Ratios of observed to expected mortality are affected by differences in case mix and quality of care. Intensive Care Med 26:1466–1472

    Article  CAS  PubMed  Google Scholar 

Download references

Acknowledgment

The study was performed without any financial support.

Conflict of interest statement

The authors do not declare any conflict of interest relevant to the subject of the manuscript. Dr. Bertolini had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Author information

Authors and Affiliations

  1. Servizio Anestesia e Rianimazione, Ospedale Civile San Martino, Belluno, Italy

    Daniele Poole

  2. Centro di Ricerche Cliniche per le Malattie Rare Aldo e Cele Daccò, Istituto di Ricerche Farmacologiche “Mario Negri”, Ranica, Bergamo, Italy

    Carlotta Rossi, Abramo Anghileri, Michele Giardino & Guido Bertolini

  3. SSVD di Neuroanestesia e Neurorianimazione, Universita’ di Brescia, Spedali Civili, Brescia, Italy

    Nicola Latronico

  4. I Servizio di Anestesia e Rianimazione, Ospedale Civile, Legnano (Mi), Italy

    Danilo Radrizzani

  5. Fondazione IRCCS Istituto Nazionale Tumori, Università degli Studi di Milano, Milan, Italy

    Martin Langer

  6. Department of Anesthesia and Intensive Care, S. Martino Hospital, Viale Europa, 22, 32100, Belluno, Italy

    Daniele Poole

Authors
  1. Daniele Poole
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Carlotta Rossi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Abramo Anghileri
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Michele Giardino
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Nicola Latronico
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Danilo Radrizzani
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Martin Langer
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Guido Bertolini
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Daniele Poole.

Additional information

On behalf of the GiViTI group Gruppo Italiano per la Valutazione degli Interventi in Terapia Intensiva (Italian Group for the Evaluation of Interventions in Intensive Care Medicine). A complete list of study participants appears in the appendix.

Appendix

Appendix

List of participating clinicians (in alphabetic order, with their location in brackets): Acciarri Claudia (Ascoli Piceno), Acquarolo Annamaria (Brescia), Adorni Adele (Como), Alberti Arnaldo (Porto Viro, RO), Alborghetti Armando (Ponte San Pietro, BG), Antonini Benvenuto (Manerbio, BS), Archi Davide (Lodi), Arditi Enrico (Genova), Badii Flavio (San Donà di Piave, VE), Balata Andrea (Sassari), Barattini Massimo (Firenze), Barbagli Remo (Firenze), Barbieri Paolo (Repubblica di San Marino), Baroncia Monica (Roma), Bartoccini Arcangelo (Borgomanero, NO), Bartoli Teresa (Antella, FI), Bassi Francesco (Cinisello Balsamo, MI), Beck Eduardo (Desio, MI), Bellato Valentina (Rozzano, MI), Bellorini Mario (Tradate, VA), Belluomo Anello Carla (Napoli), Benanti Cesare (Pescia, PT), Berardino Maurizio (Torino), Bernasconi Mara Olga (Rovigo), Besozzi Alessandra (Busto Arsizio, VA), Bianchin Andrea (Montebelluna, TV), Bindi Maria Lucia (Pisa), Biolino Piera (Chivasso, TO), Blasetti Angelo (Avezzano, AQ), Boccalatte Daniela (Lucca), Bonaccorso Giuseppina (Padova), Bonfà Andrea (Prato), Bonfiglio Monica (Lavagna, GE), Boniotti Corinna (Brescia), Bonizzoli Manuela (Firenze), Bottari Valter (Reggio Emilia), Breschi Cesare (Pesaro), Brunori Emanuela (Macerata), Buonanno Roberto (Lacco Ameno, NA), Buzzetti Virginio (Orbetello Scalo, GR), Calicchio Giuseppe (Salerno), Calva Sally (Torino), Capannolo Benita (L’Aquila), Caracciolo Adalgisa (Acquaviva delle Fonti, BA), Carnevale Livio (Pavia), Carsana Cristina (Busto Arsizio, VA), Casadei Edith (Siena), Casagli Sergio (Pisa), Casagrande Lucia (Treviso), Casalini Pierpaolo (Faenza, RA), Castiglione Giacomo (Catania), Cavallo Renata (Torino), Chiarello Marco (Camerino, MC), Chieregato Arturo (Cesena), Chinelli Elena (Bologna), Cigada Marco (Milano), Coaloa Maddalena (Savigliano, CN), Colombo Riccardo (Milano), Colombo Sergio (Milano), Corsini Walter (Carrara), Corvi Davide (Paderno Dugnano, MI), Cosentino Ercole (Cetraro, CS), Costa Roberto (Patti, ME), Costanzo Eleonora (Asti), Cottignoli Tito (Lugo di Romagna, RA), Crema Luciano (Cremona), Crestan Ezio (Lecco), Da Re Dolores (Este, PD), Dal Cero Paolo (Conegliano, TV), Dal Ferro Marino (Oderzo, TV), David Antonio (Messina), De Blasi Roberto Alberto (Roma), De Blasio Elvio (Benevento), De Luca Alessandra (Firenze), De Negri Pasquale (Rionero in Vulture, PZ), Dei Poli Marco (San Donato Milanese, MI), Del Sarto Paolo (Massa), Di Lorenzo Graziana (Chieti), Di Masi Pierfrancesco (Castellana Grotte, BA), Di Pasquale Dino (Pontedera, PI), Di Serafino Giampiero (Fermo, AP), Donato Ivan (Cantù, CO), Fabbri Lea (Firenze), Fabbri Emilio (Forlì), Fabbri Pier Giorgio (Palermo), Fabi Maria Cristina (Fano, PU), Faccio Luciana (Torino), Ferla Francesco (Cefalù, PA), Ferri Francesco (Bergamo), Fiore Gilberto (Moncalieri, TO), Fognani Giuliana (Bibbiena, AR), Galeotti Elsa (Feltre, BL), Gamberini Emiliano (Cesena), Garelli Alberto (Ravenna), Garofalo Giuseppe (Catania), Garzilli Tiziana (Perugia), Giacopuzzi Luigi (Negrar, VR), Gianni Massimo (Aosta), Giannoni Stefano (Empoli, FI), Giugiaro Pier Mario (Cirie’, TO), Gorietti Adonella (Perugia), Grassi Paolo (Figline Valdarno, FI), Greco Maurizio (Ariano Irpino, AV), Guadagnucci Alberto (Massa), Guagliardi Clementina (Gallarate, VA), Isetta Michele (Genova), Lapolla Antonio (Genova), Lefons Ugo (Poggibonsi, SI), Livigni Sergio (Torino), Madonna Roberto (Grosseto), Maestrone Carlo (Domodossola, VB), Magatti Maria Federica (Como), Maggiolo Carlo (Adria, RO), Malacarne Paolo (Pisa), Mancosu Stefano (Cagliari), Mangani Valerio (Firenze), Mantovani Giorgio (Ferrara), Marafon Silvio (Vicenza), Martínez Escobar Ricardo (Brescia), Mastroianni Alessandro (Chieri, TO), Mastropierro Rosa (Brescia), Meloni Alessandra (Sassari), Messina Marianna (Olbia), Miglioranzi Renzo (Pieve di Coriano, MN), Molino Fausto Maria (Roma), Morigi Aristide (Bologna), Mosca Carlo (Pisa), Murri Viscardo (Lanciano, CH), Nardi Giuseppe (Roma), Nascimben Ennio (Treviso), Natalini Giuseppe (Brescia), Negri Giovanni (Magenta, MI), Negro Giancarlo (Casarano, LE), Neri Massimo (Bologna), Nicolini Andrea (Lido di Camaiore, LU), Nuovo Domenico (Torino), Odetto Lorenzo (Orbassano, TO), Olivieri Carlo (Novara), Ortoleva Antonino (Ome, BS), Pacini Daniela Maria (Genova), Paganini Giorgio (Sanremo, IM), Paganoni Guido (Bergamo), Palmer Maurizio (Aversa, CE), Papiri Silvano (San Benedetto del Tronto, AP), Parma Alberto (Milano), Parrini Vieri (Borgo San Lorenzo, FI), Pastorelli Mauro (Pinerolo, TO), Pastorini Simonetta (Camposampiero, PD), Pegoraro Maurizio (Castelfranco Veneto, TV), Pezzi Angelo (Milano), Pinciroli Donatella (Legnano, MI), Piredda Giuseppe (Sassari), Piva Simone (Brescia), Pizzaballa Marialuisa (Zingonia, BG), Poole Daniele (Belluno), Possamai Clemente (Conegliano, TV), Postiglione Maurizio (Napoli), Potalivo Antonella (Bologna), Pulici Marco (Milano), Quattrocchi Pasqualino (Catania), Raffaeli Massimo (Milano), Ranzini Luisa (Milano), Righini Erminio (Lagosanto, FE), Rona Roberto (Monza, MI), Rossi Giancarlo (Livorno), Rossi Simona (Rho, MI), Rossi Maurizio (Menaggio, CO), Rottoli Federica (Bergamo), Salcuni Rosa (Ivrea, TO), Salvi Giovanni (Imperia), Savioli Monica (Milano), Scarrone Silvia (Alessandria), Schellino Maria Maddalena (Torino), Segala Vincenzo (Torino), Segalini Piero Aurelio (Piacenza), Sera Giuseppe (Reggio Calabria), Sforza Domenico (San Severo, FG), Sicignano Alberto (Milano), Sorbara Carlo (Treviso), Spadini Elisabetta (Parma), Staccioli Paola (Pistoia), Stefani Maurizio (Pisa), Sucre Maria Josè (Castellammare di Stabia, NA), Tartari Stefano (Trecenta, RO), Tavola Mario (Lecco), Tetamo Romano (Palermo), Tibaldi Giuseppe (Brescia), Tinacci Silvia (Fucecchio, FI), Todesco Livio (Cittadella, PD), Torta Mauro (Torino), Tosi Luigi (Pontremoli, MS), Tripepi Carlo (Monselice, PD), Ughi Ludovica (Milano), Vaj Monica (Torino), Vecchiarelli Pietro (Viterbo), Vespignani Maria Giovanna (Imola, BO), Visconti Maria Grazia (Cernusco sul Naviglio, MI), Vulcano Giuseppe Angelo (Rossano, CS), Zappa Sergio (Brescia), Zocaro Rosamaria (Pescara), Zuccaro Franco (Matera).

Rights and permissions

Reprints and permissions

About this article

Cite this article

Poole, D., Rossi, C., Anghileri, A. et al. External validation of the Simplified Acute Physiology Score (SAPS) 3 in a cohort of 28,357 patients from 147 Italian intensive care units. Intensive Care Med 35, 1916–1924 (2009). https://doi.org/10.1007/s00134-009-1615-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00134-009-1615-0

Keywords

Search

Navigation