Intensive Care Medicine

, Volume 43, Issue 8, pp 1123–1131 | Cite as

External validation of a biomarker and clinical prediction model for hospital mortality in acute respiratory distress syndrome

  • Zhiguo Zhao
  • Nancy Wickersham
  • Kirsten N. Kangelaris
  • Addison K. May
  • Gordon R. Bernard
  • Michael A. Matthay
  • Carolyn S. Calfee
  • Tatsuki Koyama
  • Lorraine B. WareEmail author



Mortality prediction in ARDS is important for prognostication and risk stratification. However, no prediction models have been independently validated. A combination of two biomarkers with age and APACHE III was superior in predicting mortality in the NHLBI ARDSNet ALVEOLI trial. We validated this prediction tool in two clinical trials and an observational cohort.


The validation cohorts included 849 patients from the NHLBI ARDSNet Fluid and Catheter Treatment Trial (FACTT), 144 patients from a clinical trial of sivelestat for ARDS (STRIVE), and 545 ARDS patients from the VALID observational cohort study. To evaluate the performance of the prediction model, the area under the receiver operating characteristic curve (AUC), model discrimination, and calibration were assessed, and recalibration methods were applied.


The biomarker/clinical prediction model performed well in all cohorts. Performance was better in the clinical trials with an AUC of 0.74 (95% CI 0.70–0.79) in FACTT, compared to 0.72 (95% CI 0.67–0.77) in VALID, a more heterogeneous observational cohort. The AUC was 0.73 (95% CI 0.70–0.76) when FACTT and VALID were combined.


We validated a mortality prediction model for ARDS that includes age, APACHE III, surfactant protein D, and interleukin-8 in a variety of clinical settings. Although the model performance as measured by AUC was lower than in the original model derivation cohort, the biomarker/clinical model still performed well and may be useful for risk assessment for clinical trial enrollment, an issue of increasing importance as ARDS mortality declines, and better methods are needed for selection of the most severely ill patients for inclusion.


Validation Prediction Biomarker Hospital mortality ARDS 



American European Consensus Conference


Acute lung injury


Acute physiology and chronic health evaluation


Acute respiratory distress syndrome


Area under receiver operating characteristic curve


Confidence interval


Fluid and catheter treatment trial


Intensive care unit


Integrated discrimination improvement


National Heart, Lung, and Blood Institute


Net reclassification improvement


Positive end-expiratory pressure


Receiver operating characteristic curve


Risk ratio


Sivelestat trial in ALI patients requiring mechanical ventilation


Validating acute lung injury biomarkers for diagnosis


Ventilator-free days



This manuscript was prepared using FACTT Research Materials obtained from the NHLBI Biologic Specimen and Data Repository Information Coordinating Center and does not necessarily reflect the opinions or views of the FACTT investigators or the NHLBI. We would like to thank the NHLBI BioLINCC/Biorepository for providing clinical samples and clinical data from the FACTT clinical trial. We also thank Eli Lilly and Company for providing clinical data and plasma samples from the STRIVE study.

Compliance with ethical standards


This study was supported by NIH HL112656 (LBW), 1K23HL116800-01 (KK), HL51856 (MAM), HL110969 and HL131621 (CSC), and the NHLBI BioLINCC/Biorepository.

Conflicts of interest

The authors declare that they have no conflict of interest.

Ethical approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Informed consent

Informed consent was obtained from all individual participants included in the study with the exception of some patients in the VALID study who were enrolled under an IRB-approved waiver of informed consent.

Supplementary material

134_2017_4854_MOESM1_ESM.pdf (76 kb)
Supplementary material 1 (PDF 75 kb)
134_2017_4854_MOESM2_ESM.tiff (27 mb)
Supplementary material 2 (TIFF 27685 kb)
134_2017_4854_MOESM3_ESM.doc (144 kb)
Supplementary material 3 (DOC 144 kb)


  1. 1.
    Rubenfeld GD, Caldwell E, Peabody E, Weaver J, Martin DP, Neff M, Stern EJ, Hudson LD (2005) Incidence and outcomes of acute lung injury. N Engl J Med 353:1685–1693CrossRefPubMedGoogle Scholar
  2. 2.
    Calfee CS, Delucchi K, Parsons PE, Thompson BT, Ware LB, Matthay MA, NHLBI ARDS Network (2014) Subphenotypes in acute respiratory distress syndrome: latent class analysis of data from two randomised controlled trials. Lancet Respir Med 2:611–620CrossRefPubMedPubMedCentralGoogle Scholar
  3. 3.
    Brower RG, Matthay MA, Morris A, Schoenfeld D, Thompson BT, Wheeler A, Acute Respiratory Distress Syndrome Network (2000) Ventilation with lower tidal volumes as compared with traditional tidal volumes for acute lung injury and the acute respiratory distress syndrome. N Engl J Med 342:1301–1308CrossRefPubMedGoogle Scholar
  4. 4.
    Takeda S, Ishizaka A, Fujino Y, Fukuoka T, Nagano O, Yamada Y, Takezawa J, Multicenter Clinical Trail Committee, Japan Society of Respiratory Care Medicine (2005) Time to change diagnostic criteria of ARDS: towards the disease entity-based subgrouping. Pulm Pharmacol Ther 18:115–119CrossRefPubMedGoogle Scholar
  5. 5.
    Ospina-Tascon GA, Buchele GL, Vincent JL (2008) Multicenter, randomized, controlled trials evaluating mortality in intensive care: doomed to fail? Crit Care Med 36:1311–1322CrossRefPubMedGoogle Scholar
  6. 6.
    Prescott HC, Calfee CS, Thompson BT, Angus DC, Liu VX (2016) Toward smarter lumping and smarter splitting: rethinking strategies for sepsis and acute respiratory distress syndrome clinical trial design. Am J Respir Crit Care Med 194:147–155CrossRefPubMedGoogle Scholar
  7. 7.
    Ware LB (2005) Prognostic determinants of acute respiratory distress syndrome in adults: impact on clinical trial design. Crit Care Med 33:S217–222CrossRefPubMedGoogle Scholar
  8. 8.
    Brun-Buisson C, Minelli C, Bertolini G, Brazzi L, Pimentel J, Lewandowski K, Bion J, Romand JA, Villar J, Thorsteinsson A, Damas P, Armaganidis A, Lemaire F, Alive Study Group (2004) Epidemiology and outcome of acute lung injury in European intensive care units. Results from the ALIVE study. Intensive Care Med 30:51–61CrossRefPubMedGoogle Scholar
  9. 9.
    Doyle RL, Szaflarski N, Modin GW, Wiener-Kronish JP, Matthay MA (1995) Identification of patients with acute lung injury. Predictors of mortality. Am J Respir Crit Care Med 152:1818–1824CrossRefPubMedGoogle Scholar
  10. 10.
    Monchi M, Bellenfant F, Cariou A, Joly LM, Thebert D, Laurent I, Dhainaut JF, Brunet F (1998) Early predictive factors of survival in the acute respiratory distress syndrome. A multivariate analysis. Am J Respir Crit Care Med 158:1076–1081CrossRefPubMedGoogle Scholar
  11. 11.
    Nuckton TJ, Alonso JA, Kallet RH, Daniel BM, Pittet JF, Eisner MD, Matthay MA (2002) Pulmonary dead-space fraction as a risk factor for death in the acute respiratory distress syndrome. N Engl J Med 346:1281–1286CrossRefPubMedGoogle Scholar
  12. 12.
    Venet C, Guyomarc’h S, Pingat J, Michard C, Laporte S, Bertrand M, Gery P, Page D, Vermesch R, Bertrand JC, Zeni F (2003) Prognostic factors in acute respiratory distress syndrome: a retrospective multivariate analysis including prone positioning in management strategy. Intensive Care Med 29:1435–1441CrossRefPubMedGoogle Scholar
  13. 13.
    Ware JH (2006) The limitations of risk factors as prognostic tools. N Engl J Med 355:2615–2617CrossRefPubMedGoogle Scholar
  14. 14.
    Ware LB, Matthay MA (2000) The acute respiratory distress syndrome. N Engl J Med 342:1334–1349CrossRefPubMedGoogle Scholar
  15. 15.
    Sapru A, Calfee CS, Liu KD, Kangelaris K, Hansen H, Pawlikowska L, Ware LB, Alkhouli MF, Abbott J, Matthay MA, NHLBI ARDS Network (2015) Plasma soluble thrombomodulin levels are associated with mortality in the acute respiratory distress syndrome. Intensive Care Med 41:470–478CrossRefPubMedPubMedCentralGoogle Scholar
  16. 16.
    Geboers DG, de Beer FM, Tuip-de Boer AM, van der Poll T, Horn J, Cremer OL, Bonten MJ, Ong DS, Schultz MJ, Bos LD (2015) Plasma suPAR as a prognostic biological marker for ICU mortality in ARDS patients. Intensive Care Med 41:1281–1290CrossRefPubMedGoogle Scholar
  17. 17.
    Laffey JG, Bellani G, Pham T, Fan E, Madotto F, Bajwa EK, Brochard L, Clarkson K, Esteban A, Gattinoni L, van Haren F, Heunks LM, Kurahashi K, Laake JH, Larsson A, McAuley DF, McNamee L, Nin N, Qiu H, Ranieri M, Rubenfeld GD, Thompson BT, Wrigge H, Slutsky AS, Pesenti A, LUNG SAFE Investigators and the ESICM Trials Group (2016) Potentially modifiable factors contributing to outcome from acute respiratory distress syndrome: the LUNG SAFE study. Intensive Care Med 42:1865–1876CrossRefPubMedGoogle Scholar
  18. 18.
    Cooke CR, Kahn JM, Caldwell E, Okamoto VN, Heckbert SR, Hudson LD, Rubenfeld GD (2008) Predictors of hospital mortality in a population-based cohort of patients with acute lung injury. Crit Care Med 36:1412–1420CrossRefPubMedGoogle Scholar
  19. 19.
    Brown LM, Calfee CS, Matthay MA, Brower RG, Thompson BT, Checkley W, The National Institutes of Health Acute Respiratory Distress Syndrome Network (2011) A simple classification model for hospital mortality in patients with acute lung injury managed with lung protective ventilation. Crit Care Med 39:2645–2651CrossRefPubMedPubMedCentralGoogle Scholar
  20. 20.
    Cooke CR, Shah CV, Gallop R, Bellamy S, Ancukiewicz M, Eisner MD, Lanken PN, Localio AR, Christie JD, National Heart Lung Blood Institute Acute Respiratory Distress Syndrome Network (2009) A simple clinical predictive index for objective estimates of mortality in acute lung injury. Crit Care Med 37:1913–1920CrossRefPubMedPubMedCentralGoogle Scholar
  21. 21.
    Knaus WA, Wagner DP, Draper EA, Zimmerman JE, Bergner M, Bastos PG, Sirio CA, Murphy DJ, Lotring T, Damiano A et al (1991) The APACHE III prognostic system. Risk prediction of hospital mortality for critically ill hospitalized adults. Chest 100:1619–1636CrossRefPubMedGoogle Scholar
  22. 22.
    Moreno RP, Metnitz PG, Metnitz B, Bauer P, Afonso de Carvalho S, Hoechtl A, SAPS 3 Investigators (2008) Modeling in-hospital patient survival during the first 28 days after intensive care unit admission: a prognostic model for clinical trials in general critically ill patients. J Crit Care 23:339–348CrossRefPubMedGoogle Scholar
  23. 23.
    Villar J, Ambros A, Soler JA, Martinez D, Ferrando C, Solano R, Mosteiro F, Blanco J, Martin-Rodriguez C, Fernandez MM, Lopez J, Diaz-Dominguez FJ, Andaluz-Ojeda D, Merayo E, Perez-Mendez L, Fernandez RL, Kacmarek RM, Stratification Outcome of Acute Respiratory Distress Syndrome Network (2016) Age, PaO2/FIO2, and plateau pressure score: a proposal for a simple outcome score in patients with the acute respiratory distress syndrome. Crit Care Med 44:1361–1369CrossRefPubMedGoogle Scholar
  24. 24.
    Bos LD, Schouten LR, Cremer OL, Ong DS, Schultz MJ, MARS Consortium (2016) External validation of the APPS, a new and simple outcome prediction score in patients with the acute respiratory distress syndrome. Ann Intensive Care 6:89CrossRefPubMedPubMedCentralGoogle Scholar
  25. 25.
    Ware LB, Koyama T, Billheimer DD, Wu W, Bernard GR, Thompson BT, Brower RG, Standiford TJ, Martin TR, Matthay MA, NHLBI ARDS Clinical Trials Network (2010) Prognostic and pathogenetic value of combining clinical and biochemical indices in patients with acute lung injury. Chest 137:288–296CrossRefPubMedGoogle Scholar
  26. 26.
    Ware LB, Zhao Z, Koyama T, Wickersham N, Kangelaris KN, Bernard GR, Matthay MA, Calfee CS (2015) Validation of a biomarker and clinical prediction model for mortality in ARDS in three patient cohorts. Am J Respir Crit Care Med 191:A6249Google Scholar
  27. 27.
    Brower RG, Lanken PN, MacIntyre N, Matthay MA, Morris A, Ancukiewicz M, Schoenfeld D, Thompson BT, The National Heart Lung Blood Institute Ards Clinical Trials Network (2004) Higher versus lower positive end-expiratory pressures in patients with the acute respiratory distress syndrome. N Engl J Med 351:327–336CrossRefPubMedGoogle Scholar
  28. 28.
    Wiedemann HP, Wheeler AP, Bernard GR, Thompson BT, Hayden D, deBoisblanc B, Connors AF Jr, Hite RD, Harabin AL, The National Heart Lung and Blood Institute Acute Respiratory Distress Syndrome Clinical Trials Network (2006) Comparison of two fluid-management strategies in acute lung injury. N Engl J Med 354:2564–2575CrossRefPubMedGoogle Scholar
  29. 29.
    Zeiher BG, Artigas A, Vincent JL, Dmitrienko A, Jackson K, Thompson BT, Bernard G, Strive Study Group (2004) Neutrophil elastase inhibition in acute lung injury: results of the STRIVE study. Crit Care Med 32:1695–1702CrossRefPubMedGoogle Scholar
  30. 30.
    O’Neal HR Jr, Koyama T, Koehler EA, Siew E, Curtis BR, Fremont RD, May AK, Bernard GR, Ware LB (2011) Prehospital statin and aspirin use and the prevalence of severe sepsis and acute lung injury/acute respiratory distress syndrome. Crit Care Med 39:1343–1350CrossRefPubMedPubMedCentralGoogle Scholar
  31. 31.
    Bernard GR, Artigas A, Brigham KL, Carlet J, Falke K, Hudson L, Lamy M, Legall JR, Spragg R (1994) The American-European Consensus Conference on ARDS. Definitions, mechanisms, relevant outcomes, and clinical trial coordination. Am J Respir Crit Care Med 149:818–824CrossRefPubMedGoogle Scholar
  32. 32.
    ARDS Definition Task Force, Ranieri VM, Rubenfeld GD, Thompson BT, Ferguson ND, Caldwell E, Fan E, Camporota L, Slutsky AS (2012) Acute respiratory distress syndrome: the Berlin definition. JAMA 307:2526–2533Google Scholar
  33. 33.
    Schneider AG, Lipcsey M, Bailey M, Pilcher DV, Bellomo R (2013) Simple translational equations to compare illness severity scores in intensive care trials. J Crit Care 28(885):e881–888Google Scholar
  34. 34.
    Harrell FE Jr, Lee KL, Mark DB (1996) Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat Med 15:361–387CrossRefPubMedGoogle Scholar
  35. 35.
    Janssen KJ, Moons KG, Kalkman CJ, Grobbee DE, Vergouwe Y (2008) Updating methods improved the performance of a clinical prediction model in new patients. J Clin Epidemiol 61:76–86CrossRefPubMedGoogle Scholar
  36. 36.
    Sigurdsson MI, Sigvaldason K, Gunnarsson TS, Moller A, Sigurdsson GH (2013) Acute respiratory distress syndrome: nationwide changes in incidence, treatment and mortality over 23 years. Acta Anaesthesiol Scand 57:37–45CrossRefPubMedGoogle Scholar
  37. 37.
    Frank AJ, Thompson BT (2010) Pharmacological treatments for acute respiratory distress syndrome. Curr Opin Crit Care 16:62–68CrossRefPubMedGoogle Scholar
  38. 38.
    Duggal A, Ganapathy A, Ratnapalan M, Adhikari NK (2015) Pharmacological treatments for acute respiratory distress syndrome: systematic review. Miner Anestesiol 81:567–588Google Scholar
  39. 39.
    Famous KR, Delucchi K, Ware LB, Kangelaris KN, Liu KD, Thompson BT, Calfee CS, ARDS Network (2017) Acute respiratory distress syndrome subphenotypes respond differently to randomized fluid management strategy. Am J Respir Crit Care Med 195:331–338PubMedGoogle Scholar
  40. 40.
    Taylor JM, Ankerst DP, Andridge RR (2008) Validation of biomarker-based risk prediction models. Clin Cancer Res 14:5977–5983CrossRefPubMedPubMedCentralGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg and ESICM 2017

Authors and Affiliations

  • Zhiguo Zhao
    • 1
    • 2
  • Nancy Wickersham
    • 3
  • Kirsten N. Kangelaris
    • 4
  • Addison K. May
    • 5
  • Gordon R. Bernard
    • 3
  • Michael A. Matthay
    • 6
    • 7
  • Carolyn S. Calfee
    • 6
    • 7
  • Tatsuki Koyama
    • 1
  • Lorraine B. Ware
    • 3
    • 8
    Email author
  1. 1.Department of BiostatisticsVanderbilt UniversityNashvilleUSA
  2. 2.The Institution for Medicine and Public HealthVanderbilt UniversityNashvilleUSA
  3. 3.Division of Allergy, Pulmonary and Critical Care Medicine, Department of MedicineVanderbilt UniversityNashvilleUSA
  4. 4.Division of Hospital Medicine, Department of MedicineUniversity of California San FranciscoSan FranciscoUSA
  5. 5.Division of Trauma and Surgical Critical CareVanderbilt UniversityNashvilleUSA
  6. 6.Department of MedicineUniversity of California San FranciscoSan FranciscoUSA
  7. 7.Department of Anesthesia and Cardiovascular Research InstituteUniversity of California San FranciscoSan FranciscoUSA
  8. 8.Department of Pathology, Microbiology and ImmunologyVanderbilt UniversityNashvilleUSA

Personalised recommendations