Abstract
Objective
To evaluate whether physiological trends and interaction terms could enhance predictive ability in a variant of the Richardson Score.
Method
We conducted a retrospective cohort study using data from 2,222 newborns of 34 weeks or more gestation with respiratory distress. The outcome variable was assisted ventilation longer 3 days or death. Using logistic regression and split validation we fit models for a variant of the Richardson score (gestational age, worst PaO2/FIO2 ratio, lowest MAP, and a single interaction variable relating the lowest pH and the highest PaCO2) as well as models incorporating variables for trends and additional interaction terms. We assessed discrimination using the c-statistic.
Results
The 24-h Richardson score had a c-statistic of 0.83. In the validation dataset, adding pH trend significantly increased the c-statistic to 0.87. Adding PaO2/FIO2 ratio trend increased the c-statistic to 0.86. Interactions with high significance were present in the data (e.g., adding all two-way interactions to our best trend model yielded a c-statistic of 0.92) but were unstably estimated with n = 2,222.
Conclusions
Incorporating trend and interaction terms in severity scores can enhance predictive ability. These modeling strategies have been underutilized in severity scoring, but in an era of increasing electronic availability of detailed clinical data the incorporation of trend and interaction effects in severity scoring will become both feasible and desirable.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Knaus WA, Draper EA, Wagner DP, Zimmerman JE (1985) APACHE II: a severity of disease classification system. Crit Care Med 13:818–829
Knaus WA, Wagner DP, Draper EA, Zimmerman JE, Bergner M, Bastos PG, Sirio CA, Murphy DJ, Lotring T, Damiano A (1991) The APACHE III prognostic system. Risk prediction of hospital mortality for critically ill hospitalized adults. Chest 100:1619–1636
Zimmerman JE, Kramer AA, McNair DS, Malila FM (2006) Acute Physiology and Chronic Health Evaluation (APACHE) IV: hospital mortality assessment for today's critically ill patients. Crit Care Med 34:1297–1310
Le Gall JR, Lemeshow S, Saulnier F (1993) A new Simplified Acute Physiology Score (SAPS II) based on a European/North American multicenter study. JAMA 270:2957–2963
Lemeshow S, Teres D, Klar J, Avrunin JS, Gehlbach SH, Rapoport J (1993) Mortality Probability Models (MPM II) based on an international cohort of intensive care unit patients. JAMA 270:2478–2486
Pollack MM, Ruttimann UE, Getson PR (1988) Pediatric risk of mortality (PRISM) score. Crit Care Med 16:1110–1116
Pollack MM, Patel KM, Ruttimann UE (1997) The Pediatric Risk of Mortality III-Acute Physiology Score (PRISM III-APS): a method of assessing physiologic instability for pediatric intensive care unit patients. J Pediatr 131:575–581
Shann F, Pearson G, Slater A, Wilkinson K (1997) Paediatric index of mortality (PIM): a mortality prediction model for children in intensive care. Intensive Care Med 23:201–207
Richardson DK, Gray JE, McCormick MC, Workman K, Goldmann DA (1993) Score for Neonatal Acute Physiology: a physiologic severity index for neonatal intensive care. Pediatrics 91:617–623
Richardson D, Corcoran J, Escobar G, Lee S (2001) SNAP-II and SNAPPE-II: simplified newborn illness severity and mortality risk scores. J Pediatr 138:92–100
Escobar G, Shaheen S, Breed E, Botas C, Greene JD, Yoshida CK, Zupancic J, Newman TB (2004) Richardson score predicts short-term adverse respiratory outcomes in newborns ≥ 34 weeks gestation. J Pediatr 145:754–760
Selby JV (1997) Linking automated databases for research in managed care settings. Ann Intern Med 127:719–724
Wilson A, Gardner M, Armstrong M, Folck B, Escobar G (2000) Neonatal assisted ventilation: predictors, frequency, and duration in a mature managed care organization. Pediatrics 105:822–830
Escobar GJ, Li DK, Armstrong MA, Gardner MN, Folck BF, Verdi JE, Xiong B, Bergen R (2000) Neonatal sepsis workups in babies ≥ 2000 grams at birth: A population-based study. Pediatrics 106:256–263
Metz CE (1978) Basic principles of ROC analysis. Semin Nucl Med 8:283–298
Hanley J, McNeil B (1982) The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 143:29–36
Iezzoni AA, Schwartz M (1997) Risk adjustment for measuring health care outcomes. Health Administration, Chicago, pp 207–232
DeLong ER, DeLong DM, Clarke-Pearson DL (1988) Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics 44:837–845
Lemeshow S, Hosmer DW Jr (1982) A review of goodness of fit statistics for use in the development of logistic regression models. Am J Epidemiol 115:92–106
Stone M (1974) Cross-validatory choice and assessment of statistical predictions. J R Stat Soc Ser B:111–147
Meredith JW, Kilgo PD, Osler TM (2003) Independently derived survival risk ratios yield better estimates of survival than traditional survival risk ratios when using the ICISS. J Trauma 55:933–938
Wasson JH, Sox HC, Neff RK, Goldman L (1985) Clinical prediction rules. Applications and methodological standards. N Engl J Med 313:793–799
Peduzzi P, Concato J, Kemper E, Holford TR, Feinstein AR (1996) A simulation study of the number of events per variable in logistic regression analysis. J Clin Epidemiol 49:1373–1379
Hedges L, Olkin I (1985) Statistical methods for meta-analysis. Academic, San Diego, pp 108–138
Fowlie PW, Gould CR, Tarnow-Mordi WO, Strang D (1998) Measurement properties of the Clinical Risk Index for Babies—reliabilty, validity beyond the first 12 hours, and responsiveness over 7 days. Crit Care Med 26:163–168
Stevens SM, Richardson DK, Gray JE, Goldmann DA, McCormick MC (1994) Estimating neonatal mortality risk: an analysis of clinicians' judgments. Pediatrics 93:945–950
Christensen C, Cottrell JJ, Murakami J, Mackesy ME, Fetzer AS, Elstein AS (1993) Forecasting survival in the medical intensive care unit: a comparison of clinical prognoses with formal estimates. Methods Infect Med 32:302–308
Griffin MP, Moorman JR (2001) Toward the early diagnosis of neonatal sepsis and sepsis-like illness using novel heart rate analysis. Pediatrics 107:97–104
Griffin MP, Lake DE, Moorman JR (2005) Heart rate characteristics and laboratory tests in neonatal sepsis. Pediatrics 115:937–941
Griffin MP, Lake DE, Bissonette EA, Harrell FE Jr, O'Shea TM, Moorman JR (2005) Heart rate characteristics: novel physiomarkers to predict neonatal infection and death. Pediatrics 116:1070–1074
Afessa B, Keegan MT, Gajic O, Hubmayr RD, Peters SG (2005) The influence of missing components of the Acute Physiology Score of APACHE III on the measurement of ICU performance. Intensive Care Med 31:1537–1543
Author information
Authors and Affiliations
Corresponding author
Additional information
M. K. was supported by a grant from the Glaser Pediatric Research Network. D. D. and G. J. E were supported by the Permanente Medical Group, Inc. The Kaiser Permanente Neonatal Minimum Data Set database is supported by Kaiser Foundation Hospitals, Inc., and the Permanente Medical Group, Inc.
This article is discussed in the editorial available at: http://dx.doi.org/10.1007/s00134-007-0736-6.
Rights and permissions
About this article
Cite this article
Kuzniewicz, M., Draper, D. & Escobar, G.J. Incorporation of physiological trend and interaction effects in neonatal severity of illness scores: an experiment using a variant of the Richardson score. Intensive Care Med 33, 1602–1608 (2007). https://doi.org/10.1007/s00134-007-0714-z
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00134-007-0714-z