Abstract
Recently, we devised a method to develop prognostic models incorporating patterns of sequential organ failure to predict the eventual hospital mortality at each day of intensive care stay. In this study, we aimed to understand, using a real world setting, how these models perform compared to physicians, who are exposed to additional information than the models. We found a slightly better discriminative ability for physicians (AUC range over days: 0.73-0.83 vs. 0.70-0.80) and a slightly better accuracy for the models (Brier score range: 0.14-0.19 vs. 0.16-0.19). However when we combined both sources of predictions we arrived at a significantly superior discrimination as well as accuracy (AUC range: 0.81-0.88; Brier score range: 0.11-0.15). Our results show that the models and the physicians draw on complementary information that can be best harnessed by combining both prediction sources. Extensive external validation and impact studies are imperative to further investigate the ability of the combined model.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Lucas, P.J., Abu-Hanna, A.: Prognostic methods in medicine. Artif. Intell. Med. 15, 105–119 (1999)
Abu-Hanna, A., Lucas, P.J.: Prognostic models in medicine. AI and statistical approaches. Methods Inf. Med. 40, 1–5 (2001)
Fried, T.R., Bradley, E.H., Towle, V.R., Allore, H.: Understanding the treatment preferences of seriously ill patients. N. Engl. J. Med. 346, 1061–1066 (2002)
Murphy, D.J., Burrows, D., Santilli, S., Kemp, A.W., Tenner, S., Kreling, B., et al.: The influence of the probability of survival on patients’ preferences regarding cardiopulmonary resuscitation. N. Engl. J. Med. 330, 545–549 (1994)
Rocker, G., Cook, D., Sjokvist, P., Weaver, B., Finfer, S., McDonald, E., et al.: Clinician predictions of intensive care unit mortality. Crit. Care Med. 32, 1149–1154 (2004)
Whitecotton, S.M., Sanders, D.E., Norris, K.B.: Improving Predictive Accuracy with a Combination of Human Intuition and Mechanical Decision Aids. Organ Behav. Hum. Decis. Process 76, 325–348 (1998)
Knaus, W.A., Draper, E.A., Wagner, D.P., Zimmerman, J.E.: APACHE II: a severity of disease classification system. Crit. Care Med. 13, 818–829 (1985)
Le Gall, J.R., Lemeshow, S., Saulnier, F.: A new Simplified Acute Physiology Score (SAPS II) based on a European/North American multicenter study. JAMA 270, 2957–2963 (1993)
Lemeshow, S., Teres, D., Klar, J., Avrunin, J.S., Gehlbach, S.H., Rapoport, J.: Mortality Probability Models (MPM II) based on an international cohort of intensive care unit patients. JAMA 270, 2478–2486 (1993)
Christakis, N.A., Lamont, E.B.: Extent and determinants of error in doctors’ prognoses in terminally ill patients: prospective cohort study. BMJ 320, 469–472 (2000)
McClish, D.K., Powell, S.H.: How well can physicians estimate mortality in a medical intensive care unit? Med. Decis. Making 9, 125–132 (1989)
Poses, R.M., Bekes, C., Copare, F.J., Scott, W.E.: The answer to ”What are my chances, doctor?” depends on whom is asked: prognostic disagreement and inaccuracy for critically ill patients. Crit. Care Med. 17, 827–833 (1989)
Christakis, N.A., Iwashyna, T.J.: Attitude and self-reported practice regarding prognostication in a national sample of internists. Arch. Intern. Med. 158, 2389–2395 (1998)
Scholz, N., Basler, K., Saur, P., Burchardi, H., Felder, S.: Outcome prediction in critical care: physicians’ prognoses vs. scoring systems. Eur. J. Anaesthesiol. 21, 606–611 (2004)
Garrouste-Orgeas, M., Montuclard, L., Timsit, J.F., Misset, B., Christias, M., Carlet, J.: Triaging patients to the ICU: a pilot study of factors influencing admission decisions and patient outcomes. Intensive Care Med. 29, 774–781 (2003)
Knaus, W.A., Harrell Jr., F.E., Lynn, J., Goldman, L., Phillips, R.S., Connors Jr., A.F., et al.: he SUPPORT prognostic model. Objective estimates of survival for seriously ill hospitalized adults. Study to understand prognoses and preferences for outcomes and risks of treatments. Ann. Intern. Med. 122, 191–203 (1995)
Christensen, C., Cottrell, J.J., Murakami, J., Mackesy, M.E., Fetzer, A.S., Elstein, A.S.: Forecasting survival in the medical intensive care unit: a comparison of clinical prognoses with formal estimates. Methods Inf. Med. 32, 302–308 (1993)
Marks, R.J., Simons, R.S., Blizzard, R.A., Browne, D.R.: Predicting outcome in intensive therapy units–a comparison of Apache II with subjective assessments. Intensive Care Med. 17, 159–163 (1991)
Poses, R.M., Bekes, C., Copare, F.J., Scott, W.E.: What difference do two days make? The inertia of physicians’ sequential prognostic judgments for critically ill patients. Med. Decis. Making 10, 6–14 (1990)
Poses, R.M., Bekes, C., Winkler, R.L., Scott, W.E., Copare, F.J.: Are two (inexperienced) heads better than one (experienced) head? Averaging house officers’ prognostic judgments for critically ill patients. Arch. Intern. Med. 150, 1874–1878 (1990)
Poses, R.M., McClish, D.K., Bekes, C., Scott, W.E., Morley, J.N.: Ego bias, reverse ego bias, and physicians’ prognostic. Crit. Care Med. 19, 1533–1539 (1991)
McClish, D.K., Powell, S.H.: How well can physicians estimate mortality in a medical intensive care unit? Med. Decis. Making 9, 125–132 (1989)
Brannen, A.L., Godfrey, L.J., Goetter, W.E.: Prediction of outcome from critical illness. A comparison of clinical judgment with a prediction rule. Arch. Intern. Med. 149, 1083–1086 (1989)
Kruse, J.A., Thill-Baharozian, M.C., Carlson, R.W.: Comparison of clinical assessment with APACHE II for predicting mortality risk in patients admitted to a medical intensive care unit. JAMA 260, 1739–1742 (1988)
Sinuff, T., Adhikari, N.K., Cook, D.J., Schunemann, H.J., Griffith, L.E., Rocker, G., et al.: Mortality predictions in the intensive care unit: comparing physicians with scoring systems. Crit. Care Med. 34, 878–885 (2006)
Chang, R.W., Lee, B., Jacobs, S., Lee, B.: Accuracy of decisions to withdraw therapy in critically ill patients: clinical judgment versus a computer model. Crit. Care Med. 17, 1091–1097 (1989)
Meyer, A.A., Messick, W.J., Young, P., Baker, C.C., Fakhry, S., Muakkassa, F., et al.: Prospective comparison of clinical judgment and APACHE II score in predicting the outcome in critically ill surgical patients. J. Trauma 32, 747–753 (1992)
Toma, T., Bosman, R.J., Siebes, A., Peek, N., Abu-Hanna, A.: Learning predictive models that use pattern discovery–a bootstrap evaluative approach applied in organ functioning sequences. J. Biomed. Inform. 43, 578–586 (2010)
Toma, T., Abu-Hanna, A., Bosman, R.J.: Discovery and inclusion of SOFA score episodes in mortality prediction. J. Biomed. Inform. 40, 649–660 (2007)
Toma, T., Abu-Hanna, A., Bosman, R.J.: Discovery and integration of univariate patterns from daily individual organ-failure scores for intensive care mortality prediction. Artif. Intell. Med. 43, 47–60 (2008)
de Jonge, E., Bosman, R.J., van der Voort, P.H., Korsten, H.H., Scheffer, G.J., de Keizer, N.F.: [Intensive care medicine in the Netherlands, 1997-2001. I. Patient population and treatment outcome]. Ned Tijdschr Geneeskd 147, 1013–1017 (2003)
Fretschner, R., Bleicher, W., Heininger, A., Unertl, K.: Patient data management systems in critical care. J. Am. Soc. Nephrol. 12 suppl 17, S83–S86 (2001)
Minne, L., Abu-Hanna, A., de Jonge, E.: Evaluation of SOFA-based models for predicting mortality in the ICU: A systematic review. Crit. Care 12, R161 (2008)
Burnham, K.P., Anderson, D.R.: Model selection and multimodel inference: a practical-theoretic approach. Springer, Heidelberg (2002)
Frick, S., Uehlinger, D.E., Zuercher Zenklusen, R.M.: Medical futility: predicting outcome of intensive care unit patients by nurses and doctors–a prospective comparative study. Crit. Care Med. 31, 456–461 (2003)
Moons, K.G., Royston, P., Vergouwe, Y., Grobbee, D.E., Altman, D.G.: Prognosis and prognostic research: what, why, and how? BMJ 338, b375 (2009)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Minne, L., de Jonge, E., Abu-Hanna, A. (2011). Repeated Prognosis in the Intensive Care: How Well Do Physicians and Temporal Models Perform?. In: Peleg, M., Lavrač, N., Combi, C. (eds) Artificial Intelligence in Medicine. AIME 2011. Lecture Notes in Computer Science(), vol 6747. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-22218-4_29
Download citation
DOI: https://doi.org/10.1007/978-3-642-22218-4_29
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-22217-7
Online ISBN: 978-3-642-22218-4
eBook Packages: Computer ScienceComputer Science (R0)