Prediction of Incident Delirium Using a Random Forest classifier
- 172 Downloads
Delirium is a serious medical complication associated with poor outcomes. Given the complexity of the syndrome, prevention and early detection are critical in mitigating its effects. We used Confusion Assessment Method (CAM) screening and Electronic Health Record (EHR) data for 64,038 inpatient visits to train and test a model predicting delirium arising in hospital. Incident delirium was defined as the first instance of a positive CAM occurring at least 48 h into a hospital stay. A Random Forest machine learning algorithm was used with demographic data, comorbidities, medications, procedures, and physiological measures. The data set was randomly partitioned 80% / 20% for training and validating the predictive model, respectively. Of the 51,240 patients in the training set, 2774 (5.4%) experienced delirium during their hospital stay; and of the 12,798 patients in the validation set, 701 (5.5%) experienced delirium. Under-sampling of the delirium negative population was used to address the class imbalance. The Random Forest predictive model yielded an area under the receiver operating characteristic curve (ROC AUC) of 0.909 (95% CI 0.898 to 0.921). Important variables in the model included previously identified predisposing and precipitating risk factors. This machine learning approach displayed a high degree of accuracy and has the potential to provide a clinically useful predictive model for earlier intervention in those patients at greatest risk of developing delirium.
KeywordsDelirium Prediction Decision support Machine learning Random forest
Compliance with Ethical Standards
Conflicts of Interest
The authors do not have any conflicts of interest to declare. This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.
All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.
For this type of study formal consent is not required.
- 17.van den Boogaard, M., Pickkers, P., Slooter, A. J. C. et al., Development and validation of PRE-DELIRIC (PREdiction of DELIRium in ICu patients) delirium prediction model for intensive care patients: observational multicentre study. BMJ 344:e420–e420, 2012. https://doi.org/10.1136/bmj.e420.CrossRefPubMedPubMedCentralGoogle Scholar
- 22.Wasey JO (2016) ICD: Tools for Working with ICD-9 and ICD-10 Codes, and Finding ComorbiditiesGoogle Scholar
- 23.Team TH a. (2017) h2o: R Interface for H2OGoogle Scholar
- 24.Chen C, Liaw A, Breiman L (2004) Using random forest to learn imbalanced data. Univ California, Berkeley 1–12. doi: ley.edu/sites/default/files/tech-reports/666.pdfGoogle Scholar
- 26.Platt, J., Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. Adv Large Margin Classif 10:61–74, 1999. 10.1.1.41.1639.Google Scholar
- 27.Kuhn M (2008) Building Predictive Models in R Using the caret Package. J Stat Software, Artic 28:1–26. https://doi.org/10.18637/jss.v028.i05.
- 34.Morandi, A., Han, J. H., Meagher, D. et al., Detecting Delirium Superimposed on Dementia: Evaluation of the Diagnostic Performance of the Richmond Agitation and Sedation Scale. J. Am. Med. Dir. Assoc. 17:828–833, 2016. https://doi.org/10.1016/j.jamda.2016.05.010.CrossRefPubMedPubMedCentralGoogle Scholar
- 40.Kessler, R. C., Warner, C. H., Ivany, C. et al., Predicting Suicides After Psychiatric Hospitalization in US Army Soldiers: The Army Study to Assess Risk and Resilience in Servicemembers (Army STARRS). JAMA Psychiatry 72:1–9, 2014. https://doi.org/10.1001/jamapsychiatry.2014.1754.CrossRefGoogle Scholar
- 43.Kramer, D., Veeranki, S., Hayn, D., and Quehenberger, F., Development and Validation of a Multivariable Prediction Model for the Occurrence of Delirium in Hospitalized Gerontopsychiatry and Internal Medicine Patients. Stud Health Technol Inform:32–39, 2017. https://doi.org/10.3233/978-1-61499-759-7-32.
- 44.Wong, A., Young, A. T., Liang, A. S. et al., Development and Validation of an Electronic Health Record–Based Machine Learning Model to Estimate Delirium Risk in Newly Hospitalized Patients Without Known Cognitive Impairment. JAMA Netw. Open 1:e181018, 2018. https://doi.org/10.1001/jamanetworkopen.2018.1018.CrossRefGoogle Scholar