Abstract
Hypoglycemia is a common occurrence in critically ill patients and is associated with significant mortality and morbidity. We developed a machine learning model to predict hypoglycemia by using a multicenter intensive care unit (ICU) electronic health record dataset. Machine learning algorithms were trained and tested on patient data from the publicly available eICU Collaborative Research Database. Forty-four features including patient demographics, laboratory test results, medications, and vitals sign recordings were considered. The outcome of interest was the occurrence of a hypoglycemic event (blood glucose < 72 mg/dL) during a patient’s ICU stay. Machine learning models used data prior to the second hour of the ICU stay to predict hypoglycemic outcome. Data from 61,575 patients who underwent 82,479 admissions at 199 hospitals were considered in the study. The best-performing predictive model was the eXtreme gradient boosting model (XGBoost), which achieved an area under the received operating curve (AUROC) of 0.85, a sensitivity of 0.76, and a specificity of 0.76. The machine learning model developed has strong discrimination and calibration for the prediction of hypoglycemia in ICU patients. Prospective trials of these models are required to evaluate their clinical utility in averting hypoglycemia within critically ill patient populations.
Similar content being viewed by others
Data availability
The data used in this paper is from the eICU-CRD database. The access to this dataset is controlled and researchers should request access on the PhysioNet website (https://physionet.org/about/database/). All code used in this study is available on a GitHub repository (www.github.com/SreekarMantena/hypoglycemia-modeling).
References
The NICE-SUGAR Study Investigators. Hypoglycemia and Risk of Death in Critically Ill Patients. New Engl J Med. 2012; 367:1108–18.
Hulkower RD, Pollack RM, Zonszein J. Understanding hypoglycemia in hospitalized patients. Diabetes Manag. 2014;4:165–76.
Brutsaert E, Carey M, Zonszein J. The clinical impact of inpatient hypoglycemia. J Diabetes Complications. 2014;28:565–72.
Krinsley JS, Grover A. Severe hypoglycemia in critically ill patients: Risk factors and outcomes. Crit Care Med. 2007;35:2262–7.
Cook CB, Kongable GL, Potter DJ, Abad VJ, Leija DE, Anderson M. Inpatient glucose control: a glycemic survey of 126 U.S. hospitals. J Hosp Med. 2009;4:E7–14.
Brodovicz KG, Mehta V, Zhang Q, Zhao C, Davies MJ, Chen J, et al. Association between hypoglycemia and inpatient mortality and length of hospital stay in hospitalized, insulin-treated patients. Curr Med Res Opin. 2013;29:101–7.
Kilpatrick CR, Elliott MB, Pratt E, Schafers SJ, Blackburn MC, Heard K, et al. Prevention of inpatient hypoglycemia with a real-time informatics alert. J Hosp Med. 2014;9:621–6.
Ruan Y, Bellot A, Moysova Z, Tan GD, Lumb A, Davies J, et al. Predicting the risk of inpatient hypoglycemia with machine learning using electronic health records. Diabetes Care. 2020;43:1504–11.
Mathioudakis NN, Everett E, Routh S, Pronovost PJ, Yeh H-C, Golden SH, et al. Development and validation of a prediction model for insulin-associated hypoglycemia in non-critically ill hospitalized adults. BMJ Open Diabetes Res Care. 2018;6:e000499.
Stuart K, Adderley NJ, Marshall T, Rayman G, Sitch A, Manley S, et al. Predicting inpatient hypoglycaemia in hospitalized patients with diabetes: a retrospective analysis of 9584 admissions with diabetes. Diabet Med. 2017;34:1385–91.
Elliott MB, Schafers SJ, McGill JB, Tobin GS. Prediction and prevention of treatment-related inpatient hypoglycemia. J Diabetes Sci Technol. 2012;6:302–9.
Pollard TJ, Johnson AEW, Raffa JD, Celi LA, Mark RG, Badawi O. The eICU collaborative research database, a freely available multi-center database for critical care research. Sci Data. 2018. https://doi.org/10.1038/sdata.2018.178.
Bertsimas D, Pawlowski C, Zhuo YD. From predictive methods to missing data imputation: an optimization approach. J Machine Learning Res. 2018;18(1):7133–71.
Chen T and Guestrin C. XGBoost: A scalable tree boosting system. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 2016; 785–794.
Steyerberg EW. Clinical prediction models: a practical approach to development, validation and updating. Cham: Springer; 2019.
McKinney W. Data Structures for Statistical Computing in Python. Proceedings of the 9th Python in Science Conference. 2010; 56–61.
van der Walt S, Colbert SC, Varoquaux G. The NumPy array: a structure for efficient numerical computation. Computing in Science and Engineering. 2011;13:22–30.
Pollard TJ, Johnson AEW, Raffa JD, Mark RG. tableone: an open source Python package for producing summary statistics for research papers. JAMIA Open. 2018;1:26–31.
Pedregosa F, Michel V, Grisel O, Blondel M, Prettenhofer P, Weiss R, et al. Scikit-learn: machine learning in Python. J Mach Learn Res. 2011;12:2825–30.
Lacherade JC, Jacqueminet S, Preiser JC. An overview of hypoglycemia in the critically Ill. J Diabetes Science and Technology. 2009;3:1242–9.
Pratiwi C, Mokoagow MI, Made Kshanti IA, Soewondo P. The risk factors of inpatient hypoglycemia: a systematic review. Heliyon. 2020;6(5):e03913.
Leibovitz E, Wainstein J, Boaz M. Association of albumin and cholesterol levels with incidence of hypoglycaemia in people admitted to general internal medicine units. Diabet Med. 2018;35:1735–41.
McCluskey A, Thomas AN, Bowles BJM, Kishen R. The prognostic value of serial measurements of serum albumin concentration in patients admitted to an intensive care unit. Anaesthesia. 1996;51:724–7.
Arem R. Hypoglycemia associated with renal failure. Endocrinol Metab Clin North Am. 1989;18:103–21.
Krinsley J, Schultz MJ, Spronk PE, van Braam Houckgeest F, van der Sluijs JP, Mélot C, et al. Mild hypoglycemia is strongly associated with increased intensive care unit length of stay. Ann Intensive Care. 2011. https://doi.org/10.1186/2110-5820-1-49.
Salinas PD, Mendez CE. Glucose management technologies for the critically ill. J Diabetes Sci Technol. 2019;13(4):682–90.
Finlayson SG, Subbaswamy A, Singh K, Bowers J, Kupke A, Zittrain J, et al. The clinician and dataset shift in artificial intelligence. N Engl J Med. 2021;385(3):283–6.
Funding
The work of ARA was supported by the PhD fellowship PD/BD/114107/2015 from Fundação da Ciência e da Tecnologia (FCT). The work of ARA, SMSV, and JMCS was supported through IDMEC, under LAETA, project UIDB/50022/2020; also, by the European Regional Development Fund (LISBOA-01-0145-FEDER-031474) and FCT through Programa Operacional Regional de Lisboa (PTDC/EME-SIS/31474/2017). LAC is funded by the National Institute of Health through NIBIB Grant R01 E017215.
Author information
Authors and Affiliations
Contributions
All authors contributed to writing the manuscript. SM and ARA collaborated on the data extraction, visualization, and analysis. JM, LAC, and RMC interpreted, validated results, design of the work and supervised data extraction. JMCS, SMSV and LAC reviewed the paper and supervised the work.
Corresponding author
Ethics declarations
Conflict of interest
The authors have no relevant financial or non-financial interests to disclose.
Ethical approval
The study is exempt from institutional review board approval due to its analysis of data that has been de-identified and its security schema, for which the re-identification risk was certified as meeting safe harbor standards by an independent privacy expert (Privacert, Cambridge, MA) (Health Insurance Portability and Accountability Act Certification no. 1031219-2).
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Mantena, S., Arévalo, A.R., Maley, J.H. et al. Predicting hypoglycemia in critically Ill patients using machine learning and electronic health records. J Clin Monit Comput 36, 1297–1303 (2022). https://doi.org/10.1007/s10877-021-00760-7
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10877-021-00760-7