Abstract
Frailty is associated with poor post-operative outcomes. However, frailty assessments in clinical practice are challenging due to the need for more resources and pragmatic complexities. We aimed to create a pre-operative frailty ascertainment using machine learning (ML) from electronic health record (EHR) data that is built on the Fried frailty phenotype. We leveraged a research database of 8,999 individuals aged 65 years and above who underwent a surgery. Healthcare providers administered a pre-operative frailty assessment using the Fried frailty phenotype. We built ML models to predict pre-operative frailty as a whole and by surgical service. We used the SHapley Additive exPlanations (SHAP) to interpret the results. Comparisons with the accumulated deficit approach were made using Pearson’s correlation coefficients and McNemar’s test. The ML model achieved an AUC of 0.74 in predicting pre-operative frailty phenotype. The ML models’ predictive power varied by surgical services, with AUC ranging from 0.63 to 0.81. SHAP showed that advanced age, anemia, chronic pulmonary disease, and literacy deficit were the most important features of pre-operative frailty. The ML model had similar performance to the accumulated deficit approach, with positive correlation (r: 0.36-0.58, P> 0.001) between frailty risk scores predicted by ML and frailty indices obtained from the accumulated deficit approach. Our approach provided new insights into the importance of specific EHR features in pre-operative frailty assessment. ML may offer an alternative method to predict frailty in pre-operative settings. Further validation on external data sources is required to generalize our ML models.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Availability of data and materials
Public data sharing was not an element approved by the UF IRB at the time of study initiation. Researchers with questions regarding a minimal dataset should contact the corresponding author; qualified researchers can apply for access to the dataset by contacting the corresponding author.
Code Availability
Python codes used for the study is publicly available at https://github.com/ufdsat/Frailty-Prediction-EHR.
References
AHRQ (2016). Clinical classifications software for icd-9 diagnoses. Retrieved 24-June-2023, from http://www.ahrq.gov/data/hcup/
Amini, S., Crowley, S., Hizel, L., et al. (2019). Feasibility and rationale for incorporating frailty and cognitive screening protocols in a preoperative anesthesia clinic. Anesthesia and Analgesia, 129(3), 830–838. https://doi.org/10.1213/ANE.0000000000004190
ASA (2020). Asa physical status classification system. Retrieved 24-June-2023, from https://www.asahq.org/standards-and-guidelines/asa-physical-statusclassification-system
Cabitza, F., Rasoini, R., & Gensini, G. F. (2017). Unintended consequences of machine learning in medicine. JAMA - Journal of the American Medical Association, 318(6), 517–518. https://doi.org/10.1001/jama.2017.7797
Cawley, G. C., & Talbot, N. L. (2010). On over-fitting in model selection and subsequent selection bias in performance evaluation. Journal of Machine Learning Research, 11, 2079–2107. https://doi.org/10.5555/1756006.1859921
Chen, T., & Guestrin, C. (2016). Xgboost: A scalable tree boosting system. Proceedings of the 22nd ACM sigkdd International Conference On Knowledge Discovery And Data Mining (p. 785–794). New York, NY, USA: Association for Computing Machinery.
Chen, C. L., Chen, C. M., Wang, C. Y., et al. (2019). Frailty is associated with an increased risk of major adverse outcomes in elderly patients following surgical treatment of hip fracture. Scientific Reports,9(1), https://doi.org/10.1038/s41598-019-55459-2
Choudhury, A., Renjilian, E., & Asan, O. (2020). Use of machine learning in geriatric clinical care for chronic diseases: A systematic literature review. JAMIA Open, 3(3), 459–471. https://doi.org/10.1093/JAMIAOPEN/OOAA034
Clegg, A., Bates, C., Young, J., et al. (2016). Development and validation of an electronic frailty index using routine primary care electronic health record data. Age and Ageing, 45(3), 353–360. https://doi.org/10.1093/ageing/afw039
Daabiss, M. (2011). American society of anaesthesiologists physical status classification. Indian Journal of Anaesthesia, 55(2), 111–115. https://doi.org/10.4103/0019-5049.79879
DeLong, E. R., DeLong, D. M., & Clarke-Pearson, D. L. (1988). Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics, 44(3), 837–845.
Dodds, R. M., Syddall, H. E., Cooper, R., et al. (2014). Grip strength across the life course: Normative data from twelve british studies. PLoS ONE,9(12). https://doi.org/10.1371/journal.pone.0113637
Du, Z., Yang, Y., Zheng, J., et al. (2020). Accurate prediction of coronary heart disease for patients with hypertension from electronic health records with big data and machine-learning methods: Model development and performance evaluation. JMIR Medical Informatics,8(7). https://doi.org/10.2196/17257
Fried, L. P., Tangen, C. M., Walston, J., et al. (2001). Frailty in older adults: Evidence for a phenotype. Journals of Gerontology - Series A Biological Sciences and Medical Sciences, 56(3), 146–157. https://doi.org/10.1093/gerona/56.3.m146
Gilbert, T., Neuburger, J., Kraindler, J., et al. (2018). Development and validation of a hospital frailty risk score focusing on older people in acute care settings using electronic hospital records: an observational study. The Lancet, 391(10132), 1775–1782. https://doi.org/10.1016/S0140-6736(18)30668-8
Hlatky, M. A., Boineau, R. E., Higginbotham, M. B., et al. (1989). A brief selfadministered questionnaire to determine functional capacity (the duke activity status index). The American Journal of Cardiology, 64(10), 651–654. https://doi.org/10.1016/0002-9149(89)90496-7
Javadi-Pashaki, N., Ghazanfari, M. J., & Karkhah, S. (2021). Machine learning for geriatric clinical care: Opportunities and challenges. Annals of Geriatric Medicine and Research, 25(2), 137–138. https://doi.org/10.4235/agmr.21.0054
Jiménez, Álvaro Barbero, Lázaro, J. L., & Dorronsoro, J. R. (2007). Finding optimal model parameters by discrete grid search. Innovations in Hybrid Intelligent Systems, 44, 120–127. https://doi.org/10.1007/978-3-540-74972-117
Kan, G. A. V., Rolland, Y., Bergman, H., et al. (2008). The i.a.n.a task force on frailty assessment of older people in clinical practice. J Nutr Health Aging, 12(1), 29–37. https://doi.org/10.1007/BF02982161
Kim, Y., Song, K., Kang, C. M., et al. (2022). Impact of preoperative laboratory frailty index on mortality and clinical outcomes in older surgical patients with cancer. Scientific Reports,12(1), https://doi.org/10.1038/s41598-022-13426-4
Kim, D. H., Glynn, R. J., Avorn, J., et al. (2019). Validation of a claims-based frailty index against physical performance and adverse health outcomes in the health and retirement study. Journals of Gerontology - Series A Biological Sciences and Medical Sciences, 74(8), 1271–1276. https://doi.org/10.1093/gerona/gly197
Kim, D. H., & Schneeweiss, S. (2014). Measuring frailty using claims data for pharmacoepidemiologic studies of mortality in older adults: Evidence and recommendations. Pharmacoepidemiology and Drug Safety, 23(9), 891–901. https://doi.org/10.1002/pds.3674
Kim, D. H., Schneeweiss, S., Glynn, R. J., et al. (2018). Measuring frailty in medicare data: Development and validation of a claims-based frailty index. Journals of Gerontology - Series A Biological Sciences and Medical Sciences, 73(7), 980–987. https://doi.org/10.1093/gerona/glx229
Lancet, T. (2017). Artificial intelligence in health care: within touching distance. The Lancet, 390(10114), 2739. https://doi.org/10.1016/S0140-6736(17)31540-4
Leigh, J. P., & Fries, J. F. (2002). Frailty and education in the hispanic health and nutrition examination survey. Journal of Health Care for the Poor and Underserved, 13(1), 112–127. https://doi.org/10.1353/hpu.2010.0307
Li, Z., Dawson, E., Moodie, J., et al. (2021). Measurement and prognosis of frail patients undergoing transcatheter aortic valve implantation: A systematic review and meta-analysis. BMJ Open,11(3), https://doi.org/10.1136/bmjopen-2020-040459
Lin, H. S., Watts, J. N., Peel, N. M., et al. (2016). Frailty and post-operative outcomes in older surgical patients: A systematic review. BMC Geriatrics,16(1). https://doi.org/10.1186/s12877-016-0329-8
Lundberg, S.M., & Lee, S.-I. (2017). A unified approach to interpreting model predictions. Proceedings of the 31st International Conference On Neural Information Processing Systems (p. 4768–4777). Red Hook, NY, USA: Curran Associates Inc.
Makary, M. A., Segev, D. L., Pronovost, P. J., et al. (2010). Frailty as a predictor of surgical outcomes in older patients. Journal of the American College of Surgeons, 210(6), 901–908. https://doi.org/10.1016/j.jamcollsurg.2010.01.028
McIsaac, D. I., Wong, C. A., Huang, A., et al. (2019). Derivation and validation of a generalizable preoperative frailty index using population-based health administrative data. Annals of Surgery, 270(1), 102–108. https://doi.org/10.1097/SLA.0000000000002769
Mcnemar, Q. (1947). Note on the sampling error of the difference between correlated proportions or percentages. Psychometrika, 12(2), 1947. https://doi.org/10.1007/BF02295996
Morley, J. E., Vellas, B., van Kan, G. A., et al. (2013). Frailty consensus: A call to action. Journal of the American Medical Directors Association, 14(6), 392–397. https://doi.org/10.1016/j.jamda.2013.03.022
Orkaby, A. R., Nussbaum, L., Ho, Y. L., et al. (2019). The burden of frailty among u.s. veterans and its association with mortality, 2002–2012. Journals of Gerontology- Series A Biological Sciences and Medical Sciences, 74(8), 1257–1264. https://doi.org/10.1093/gerona/gly232
Ozge Dokuzlar, Soyal, P., & Isik, A.T. (2017). Association between serum vitamin b12 levels and frailty in older adults. Northern Clinics of Istanbul, https://doi.org/10.14744/nci.2017.82787
Pajewski, N. M., Lenoir, K., Wells, B. J., et al. (2019). Frailty screening using the electronic health record within a medicare accountable care organization. Journals of Gerontology - Series A Biological Sciences and Medical Sciences, 74(11), 1771–1777. https://doi.org/10.1093/gerona/glz017
Pajewski, N. M., Williamson, J. D., Applegate, W. B., et al. (2016). Characterizing frailty status in the systolic blood pressure intervention trial. Journals of Gerontology - Series A Biological Sciences and Medical Sciences, 71(5), 649–655. https://doi.org/10.1093/gerona/glv228
Pogam, M.-A.L., Seematter-Bagnoud, L., Niemi, T., et al. (2022). Development and validation of a knowledge-based score to predict fried’s frailty phenotype across multiple settings using one-year hospital discharge data: The electronic frailty score. eClinicalMedicine, 44, 101260. https://doi.org/10.1016/j.eclinm.2021.101260
Robinson, T. N., Wallace, J. I., Wu, D. S., et al. (2011). Accumulated frailty characteristics predict postoperative discharge institutionalization in the geriatric patient. Journal of the American College of Surgeons, 213(1), 37–42. https://doi.org/10.1016/j.jamcollsurg.2011.01.056
Robinson, T. N., Wu, D. S., Pointer, L., et al. (2013). Simple frailty score predicts postoperative complications across surgical specialties. American Journal of Surgery, 206(4), 544–550. https://doi.org/10.1016/j.amjsurg.2013.03.012
Rockwood, K., Andrew, M., & Mitnitski, A. (2007). Unconventional views of frailty a comparison of two approaches to measuring frailty in elderly people. J Gerontol A Biol Sci Med Sci, 62(7), 738–743. https://doi.org/10.1093/gerona/62.7.738
Rockwood, K., & Mitnitski, A. (2007). Frailty in relation to the accumulation of deficits. Journals of Gerontology - Series A Biological Sciences and Medical Sciences, 62(7), 722–727. https://doi.org/10.1093/gerona/62.7.722
Segal, J. B., Chang, H.-Y., Du, Y., et al. (2017). Development of a claims-based frailty indicator anchored to a well-established frailty phenotype. Med Care,55(7), 716–722. https://doi.org/10.1097/MLR.0000000000000729www.lww-medicalcare.com
Shinall, M. C., Arya, S., Youk, A., et al. (2020). Association of preoperative patient frailty and operative stress with postoperative mortality. JAMA Surgery,155(1). https://doi.org/10.1001/jamasurg.2019.4620
Smetana, G.W. (2022). Preoperative medical evaluation of the healthy adult patient. Retrieved 24-June-2023, from https://www.uptodate.com/contents/preoperativemedical-evaluation-of-the-healthy-adult-patient
Steele, A. J., Denaxas, S. C., Shah, A. D., et al. (2018). Machine learning models in electronic health records can outperform conventional survival models for predicting patient mortality in coronary artery disease. PLoS ONE,13(8). https://doi.org/10.1371/journal.pone.0202344
Strom, J. B., Xu, J., Orkaby, A. R., et al. (2021). Role of frailty in identifying benefit from transcatheter versus surgical aortic valve replacement. Circulation. Cardiovascular quality and outcomes, 14(12), e008566. https://doi.org/10.1161/CIRCOUTCOMES.121.008566
Theou, O., Brothers, T. D., Mitnitski, A., et al. (2013). Operationalization of frailty using eight commonly used scales and comparison of their ability to predict allcause mortality. Journal of the American Geriatrics Society, 61(9), 1537–1551. https://doi.org/10.1111/jgs.12420
Trevisan, C., Veronese, N., Maggi, S., et al. (2017). Factors influencing transitions between frailty states in elderly adults: The progetto veneto anziani longitudinal study. Journal of the American Geriatrics Society, 65(1), 179–184. https://doi.org/10.1111/jgs.14515
Troyanskaya, O., Cantor, M., Sherlock, G., et al. (2001). Missing value estimation methods for dna microarrays. Bioinformatics, 17(6), 520–525. https://doi.org/10.1093/bioinformatics/17.6.520
Turner, G., & Clegg, A. (2014). Best practice guidelines for the management of frailty: A british geriatrics society, age uk and royal college of general practitioners report. Age and Ageing, 43(6), 744–747. https://doi.org/10.1093/ageing/afu138
Zhang, Y., Zhou, H., & Li, J., et al. (2018). A time-sensitive hybrid learning model for patient subgrouping. Proceedings of the International Joint Conference on Neural Networks, 2018-July, https://doi.org/10.1109/IJCNN.2018.8488991
Acknowledgements
We would like to acknowledge the Perioperative Cognitive Anesthesia Network (PeCAN) program investigative team.
Funding
Support was provided by the University of Florida Claude D. Pepper Older Americans Independence Center (P30AG028740, MM) and the National Institutes of Health (R01AG055337, CP/TP; K07AG066813, CP). The content is solely the responsibility of the authors. The funder of the study had no role in study design, data collection, data analysis, data interpretation, writing of the report, and in the decision to submit the paper for publication.
Author information
Authors and Affiliations
Contributions
MM, TM, and CB designed the study. MM, CP, SA, and PT acquired the data. CB and MM performed the literature search, analyzed and interpreted data, wrote the original draft. MA reviewed the chosen deficits for construction of frailty index. All authors revised the manuscript.
Corresponding author
Ethics declarations
Competing interest
The authors declare no competing interests.
Ethical approval
The study was approved by University of Florida’s institutional review board.
Consent to participate
Written informed consent was obtained from the participants during data collection.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
Below is the link to the electronic supplementary material.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Bai, C., Al-Ani, M., Amini, S. et al. Developing and validating an electronic health record-based frailty index in pre-operative settings using machine learning. J Intell Inf Syst (2023). https://doi.org/10.1007/s10844-023-00818-9
Received:
Revised:
Accepted:
Published:
DOI: https://doi.org/10.1007/s10844-023-00818-9