Skip to main content

Advertisement

SpringerLink
Log in

Machine learning algorithms for improved prediction of in-hospital outcomes after moderate-to-severe traumatic brain injury: a Chinese retrospective cohort study

  • Original Article
  • Published:
Acta Neurochirurgica Aims and scope Submit manuscript

Abstract

Aim

Controversy remains high over the superiority of advanced machine learning (ML) algorithms to conventional logistic regression (LR) in the prediction of prognosis after traumatic brain injury (TBI). This study aimed to compare the performance of ML and LR models in predicting in-hospital prognosis after TBI.

Method

In a single-center retrospective cohort of adult patients hospitalized for moderate-to-severe TBI (Glasgow coma score ≤12) in our hospital from 2011 to 2020, LR and three ML algorithms (XGboost, lightGBM, and FT-transformer) were run to build prediction models for in-hospital mortality and the Glasgow Outcome Scale (GOS) functional outcomes using either all 19 clinical and laboratory features or the 10 non-laboratory ones collected at admission to the neurological intensive care unit. The Shapley (SHAP) value was used for model interpretation.

Result

In total, 482 patients had an in-hospital mortality rate of 11.0%. A total of 23.0% of the patients had good functional scores (GOS ≥ 4) at discharge. All ML models performed better than the LR model in predicting in-hospital prognosis after TBI, among which the lightGBM model showed the best performance: When predicting mortality, the lightGBM model yielded an area under the curve (AUC) of 0.953 using all 19 features (the LR model: 0.813) and an AUC of 0.935 using 10 non-laboratory features (the LR model: 0.803); when predicting GOS functional outcomes, it yielded an AUC of 0.913 using all 19 features (the LR model: 0.832) and an AUC of 0.889 using non-laboratory data (the LR model: 0.818). The SHAP method identified key contributors to explain the lightGBM models. Finally, the integration of the lightGBM models with different prediction purposes was found to provide refined prognostic information, particularly for patients who survived moderate-to-severe TBI.

Conclusion

The study supported the superiority of ML to LR in predicting prognosis after moderate-to-severe TBI and highlighted its potential use for clinical application.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Abujaber A, Fadlalla A, Gammoh D, Abdelrahman H, Mollazehi M, El-Menyar A (2020) Prediction of in-hospital mortality in patients with post traumatic brain injury using national trauma registry and machine learning approach. Scand J Trauma Resusc Emerg Med 28(1):44

    Article  PubMed  PubMed Central  Google Scholar 

  2. Bergstra J, Bardenet R, Bengio Y, Kégl B (2011) Algorithms for hyper-parameter optimization. Advances in Neural Information Processing Systems 24

  3. Bruschetta R, Tartarisco G, Lucca LF et al (2022) Predicting outcome of traumatic brain injury: is machine learning the best way? Biomedicines 10(3)

  4. Carney N, Totten AM, O'Reilly C et al (2017) Guidelines for the management of severe traumatic brain injury, fourth edition. Neurosurgery 80(1):6–15

    Article  PubMed  Google Scholar 

  5. Chen T, Guestrin C (2016) Xgboost: a scalable tree boosting system. In: Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining, pp 785–794

    Chapter  Google Scholar 

  6. Cerasa A, Tartarisco G, Bruschetta R et al (2022) Predicting outcome in patients with brain injury: differences between machine learning versus conventional statistics. Biomedicines 10(9)

  7. Dijkland SA, Foks KA, Polinder S et al (2020) Prognosis in moderate and severe traumatic brain injury: a systematic review of contemporary models and validation studies. J Neurotrauma 37(1):1–13

    Article  PubMed  Google Scholar 

  8. (2020) Expert consensus on the standard technique of large craniotomy for adult severe traumatic brain injury in China. Chin J Neurotrauma Surg (Electronic Edition) 2(6):68–75

  9. Gorishniy Y, Rubachev I, Khrulkov V, Babenko A (2021) Revisiting deep learning models for tabular data. Advances in Neural Information Processing Systems 34:18932–18943

    Google Scholar 

  10. (2007) Guidelines for the management of severe traumatic brain injury. J Neurotrauma 24(Suppl 1):S1–S106

  11. He YL, Liu WB (2020) Experts consensus on the management of factors of secondary brain insults following traumatic brain injury. J Clin Neurosurg 3(17):241–253

    Google Scholar 

  12. Hsu SD, Chao E, Chen SJ, Hueng DY, Lan HY, Chiang HH (2021) Machine learning algorithms to predict in-hospital mortality in patients with traumatic brain injury. J Pers Med 11(11)

  13. Ke G, Meng Q, Finley T et al (2017) Lightgbm: a highly efficient gradient boosting decision tree. Advances in Neural Information Processing Systems 30

  14. Lipkova J, Chen RJ, Chen B et al (2022) Artificial intelligence for multimodal data integration in oncology. Cancer Cell 40(10):1095–1110

    Article  CAS  PubMed  Google Scholar 

  15. Loh HW, Ooi CP, Seoni S, Barua PD, Molinari F, Acharya UR (2022) Application of explainable artificial intelligence for healthcare: a systematic review of the last decade (2011-2022). Comput Methods Programs Biomed 226:107161

    Article  PubMed  Google Scholar 

  16. Lundberg SM, Lee S (2017) A unified approach to interpreting model predictions. Advances in Neural Information Processing Systems 30

  17. Maas A, Menon DK, Manley GT et al (2022) Traumatic brain injury: progress and challenges in prevention, clinical care, and research. Lancet Neurol 21(11):1004–1060

    Article  PubMed  Google Scholar 

  18. McMillan T, Wilson L, Ponsford J, Levin H, Teasdale G, Bond M (2016) The Glasgow outcome scale – 40 years of application and refinement. Nat Rev Neurol 12(8):477–485

    Article  PubMed  Google Scholar 

  19. Meyfroidt G, Bouzat P, Casaer MP et al (2022) Management of moderate to severe traumatic brain injury: an update for the intensivist. Intensive Care Med 48(6):649–666

    Article  PubMed  Google Scholar 

  20. Nijman S, Leeuwenberg AM, Beekers I et al (2022) Missing data is poorly handled and reported in prediction model studies using machine learning: a literature review. J Clin Epidemiol 142:218–229

    Article  PubMed  Google Scholar 

  21. Park SH, Han K, Jang HY et al (2022) Methods for clinical evaluation of artificial intelligence algorithms for medical diagnosis. Radiology:220182

  22. Pes B, Lai G (2021) Cost-sensitive learning strategies for high-dimensional and imbalanced data: a comparative study. PeerJ Comput Sci 7:e832

    Article  PubMed  PubMed Central  Google Scholar 

  23. Raj R, Wennervirta JM, Tjerkaski J et al (2022) Dynamic prediction of mortality after traumatic brain injury using a machine learning algorithm. NPJ Digit Med 5(1):96

    Article  PubMed  PubMed Central  Google Scholar 

  24. Rau CS, Kuo PJ, Chien PC, Huang CY, Hsieh HY, Hsieh CH (2018) Mortality prediction in patients with isolated moderate and severe traumatic brain injury using machine learning models. PloS One 13(11):e207192

    Article  Google Scholar 

  25. Tu KC, Eric NT, Wang CC et al (2022) A computer-assisted system for early mortality risk prediction in patients with traumatic brain injury using artificial intelligence algorithms in emergency room triage. Brain Sci 12(5)

  26. Xinchuan Z, Tony RM (2000) Distribution-balanced stratified cross-validation for accuracy estimation. J Exp Theor Artif in 12(1)

  27. Wang QQ, Yu SC, Qi X et al (2019) Overview of logistic regression model analysis and application. Zhonghua Yu Fang Yi Xue Za Zhi 53(9):955–960

    CAS  PubMed  Google Scholar 

  28. Wang R, Wang L, Zhang J, He M, Xu J (2022) Xgboost machine learning algorism performed better than regression models in predicting mortality of moderate-to-severe traumatic brain injury. World Neurosurg 163:e617–e622

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

Dr. Anan Yin wants to thank his daughter (Jiayan Yin) and his wife (Dr. Yu Dong) for their great support.

Funding

This work was partially funded by grants from the National Natural Science Foundation of China (No. 81402049, 81802486), the Shaanxi Province Natural Science Foundation (No.2023-JC-YB-641) and the Shandong Province Natural Science Foundation (No.ZR2020QH0233).

Author information

Author notes

  1. An-an Yin

    Present address: Department of Plastic and Reconstructive Surgery, Xijing Hospital, Fourth Military Medical University, Xi’an, 710032, China

  2. Zan Zhang, Sheng-ju Wang and Kun Chen contributed equally to this work.

Authors and Affiliations

  1. School of Electronic and Control Engineering, Chang’an University, Xi’an, 710064, China

    Zan Zhang & Sheng-ju Wang

  2. Department of Anatomy, Histology and Embryology and K.K. Leung Brain Research Centre, School of Basic Medicine, Fourth Military Medical University, Xi’an, 710032, China

    Kun Chen

  3. Shaanxi Provincial Key Laboratory of Clinic Genetics, Fourth Military Medical University, Xi’an, 710032, China

    Kun Chen

  4. Department of Neurosurgery, Xijing Institute of Clinical Neuroscience, Xijing Hospital, Fourth Military Medical University, Xi’an, 710032, China

    An-an Yin, Lin Wei & Ya-long He

  5. Department of Biochemistry and Molecular Biology, Fourth Military Medical University, Xi’an, 710032, China

    An-an Yin

Authors
  1. Zan Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sheng-ju Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kun Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. An-an Yin
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Lin Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Ya-long He
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Conception and design of the study: AAY, WL, and YLH

Provision of study material or patients: AAY, WL, and YLH

Acquisition and assembly of data: ZZ, SJW, AAY, WL, and YLH

Project administration, Software, methodology: ZZ, SJW, KC, and AAY

Analysis and interpretation of results: AAY, WL, and YLH

Manuscript writing: all authors

Final approval of manuscript: all authors

Corresponding authors

Correspondence to An-an Yin, Lin Wei or Ya-long He.

Ethics declarations

Ethical approval

Approved.

Informed consent

Informed consent was obtained for all participants from the Neurosurgery Department, Xijing Hospital.

Registry and the registration no. of the study/trial

N/A

Animal studies

N/A

Conflict of interest

The authors declare no competing interests.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

ESM 1

Supplementary figure S1: ROCs of the 5-fold cross-validation in each model for prediction of in-hospital mortality using (A) all 19 features and (B) the 10 non-laboratory features (PDF 1768 kb)

ESM 2

Supplementary figure S2: ROCs of the 5-fold cross-validation in each model for prediction of favorable GOS functional outcomes using (A) all 19 features and (B) the 10 non-laboratory features (PDF 1868 kb)

ESM 3

Supplementary figure S3: Confusion matrice of the ML versus LR models in prediction of (A) In-hospital mortality and (B) favorable GOS outcomes at discharge, using the 10 non-laboratory features; LR= logistic regression; ML=machine learning; GOS= Glasgow Outcome Scale; (PDF 233 kb)

ESM 4

Supplementary figure S4: Interpretations of the importance of the 10 non-laboratory features within the best-performing lightGBM models using the SHAP plot; (A) In-hospital mortality; (B) favorable GOS outcomes at discharge; feature importance was ranked according to the mean SHAP values. In the SHAP plot, the red and blue colors indicate feature values of high and low levels. For example, a low potassium level had a strong and positive contribution to a high probability of death; DBP= diastolic blood pressure; GCS= Glasgow coma score; SBP= systolic blood pressure; MAE= mean arterial pressure; (PDF 2947 kb)

ESM 5

Supplementary figure S5: Integrated confusion matrice of the best-performing lightGBM models in prediction of patients (A) who may survival but have poor GOS scores (<4) and (B) who may survival and have good GOS scores (≥4), using the 10 non-laboratory features; GOS= Glasgow Outcome Scale (PDF 227 kb)

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, Z., Wang, Sj., Chen, K. et al. Machine learning algorithms for improved prediction of in-hospital outcomes after moderate-to-severe traumatic brain injury: a Chinese retrospective cohort study. Acta Neurochir 165, 2237–2247 (2023). https://doi.org/10.1007/s00701-023-05647-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00701-023-05647-x

Keywords

Search

Navigation