Development and validation of machine learning prediction model based on computed tomography angiography–derived hemodynamics for rupture status of intracranial aneurysms: a Chinese multicenter study

Abstract

Objectives

To build models based on conventional logistic regression (LR) and machine learning (ML) algorithms combining clinical, morphological, and hemodynamic information to predict individual rupture status of unruptured intracranial aneurysms (UIAs), afterwards tested in internal and external validation datasets.

Methods

Patients with intracranial aneurysms diagnosed by computed tomography angiography and confirmed by invasive cerebral angiograph or clipping surgery were included. The prediction models were developed based on clinical, aneurysm morphological, and hemodynamic parameters by conventional LR and ML methods.

Results

The training, internal validation, and external validation cohorts were composed of 807 patients, 200 patients, and 108 patients, respectively. The area under curves (AUCs) of conventional LR models 1 (clinical), 2 (clinical and aneurysm morphological), and 3 (clinical, aneurysm morphological and hemodynamic characteristics) were 0.608, 0.765, and 0.886, respectively (all p < 0.05). The AUCs of ML models using random forest (RF), multilayer perceptron (MLP), and support vector machine (SVM) were 0.871, 0.851, and 0.863, respectively. There were no difference among AUCs of conventional LR, RF, and SVM (all p > 0.05/6), while the AUC of MLP was lower than that of conventional LR (p = 0.0055).

Conclusion

Hemodynamic parameters play an important role in the prediction performance of the models. ML methods cannot outperform conventional LR in prediction models for rupture status of UIAs integrating clinical, aneurysm morphological, and hemodynamic parameters.

Key Points

• The addition of hemodynamic parameters can improve prediction performance for rupture status of unruptured intracranial aneurysms.

• Machine learning algorithms cannot outperform conventional logistic regression in prediction models for rupture status integrating clinical, aneurysm morphological, and hemodynamic parameters.

• Models integrating clinical, aneurysm morphological, and hemodynamic parameters may help choose the optimal management.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Abbreviations

ACoA:

Anterior communication artery

AFI:

Aneurysm formation index

AUC:

Area under curve

AWSSABS:

Averaged WSS-absolute

AWSSG:

Averaged WSS gradient

AWSSMEAN:

Time-average of the mean WSS

CTA:

CT angiography

GON:

Gradient oscillatory number

ICA:

Internal carotid artery

LR:

Logistic regression

MCA:

Middle cerebral artery

ML:

Machine learning

MLP:

Multilayer perceptron

OSI:

Oscillatory shear index

PCoA:

Posterior communication artery

RF:

Random forest

ROC:

Receiver operation characteristic curve

RRT:

Relative residence time

SVM:

Support vector machine

UIAs:

Unruptured intracranial aneurysms

WSS:

Wall shear stress

WSSG:

WSS gradient

References

  1. 1.

    Brown RD Jr, Broderick JP (2014) Unruptured intracranial aneurysms: epidemiology, natural history, management options, and familial screening. Lancet Neurol 13:393–404

    PubMed  Article  Google Scholar 

  2. 2.

    Vlak MH, Algra A, Brandenburg R, Rinkel GJ (2011) Prevalence of unruptured intracranial aneurysms, with emphasis on sex, age, comorbidity, country, and time period: a systematic review and meta-analysis. Lancet Neurol 10:626–636

    PubMed  Article  Google Scholar 

  3. 3.

    Rinkel GJ, Algra A (2011) Long-term outcomes of patients with aneurysmal subarachnoid haemorrhage. Lancet Neurol 10:349–356

    PubMed  Article  Google Scholar 

  4. 4.

    Naggara ON, White PM, Guilbert F, Roy D, Weill A, Raymond J (2010) Endovascular treatment of intracranial unruptured aneurysms: systematic review and meta-analysis of the literature on safety and efficacy. Radiology 256:887–897

  5. 5.

    Thompson BG, Brown RD Jr, Amin-Hanjani S et al (2015) Guidelines for the management of patients with unruptured intracranial aneurysms: a guideline for healthcare professionals from the American Heart Association/American Stroke Association. Stroke 46:2368–2400

    PubMed  Article  Google Scholar 

  6. 6.

    UCAS Japan Investigators, Morita A, Kirino T et al (2012) The natural course of unruptured cerebral aneurysms in a Japanese cohort. N Engl J Med 366:2474–2482

    Article  Google Scholar 

  7. 7.

    Greving JP, Wermer MJ, Brown RD Jr et al (2014) Development of the PHASES score for prediction of risk of rupture of intracranial aneurysms: a pooled analysis of six prospective cohort studies. Lancet Neurol 13:59–66

    PubMed  Article  Google Scholar 

  8. 8.

    Frösen J, Tulamo R, Paetau A et al (2012) Saccular intracranial aneurysm: pathology and mechanisms. Acta Neuropathol 123:6773–6786

    Article  Google Scholar 

  9. 9.

    Turjman AS, Turjman F, Edelman ER (2014) Role of fluid dynamics and inflammation in intracranial aneurysm formation. Circulation 129:373–382

    PubMed  PubMed Central  Article  Google Scholar 

  10. 10.

    Takao H, Murayama Y, Otsuka S et al (2012) Hemodynamic differences between unruptured and ruptured intracranial aneurysms during observation. Stroke 43:1436–1439

    PubMed  Article  Google Scholar 

  11. 11.

    Xiang J, Natarajan SK, Tremmel M et al (2011) Hemodynamic−morphologic discriminants for intracranial aneurysm rupture. Stroke 42:144–152

    PubMed  Article  Google Scholar 

  12. 12.

    Tominari S, Morita A, Ishibashi T et al (2015) Prediction model for three-year rupture risk of unruptured cerebral aneurysms in Japanese patients. Ann Neurol 77:1050–1059

    PubMed  Article  Google Scholar 

  13. 13.

    Ren Y, Chen GZ, Liu Z, Cai Y, Lu GM, Li ZY (2016) Reproducibility of image-based computational models of intracranial aneurysm: a comparison between 3D rotational angiography, CT angiography and MR angiography. Biomed Eng Online 15:50

  14. 14.

    Riccardello GJ Jr, Shastri DN, Changa AR et al (2018) Influence of relative residence time on side-wall aneurysm inception. Neurosurgery 83:574–581

    PubMed  Article  Google Scholar 

  15. 15.

    Cebral JR, Mut F, Weir J, Putman CM (2011) Association of hemodynamic characteristics and cerebral aneurysm rupture. AJNR Am J Neuroradiol 32:264–270

    CAS  PubMed  Article  Google Scholar 

  16. 16.

    Topol EJ (2019) High-performance medicine: the convergence of human and artificial intelligence. Nat Med 25:44–56

    CAS  PubMed  Article  Google Scholar 

  17. 17.

    Liu J, Chen Y, Lan L et al (2018) Prediction of rupture risk in anterior communicating artery aneurysms with a feed-forward artificial neural network. Eur Radiol 28:3268–3275

    PubMed  Article  Google Scholar 

  18. 18.

    Kim HC, Rhim JK, Ahn JH et al (2019) Machine learning application for rupture risk assessment in small-sized intracranial aneurysm. J Clin Med 8:683

    PubMed Central  Article  Google Scholar 

  19. 19.

    Liu Q, Jiang P, Jiang Y et al (2019) Prediction of aneurysm stability using a machine learning model based on PyRadiomics-derived morphological features. Stroke 10:STROKEAHA119025777

    Google Scholar 

  20. 20.

    Can A, Du R (2016) Association of hemodynamic factors with intracranial aneurysm formation and rupture: systematic review and meta-analysis. Neurosurgery 78:510–520

    PubMed  Article  Google Scholar 

  21. 21.

    Landis JR, Koch GG (1977) The measurement of observer agreement for categorical data. Biometrics 33(1):159–174

    CAS  PubMed  Article  Google Scholar 

  22. 22.

    Schneiders JJ, Marquering HA, van Ooij P et al (2015) Additional value of intra-aneurysmal hemodynamics in discriminating ruptured versus unruptured intracranial aneurysms. AJNR Am J Neuroradiol 36:1920–1926

    CAS  PubMed  Article  Google Scholar 

  23. 23.

    Jou LD, Lee DH, Morsi H, Mawad ME (2008) Wall shear stress on ruptured and unruptured intracranial aneurysms at the internal carotid artery. AJNR Am J Neuroradiol 29:1761–1767

    PubMed  Article  Google Scholar 

  24. 24.

    Miura Y, Ishida F, Umeda Y et al (2013) Low wall shear stress is independently associated with the rupture status of middle cerebral artery aneurysms. Stroke 44:519–521

    PubMed  Article  Google Scholar 

  25. 25.

    Lauric A, Hippelheuser J, Cohen AD, Kadasi LM, Malek AM (2014) Wall shear stress association with rupture status in volume matched sidewall aneurysms. J Neurointerv Surg 6:466–473

    PubMed  Article  Google Scholar 

  26. 26.

    Lu G, Huang L, Zhang XL et al (2011) Influence of hemodynamic factors on rupture of intracranial aneurysms: patient-specific 3D mirror aneurysms model computational fluid dynamics simulation. AJNR Am J Neuroradiol 32:1255–1261

    CAS  PubMed  Article  Google Scholar 

  27. 27.

    Pereira VM, Brina O, Bijlenga P et al (2014) Wall shear stress distribution of small aneurysms prone to rupture: a case-control study. Stroke 45:261–264

    PubMed  Article  Google Scholar 

  28. 28.

    Meng H, Tutino VM, Xiang J, Siddiqui (2014) A high WSS or low WSS? Complex interactions of hemodynamics with intracranial aneurysm initiation, growth, and rupture: toward unifying hypothesis. AJNR Am J Neuroradiol 35:1254–1262

    CAS  PubMed  Article  Google Scholar 

  29. 29.

    Varble N, Tutino VM, Yu J et al (2018) Shared and distinct rupture discriminants of small and large intracranial aneurysms. Stroke 49:856–864

    PubMed  PubMed Central  Article  Google Scholar 

  30. 30.

    Beam AL, Kohane IS (2018) Big data and machine learning in health care. JAMA 319:1317–1318

    PubMed  Article  Google Scholar 

  31. 31.

    Christodoulou E, Ma J, Collins GS, Steyerberg EW, Verbakel JY, Van Calster B (2019) A systematic review shows no performance benefit of machine learning over logistic regression for clinical prediction models. J Clin Epidemiol S0895-4356(18):31081–31083

  32. 32.

    Frizzell JD, Liang L, Schulte PJ et al (2017) Prediction of 30-day all-cause readmissions in patients hospitalized for heart failure: comparison of machine learning and other statistical approaches. JAMA Cardiol 2:204–209

    PubMed  Article  Google Scholar 

  33. 33.

    Schneiders JJ, Marquering HA, van den Berg R et al (2014) Rupture-associated changes of cerebral aneurysm geometry: high-resolution 3D imaging before and after rupture. AJNR Am J Neuroradiol 35:1358–1362

    CAS  PubMed  Article  Google Scholar 

Download references

Funding

Supported by The National Key Research and Development Program of China (2017YFC0113400 for L.J.Z.), The Key Projects of the National Natural Science Foundation of China (81830057 for L.J.Z.) and The National Natural Science Foundation of China (No.81803338 for L.M.J.)

Author information

Affiliations

Authors

Corresponding authors

Correspondence to Long Jiang Zhang or Guang Ming Lu.

Ethics declarations

Guarantor

The scientific guarantors of this publication are Long Jiang Zhang and Guang Ming Lu.

Conflict of interest

The authors of this manuscript declare no relationships with any companies whose products or services may be related to the subject matter of the article.

Statistics and biometry

No complex statistical methods were necessary for this paper.

Informed consent

The informed consent form was waived because this was a retrospective study.

Ethical approval

Institutional Review Board approval was obtained.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

ESM 1

(DOCX 1470 kb).

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Chen, G., Lu, M., Shi, Z. et al. Development and validation of machine learning prediction model based on computed tomography angiography–derived hemodynamics for rupture status of intracranial aneurysms: a Chinese multicenter study. Eur Radiol 30, 5170–5182 (2020). https://doi.org/10.1007/s00330-020-06886-7

Download citation

Keywords

  • Intracranial aneurysm
  • Tomography, X-ray computed
  • Machine learning
  • Angiography
  • Rupture