Skip to main content
SpringerLink
Log in

External validation of cerebral aneurysm rupture probability model with data from two patient cohorts

  • Original Article - Vascular Neurosurgery - Aneurysm
  • Published:
Acta Neurochirurgica Aims and scope Submit manuscript

Abstract

Background

For a treatment decision of unruptured cerebral aneurysms, physicians and patients need to weigh the risk of treatment against the risk of hemorrhagic stroke caused by aneurysm rupture. The aim of this study was to externally evaluate a recently developed statistical aneurysm rupture probability model, which could potentially support such treatment decisions.

Methods

Segmented image data and patient information obtained from two patient cohorts including 203 patients with 249 aneurysms were used for patient-specific computational fluid dynamics simulations and subsequent evaluation of the statistical model in terms of accuracy, discrimination, and goodness of fit. The model’s performance was further compared to a similarity-based approach for rupture assessment by identifying aneurysms in the training cohort that were similar in terms of hemodynamics and shape compared to a given aneurysm from the external cohorts.

Results

When applied to the external data, the model achieved a good discrimination and goodness of fit (area under the receiver operating characteristic curve AUC = 0.82), which was only slightly reduced compared to the optimism-corrected AUC in the training population (AUC = 0.84). The accuracy metrics indicated a small decrease in accuracy compared to the training data (misclassification error of 0.24 vs. 0.21). The model’s prediction accuracy was improved when combined with the similarity approach (misclassification error of 0.14).

Conclusions

The model’s performance measures indicated a good generalizability for data acquired at different clinical institutions. Combining the model-based and similarity-based approach could further improve the assessment and interpretation of new cases, demonstrating its potential use for clinical risk assessment.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Notes

  1. http://ecm2.mathcs.emory.edu/aneuriskweb/index

  2. The AneuX dataset was processed under supervision of Vitor Mendes Pereira, Philippe Bijlenga, Rafik Ouared, Norman Juchler, and Sven Hirsch. It is maintained within the scope of the SystemsX.ch project AneuX.

References

  1. Austin PC, Steyerberg EW (2014) Graphical assessment of internal and external calibration of logistic regression models by using loess smoothers. Stat Med 33(3):517–535

    Article  Google Scholar 

  2. Bijlenga P, Gondar R, Schilling S, Morel S, Hirsch S, Cuony J, Corniola M-V, Perren F, Rüfenacht D, Schaller K (2017) PHASES score for the management of intracranial aneurysm: a cross-sectional population-based retrospective study. Stroke 48(8):2105–2112

    Article  Google Scholar 

  3. Cebral JR, Castro MA, Appanaboyina S, Putman CM, Millan D, Frangi AF (2005) Efficient pipeline for image-based patient-specific analysis of cerebral aneurysm hemodynamics: technique and sensitivity. IEEE Trans Med Imaging 24(4):457–467

    Article  Google Scholar 

  4. Cebral JR, Castro MA, Putman CM, Alperin N (2008) Flow-area relationship in internal carotid and vertebral arteries. Physiol Meas 29:585–594

    Article  CAS  Google Scholar 

  5. Cebral JR, Sheridan MJ, Putman CM (2010) Hemodynamics and Bleb formation in intracranial aneurysms. AJNR Am J Neuroradiol 31:304–310

    Article  CAS  Google Scholar 

  6. Cebral J, Ollikainen E, Chung BJ, Mut F, Sippola V, Jahromi BR, Tulamo R, Hernesniemi J, Niemela M, Robertson AM, Frösen J (2017) Flow conditions in the intracranial aneurysm lumen are associated with inflammation and degenerative changes of the aneurysm wall. AJNR Am J Neuroradiol 38:119–126

    Article  CAS  Google Scholar 

  7. Chalouhi N, Ali MS, Jabbour PM, Tjoumakaris SI, Gonzalez LF, Rosenwasser RH, Koch WJ, Dumont AS (2012) Biology of intracranial aneurysms: role of inflammation. J Cereb Blood Flow Metab. https://doi.org/10.1038/jcbfm.2012.84

    Article  CAS  Google Scholar 

  8. Chung BJ, Mut F, Putman C, Hamzei-Sichani F, Brinjikji W, Kallmes DF, Cebral JR (2018) Identification of hostile hemodynamics and geometries of cerebral aneurysms: a case-control study. AJNR Am J Neuroradiol. https://doi.org/10.3174/ajnr.A5764

    Article  CAS  Google Scholar 

  9. Collins GS, Reitsma JB, Altman DG, Moons K (2015) Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement. BMC Med 13(1):1

    Article  Google Scholar 

  10. Detmer FJ, Chung BJ, Mut F, Slawski M, Hamzei-Sichani F, Putman C, Jiménez C, Cebral JR (2018) Development and internal validation of an aneurysm rupture probability model based on patient characteristics and aneurysm location, morphology, and hemodynamics. Int J Comput Assist Radiol Surg. https://doi.org/10.1007/s11548-018-1837-0

    Article  Google Scholar 

  11. Gabriel RA, Kim H, Sidney S, McCulloch CE, Singh V, Johnston SC, Ko NU, Achrol AS, Zaroff JG, Young WL (2010) Ten-year detection rate of brain arteriovenous malformations in a large, multiethnic, defined population. Stroke 41(1):21–26

    Article  Google Scholar 

  12. Gillani RL, Podraza KM, Luthra N, Origitano TC, Schneck MJ (2016) Factors influencing the management of unruptured intracranial aneurysms. Cureus. https://doi.org/10.7759/cureus.601

  13. Greving JP, Wermer MJ, Brown RD, Morita A, Juvela S, Yonekura M, Ishibashi T, Torner JC, Nakayama T, Rinkel GJ, Algra A (2014) Development of the PHASES score for prediction of risk of rupture of intracranial aneurysms: a pooled analysis of six prospective cohort studies. Lancet Neurol 13:59–66

    Article  Google Scholar 

  14. Hernandez M, Frangi AF (2007) Non-parametric geodesic active regions: method and evaluation for cerebral aneurysms segmentation in 3DRA and CTA. Med Image Anal 11:224–241

    Article  Google Scholar 

  15. Huang MC, Baaj AA, Downes K, Youssef AS, Sauvageau E, van Loveren HR, Agazzi S (2011) Paradoxical trends in the management of unruptured cerebral aneurysms in the United States: analysis of nationwide database over a 10-year period. Stroke 42(6):1730–1735

    Article  Google Scholar 

  16. Jabbarli R, Dinger TF, Darkwah Oppong M, Pierscianek D, Dammann P, Wrede KH, Kaier K, Köhrmann M, Forsting M, Kleinschnitz C, Sure U (2018) Risk factors for and clinical consequences of multiple intracranial aneurysms: a systematic review and meta-analysis. Stroke 49(4):848–855

    Article  Google Scholar 

  17. Japan Investigators UCAS, Morita A, Kirino T, Hashi K, Aoki N, Fukuhara S, Hashimoto N, Nakayama T, Sakai M, Teramoto A, Tominari S, Yoshimoto T (2012) The natural course of unruptured cerebral aneurysms in a Japanese cohort. N Engl J Med 366:2474–2482

    Article  Google Scholar 

  18. Juvela S, Poussa K, Lehto H, Porras M (2013) Natural history of unruptured intracranial aneurysms: a long-term follow-up study. Stroke 44:2414–2421

    Article  Google Scholar 

  19. Kleinloog R, de Mul N, Verweij BH, Post JA, Rinkel GJE, Ruigrok YM (2017) Risk factors for intracranial aneurysm rupture: a systematic review. Neurosurgery. https://doi.org/10.1093/neuros/nyx238

  20. Korja M, Kaprio J (2016) Controversies in epidemiology of intracranial aneurysms and SAH. Nat Rev Neurol 12(1):50–55

    Article  Google Scholar 

  21. Korja M, Lehto H, Juvela S (2014) Lifelong rupture risk of intracranial aneurysms depends on risk factors: a prospective Finnish cohort study. Stroke 45:1958–1963

    Article  Google Scholar 

  22. Korja M, Lehto H, Juvela S, Kaprio J (2016) Incidence of subarachnoid hemorrhage is decreasing together with decreasing smoking rates. Neurology 87(11):1118–1123

    Article  Google Scholar 

  23. Lindgren AE, Koivisto T, Björkman J, von und zu Fraunberg M, Helin K, Jääskeläinen JE, Frösen J (2016) Irregular shape of intracranial aneurysm indicates rupture risk irrespective of size in a population-based cohort. Stroke 47(5):1219–1226

    Article  Google Scholar 

  24. Piccinelli M, Bacigaluppi S, Boccardi E, Ene-Iordache B, Remuzzi A, Veneziani A, Antiga L (2011) Geometry of the internal carotid artery and recurrent patterns in location, orientation, and rupture status of lateral aneurysms: an image-based computational study. Neurosurgery 68(5):1270–1285

    Article  Google Scholar 

  25. R Core Team (2017) R: a language and environment for statistical computing. Version 3.3.3. R Foundation for Statistical Computing, Vienna https://www.R-project.org/. Accessed on Sept 2018

    Google Scholar 

  26. Rinkel GJ, Djibuti M, van Gijn J (1998) Prevalence and risk of rupture of intracranial aneurysms: a systematic review. Stroke 29:251–259

    Article  CAS  Google Scholar 

  27. Rivero-Arias O, Gray A, Wolstenholme J (2010) Burden of disease and costs of aneurysmal subarachnoid haemorrhage (aSAH) in the United Kingdom. Cost Eff Resour Alloc 8(1):6

    Article  Google Scholar 

  28. Sangalli LM, Secchi P, Vantini S (2014) AneuRisk65: a dataset of three-dimensional cerebral vascular geometries. Electron J Stat 8(2):1879–1890

    Article  Google Scholar 

  29. Steiner T, Juvela S, Unterberg A, Jung C, Forsting M, Rinkel G (2013) European stroke organization guidelines for the management of intracranial aneurysms and subarachnoid haemorrhage. Cerebrovasc Dis 35(2):93–112

    Article  Google Scholar 

  30. Thompson BG, Brown RD, Amin-Hanjani S, Broderick JP, Cockroft KM, Connolly ES, Duckwiler GR, Harris CC, Howard VJ, Johnston SC, Meyers PM, Molyneux A, Ogilvy CS, Ringer AJ, Torner J (2015) Guidelines for the management of patients with unruptured intracranial aneurysms: a guideline for healthcare professionals from the American Heart Association/American Stroke Association. Stroke 46(8):2368–2400

    Article  Google Scholar 

  31. Tulamo R, Frosen J, Hernesniemi J, Niemela M (2010) Inflammatory changes in the aneurysm wall: a review. J Neurointerv Surg 2:120–130

    Article  Google Scholar 

  32. Villa-Uriol MC, Berti G, Hose DR, Marzo A, Chiarini A, Penrose J, Pozo J, Schmidt JG, Singh P, Lycett R, Larrabide I, Frangi AF (2011) @neurIST complex information processing toolchain for the integrated management of cerebral aneurysms. Interface Focus 1(3):308–319

    Article  CAS  Google Scholar 

  33. Vlak MH, Algra A, Brandenburg R, Rinkel GJ (2011) Prevalence of unruptured intracranial aneurysms, with emphasis on sex, age, comorbidity, country, and time period: a systematic review and meta-analysis. Lancet Neurol 10:626–636

    Article  Google Scholar 

  34. Wang G, Zhang Z, Ayala C, Dunet DO, Fang J, George MG (2014) Costs of hospitalization for stroke patients aged 18-64 years in the United States. J Stroke Cerebrovasc Dis 23(5):861–868

    Article  Google Scholar 

  35. Wermer MJ, van der Schaaf IC, Algra A, Rinkel GJ (2007) Risk of rupture of unruptured intracranial aneurysms in relation to patient and aneurysm characteristics: an updated meta-analysis. Stroke 38:1404–1410

    Article  Google Scholar 

  36. Wiebers DO, Whisnant JP, Huston J, Meissner I, Brown RD, Piepgras DG, Forbes GS, Thielen K, Nichols D, O’Fallon WM, Peacock J, Jaeger L, Kassell NF, Kongable-Beckman GL, Torner JC (2003) Unruptured intracranial aneurysms: natural history, clinical outcome, and risks of surgical and endovascular treatment. Lancet 362:103–110

    Article  Google Scholar 

  37. Xiang J, Varble N, Davies JM, Rai AT, Kono K, Sugiyama S, Binning MJ, Tawk RG, Choi H, Ringer AJ, Snyder KV, Levy EI, Hopkins LN, Siddiqui AH, Meng H (2017) Initial clinical experience with AView—a clinical computational platform for intracranial aneurysm morphology, hemodynamics, and treatment management. World Neurosurg 108:534–542

    Article  Google Scholar 

  38. Yang X, Liu C, Le Minh H, Wang Z, Chien A, Cheng K-T (2017) An automated method for accurate vessel segmentation. Phys Med Biol 62(9):3757–3778

    Article  Google Scholar 

Download references

Acknowledgements

The authors would like to thank Rafik Ouared and Olivier Brina for helping with the image segmentation of the AneuX data.

Funding

This study was funded by the National Institutes of Health/National Institute of Neurological Disorders and Stroke (NIH-NINDS, grant no. R21NS094780). NJ, SH, VMP, and PB were supported by SystemsX.ch project AneuX evaluated by the Swiss National Science Foundation.

Author information

Authors and Affiliations

  1. Bioengineering Department, Volgenau School of Engineering, George Mason University, 4400 University Drive, Fairfax, VA, 22030, USA

    Felicitas J. Detmer, Daniel Fajardo-Jiménez, Fernando Mut & Juan R. Cebral

  2. Institute of Applied Simulation, ZHAW University of Applied Sciences, Waedenswil, Switzerland

    Norman Juchler & Sven Hirsch

  3. Institute of Physiology, University of Zurich, Zurich, Switzerland

    Norman Juchler

  4. Interventional Neuroradiology Unit, Service of Neuroradiology, Faculty of Medicine, University of Geneva, Geneva, Switzerland

    Vitor Mendes Pereira

  5. Neurosurgery, Clinical Neurosciences Department, Faculty of Medicine, University of Geneva, Geneva, Switzerland

    Philippe Bijlenga

Authors
  1. Felicitas J. Detmer
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Daniel Fajardo-Jiménez
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Fernando Mut
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Norman Juchler
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Sven Hirsch
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Vitor Mendes Pereira
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Philippe Bijlenga
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Juan R. Cebral
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Felicitas J. Detmer.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. For this type of study, formal consent is not required.

Additional information

This article is part of the Topical Collection on Vascular Neurosurgery - Aneurysm

Electronic supplementary material

ESM 1

(PDF 457 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Detmer, F.J., Fajardo-Jiménez, D., Mut, F. et al. External validation of cerebral aneurysm rupture probability model with data from two patient cohorts. Acta Neurochir 160, 2425–2434 (2018). https://doi.org/10.1007/s00701-018-3712-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00701-018-3712-8

Keywords

Search

Navigation