Acta Neurochirurgica

, Volume 160, Issue 12, pp 2425–2434 | Cite as

External validation of cerebral aneurysm rupture probability model with data from two patient cohorts

  • Felicitas J. DetmerEmail author
  • Daniel Fajardo-Jiménez
  • Fernando Mut
  • Norman Juchler
  • Sven Hirsch
  • Vitor Mendes Pereira
  • Philippe Bijlenga
  • Juan R. Cebral
Original Article - Vascular Neurosurgery - Aneurysm
Part of the following topical collections:
  1. Vascular Neurosurgery – Aneurysm



For a treatment decision of unruptured cerebral aneurysms, physicians and patients need to weigh the risk of treatment against the risk of hemorrhagic stroke caused by aneurysm rupture. The aim of this study was to externally evaluate a recently developed statistical aneurysm rupture probability model, which could potentially support such treatment decisions.


Segmented image data and patient information obtained from two patient cohorts including 203 patients with 249 aneurysms were used for patient-specific computational fluid dynamics simulations and subsequent evaluation of the statistical model in terms of accuracy, discrimination, and goodness of fit. The model’s performance was further compared to a similarity-based approach for rupture assessment by identifying aneurysms in the training cohort that were similar in terms of hemodynamics and shape compared to a given aneurysm from the external cohorts.


When applied to the external data, the model achieved a good discrimination and goodness of fit (area under the receiver operating characteristic curve AUC = 0.82), which was only slightly reduced compared to the optimism-corrected AUC in the training population (AUC = 0.84). The accuracy metrics indicated a small decrease in accuracy compared to the training data (misclassification error of 0.24 vs. 0.21). The model’s prediction accuracy was improved when combined with the similarity approach (misclassification error of 0.14).


The model’s performance measures indicated a good generalizability for data acquired at different clinical institutions. Combining the model-based and similarity-based approach could further improve the assessment and interpretation of new cases, demonstrating its potential use for clinical risk assessment.


Cerebral aneurysm Risk factors Hemodynamics Shape Rupture Prediction 



The authors would like to thank Rafik Ouared and Olivier Brina for helping with the image segmentation of the AneuX data.


This study was funded by the National Institutes of Health/National Institute of Neurological Disorders and Stroke (NIH-NINDS, grant no. R21NS094780). NJ, SH, VMP, and PB were supported by project AneuX evaluated by the Swiss National Science Foundation.

Compliance with ethical standards

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. For this type of study, formal consent is not required.

Supplementary material

701_2018_3712_MOESM1_ESM.pdf (457 kb)
ESM 1 (PDF 457 kb)


  1. 1.
    Austin PC, Steyerberg EW (2014) Graphical assessment of internal and external calibration of logistic regression models by using loess smoothers. Stat Med 33(3):517–535CrossRefGoogle Scholar
  2. 2.
    Bijlenga P, Gondar R, Schilling S, Morel S, Hirsch S, Cuony J, Corniola M-V, Perren F, Rüfenacht D, Schaller K (2017) PHASES score for the management of intracranial aneurysm: a cross-sectional population-based retrospective study. Stroke 48(8):2105–2112CrossRefGoogle Scholar
  3. 3.
    Cebral JR, Castro MA, Appanaboyina S, Putman CM, Millan D, Frangi AF (2005) Efficient pipeline for image-based patient-specific analysis of cerebral aneurysm hemodynamics: technique and sensitivity. IEEE Trans Med Imaging 24(4):457–467CrossRefGoogle Scholar
  4. 4.
    Cebral JR, Castro MA, Putman CM, Alperin N (2008) Flow-area relationship in internal carotid and vertebral arteries. Physiol Meas 29:585–594CrossRefGoogle Scholar
  5. 5.
    Cebral JR, Sheridan MJ, Putman CM (2010) Hemodynamics and Bleb formation in intracranial aneurysms. AJNR Am J Neuroradiol 31:304–310CrossRefGoogle Scholar
  6. 6.
    Cebral J, Ollikainen E, Chung BJ, Mut F, Sippola V, Jahromi BR, Tulamo R, Hernesniemi J, Niemela M, Robertson AM, Frösen J (2017) Flow conditions in the intracranial aneurysm lumen are associated with inflammation and degenerative changes of the aneurysm wall. AJNR Am J Neuroradiol 38:119–126CrossRefGoogle Scholar
  7. 7.
    Chalouhi N, Ali MS, Jabbour PM, Tjoumakaris SI, Gonzalez LF, Rosenwasser RH, Koch WJ, Dumont AS (2012) Biology of intracranial aneurysms: role of inflammation. J Cereb Blood Flow Metab. CrossRefGoogle Scholar
  8. 8.
    Chung BJ, Mut F, Putman C, Hamzei-Sichani F, Brinjikji W, Kallmes DF, Cebral JR (2018) Identification of hostile hemodynamics and geometries of cerebral aneurysms: a case-control study. AJNR Am J Neuroradiol. CrossRefGoogle Scholar
  9. 9.
    Collins GS, Reitsma JB, Altman DG, Moons K (2015) Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement. BMC Med 13(1):1CrossRefGoogle Scholar
  10. 10.
    Detmer FJ, Chung BJ, Mut F, Slawski M, Hamzei-Sichani F, Putman C, Jiménez C, Cebral JR (2018) Development and internal validation of an aneurysm rupture probability model based on patient characteristics and aneurysm location, morphology, and hemodynamics. Int J Comput Assist Radiol Surg. CrossRefGoogle Scholar
  11. 11.
    Gabriel RA, Kim H, Sidney S, McCulloch CE, Singh V, Johnston SC, Ko NU, Achrol AS, Zaroff JG, Young WL (2010) Ten-year detection rate of brain arteriovenous malformations in a large, multiethnic, defined population. Stroke 41(1):21–26CrossRefGoogle Scholar
  12. 12.
    Gillani RL, Podraza KM, Luthra N, Origitano TC, Schneck MJ (2016) Factors influencing the management of unruptured intracranial aneurysms. Cureus.
  13. 13.
    Greving JP, Wermer MJ, Brown RD, Morita A, Juvela S, Yonekura M, Ishibashi T, Torner JC, Nakayama T, Rinkel GJ, Algra A (2014) Development of the PHASES score for prediction of risk of rupture of intracranial aneurysms: a pooled analysis of six prospective cohort studies. Lancet Neurol 13:59–66CrossRefGoogle Scholar
  14. 14.
    Hernandez M, Frangi AF (2007) Non-parametric geodesic active regions: method and evaluation for cerebral aneurysms segmentation in 3DRA and CTA. Med Image Anal 11:224–241CrossRefGoogle Scholar
  15. 15.
    Huang MC, Baaj AA, Downes K, Youssef AS, Sauvageau E, van Loveren HR, Agazzi S (2011) Paradoxical trends in the management of unruptured cerebral aneurysms in the United States: analysis of nationwide database over a 10-year period. Stroke 42(6):1730–1735CrossRefGoogle Scholar
  16. 16.
    Jabbarli R, Dinger TF, Darkwah Oppong M, Pierscianek D, Dammann P, Wrede KH, Kaier K, Köhrmann M, Forsting M, Kleinschnitz C, Sure U (2018) Risk factors for and clinical consequences of multiple intracranial aneurysms: a systematic review and meta-analysis. Stroke 49(4):848–855CrossRefGoogle Scholar
  17. 17.
    Japan Investigators UCAS, Morita A, Kirino T, Hashi K, Aoki N, Fukuhara S, Hashimoto N, Nakayama T, Sakai M, Teramoto A, Tominari S, Yoshimoto T (2012) The natural course of unruptured cerebral aneurysms in a Japanese cohort. N Engl J Med 366:2474–2482CrossRefGoogle Scholar
  18. 18.
    Juvela S, Poussa K, Lehto H, Porras M (2013) Natural history of unruptured intracranial aneurysms: a long-term follow-up study. Stroke 44:2414–2421CrossRefGoogle Scholar
  19. 19.
    Kleinloog R, de Mul N, Verweij BH, Post JA, Rinkel GJE, Ruigrok YM (2017) Risk factors for intracranial aneurysm rupture: a systematic review. Neurosurgery.
  20. 20.
    Korja M, Kaprio J (2016) Controversies in epidemiology of intracranial aneurysms and SAH. Nat Rev Neurol 12(1):50–55CrossRefGoogle Scholar
  21. 21.
    Korja M, Lehto H, Juvela S (2014) Lifelong rupture risk of intracranial aneurysms depends on risk factors: a prospective Finnish cohort study. Stroke 45:1958–1963CrossRefGoogle Scholar
  22. 22.
    Korja M, Lehto H, Juvela S, Kaprio J (2016) Incidence of subarachnoid hemorrhage is decreasing together with decreasing smoking rates. Neurology 87(11):1118–1123CrossRefGoogle Scholar
  23. 23.
    Lindgren AE, Koivisto T, Björkman J, von und zu Fraunberg M, Helin K, Jääskeläinen JE, Frösen J (2016) Irregular shape of intracranial aneurysm indicates rupture risk irrespective of size in a population-based cohort. Stroke 47(5):1219–1226CrossRefGoogle Scholar
  24. 24.
    Piccinelli M, Bacigaluppi S, Boccardi E, Ene-Iordache B, Remuzzi A, Veneziani A, Antiga L (2011) Geometry of the internal carotid artery and recurrent patterns in location, orientation, and rupture status of lateral aneurysms: an image-based computational study. Neurosurgery 68(5):1270–1285CrossRefGoogle Scholar
  25. 25.
    R Core Team (2017) R: a language and environment for statistical computing. Version 3.3.3. R Foundation for Statistical Computing, Vienna Accessed on Sept 2018Google Scholar
  26. 26.
    Rinkel GJ, Djibuti M, van Gijn J (1998) Prevalence and risk of rupture of intracranial aneurysms: a systematic review. Stroke 29:251–259CrossRefGoogle Scholar
  27. 27.
    Rivero-Arias O, Gray A, Wolstenholme J (2010) Burden of disease and costs of aneurysmal subarachnoid haemorrhage (aSAH) in the United Kingdom. Cost Eff Resour Alloc 8(1):6CrossRefGoogle Scholar
  28. 28.
    Sangalli LM, Secchi P, Vantini S (2014) AneuRisk65: a dataset of three-dimensional cerebral vascular geometries. Electron J Stat 8(2):1879–1890CrossRefGoogle Scholar
  29. 29.
    Steiner T, Juvela S, Unterberg A, Jung C, Forsting M, Rinkel G (2013) European stroke organization guidelines for the management of intracranial aneurysms and subarachnoid haemorrhage. Cerebrovasc Dis 35(2):93–112CrossRefGoogle Scholar
  30. 30.
    Thompson BG, Brown RD, Amin-Hanjani S, Broderick JP, Cockroft KM, Connolly ES, Duckwiler GR, Harris CC, Howard VJ, Johnston SC, Meyers PM, Molyneux A, Ogilvy CS, Ringer AJ, Torner J (2015) Guidelines for the management of patients with unruptured intracranial aneurysms: a guideline for healthcare professionals from the American Heart Association/American Stroke Association. Stroke 46(8):2368–2400CrossRefGoogle Scholar
  31. 31.
    Tulamo R, Frosen J, Hernesniemi J, Niemela M (2010) Inflammatory changes in the aneurysm wall: a review. J Neurointerv Surg 2:120–130CrossRefGoogle Scholar
  32. 32.
    Villa-Uriol MC, Berti G, Hose DR, Marzo A, Chiarini A, Penrose J, Pozo J, Schmidt JG, Singh P, Lycett R, Larrabide I, Frangi AF (2011) @neurIST complex information processing toolchain for the integrated management of cerebral aneurysms. Interface Focus 1(3):308–319CrossRefGoogle Scholar
  33. 33.
    Vlak MH, Algra A, Brandenburg R, Rinkel GJ (2011) Prevalence of unruptured intracranial aneurysms, with emphasis on sex, age, comorbidity, country, and time period: a systematic review and meta-analysis. Lancet Neurol 10:626–636CrossRefGoogle Scholar
  34. 34.
    Wang G, Zhang Z, Ayala C, Dunet DO, Fang J, George MG (2014) Costs of hospitalization for stroke patients aged 18-64 years in the United States. J Stroke Cerebrovasc Dis 23(5):861–868CrossRefGoogle Scholar
  35. 35.
    Wermer MJ, van der Schaaf IC, Algra A, Rinkel GJ (2007) Risk of rupture of unruptured intracranial aneurysms in relation to patient and aneurysm characteristics: an updated meta-analysis. Stroke 38:1404–1410CrossRefGoogle Scholar
  36. 36.
    Wiebers DO, Whisnant JP, Huston J, Meissner I, Brown RD, Piepgras DG, Forbes GS, Thielen K, Nichols D, O’Fallon WM, Peacock J, Jaeger L, Kassell NF, Kongable-Beckman GL, Torner JC (2003) Unruptured intracranial aneurysms: natural history, clinical outcome, and risks of surgical and endovascular treatment. Lancet 362:103–110CrossRefGoogle Scholar
  37. 37.
    Xiang J, Varble N, Davies JM, Rai AT, Kono K, Sugiyama S, Binning MJ, Tawk RG, Choi H, Ringer AJ, Snyder KV, Levy EI, Hopkins LN, Siddiqui AH, Meng H (2017) Initial clinical experience with AView—a clinical computational platform for intracranial aneurysm morphology, hemodynamics, and treatment management. World Neurosurg 108:534–542CrossRefGoogle Scholar
  38. 38.
    Yang X, Liu C, Le Minh H, Wang Z, Chien A, Cheng K-T (2017) An automated method for accurate vessel segmentation. Phys Med Biol 62(9):3757–3778CrossRefGoogle Scholar

Copyright information

© Springer-Verlag GmbH Austria, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Bioengineering Department, Volgenau School of EngineeringGeorge Mason UniversityFairfaxUSA
  2. 2.Institute of Applied SimulationZHAW University of Applied SciencesWaedenswilSwitzerland
  3. 3.Institute of PhysiologyUniversity of ZurichZurichSwitzerland
  4. 4.Interventional Neuroradiology Unit, Service of Neuroradiology, Faculty of MedicineUniversity of GenevaGenevaSwitzerland
  5. 5.Neurosurgery, Clinical Neurosciences Department, Faculty of MedicineUniversity of GenevaGenevaSwitzerland

Personalised recommendations