Annals of Biomedical Engineering

, Volume 43, Issue 1, pp 154–167 | Cite as

Accuracy and Reproducibility of Patient-Specific Hemodynamic Models of Stented Intracranial Aneurysms: Report on the Virtual Intracranial Stenting Challenge 2011

  • S. CitoEmail author
  • A. J. GeersEmail author
  • M. P. Arroyo
  • V. R. Palero
  • J. Pallarés
  • A. Vernet
  • J. Blasco
  • L. San Román
  • W. Fu
  • A. Qiao
  • G. Janiga
  • Y. Miura
  • M. Ohta
  • M. Mendina
  • G. Usera
  • A. F. Frangi


Validation studies are prerequisites for computational fluid dynamics (CFD) simulations to be accepted as part of clinical decision-making. This paper reports on the 2011 edition of the Virtual Intracranial Stenting Challenge. The challenge aimed to assess the reproducibility with which research groups can simulate the velocity field in an intracranial aneurysm, both untreated and treated with five different configurations of high-porosity stents. Particle imaging velocimetry (PIV) measurements were obtained to validate the untreated velocity field. Six participants, totaling three CFD solvers, were provided with surface meshes of the vascular geometry and the deployed stent geometries, and flow rate boundary conditions for all inlets and outlets. As output, they were invited to submit an abstract to the 8th International Interdisciplinary Cerebrovascular Symposium 2011 (ICS’11), outlining their methods and giving their interpretation of the performance of each stent configuration. After the challenge, all CFD solutions were collected and analyzed. To quantitatively analyze the data, we calculated the root-mean-square error (RMSE) over uniformly distributed nodes on a plane slicing the main flow jet along its axis and normalized it with the maximum velocity on the slice of the untreated case (NRMSE). Good agreement was found between CFD and PIV with a NRMSE of 7.28%. Excellent agreement was found between CFD solutions, both untreated and treated. The maximum difference between any two groups (along a line perpendicular to the main flow jet) was 4.0 mm/s, i.e. 4.1% of the maximum velocity of the untreated case, and the average NRMSE was 0.47% (range 0.28–1.03%). In conclusion, given geometry and flow rates, research groups can accurately simulate the velocity field inside an intracranial aneurysm—as assessed by comparison with in vitro measurements—and find excellent agreement on the hemodynamic effect of different stent configurations.


Intracranial aneurysm Stents Computational fluid dynamics Particle imaging velocimetry Challenge 



S. Cito, J. Pallares and A. Vernet received funding from projects DPI2010-17212 and CTQ2013-46799-C2-1-P of the Spanish Ministerio de Economía y Competitividad, S. Cito, A.J. Geers and A.F. Frangi received funding through the Spanish project cvREMOD (CEN-20091044, funded by the CENIT programme of the Industrial and Technological Development Center) and A. Qiao received funding through the National Natural Science Foundation of China (81171107).

Conflict of interest

None of the authors in this work has conflict of interests with other people and organizations.


  1. 1.
    Antiga, L., M. Piccinelli, L. A. Botti, B. Ene-Iordache, A. Remuzzi, and D. A. Steinman. An image-based modeling framework for patient-specific computational hemodynamics. Med. Biol. Eng. Comput. 46(11):1097–1112, 2008.PubMedCrossRefGoogle Scholar
  2. 2.
    Appanaboyina, S., F. Mut, R. Löhner, C. M. Putman, and J. R. Cebral. Simulation of intracranial aneurysm stenting: techniques and challenges. Comput. Methods Appl. Mech. Eng. 198(45–46):3567–3582, 2009.CrossRefGoogle Scholar
  3. 3.
    Augsburger, L., P. Reymond, D. A. Rüfenacht, and N. Stergiopulos. Intracranial stents being modeled as a porous medium: flow simulation in stented cerebral aneurysms. Ann. Biomed. Eng. 39(2):850–863, 2011.PubMedCrossRefGoogle Scholar
  4. 4.
    Babiker, M. H., B. Chong, L. F. Gonzalez, S. Cheema, and D. H. Frakes. Finite element modeling of embolic coil deployment: multifactor characterization of treatment effects on cerebral aneurysm hemodynamics. J. Biomech. 46(16):2809–2816, 2013.PubMedCrossRefGoogle Scholar
  5. 5.
    Babiker, M. H., L. F. Gonzalez, J. Ryan, F. Albuquerque, D. Collins, A. Elvikis, and D. H. Frakes. Influence of stent configuration on cerebral aneurysm fluid dynamics. J. Biomech. 45(3):440–447, 2012.PubMedCrossRefGoogle Scholar
  6. 6.
    Bernardini, A., I. Larrabide, H. G. Morales, G. Pennati, S. Cito L. Petrini, and A. F. Frangi. Influence of different computational approaches for stent deployment on cerebral aneurysm haemodynamics. Interface Focus 1(3):338–348, 2011.PubMedCentralPubMedCrossRefGoogle Scholar
  7. 7.
    Cavazzuti, M., M. Atherton, M. Collins, and G. Barozzi. Beyond the Virtual Intracranial Stenting Challenge 2007: non-Newtonian and flow pulsatility effects. J. Biomech. 43(13):2645–2647, 2010.PubMedCrossRefGoogle Scholar
  8. 8.
    Cebral, J. R., F. Mut, M. Raschi, E. Scrivano, R. Ceratto, P. Lylyk, and C. M. Putman. Aneurysm rupture following treatment with flow-diverting stents: computational hemodynamics analysis of treatment. Am. J. Neuroradiol. 32(1):27–33, 2011.PubMedGoogle Scholar
  9. 9.
    Cekirge, H. S., K. Yavuz, S. Geyik, and I. Saatci. A novel “Y” stent flow diversion technique for the endovascular treatment of bifurcation aneurysms without endosaccular coiling. Am. J. Neuroradiol. 32(7):1262–1268, 2011.PubMedCrossRefGoogle Scholar
  10. 10.
    Cito, S., A. J. Geers, M. P. Arroyo, J. Pallares, A. Vernet, V. R. Palero, J. Lobera, J. Blasco, L. San-Roman, and A. F. Frangi. Virtual Intracranial Stenting Challenge 2011. 8th International Interdisciplinary Cerebrovascular Symposium, Shanghai, China, 2011.Google Scholar
  11. 11.
    Ford, M. D., H. N. Nikolov, J. S. Milner, S. P. Lownie, E. M. Demont, W. Kalata, F. Loth, D. W. Holdsworth, and D. A. Steinman. PIV-measured vs. CFD-predicted flow dynamics in anatomically realistic cerebral aneurysm models. J. Biomech. Eng. 130(2):21015, 2008.CrossRefGoogle Scholar
  12. 12.
    Fu, W., Z. Gu, X. Meng, B. Chu, and A. K. Qiao. Numerical simulation of hemodynamics in stented internal carotid aneurysm based on patient-specific model. J. Biomech. 43(7):1337–1342, 2010.PubMedCrossRefGoogle Scholar
  13. 13.
    Geers, A. J., I. Larrabide, A. G. Radaelli, H. Bogunović, M. Kim, H. A. F. Gratama van Andel, C. B. Majoie, E. VanBavel, and A. F. Frangi. Patient-specific computational hemodynamics of intracranial aneurysms from 3D rotational angiography and CT angiography: an in vivo reproducibility study. Am. J. Neuroradiol. 32(3):581–586, 2011.PubMedCrossRefGoogle Scholar
  14. 14.
    Grinberg L., and G. E. Karniadakis. Outflow boundary conditions for arterial networks with multiple outlets. Ann. Biomed. Eng. 36(9):1496–1514, 2008.PubMedCrossRefGoogle Scholar
  15. 15.
    Hoi, Y., S. H. Woodward, M. Kim, D. B. Taulbee, and H. Meng. Validation of CFD simulations of cerebral aneurysms with implication of geometric variations. J. Biomech. Eng. 128(6):844–851, 2006.PubMedCentralPubMedCrossRefGoogle Scholar
  16. 16.
    Hunter, J. D. Matplotlib: a 2D graphics environment. Comput. Sci. Eng. 9(3):90–95, 2007.CrossRefGoogle Scholar
  17. 17.
    Izar, B., A. Rai, K. Raghuram, J. Rotruck, and J. Carpenter. Comparison of devices used for stent-assisted coiling of intracranial aneurysms. PLOS ONE 6(9):e24875, 2011.PubMedCentralPubMedCrossRefGoogle Scholar
  18. 18.
    Janiga, G., C. Rössl, M. Skalej, and D. Thévenin. Realistic virtual intracranial stenting and computational fluid dynamics for treatment analysis. J. Biomech. 46(1):7–12, 2013.PubMedCrossRefGoogle Scholar
  19. 19.
    Kakalis, N. M., A. P. Mitsos, J. V. Byrne, and Y. P. Ventikos. The haemodynamics of endovascular aneurysm treatment: a computational modelling approach for estimating the influence of multiple coil deployment. IEEE Trans. Med. Imaging 27(6):814–824, 2008.PubMedCrossRefGoogle Scholar
  20. 20.
    Karmonik, C., C. Yen, O. Diaz, R. Klucznik, R. G. Grossman, and G. Benndorf. Temporal variations of wall shear stress parameters in intracranial aneurysms-importance of patient-specific inflow waveforms for CFD calculations. Acta Neurochirurgica 152(8):1391–1398, 2010.PubMedCrossRefGoogle Scholar
  21. 21.
    Kim, M., E. I. Levy, H. Meng, and L. N. Hopkins. Quantification of hemodynamic changes induced by virtual placement of multiple stents across a wide-necked basilar trunk aneurysm. Neurosurgery 61(6):1305–1312, 2007.PubMedCentralPubMedCrossRefGoogle Scholar
  22. 22.
    Kono K., and T. Terada. Hemodynamics of 8 different configurations of stenting for bifurcation aneurysms. Am. J. Neuroradiol. 34(10):1980–1986, 2013.PubMedCrossRefGoogle Scholar
  23. 23.
    Larrabide, I., M. L. Aguilar, H. G. Morales, A. J. Geers, Z. Kulcsár, D. Rüfenacht, and A.F. Frangi. Intra-aneurysmal pressure and flow changes induced by flow diverters: relation to aneurysm size and shape. Am. J. Neuroradiol. 34(4):816–822, 2013.PubMedCrossRefGoogle Scholar
  24. 24.
    Larrabide, I., M. Kim, L. Augsburger, M. C. Villa-Uriol, D. A. Rüfenacht, and A.F. Frangi. Fast virtual deployment of self-expandable stents: method and in vitro evaluation for intracranial aneurysmal stenting. Med. Image Anal. 16(3):721–730, 2012.PubMedCrossRefGoogle Scholar
  25. 25.
    Lawson, M. F., W. C. Newman, Y. Chi, J. D. Mocco, and B. L. Hoh. Stent-associated flow remodeling causes further occlusion of incompletely coiled aneurysms. Neurosurgery 69(3):598–603, 2011.PubMedCrossRefGoogle Scholar
  26. 26.
    Ma, D., J. Xiang, H. Choi, T. M. Dumont, S. K. Natarajan, A. H. Siddiquiand, and H. Meng. Enhanced aneurysmal flow diversion using a dynamic push-pull technique: an experimental and modeling study. Am. J. Neuroradiol. 2014. DOI:  10.3174/ajnr.A3933.Google Scholar
  27. 27.
    McGah, P. M., M. R. Levitt, M. C. Barbour, R. P. Morton, J. D. Nerva, P. D. Mourad, B. V. Ghodke, D. K.Hallam, L. N. Sekhar, L. J. Kim, and A. Aliseda. Accuracy of computational cerebral aneurysm hemodynamics using patient-specific endovascular measurements. Am. J. Neuroradiol. 42(3):503–514, 2014.Google Scholar
  28. 28.
    Metcalfe, R. W. The promise of computational fluid dynamics as a tool for delineating therapeutic options in the treatment of aneurysms. Am. J. Neuroradiol. 24(4):553–554, 2003.PubMedGoogle Scholar
  29. 29.
    Morales, H. G., I. Larrabide, A. J. Geers, L. San Roman, J. Blasco, J. M. Macho, and A. F. Frangi. A virtual coiling technique for image-based aneurysm models by dynamic path planning. IEEE Trans. Med. Imaging 32(1):119–129, 2013.PubMedCrossRefGoogle Scholar
  30. 30.
    Nealen, A., T. Igarashi, O. Sorkine, and M. Alexa. Laplacian mesh optimization. Proceedings of the 4th International Conference on Computer Graphics and Interactive Techniques in Australasia and Southeast Asia (GRAPHITE), Kuala Lumpur, Malaysia, 2006, pp. 381–389.Google Scholar
  31. 31.
    Palero, V., J. Lobera, and M. Arroyo. Three-component velocity field measurement in confined liquid flows with high-speed digital image plane holography. Exp. Fluids 49(2):471–483, 2010.CrossRefGoogle Scholar
  32. 32.
    Radaelli, A. G., L. Augsburger, J. R. Cebral, M. Ohta, D. A. Rüfenacht, R. Balossino, G. Benndorf, D. R. Hose, A. Marzo, R. W. Metcalfe, P. Mortier, F. Mut, P. Reymond, L. Socci, B. Verhegghe, and A. F. Frangi. Reproducibility of haemodynamical simulations in a subject-specific stented aneurysm model: a report on the Virtual Intracranial Stenting Challenge 2007. J. Biomech. 41(10):2069–2081, 2008.PubMedCrossRefGoogle Scholar
  33. 33.
    Raffel, M., C. E. Willert, S. T. Wereley, and J. Kompenhans. Particle image velocimetry: a practical guide, 2nd ed. Berlin: Springer, 2007.Google Scholar
  34. 34.
    Raschi, M., F. Mut, G. Byrne, C. M. Putman, S. Tateshima, F. Viñuela, T. Tanoue, K. Tanishita, and J. R. Cebral. CFD and PIV analysis of hemodynamics in a growing intracranial aneurysm. Int. J. Numer. Methods Biomed. Eng. 28(2):214–228, 2012.PubMedCentralPubMedCrossRefGoogle Scholar
  35. 35.
    Schneiders, J. J., H. A. Marquering, L. Antiga, R. Van den Berg, E. VanBavel, and C. B. Majoie. Intracranial aneurysm neck size overestimation with 3D rotational angiography: the impact on intra-aneurysmal hemodynamics simulated with computational fluid dynamics. Am. J. Neuroradiol. 34(1):121–128, 2013.PubMedCrossRefGoogle Scholar
  36. 36.
    Sforza, D. M., C. M. Putman, and J. R. Cebral. Hemodynamics of cerebral aneurysms. Annu. Rev. Fluid Mech. 41:91–107, 2009.PubMedCentralPubMedCrossRefGoogle Scholar
  37. 37.
    Shimogonya, Y., T. Ishikawa, Y. Imai, N. Matsuki, and T. Yamaguchi. Can temporal fluctuation in spatial wall shear stress gradient initiate a cerebral aneurysm? A proposed novel hemodynamic index, the gradient oscillatory number (GON). J. Biomech. 42(4):550–554, 2009.PubMedCrossRefGoogle Scholar
  38. 38.
    Steinman, D. A., Y. Hoi, P. Fahy, L. Morris, M. T. Walsh, N. Aristokleous, A. S. Anayiotos, Y. Papaharilaou, A. Arzani, S. C. Shadden, P. Berg, G. Janiga, J. Bols, P. Segers, N. W. Bressloff, M. Cibis, F. H. Gijsen, S. Cito, J. Pallareacutes, L. D. Browne, J. A. Costelloe, A. G. Lynch, J. Degroote, J. Vierendeels, W. Fu, A. Qiao, S. Hodis, D. F. Kallmes, H. Kalsi, Q. Long, V. O. Kheyfets, E. A. Finol, K. Kono, A. M. Malek, A. Lauric, P. G. Menon, K. Pekkan, M. Esmaily Moghadam, A. L. Marsden, M. Oshima, K. Katagiri, V. Peiffer, Y. Mohamied, S. J. Sherwin, J. Schaller, L. Goubergrits, G. Usera, M. Mendina, K. Valen-Sendstad, D. F. Habets, J. Xiang, H. Meng, Y. Yu, G. E. Karniadakis, N. Shaffer, and F. Loth. Variability of computational fluid dynamics solutions for pressure and flow in a giant aneurysm: the ASME 2012 Summer Bioengineering Conference CFD challenge. J. Biomech. Eng. 135(2):021016, 2013.Google Scholar
  39. 39.
    Steinman, D. A., and C. A. Taylor. Flow imaging and computing: large artery hemodynamics. Ann. Biomed. Eng. 33(12):1704–1709, 2005.PubMedCrossRefGoogle Scholar
  40. 40.
    Tremmel, M., J. Xiang, S. K. Natarajan, L. N. Hopkins, A. H. Siddiqui, E. I. Levy, and H. Meng. Alteration of intra-aneurysmal hemodynamics for flow diversion using enterprise and vision stents. World Neurosurg. 74(716):306–315, 2010.PubMedCentralPubMedCrossRefGoogle Scholar
  41. 41.
    Ugron, Á., M. Farinas, L. Kiss, and G. Paál. Unsteady velocity measurements in a realistic intracranial aneurysm model. Exp. Fluids 52(1):37–52, 2011.Google Scholar
  42. 42.
    Usera, G., A. Vernet, and J. A. Ferré. A parallel block-structured finite volume method for flows in complex geometry with sliding interfaces. Flow Turbul. Combust. 81(3):471–495, 2008.CrossRefGoogle Scholar
  43. 43.
    van Ooij, P., A. Guédon, C. Poelma, J. Schneiders, M. C. M. Rutten, H. A. Marquering, C. B. Majoie, E. VanBavel, and A. J. Nederveen. Complex flow patterns in a real-size intracranial aneurysm phantom: phase contrast MRI compared with particle image velocimetry and computational fluid dynamics. NMR Biomed. 25(1):14–26, 2012.PubMedCrossRefGoogle Scholar
  44. 44.
    Wootton, D. M., and D. N. Ku. Fluid mechanics of vascular systems, diseases, and thrombosis. Annu. Rev. Biomed. Eng. 1:299–329, 1999.PubMedCrossRefGoogle Scholar
  45. 45.
    Zenteno, M. A., J. A. Santos-Franco, J. M. Freitas-Modenesi, C. Gómez, L. Murillo-Bonilla, Y. Aburto-Murrieta, R. Díaz-Romero, E. Nathal, S. Gómez-Llata, and A. Lee. Use of the sole stenting technique for the management of aneurysms in the posterior circulation in a prospective series of 20 patients. J. Neurosurg. 108(6):1104–18, 2008.PubMedCrossRefGoogle Scholar

Copyright information

© Biomedical Engineering Society 2014

Authors and Affiliations

  • S. Cito
    • 1
    • 2
    • 3
    Email author
  • A. J. Geers
    • 3
    Email author
  • M. P. Arroyo
    • 4
  • V. R. Palero
    • 4
  • J. Pallarés
    • 2
  • A. Vernet
    • 2
  • J. Blasco
    • 5
  • L. San Román
    • 5
  • W. Fu
    • 6
  • A. Qiao
    • 7
  • G. Janiga
    • 8
  • Y. Miura
    • 9
  • M. Ohta
    • 9
  • M. Mendina
    • 10
  • G. Usera
    • 10
  • A. F. Frangi
    • 3
    • 11
  1. 1.University of HelsinkiHelsinkiFinland
  2. 2.Universitat Rovira i VirgiliTarragonaSpain
  3. 3.Universitat Pompeu FabraBarcelonaSpain
  4. 4.Universidad de ZaragozaZaragozaSpain
  5. 5.Hospital ClínicBarcelonaSpain
  6. 6.Beijing Union UniversityBeijingChina
  7. 7.Beijing University of TechnologyBeijingChina
  8. 8.University of MagdeburgMagdeburgGermany
  9. 9.Tohoku UniversityTohokuJapan
  10. 10.Universidad de la RepúblicaMontevideoUruguay
  11. 11.University of SheffieldSheffieldUK

Personalised recommendations