Computational Mechanics

, Volume 46, Issue 1, pp 71–82 | Cite as

Cardiovascular flow simulation at extreme scale

  • Min Zhou
  • Onkar Sahni
  • H. Jin Kim
  • C. Alberto Figueroa
  • Charles A. Taylor
  • Mark S. Shephard
  • Kenneth E. JansenEmail author
Original Paper


As cardiovascular models grow more sophisticated in terms of the geometry considered, and more physiologically realistic boundary conditions are applied, and fluid flow is coupled to structural models, the computational complexity grows. Massively parallel adaptivity and flow solvers with extreme scalability enable cardiovascular simulations to reach an extreme scale while keeping the time-to-solution reasonable. In this paper, we discuss our efforts in this area and provide two demonstrations: one on an extremely large and complex geometry (including many of the major arteries below the neck) where the solution is efficiently captured with anisotropic adaptivity and another case (severe abdominal aorta aneurysm) where the transitional flow leads to extremely large meshes (O(109)) and scalability to extremely large core counts (O(105)) for both rigid and deforming wall simulations.


Wall Shear Stress Abdominal Aorta Aneurysm Strong Scaling Extreme Scale Boundary Layer Mesh 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Bazilevs Y, Gohean JR, Hughes TJR, Moser RD, Zhang Y (2009) Patientspecific isogeometric fluid–structure interaction analysis of thoracic aortic blood flow due to implantation of the jarvik 2000 left ventricular assist device. Comp Methods Appl Mech Eng 198: 3534–3550CrossRefMathSciNetzbMATHGoogle Scholar
  2. 2.
    Bazilevs Y, Hsu M-C, Benson DJ, Sankaran S, Marsden AL (2009) Computational fluidstructure interaction: methods and application to a total cavopulmonary connection. Comput Mech (submitted)Google Scholar
  3. 3.
    Boman E, Devine K, Fisk LA, Heaphy R, Hendrickson B, Leung V, Vaughan C, Catalyurek U, Bozdag D, Mitchell W (1999) Zoltan home page.
  4. 4.
    Brooks AN, Hughes TJR (1982) Streamline upwind/petrov-galerkin formulations for convection dominated flows with particular emphasis on the incompressible Navier–Stokes equations. Comp Methods Appl Mech Eng 32: 199–259CrossRefMathSciNetzbMATHGoogle Scholar
  5. 5.
    Cebral JR, Castro MA, Burgess JE, Pergolizzi RS, Sheridan MJ, Putman CM (2005) Characterization of cerebral aneurysms for assessing risk of rupture by using patient-specific computational hemodynamics models. AJNR Am J Neuroradiol 26(10): 2550–2559Google Scholar
  6. 6.
    Farhat C, Geuzaine P (2004) Design and analysis of robust ale time-integrators for the solution of unsteady flow problems on moving grids. Comp Methods Appl Mech Eng 193: 4073–4095CrossRefMathSciNetzbMATHGoogle Scholar
  7. 7.
    Fernandez MA, Le Tallec P (2003) Linear stability analysis in fluid–structure interaction with transpiration. Part ii: numerical analysis and applications. Comp Methods Appl Mech Eng 192: 4837–4873CrossRefMathSciNetzbMATHGoogle Scholar
  8. 8.
    Figueroa CA, Vignon-Clementel IE, Jansen KE, Hughes TJR, Taylor CA (2006) A coupled momentum method for modeling blood flow in three-dimensional deformable arteries. Comp Methods Appl Mech Eng 195(41–43): 5685–5706CrossRefMathSciNetzbMATHGoogle Scholar
  9. 9.
    Formaggia L, Gerbeau JF, Nobile F, Quarteroni A (2002) Numerical treatment of defective boundary conditions for the Navier–Stokes equations. SIAM J Numer Anal 40(1): 376–401CrossRefMathSciNetzbMATHGoogle Scholar
  10. 10.
    Franca LP, Frey SL (1992) Stabilized finite element methods ii. the incompressible Navier–Stokes equations. Comp Methods Appl Mech Eng 99(2–3): 209–233CrossRefMathSciNetzbMATHGoogle Scholar
  11. 11.
    Jansen KE, Whiting CH, Hulbert GM (2000) Generalized-α method for integrating the filtered Navier–Stokes equations with a stabilized finite element method. Comp Methods Appl Mech Eng 190(3–4): 305–319MathSciNetzbMATHGoogle Scholar
  12. 12.
    Karypis G, Kumar V (1999) Parallel multilevel k-way partitioning scheme for irr. graphs. SIAM Rev 41: 278–300CrossRefMathSciNetzbMATHGoogle Scholar
  13. 13.
    LaDisa JF, Olson LE, Molthen RC, Hettrick DA, Pratt PF, Hardel MD, Kersten JR, Warltier DC, Pagel PS (2005) Alterations in wall shear stress predict sites of neointimal hyperplasia after stent implantation in rabbit iliac arteries. Am J Phys Heart Circ Phys 288: H2465–H2475CrossRefGoogle Scholar
  14. 14.
    Laskey WK, Parker HG, Ferrari VA, Kussmaul WG, Noordergraaf A (1990) Estimation of total systemic arterial compliance in humans. J Appl Physiol 69(1): 112–119Google Scholar
  15. 15.
    Les AS, Shadden SC, Figueroa CA, Park JM, Tedesco MM, Herfkens RJ, Taylor CA, Dalman RL (2009) Quantification of hemodynamics in abdominal aortic aneurysms during rest and exercise using magnetic resonance imaging and computational fluid dynamic. Ann Biomed Eng (submitted)Google Scholar
  16. 16.
    Li Z, Kleinstreuer C (2005) Blood flow and structure interactions in a stented abdominal aortic aneurysm model. Med Eng Phys 27(5): 369–382CrossRefGoogle Scholar
  17. 17.
    Migliavacca F, Balossino R, Pennati G, Dubini G, Hsia TY, de Leval MR, Bove EL (2006) Multiscale modelling in biofluidynamics: application to reconstructive paediatric cardiac surgery. J Biomech 39(6): 1010–1020CrossRefGoogle Scholar
  18. 18.
    Perktold K, Peter R, Resch M (1989) Pulsatile non-newtonian blood flow simulation through a bifurcation with an aneurysm. Biorheology 26(6): 1011–1030Google Scholar
  19. 19.
    Peskin CS, McQueen DM (1995) A general method for the computer simulation of biological systems interacting with fluids. Symp Soc Exp Biol 49: 265–276Google Scholar
  20. 20.
    Quarteroni A, Ragni S, Veneziani A (2001) Coupling between lumped and distributed models for blood flow problems. Comp Vis Sci 4(2): 111–124CrossRefMathSciNetzbMATHGoogle Scholar
  21. 21.
    Quarteroni A, Tuveri M, Veneziani A (2000) Computational vascular fluid dynamics: problems, models and methods. Comp Vis Sci 2(4): 163–197CrossRefzbMATHGoogle Scholar
  22. 22.
    Saad Y, Schultz MH (1986) GMRES: a generalized minimal residual algorithm for solving nonsymmetric linear systems. SIAM J Sci Stat Comput 7: 856–869CrossRefMathSciNetzbMATHGoogle Scholar
  23. 23.
    Sahni O, Müller Y, Jansen KE, Shephard MS, Taylor CA (2006) Efficient anisotropic adaptive discretization of the cardiovascular system. Comp Methods Appl Mech Eng 195: 5634–5655CrossRefzbMATHGoogle Scholar
  24. 24.
    Sahni O, Zhou M, Shephard MS, Jansen KE (2009) Scalable implicit finite element solver for massively parallel processing with demonstration to 160k cores. In: Proceedings of IEEE/ACM SC’09, Finalist paper for the Gordon Bell PrizeGoogle Scholar
  25. 25.
    Shephard MS, Jansen KE, Sahni O, Diachin LA (2007) Parallel adaptive simulations on unstructured meshes. J Phys Conf Ser 78. doi: 10.1088/1742-6596/78/1/012053
  26. 26.
    Soerensen DD, Pekkan K, de Zelicourt D, Sharma S, Kanter K, Fogel M, Yoganathan AP (2007) Introduction of a new optimized total cavopulmonary connection. Ann Thorac Surg 83(6): 2182–2190CrossRefGoogle Scholar
  27. 27.
    Stergiopulos N, Segers P, Westerhof N (1999) Use of pulse pressure method for estimating total arterial compliance in vivo. Am J Physiol Heart Circ Physiol 276(2): H424–H428Google Scholar
  28. 28.
    Stuhne GR, Steinman DA (2004) Finite-element modeling of the hemodynamics of stented aneurysms. J Biomech Eng 126(3): 382–387CrossRefGoogle Scholar
  29. 29.
    Tang BT, Cheng CP, Draney MT, Wilson NM, Tsao PS, Herfkens RJ, Taylor CA (2006) Abdominal aortic hemodynamics in young healthy adults at rest and during lower limb exercise: quantification using image-based computer modeling. Am J Physiol Heart Circ Physiol 291(2): H668–H676CrossRefGoogle Scholar
  30. 30.
    Taylor CA, Hughes TJR, Zarins CK (1998) Finite element modeling of blood flow in arteries. Comput Methods Appl Mech Eng 158(1–2): 155–196CrossRefMathSciNetzbMATHGoogle Scholar
  31. 31.
    Taylor CA, Draney MT (2004) Experimental and computational methods in cardiovascular fluid mechanics. Annu Rev Fluid Mech 36: 197–231CrossRefMathSciNetGoogle Scholar
  32. 32.
    Taylor CA, Draney MT, Ku JP, Parker D, Steele BN, Wang K, Zarins CK (1999) Predictive medicine: computational techniques in therapeutic decision-making. Comp Aided Surg 4(5): 231–247CrossRefGoogle Scholar
  33. 33.
    Tezduyar TE, Behr M, Liou J (1992) A new strategy for finite element computations involving moving boundaries and interfaces -the deforming-spatial-domain/space–time procedure: I. the concept and the preliminary numerical tests. Comput Methods Appl Mech Eng 94: 339–351CrossRefMathSciNetzbMATHGoogle Scholar
  34. 34.
    Vignon-Clementel IE, Figueroa CA, Jansen KE, Taylor CA (2008) Outflow boundary conditions for three-dimensional simulations of non-periodic blood flow and pressure fields in deformable arteries. Comput Meth Biomech Biomed Eng (submitted)Google Scholar
  35. 35.
    Vignon-Clementel IE, Figueroa CA, Jansen KE, Taylor CA (2006) Outflow boundary conditions for three-dimensional finite element modeling of blood flow and pressure in arteries. Comp Methods Appl Mech Eng 195(29–32): 3776–3796CrossRefMathSciNetzbMATHGoogle Scholar
  36. 36.
    Whiting CH, Jansen KE (2001) A stabilized finite element method for the incompressible Navier–Stokes equations using a hierarchical basis. Int J Numer Meth Fluids 35: 93–116CrossRefzbMATHGoogle Scholar
  37. 37.
    Womersley J (1955) Method for the calculation of velocity, rate of flow and viscous drag in arteries when the pressure gradient is known. J Physiol 127: 553–563Google Scholar
  38. 38.
    Zamir M, Sinclair P, Wonnacott TH (1992) Relation between diameter and flow in major branches of the arch of the aorta. J Biomech 25(11): 1303–1310CrossRefGoogle Scholar
  39. 39.
    Zhou M (2009) Petascale adaptive computational fluid dynamics. Ph.D. thesis, Rensselaer Polytechnic InstituteGoogle Scholar

Copyright information

© Springer-Verlag 2009

Authors and Affiliations

  • Min Zhou
    • 1
  • Onkar Sahni
    • 1
  • H. Jin Kim
    • 1
  • C. Alberto Figueroa
    • 2
  • Charles A. Taylor
    • 2
  • Mark S. Shephard
    • 1
  • Kenneth E. Jansen
    • 1
    Email author
  1. 1.SCOREC, Department of Mechanical, Aerospace, and Nuclear EngineeringRensselaer Polytechnic InstituteTroyUSA
  2. 2.Bioengineering DepartmentStanford UniversityStanfordUSA

Personalised recommendations