Skip to main content
SpringerLink
Log in

Alya: Computational Solid Mechanics for Supercomputers

  • Original Paper
  • Published:
Archives of Computational Methods in Engineering Aims and scope Submit manuscript

Abstract

While solid mechanics codes are now conventional tools both in industry and research, the increasingly more exigent requirements of both sectors are fuelling the need for more computational power and more advanced algorithms. For obvious reasons, commercial codes are lagging behind academic codes often dedicated either to the implementation of one new technique, or the upscaling of current conventional codes to tackle massively large scale computational problems. Only in a few cases, both approaches have been followed simultaneously. In this article, a solid mechanics simulation strategy for parallel supercomputers based on a hybrid approach is presented. Hybrid parallelization exploits the thread-level parallelism of multicore architectures, combining MPI tasks with OpenMP threads. This paper describes the proposed strategy, programmed in Alya, a parallel multi-physics code. Hybrid parallelization is specially well suited for the current trend of supercomputers, namely large clusters of multicores. The strategy is assessed through transient non-linear solid mechanics problems, both for explicit and implicit schemes, running on thousands of cores. In order to demonstrate the flexibility of the proposed strategy under advance algorithmic evolution of computational mechanics, a non-local parallel overset meshes method (Chimera-like) is implemented and the conservation of the scalability is demonstrated.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24

Similar content being viewed by others

References

  1. Alya system. http://www.bsc.es/computer-applications/alya-system

  2. Bigdft. http://bigdft.org/Wiki/index.php?title=Presenting_BigDFT

  3. Codeaster. http://www.code-aster.org/

  4. CODE\_SATURN. http://code-saturne.org

  5. Febio. http://febio.org/

  6. Metis, family of multilevel partitioning algorithms. http://glaros.dtc.umn.edu/gkhome/views/metis

  7. Openacc. https://developer.nvidia.com/openacc

  8. Opencl. https://developer.nvidia.com/opencl

  9. Openfoam. http://www.openfoam.com/

  10. Openmp. http://openmp.org/

  11. Openmp. http://openmp.org/wp/

  12. Summary of available software for sparse direct methods. http://www.cise.ufl.edu/research/sparse/codes/

  13. Adams M (1999) Parallel multigrid algorithms for unstructured 3d large deformation elasticity and plasticity finite element problems. Technical report UCB/CSD-99-1036, EECS Department, University of California, Berkeley. http://www.eecs.berkeley.edu/Pubs/TechRpts/1999/5398.html

  14. Adams MF (2007) Algebraic multigrid methods for direct frequency response analyses in solid mechanics. Comput Mech 39:497–507

    Article  MATH  Google Scholar 

  15. Malan AG, Oxtoby O (2013) An accelerated, fully-coupled, parallel 3d hybrid finite-volume fluid–structure interaction scheme. Comput Methods Appl Mech Eng 253:426–438

    Article  MATH  MathSciNet  Google Scholar 

  16. Amestoy P, Duff I, L’Excellent JY (2000) Multifrontal parallel distributed symmetric and unsymmetric solvers. Comput Methods Appl Mech Eng 184(2–4):501–520

    Article  MATH  Google Scholar 

  17. Arbenz P, van Lenthe G, Mennel U, Müller R, Sala M (2008) A scalable multi-level preconditioner for matrix-free \(\mu \)-finite element analysis of human bone structures. Int J Numer Methods Eng 73:937–947. doi: 10.1002/nme.2101

    Article  Google Scholar 

  18. Badia S, Martin A, Principe J (2014) A highly scalable parallel implementation of balancing domain decomposition by constraints. SIAM J Sci Comput 36(2):C190–C218

    Article  MathSciNet  Google Scholar 

  19. Balay S, Abhyankar S, Adams MF, Brown J, Brune P, Buschelman K, Eijkhout V, Gropp WD, Kaushik D, Knepley MG, McInnes LC, Rupp K, Smith BF, Zhang H (2014) PETSc Web page. http://www.mcs.anl.gov/petsc

  20. Balay S, Adams MF, Brown J, Brune P, Buschelman K, Eijkhout V, Gropp WD, Kaushik D, Knepley MG, McInnes LC, Rupp K, Smith BF, Zhang H (2013) PETSc users manual. Technical report ANL-95/11—revision 3.4, Argonne National Laboratory. http://www.mcs.anl.gov/petsc

  21. Becker G, Noels L (2013) A full-discontinuous galerkin formulation of nonlinear kirchhoff-love shells: elasto–plastic finite deformations, parallel computation, and fracture applications. Int J Numer Methods Eng 93:80–117. doi:10.1002/nme.4381

    Article  MathSciNet  Google Scholar 

  22. Benek J (1986) Chimera. A grid-embedding technique. Technical report, DTIC Document

  23. Bhardwaj M, Pierson K, Reese G, Walsh T, Day D, Alvin K, Peery J, Farhat C, Lesoinne M (2002) Salinas: a scalable software for high-performance structural and solid mechanics simulations. In: Proceedings of the 2002 ACM/IEEE conference on supercomputing., SC’02IEEE Computer Society Press, Los Alamitos, pp 1–19

  24. Blatt M (2009) Dune on bluegeen/p. In: Proceedings of 15th SciComp. University Press

  25. Casasdei F, Avotins J (1997) A language for implementing computational mechanics applications. In: Technology of object-oriented languages and systems, 1997. TOOLS 25, Proceedings, pp 52–67

  26. Ciccozzi F (2013) Towards code generation from design models for embedded systems on heterogeneous cpu-gpu platforms. In: 2013 IEEE 18th conference on emerging technologies factory automation (ETFA), pp 1–4

  27. Cirak F, Deiterding R, Mauch S (2007) Large-scale fluidstructure interaction simulation of viscoplastic and fracturing thin-shells subjected to shocks and detonations. Comput Struct 85:1049–1065. doi:10.1016/j.compstruc.2006.11.014

    Article  Google Scholar 

  28. Duff I, Reid J (1983) The multifrontal solution of indefinite sparse symmetric linear-equations. ACM Trans Math Softw 9(3):302–325

    Article  MATH  MathSciNet  Google Scholar 

  29. Eguzkitza B, Houzeaux G, Aubry R, Owen H, Vázquez M (2013) A parallel coupling strategy for the Chimera and domain decomposition methods in computational mechanics. Comput Fluids 80:128–141

    Article  MATH  MathSciNet  Google Scholar 

  30. El maliki A, Fortin M, Tardieu N, Fortin A (2010) Iterative solvers for 3d linear and nonlinear elasticity problems: Displacement and mixed formulations. Int J Numer Methods Eng 83(13):1780– 1802

  31. Falgout RD, Yang UM (2002) hypre: a library of high performance preconditioners. In: Preconditioners, lecture notes in computer science, pp 632–641

  32. Farhat C, Roux FX, Oden JT (1994) Implicit parallel processing in structural mechanics. Elsevier Science SA, Amsterdam

    Google Scholar 

  33. Flaig C, Arbenz P (2011) A scalable memory efficient multigrid solver for micro-finite element analyses based on CT images. Parallel Comput 37:846–854. doi:10.1016/j.parco.2011.08.001

    Article  Google Scholar 

  34. George A, Liu JW (1981) Computer solution of large sparse positive definite. Prentice Hall, Englewood cliffs (Professional technical reference)

  35. Gerstenberger A, Tuminaro R (2013) An algebraic multigrid approach to solve extended finite element method based fracture problems. Int J Numer Methods Eng 94:248–272. doi:10.1002/nme.4442

    Article  MathSciNet  Google Scholar 

  36. Geuzaine C, Remacle JF (2009) Gmsh: a 3-d finite element mesh generator with built-in pre-and post-processing facilities. Int J Numer Methods Eng 79(11):1309–1331

    Article  MATH  MathSciNet  Google Scholar 

  37. Goudreau G, Hallquist J (1982) Recent developments in large-scale finite element Lagrangian hydrocode technology. Comput Methods Appl Mech Eng 33:725–757. doi:10.1016/0045-7825(82)90129-3

    Article  MATH  MathSciNet  Google Scholar 

  38. Gould NIM, Scott JA, Hu Y (2007) A numerical evaluation of sparse direct solvers for the solution of large sparse symmetric linear systems of equations. ACM Trans Math Softw 33(2):300–331

  39. Grinberg L, Pekurovsky D, Sherwin SJ, Karniadakis GE (2009) Parallel performance of the coarse space linear vertex solver and low energy basis preconditioner for spectral/hp elements. Parallel Comput 35(5):284–304

    Article  MathSciNet  Google Scholar 

  40. Gupta A, Koric S, George T (2009) Sparse matrix factorization on massively parallel computers. In: Proceedings of the conference on high performance computing networking, storage and analysis, SC’09, ACM, New York, pp 1:1–1:12

  41. Goddeke D, Wobker H, Strzodka R, Mohd-Yusof J, McCormick P, Turek S (2009) Co-processor acceleration of an unmodified parallel solid mechanics code with feastgpu. Int J Comput Sci Eng 4(4):254–269

  42. Hales JD, Novascone SR, Williamson RL, Gaston DR, Tonks MR (2012) Solving nonlinear solid mechanics problems with the Jacobian-free Newton Krylov method. Comput Model Eng Sci 84:84–123

    MathSciNet  Google Scholar 

  43. Heath M, Ng E, Peyton B (1991) Parallel algorithms for sparse linear systems. SIAM Rev 33(3):420–460

    Article  MATH  MathSciNet  Google Scholar 

  44. Heil M, Hazel A, Boyle J (2008) Solvers for large-displacement fluid–structure interaction problems: segregated versus monolithic approaches. Comput Mech 43(1):91–101

    Article  MATH  Google Scholar 

  45. Heroux MA, Bartlett RA, Howle VE, Hoekstra RJ, Hu JJ, Kolda TG, Lehoucq RB, Long KR, Pawlowski RP, Phipps ET, Salinger AG, Thornquist HK, Tuminaro RS, Willenbring JM, Williams A, Stanley KS (2005) An overview of the Trilinos project. ACM Trans Math Softw 31(3):397–423

    Article  MATH  MathSciNet  Google Scholar 

  46. Hess B, Kutzner C, van der Spoel D, Lindahl E (2008) GROMACS 4: algorithms for highly efficient, load-balanced, and scalable molecular simulation. J Chem Theory Comput 4(3), 435–447 (2008) doi:10.1021/ct700301q

  47. Houzeaux G, de la Cruz R, Owen H, Vázquez M (2013) Parallel uniform mesh multiplication applied to a Navier–Stokes solver. Comput Fluids 80:142–151

    Article  MATH  MathSciNet  Google Scholar 

  48. Houzeaux G, Vázquez M, Aubry R, Cela J (2009) A massively parallel fractional step solver for incompressible flows. JCP 228(17):6316–6332

    Article  MATH  Google Scholar 

  49. Hughes T, Ferencz R (1987) Large-scale vectorized implicit calculations in solid mechanics on a CrayX-MP/48 utilizing EBE preconditioned conjugate gradients. Comput Methods Appl Mech Eng 61:215–248. doi:10.1016/0045-7825(87)90005-3

    Article  MATH  MathSciNet  Google Scholar 

  50. Hughes TJ (2012) The finite element method: linear static and dynamic finite element analysis. DoverPublications.com

  51. Hussain M, Abid M, Ahmad M, Khokhar A, Masud A (2011) A parallel implementation of ALE moving mesh technique for FSI problems using OpenMP. Int J Parallel Program 39:717–745. doi:10.1007/s10766-011-0168-3

    Article  Google Scholar 

  52. Jouglard C, Coutinho A (1998) A comparison of iterative multi-level finite element solvers. Comput Struct 69(5):655–670

    Article  MATH  Google Scholar 

  53. Kilic SA, Saied F, Sameh A (2004) Efficient iterative solvers for structural dynamics problems. Comput Struct 82(28):2363–2375

    Article  Google Scholar 

  54. Knoll D, Keyes D (2004) Jacobian-free Newton Krylov methods: a survey of approaches and applications. J Comput Phys 193(2):357–397

    Article  MATH  MathSciNet  Google Scholar 

  55. Komatitsch D, Erlebacher G, Göddeke D, Michéa D (2010) High-order finite-element seismic wave propagation modeling with MPI on a large GPU cluster. J Comput Phys 229:7692–7714. doi:10.1016/j.jcp.2010.06.024

    Article  MATH  MathSciNet  Google Scholar 

  56. Lafortune P, Arís R, Vázquez M, Houzeaux G (2012) Coupled electromechanical model of the heart: parallel finite element formulation. Int J Numer Methods Biomed Eng 28:72–86. doi:10.1002/cnm.1494

    Article  MATH  MathSciNet  Google Scholar 

  57. Lang S, Wieners C, Wittum G (2002) The application of adaptive parallel multigrid methods to problems in nonlinear solid mechanics. In: Ramm E, Rank E, Rannacher R, Schweizerhof K, Stein E, Wendland W, Wittum G, Wriggers P, Wunderlich W (eds) Error-controlled adaptive finite elements in solid mechanics, 422 pp, ISBN: 978-0-471-49650-2

  58. Li X, Demmel JW (2003) Superlu dist: a scalable distributed-memory sparse direct solver for unsymmetric linear systems. ACM Trans Math Softw 29:110–140

    Article  MATH  MathSciNet  Google Scholar 

  59. Liu WK, Belytschko T, Moran B (2000) Nonlinear finite elements for continua and structures. Wiley, New York

    MATH  Google Scholar 

  60. Liu Y, Zhou W, Yang Q (2007) A distributed memory parallel element-by-element scheme based on Jacobi-conditioned conjugate gradient for 3D finite element analysis. Finite Elem Anal Design 43:494–503. doi:10.1016/j.finel.2006.12.007

    Article  Google Scholar 

  61. Logg A, Mardal KA, Wells GN (eds) (2012) Automated solution of differential equations by the finite element method. Lecture notes in computational science and engineering, vol 84. Springer, Berlin. doi:10.1007/978-3-642-23099-8

  62. Lohner R, Mut F, Cebral J, Aubry R, Houzeaux G (2010) Deflated preconditioned conjugate gradient solvers for the pressure-poisson equation: Extensions and improvements. Int J Numer Meth Eng 10196–10208

  63. Luebke D (2008) Cuda: scalable parallel programming for high-performance scientific computing. In: 5th IEEE international symposium on biomedical imaging: from nano to macro, 2008. ISBI 2008, pp 836–838

  64. Maday Y, Magoulès F (2006) Absorbing interface conditions for domain decomposition methods: a general presentation. Comput Methods Appl Mech Eng 195(29–32):3880–3900

    Article  MATH  Google Scholar 

  65. Maurer D, Wieners C (2011) A parallel block LU decomposition method for distributed finite element matrices. Parallel Comput 37(12):742–758

    Article  Google Scholar 

  66. McCormick SF (1987) Multigrid methods. Frontiers in applied mathematics. Philadelphia, Pa. Society for Industrial and Applied Mathematics

  67. Moore D, Jérusalem A, Nyein M, Noels L, Jaffee M, Radovitzky R (2009) Computational biology—modeling of primary blast effects on the central nervous system. NeuroImage 47(2):T10–T20. doi:10.1016/j.neuroimage.2009.02.019

    Article  Google Scholar 

  68. Owens J, Houston M, Luebke D, Green S, Stone J, Phillips J (2008) Gpu computing. Proc IEEE 96(5):879–899

    Article  Google Scholar 

  69. Paszyski M, Jurczyk T, Pardo D (2013) Multi-frontal solver for simulations of linear elasticity coupled with acoustics. Comput Sci 12(0). http://journals.agh.edu.pl/csci/article/view/102

  70. Quey R, Dawson PR, Barbe F (2011) Large-scale 3D random polycrystals for the finite element method: Generation, meshing and remeshing. Comput Methods Appl Mech Eng 200:1729–1745. doi:10.1016/j.cma.2011.01.002

    Article  MATH  Google Scholar 

  71. Radovitzky R, Seagraves A, Tupek M, Noels L (2011) A scalable 3D fracture and fragmentation algorithm based on a hybrid, discontinuous galerkin, cohesive element method. Comput Methods Appl Mech Eng 200:326–344. doi:10.1016/j.cma.2010.08.014

    Article  MathSciNet  Google Scholar 

  72. Saad Y (2003) Iterative methods for sparse linear systems. Society for Industrial and Applied Mathematics

  73. Smith BF (1995) Domain decomposition methods for partial differential equations. In: Proceedings of ICASE/LaRC Workshop on Parallel Numerical Algorithms. University Press

  74. Soto O, Löhner R, Camelli F (2003) A linelet preconditioner for incompressible flow solvers. Int J Numer Meth Heat Fluid Flow 13(1):133–147

    Article  MATH  Google Scholar 

  75. Steger J, Benek FDJ (1983) A chimera grid scheme. Adv Grid Gener 5:59–69

    Google Scholar 

  76. Steger J, Benek J (1987) On the use of composite grid schemes in computational aerodynamics. Comput Meth Appl Mech Eng 64:301–320

    Article  MATH  MathSciNet  Google Scholar 

  77. Stewart JR, Edwards H (2004) A framework approach for developing parallel adaptive multiphysics applications. Finite Elem Anal Design 40(12):1599–1617 (The Fifteenth Annual Robert J. Melosh Competition)

  78. Kalro V, Tezduyar TE (2000) A parallel 3d computational method for fluid–structure interactions in parachute systems. Comput Methods Appl Mech Eng 190:1467–1482

    Article  Google Scholar 

  79. van Rietbergen B, Weinans H, Huiskes R, Polman B (1996) Computational strategies for iterative solutions of large FEM applications employing voxel data. Int J Numer Methods Eng 39:2743–2767. doi:10.1002/(SICI)1097-0207(19960830)39:162743:AID-NME9743.3.CO;2-1

  80. Vázquez M, Arís R, Houzeaux G, Aubry R, Villar P, Garcia-Barnós J, Gil D, Carreras F (2011) A massively parallel computational electrophysiology model of the heart. Int J Numer Methods Biomed Eng 27(12):1911–1929. doi:10.1002/cnm.1443

    Article  MATH  Google Scholar 

  81. Vázquez M, Houzeaux G, Grima R, Cela J (2007) Applications of parallel computational fluid mechanics in MareNostrum supercomputer: low-mach compressible flows. In: PARCFD2007. Antalya (Turkey)

  82. Vázquez M, Rubio F, Houzeaux G, González J, Giménez J, Beltran V, de la Cruz R, Folch A (2014) Xeon phi performance for hpc-based computational mechanics codes

  83. Waisman H, Berger-Vergiat L (2013) An adaptive domain decomposition preconditioner for crack propagation problems modeled by XFEM. Int J Multiscale Comput Eng 11(6):633–654

    Article  Google Scholar 

  84. Wall WA, Ramm E. Fluid–structure interaction based upon stabilized (ale) finite element method. In: IV World congress on computational mechanics. Barcelona. CIMNE.

  85. Wieners C (2001) The application of multigrid methods to plasticity at finite strains. ZAMM J Appl Math Mech [Zeitschrift fr Angewandte Mathematik und Mechanik] 81(S3):733–734

    Article  MATH  Google Scholar 

  86. Zienkiewicz O, Taylor R (2000) The Finite Elements method for Solid and Structural Mechanics, 5th edn. Butterworth-Heinermann, Boston

    Google Scholar 

Download references

Acknowledgments

D.D.T and A.J acknowledge funding through SIMUCOMP and ERA-NET MATERA+ project financed by the Consejería de Educación y Empleo of the Comunidad de Madrid and by the European Union’s Seventh Framework Programme (FP7/2007-2013). This work was partially supported by the grant SEV-2011-00067, Severo Ochoa Program, awarded by the Spanish Government. The authors’ would like to acknowledge PRACE infrastructure support.

Author information

Authors and Affiliations

  1. Barcelona Supercomputing Center (BSC-CNS), Edificio NEXUS I, Campus Nord UPC Gran Capitán 2–4, 08034, Barcelona, Spain

    E. Casoni, C. Samaniego, B. Eguzkitza, X. Sáez, G. Houzeaux & M. Vázquez

  2. Artificial Intelligence Research Institute CSIC (IIIA-CSIC), Campus de la UAB, 08193, Bellaterra, Spain

    M. Vázquez

  3. IMDEA Materials Institute, Tecnogetafe, C/ Eric Kandel 2, 28906, Getafe, Spain

    A. Jérusalem & D. D. Tjahjanto

  4. Department of Solid Mechanics, KTH Royal Institute of Technology, Teknikringen 8D, 10044, Stockholm, Sweden

    D. D. Tjahjanto

  5. Idra Simulation S.C.P., 08004, Barcelona, Spain

    P. Lafortune

  6. Department of Engineering Science, University of Oxford, Parks Road, Oxford, OX1 3PJ, UK

    A. Jérusalem

Authors
  1. E. Casoni
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. A. Jérusalem
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. C. Samaniego
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. B. Eguzkitza
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. P. Lafortune
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. D. D. Tjahjanto
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. X. Sáez
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. G. Houzeaux
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. M. Vázquez
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to E. Casoni.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Casoni, E., Jérusalem, A., Samaniego, C. et al. Alya: Computational Solid Mechanics for Supercomputers. Arch Computat Methods Eng 22, 557–576 (2015). https://doi.org/10.1007/s11831-014-9126-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11831-014-9126-8

Keywords

Search

Navigation