Abstract
We present parallel implementations of Newton–Raphson iterative and spectral based non-iterative solvers for single-crystal visco-plasticity models on a specialized computer hardware integrating a graphics-processing unit (GPU). We explore two implementations for the iterative solver on GPU multiprocessors: one based on a thread per crystal parallelization on local memory and another based on multiple threads per crystal on shared memory. The non-iterative solver implementation on the GPU hardware is based on a divide-conquer approach for matrix operations. The reduction of computational time for the iterative scheme was found to approach one order of magnitude. From detailed performance comparisons of the developed GPU iterative and non-iterative implementations, we conclude that the spectral non-iterative solver programed on a GPU platform is superior over the iterative implementation in terms of runtime as well as ease of implementation. It provides remarkable speedup factors exceeding three orders of magnitude over the iterative scalar version of the solver.
Similar content being viewed by others
References
Taylor GI (1938) Plastic strain in metals. J Inst Met 62:307–324
Lebensohn RA, Tomé CN (1993) A self-consistent anisotropic approach for the simulation of plastic deformation and texture development of polycrystals: Application to zirconium alloys. Acta Metall Mater 41:2611–2624
Lebensohn RA, Tomé CN, Castaneda PP (2007) Self-consistent modelling of the mechanical behaviour of viscoplastic polycrystals incorporating intragranular field fluctuations. Philos Mag 87:4287–4322
Knezevic M, Beyerlein IJ, Nizolek T, Mara NA, Pollock TM (2013) Anomalous basal slip activity in zirconium under high-strain deformation. Mater Res Lett 1:133–140
Roters F, Eisenlohr P, Hantcherli L, Tjahjanto DD, Bieler TR, Raabe D (2010) Overview of constitutive laws, kinematics, homogenization and multiscale methods in crystal plasticity finite-element modeling: theory, experiments, applications. Acta Mater 58:1152–1211
Kalidindi SR, Bronkhorst CA, Anand L (1992) Crystallographic texture evolution in bulk deformation processing of FCC metals. J Mech Phys Solids 40:537–569
Knezevic M, Drach B, Ardeljan M, Beyerlein IJ (2014) Three dimensional predictions of grain scale plasticity and grain boundaries using crystal plasticity finite element models. Comput Methods Appl Mech Eng 277:239–259
Jahedi M, Ardeljan M, Beyerlein IJ, Paydar MH, Knezevic M (2015) Enhancement of orientation gradients during simple shear deformation by application of simple compression. J Appl Phys 117:214309
Ardeljan M, Knezevic M, Nizolek T, Beyerlein IJ, Mara NA, Pollock TM (2015) A study of microstructure-driven strain localizations in two-phase polycrystalline HCP/BCC composites using a multi-scale model. Int J Plasticity 74:35–57
Ardeljan M, McCabe RJ, Beyerlein IJ, Knezevic M (2015) Explicit incorporation of deformation twins into crystal plasticity finite element models. Comput Methods Appl Mech Eng 295:396–413
Lebensohn RA, Kanjarla AK, Eisenlohr P (2012) An elasto-viscoplastic formulation based on fast Fourier transforms for the prediction of micromechanical fields in polycrystalline materials. Int J Plast 32–33:59–69
Eisenlohr P, Diehl M, Lebensohn RA, Roters F (2013) A spectral method solution to crystal elasto-viscoplasticity at finite strains. Int J Plast 46:37–53
Beyerlein IJ, Tomé CN (2007) Modeling transients in the mechanical response of copper due to strain path changes. Int J Plast 23:640–664
Tomé C, Necker C, Lebensohn R (2002) Mechanical anisotropy and grain interaction in recrystallized aluminum. Metall Mater Trans A 33:2635–2648
Zecevic M, Knezevic M (2015) A dislocation density based elasto-plastic self-consistent model for the prediction of cyclic deformation: Application to Al6022-T4. Int J Plast 72:200–217
Peeters B, Seefeldt M, Van Houtte P, Aernoudt E (2001) Taylor ambiguity in BCC polycrystals: a non-problem if substructural anisotropy is considered. Scripta Mater 45:1349–1356
Fast T, Knezevic M, Kalidindi SR (2008) Application of microstructure sensitive design to structural components produced from hexagonal polycrystalline metals. Comput Mater Sci 43:374–383
Wu X, Proust G, Knezevic M, Kalidindi SR (2007) Elastic-plastic property closures for hexagonal close-packed polycrystalline metals using first-order bounding theories. Acta Mater 55:2729–2737
Fromm BS, Adams BL, Ahmadi S, Knezevic M (2009) Grain size and orientation distributions: application to yielding of \(\alpha \)-titanium. Acta Mater 57:2339–2348
Proust G, Tomé CN, Kaschner GC (2007) Modeling texture, twinning and hardening evolution during deformation of hexagonal materials. Acta Mater 55:2137–2148
Knezevic M, Nizolek T, Ardeljan M, Beyerlein IJ, Mara NA, Pollock TM (2014) Texture evolution in two-phase Zr/Nb lamellar composites during accumulative roll bonding. Int J Plast 57:16–28
Knezevic M, Zecevic M, Beyerlein IJ, Bingert JF, McCabe RJ (2015) Strain rate and temperature effects on the selection of primary and secondary slip and twinning systems in HCP Zr. Acta Mater 88:55–73
Knezevic M, Capolungo L, Tomé CN, Lebensohn RA, Alexander DJ, Mihaila B, McCabe RJ (2012) Anisotropic stress-strain response and microstructure evolution of textured \(\alpha \)-uranium. Acta Mater 60:702–715
Knezevic M, McCabe RJ, Tomé CN, Lebensohn RA, Chen SR, Cady CM, Gray Iii GT, Mihaila B (2013) Modeling mechanical response and texture evolution of \(\alpha \)-uranium as a function of strain rate and temperature using polycrystal plasticity. Int J Plast 43:70–84
Knezevic M, Beyerlein IJ, Lovato ML, Tomé CN, Richards AW, McCabe RJ (2014) A strain-rate and temperature dependent constitutive model for BCC metals incorporating non-Schmid effects: application to tantalum-tungsten alloys. Int J Plast 62:93–104
Bhattacharyya A, Knezevic M, Abouaf M (2015) Characterization of crystallographic texture and intra-grain morphology in cross-rolled tantalum. Metall Mater Trans A 46:1085–1096
Ardeljan M, Knezevic M, Nizolek T, Beyerlein IJ, Zheng SJ, Carpenter JS, McCabe RJ, Mara NA, Pollock TM (2014) A multi-scale model for texture development in Zr/Nb nanolayered composites processed by accumulative roll bonding. IOP Conf Ser Mater Sci Eng 63:012170
Zecevic M, Knezevic M, Beyerlein IJ, Tomé CN (2015) An elasto-plastic self-consistent model with hardening based on dislocation density, twinning and de-twinning: application to strain path changes in HCP metals. Mater Sci Eng A 638:262–274
Knezevic M, Beyerlein IJ, Brown DW, Sisneros TA, Tomé CN (2013) A polycrystal plasticity model for predicting mechanical response and texture evolution during strain-path changes: application to beryllium. Int J Plast 49:185–198
Oppedal AL, El Kadiri H, Tomé CN, Kaschner GC, Vogel SC, Baird JC, Horstemeyer MF (2012) Effect of dislocation transmutation on modeling hardening mechanisms by twinning in magnesium. Int J Plast 30–31:41–61
Knezevic M, Levinson A, Harris R, Mishra RK, Doherty RD, Kalidindi SR (2010) Deformation twinning in AZ31: influence on strain hardening and texture evolution. Acta Mater 58:6230–6242
Lentz M, Klaus M, Beyerlein IJ, Zecevic M, Reimers W, Knezevic M (2015) In situ X-ray diffraction and crystal plasticity modeling of the deformation behavior of extruded Mg-Li-(Al) alloys: an uncommon tension-compression asymmetry. Acta Mater 86:254–268
Lentz M, Klaus M, Wagner M, Fahrenson C, Beyerlein IJ, Zecevic M, Reimers W, Knezevic M (2015) Effect of age hardening on the deformation behavior of an Mg-Y-Nd alloy: In-situ X-ray diffraction and crystal plasticity modeling. Mater Sci Eng A 628:396–409
Simo JC, Taylor RL (1985) Consistent tangent operators for rate-independent elastoplasticity. Comput Methods Appl Mech Eng 48:101–118
Von Mises R (1928) Mechanik der plastischen formanderung von kristallen. Math Mech 8:161–185
Cazacu O, Plunkett B, Barlat F (2006) Orthotropic yield criterion for hexagonal closed packed metals. Int J Plast 22:1171–1194
Hill R (1948) A theory of the yielding and plastic flow of anisotropic metals. In: Proceedings of the royal society of London. Series A, mathematical and physical sciences, vol 193. pp 281–297
Asaro RJ, Needleman A (1985) Texture development and strain hardening in rate dependent polycrystals. Acta Metall Mater 33:923–953
Beaudoin AJ, Mathur KK, Dawson PR, Johnson GC (1993) Three-dimensional deformation process simulation with explicit use of polycrystal plasticity models. Int J Plast 9:833–860
Knezevic M, McCabe RJ, Lebensohn RA, Tomé CN, Liu C, Lovato ML, Mihaila B (2013) Integration of self-consistent polycrystal plasticity with dislocation density based hardening laws within an implicit finite element framework: Application to low-symmetry metals. J Mech Phys Solids 61:2034–2046
Knezevic M, McCabe RJ, Lebensohn RA, Tomé CN, Mihaila B (2012) Finite element implementation of a self-consistent polycrystal plasticity model: application to \(\alpha \)-uranium. In: Proceedings of the materials properties, characterization, and modeling TMS, vol 2 . The Minerals, Metals & Materials Society, pp 789-796
Knezevic M, Lebensohn RA, Cazacu O, Revil-Baudard B, Proust GNL, Vogel SC, Nixon ME (2013) Modeling bending of \(\alpha \)-titanium with embedded polycrystal plasticity in implicit finite elements. Mater Sci Eng A 564:116–126
Segurado J, Lebensohn RA, Llorca J, Tomé CN (2012) Multiscale modeling of plasticity based on embedding the viscoplastic self-consistent formulation in implicit finite elements. Int J Plast 28:124–140
Balasubramanian S, Anand L (1996) Single crystal and polycrystal elasto-viscoplasticity: application to earing in cup drawing of F.C.C. materials. Comput Mech 17:209–225
Maudlin PJ, Schiferl SK (1996) Computational anisotropic plasticity for high-rate forming applications. Comput Methods Appl Mech Eng 131:1–30
Beaudoin AJ, Dawson PR, Mathur KK, Kocks UF, Korzekwa DA (1994) Application of polycrystal plasticity to sheet forming. Comput Methods Appl Mech Eng 117:49–70
Kumar A, Dawson PR (1995) Polycrystal plasticity modeling of bulk forming with finite elements over orientation space. Comput Mech 17:10–25
Knezevic M, Jahedi M, Korkolis YP, Beyerlein IJ (2014) Material-based design of the extrusion of bimetallic tubes. Comput Mater Sci 95:63–73
Zecevic M, McCabe RJ, Knezevic M (2015) Spectral database solutions to elasto-viscoplasticity within finite elements: application to a cobalt-based FCC superalloy. Int J Plast 70:151–165
Jahedi M, Knezevic M, Paydar M (2015) High-pressure double torsion as a severe plastic deformation process: experimental procedure and finite element modeling. J Mate Eng Perform 24:1471–1482
Jahedi M, Paydar MH, Zheng S, Beyerlein IJ, Knezevic M (2014) Texture evolution and enhanced grain refinement under high-pressure-double-torsion. Mater Sci Eng A 611:29–36
Jahedi M, Paydar MH, Knezevic M (2015) Enhanced microstructural homogeneity in metal-matrix composites developed under high-pressure-double-torsion. Mater Charact 104:92–100
Barton N, Bernier J, Knap J, Sunwoo A, Cerreta E, Turner T (2011) A call to arms for task parallelism in multi-scale materials modeling. Int J Numer Methods Eng 86:744–764
Panchal JH, Kalidindi SR, McDowell DL (2013) Key computational modeling issues in integrated computational materials engineering. Comput Aided Des 45:4–25
Knezevic M, Savage DJ (2014) A high-performance computational framework for fast crystal plasticity simulations. Comput Mater Sci 83:101–106
Mellbin Y, Hallberg H, Ristinmaa M (2014) Accelerating crystal plasticity simulations using GPU multiprocessors. Int J Numer Methods Eng 100:111–135
Knezevic M, Savage DJ, Landry NW (2014) Towards computationally tractable simulations of metal forming processes with evolving microstructures. In: ASME 2014 international manufacturing science and engineering conference collocated with the JSME 2014 international conference on materials and processing and the 42nd north american manufacturing research conference. American Society of Mechanical Engineers, p V002T002A070
Li DS, Garmestani H, Schoenfeld S (2003) Evolution of crystal orientation distribution coefficients during plastic deformation. Scripta Mater 49:867–872
Shaffer JB, Knezevic M, Kalidindi SR (2010) Building texture evolution networks for deformation processing of polycrystalline fcc metals using spectral approaches: applications to process design for targeted performance. Int J Plast 26:1183–1194
Knezevic M, Kalidindi SR, Mishra RK (2008) Delineation of first-order closures for plastic properties requiring explicit consideration of strain hardening and crystallographic texture evolution. Int J Plast 24:327–342
Kalidindi SR, Duvvuru HK, Knezevic M (2006) Spectral calibration of crystal plasticity models. Acta Mater 54:1795–1804
Knezevic M, Kalidindi SR (2007) Fast computation of first-order elastic-plastic closures for polycrystalline cubic-orthorhombic microstructures. Comput Mater Sci 39:643–648
Knezevic M, Landry NW (2015) Procedures for reducing large datasets of crystal orientations using generalized spherical harmonics. Mech Mater 88:73–86
Duvvuru HK, Knezevic M, Mishra RK, Kalidindi SR (2007) Application of microstructure sensitive design to FCC polycrystals. Mater Sci Forum 546:675–680
Knezevic M, Al-Harbi HF, Kalidindi SR (2009) Crystal plasticity simulations using discrete Fourier transforms. Acta Mater 57:1777–1784
Al-Harbi HF, Knezevic M, Kalidindi SR (2010) Spectral approaches for the fast computation of yield surfaces and first-order plastic property closures for polycrystalline materials with cubic-triclinic textures. Comput Mater Contin (CMC) 15:153–172
Knezevic M, Kalidindi SR, Fullwood D (2008) Computationally efficient database and spectral interpolation for fully plastic Taylor-type crystal plasticity calculations of face-centered cubic polycrystals. Int J Plast 24:1264–1276
Kalidindi SR, Knezevic M, Niezgoda S, Shaffer J (2009) Representation of the orientation distribution function and computation of first-order elastic properties closures using discrete Fourier transforms. Acta Mater 57:3916–3923
Zecevic M, McCabe RJ, Knezevic M (2015) A new implementation of the spectral crystal plasticity framework in implicit finite elements. Mech Mater 84:114–126
Barton NR, Knap J, Arsenlis A, Becker R, Hornung RD, Jefferson DR (2008) Embedded polycrystal plasticity and adaptive sampling. Int J Plast 24:242–266
Barton NR, Bernier JV, Lebensohn RA, Boyce DE (2015) The use of discrete harmonics in direct multi-scale embedding of polycrystal plasticity. Comput Methods Appl Mech Eng 283:224–242
Chockalingam K, Tonks MR, Hales JD, Gaston DR, Millett PC, Zhang L (2013) Crystal plasticity with Jacobian-Free Newton–Krylov. Comput Mech 51:617–627
Yamanaka A, Takaki T, Tomita Y (2012) Simulation of austenite-to-ferrite transformation in deformed austenite by crystal plasticity finite element method and multi-phase-field method. ISIJ Int 52:659–668
Anderson JA, Lorenz CD, Travesset A (2008) General purpose molecular dynamics simulations fully implemented on graphics processing units. J Comput Phys 227:5342–5359
Baker JA, Hirst JD (2011) Molecular dynamics simulations using graphics processing units. Mol Inform 30:498–504
Corrigan A, Camelli FF, Löhner R, Wallin J (2011) Running unstructured gridbased CFD solvers on modern graphics hardware. Int J Numer Methods Fluids 66:221–229
Mossaiby F, Rossi R, Dadvand P, Idelsohn S (2012) OpenCL-based implementation of an unstructured edge-based finite element convection-diffusion solver on graphics hardware. Int J Numer Methods Eng 89:1635–1651
Cecka C, Lew AJ, Darve E (2011) Assembly of finite element methods on graphics processors. Int J Numer Methods Eng 85:640–669
Dziekonski A, Sypek P, Lamecki A, Mrozowski M (2013) Generation of large finiteelement matrices on multiple graphics processors. Int J Numer Methods Eng 94:204–220
Mihaila B, Knezevic M, Cardenas A (2014) Three orders of magnitude improved efficiency with highperformance spectral crystal plasticity on GPU platforms. Int J Numer Methods Eng 97:785–798
Beyerlein IJ, Tomé CN (2008) A dislocation-based constitutive law for pure Zr including temperature effects. Int J Plast 24:867–895
Ardeljan M, Beyerlein IJ, Knezevic M (2014) A dislocation density based crystal plasticity finite element model: application to a two-phase polycrystalline HCP/BCC composites. J Mech Phys Solids 66:16–31
Hutchinson JW (1976) Bounds and self-consistent estimates for creep of polycrystalline materials. In: Proceedings of the royal society of London. Series A, mathematical and physical sciences. pp 101–126
Armijo L (1966) Minimization of functions having Lipschitz continuous first partial derivatives. Pac J Math 16:1–3
Van Houtte P (1994) Application of plastic potentials to strain rate sensitive and insensitive anisotropic materials. Int J Plast 10:719–748
Bunge H-J (1993) Texture analysis in materials science., Mathematical methodsCuvillier Verlag, Göttingen
Voce E (1948) The relationship between stress and strain for homogeneous deformation. J Inst Met 74:537–562
Tomé C, Canova GR, Kocks UF, Christodoulou N, Jonas JJ (1984) The relation between macroscopic and microscopic strain hardening in F.C.C. polycrystals. Acta Metall 32:1637–1653
Sarma G, Zacharia T, Miles D (1998) Using high performance Fortran for parallel programming. Comput Math Appl 35:41–57
Ruetsch G, Fatica M (2013) CUDA Fortran for scientists and engineers: best practices for efficient CUDA Fortran programming, 1st edn. Elsevier Science, Amsterdam
2007–2014 NVIDIA Corporation, CUDA Toolkit Documentation v6.5, http://docs.nvidia.com/cuda/#axzz3MMC3iZGv
CUDA FORTRAN Programming Guide and Reference, The Portland Group, Version 2014, 2014
2014 Khronos Group. https://www.khronos.org/opencl/
2011–2014 OpenACC.org. http://www.openacc-standard.org/
Patterson DA, Hennessy JL (1998) Computer organization and design: the hardware/software interface, 2nd edn. Morgan Kaufmann Publishers Inc, San Francisco
Hennessy JL, Patterson DA (2012) Computer architecture: a quantitative approach, 5th edn. Elsevier, Inc., Waltham
NVIDIA’s Next Generation CUDA Compute Architecture: Kepler GK110, NVIDIA Corporation, 2012
Press WH, Teukolsky SA, Vetterling WT, Flannery BP (2002) Numerical recipes in C++: the art of scientific computing. Cambridge University Press, Cambridge
Junjie SRL, Sahni S (2011) Strassen’s matrix multiplication on GPUs. Parallel and distributed systems (ICPADS). In: Proceedings of the 2011 IEEE 17th international conference. pp 157–164
Baudin T, Jura J, Penelle R, Pospiech J (1995) Estimation of the minimum grain number for the orientation distribution function calculation from individual orientation measurements on Fe-3% Si and Ti-4Al-6V alloys. J Appl Crystallogr 28:582–589
Pospiech J, Jura J, Gottstein G (1994) Statistical analysis of single grain orientation data generated from model textures. Mater Sci Forum Trans Tech Publ 157:407–412
ABAQUS Version 6.11-2, Dassault Systèmes, Providence, RI, USA (2013)
Beyerlein IJ, Li S, Necker CT, Alexander DJ, Tomé CN (2005) Non-uniform microstructure and texture evolution during equal channel angular extrusion. Philos Mag 85:1359–1394
Acknowledgments
This work is based upon a project supported by the U.S. National Science Foundation under Grant No. CMMI-1541918. The authors gratefully acknowledge this support.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Savage, D.J., Knezevic, M. Computer implementations of iterative and non-iterative crystal plasticity solvers on high performance graphics hardware. Comput Mech 56, 677–690 (2015). https://doi.org/10.1007/s00466-015-1194-6
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00466-015-1194-6