Implicit Low-Order Unstructured Finite-Element Multiple Simulation Enhanced by Dense Computation Using OpenACC

  • Takuma YamaguchiEmail author
  • Kohei Fujita
  • Tsuyoshi Ichimura
  • Muneo Hori
  • Maddegedara Lalith
  • Kengo Nakajima
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10732)


In this paper, we develop a low-order three-dimensional finite-element solver for fast multiple-case crust deformation computation on GPU-based systems. Based on a high-performance solver designed for massively parallel CPU-based systems, we modify the algorithm to reduce random data access, and then insert OpenACC directives. By developing algorithm appropriate for each computer architecture, we enable to exhibit higher performance. The developed solver on ten Reedbush-H nodes (20 P100 GPUs) attained speedup of 14.2 times from the original solver on 20 K computer nodes. On the newest Volta generation V100 GPUs, the solver attained a further 2.52 times speedup with respect to P100 GPUs. As a demonstrative example, we computed 368 cases of crustal deformation analyses of northeast Japan with 400 million degrees of freedom. The total procedure of algorithm modification and porting implementation took only two weeks; we can see that high performance improvement was achieved with low development cost. With the developed solver, we can expect improvement in reliability of crust-deformation analyses by many-case analyses on a wide range of GPU-based systems.



We thank Mr. Craig Toepfer (NVIDIA) and Mr. Yukihiko Hirano (NVIDIA) for the generous support and performance analyses concerning the use of NVIDIA DGX-1 (Volta V100 GPU) and NVIDIA DGX-1 (Pascal P100 GPU) environment. Part of the results were obtained using the K computer at the RIKEN Advanced Institute for Computational Science (Proposal numbers: hp160221, hp160160, 160157, and hp170249). This work was supported by Post K computer project (priority issue 3: Development of Integrated Simulation Systems for Hazard and Disaster Induced by Earthquake and Tsunami), Japan Society for the Promotion of Science (KAKENHI Grant Numbers 15K18110, 26249066, 25220908, and 17K14719) and FOCUS Establishing Supercomputing Center of Excellence.


  1. 1.
    Miyazaki, H., Kusano, Y., Shinjou, N., Shoji, F., Yokokawa, M., Watanabe, T.: Overview of the K computer system. FUJITSU Sci. Tech. J. 48(3), 302–309 (2012)Google Scholar
  2. 2.
    Ichimura, T., Fujita, K., Tanaka, S., Hori, M., Maddegedara, L., Shizawa, Y., Kobayashi, H.: Physics-based urban earthquake simulation enhanced by 10.7 blndof x 30 K time-step unstructured fe non-linear seismic wave simulation. In: Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis, pp. 15–26 (2014)Google Scholar
  3. 3.
    Ichimura, T., Fujita, K., Quinay, P.E.B., Maddegedara, L., Hori, M., Tanaka, S., Shizawa, Y., Kobayashi, H., Minami, K.: Implicit nonlinear wave simulation with 1.08t dof and 0.270t unstructured finite elements to enhance comprehensive earthquake simulation. In: Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis, pp. 1–12 (2015)Google Scholar
  4. 4.
  5. 5.
    Fujita, K., Yamaguchi, T., Ichimura, T., Hori, M., Maddegedara, L.: Acceleration of element-by-element kernel in unstructured implicit low-order finite-element earthquake simulation using openacc on pascal gpus. In: Proceedings of the Third International Workshop on Accelerator Programming Using Directives, pp. 1–12 (2016)Google Scholar
  6. 6.
    Fujita, K., Ichimura, T., Koyama, K., Inoue, H., Hori, M., Maddegedara, L.: Fast and scalable low-order implicit unstructured finite-element solver for earth’s crust deformation problem. In: Proceedings of the Platform for Advanced Scientific Computing Conference, pp. 11–20 (2017)Google Scholar
  7. 7.
  8. 8.
  9. 9.
    Melosh, H.J., Raefsky, A.: A simple and efficient method for introducing faults into finite element computations. Bull. Seismol. Soc. Am. 71(5), 1391–1400 (1981)Google Scholar
  10. 10.
    Golub, G.H., Ye, Q.: Inexact conjugate gradient method with inner-outer iteration. SIAM J. Sci. Comput. 21(4), 1305–1320 (1997)MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    Brandt, A.: Multi-level adaptive solutions to boundary-value problems. Math. Comput. 31(138), 333–390 (1977)MathSciNetCrossRefzbMATHGoogle Scholar
  12. 12.
    Winget, J.M., Hughes, T.J.R.: Solution algorithms for nonlinear transient heat conduction analysis employing element-by-element iterative strategies. Comput. Methods Appl. Mech. Eng. 52, 711–815 (1985)MathSciNetCrossRefzbMATHGoogle Scholar
  13. 13.
  14. 14.
  15. 15.
    Masterlark, T.: Finite element model predictions of static deformation from dislocation sources in a subduction zone: sensitivities to homogeneous, isotropic, poisson-solid, and half-space assumptions. J. Geophys. Res. 108(B11), 2540 (2003)CrossRefGoogle Scholar
  16. 16.
    Hughes, K.L., Masterlark, T., Mooney, W.D.: Poroelastic stress-triggering of the 2005 m8.7 nias earthquake by the 2004 m9.2 sumatra-andaman earthquake. Earth planet. Sci. Lett. 293, 289–299 (2010)CrossRefGoogle Scholar
  17. 17.
    Agata, R., Ichimura, T., Hirahara, K., Hyodo, M., Hori, T., Hori, M.: Robust and portable capacity computing method for many finite element analyses of a high-fidelity crustal structure model aimed for coseismic slip estimation. Comput. Geosci. 94, 121–130 (2016)CrossRefGoogle Scholar
  18. 18.
    Hansen, P.C.: Analysis of discrete ill-posed problems by means of the l-curve. SIAM Rev. 34(4), 561–580 (1992)MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© Springer International Publishing AG 2018

Authors and Affiliations

  1. 1.Department of Civil Engineering, Earthquake Research InstituteThe University of TokyoTokyoJapan
  2. 2.Advanced Institute for Computational Science, RIKENKobeJapan
  3. 3.Information Technology CenterThe University of TokyoTokyoJapan

Personalised recommendations