ISC High Performance 2016: High Performance Computing pp 343-362 | Cite as

High Order Seismic Simulations on the Intel Xeon Phi Processor (Knights Landing)

  • Alexander Heinecke
  • Alexander Breuer
  • Michael Bader
  • Pradeep Dubey
Conference paper
First Online:
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9697)

Abstract

We present a holistic optimization of the ADER-DG finite element software SeisSol targeting the Intel\(^{\textregistered }\) Xeon Phi\(^\mathrm{TM}\) x200 processor, codenamed Knights Landing (KNL). SeisSol is a multi-physics software package performing earthquake simulations by coupling seismic wave propagation and the rupture process. The code was shown to scale beyond 1.5 million cores and achieved petascale performance when using local time stepping for the computationally heavy seismic wave propagation. Advancing further along these lines, we discuss the utilization of KNL’s core features, the exploitation of its two-level memory subsystem (which allows for efficient out-of-core implementations), and optimizations targeting at KNL’s 2D mesh on-die interconnect. Our performance comparisons demonstrate that KNL is able to outperform its previous generation, the Intel\(^{\textregistered }\) Xeon Phi™ coprocessor x100 family, by more than 2.9\(\times \) in time-to-solution. Additionally, our results show a 3.4\(\times \) speedup compared to latest Intel\(^{\textregistered }\) Xeon\(^{\textregistered }\) E5v3 CPUs.

Keywords

High-order Vectorization ADER Discontinuous Galerkin Finite Element Method Intel Xeon Phi Knights landing KNL 
This is a preview of subscription content, log in to check access.

References

  1. 1.
    Benjemaa, M., et al.: 3-D dynamic rupture simulations by a finite volume method. Geophys. J. Int. 178, 541–560 (2009)CrossRefGoogle Scholar
  2. 2.
    Bielak, J., et al.: Parallel octree-based finite element method for large-scale earthquake ground motion simulation. Comput. Model. Eng. Sci. 10(2), 99 (2005)MathSciNetMATHGoogle Scholar
  3. 3.
    Bielak, J., et al.: The shakeout earthquake scenario: verification of three simulation sets. Geophys. J. Int. 180(1), 375–404 (2010)CrossRefGoogle Scholar
  4. 4.
    Borstnik, U., et al.: Sparse matrix multiplication: the distributed block-compressed sparse row library. Parallel Comput. 40(5–6), 47–58 (2014)MathSciNetCrossRefGoogle Scholar
  5. 5.
    Breuer, A., et al.: High-order ADER-DG minimizes energy- and time-to-solution of SeisSol. In: Kunkel, J.M., Ludwig, T. (eds.) ISC High Performance 2015. LNCS, vol. 9137, pp. 340–357. Springer, Heidelberg (2015)CrossRefGoogle Scholar
  6. 6.
    Breuer, A., et al.: Petascale local time stepping for the ADER-DG finite element method. In: Proceedings of IPDPS 2016 (2016). To appearGoogle Scholar
  7. 7.
    Breuer, A., et al.: Sustained petascale performance of seismic simulations with SeisSol on SuperMUC. In: Kunkel, J.M., Ludwig, T., Meuer, H.W. (eds.) ISC 2014. LNCS, vol. 8488, pp. 1–18. Springer, Heidelberg (2014)Google Scholar
  8. 8.
    Carrington, L., et al.: High-frequency simulations of global seismic wave propagation using SPECFEM3D_GLOBE on 62K processors. In: Proceedings of SC 2008 (2008)Google Scholar
  9. 9.
    Cui, Y., et al.: Physics-based seismic hazard analysis on petascale heterogeneous supercomputers. In: Proceedings of SC 2013 (2013)Google Scholar
  10. 10.
    Cui, Y., et al.: Scalable earthquake simulation on petascale supercomputers. In: Proceedings of SC 2010 (2010)Google Scholar
  11. 11.
    Day, S.M., et al.: Tests of 3D elastodynamic codes: final report for lifelines project 1A02. Pacific Earthquake Engineering Research Center (2003)Google Scholar
  12. 12.
    de la Puente, J., et al.: An arbitrary high-order discontinuous galerkin method for elastic waves on unstructured meshes-IV. Anisotropy. Geophys. J. Int. 169(3), 1210–1228 (2007)CrossRefGoogle Scholar
  13. 13.
    de la Puente, L., et al.: Dynamic rupture modeling on unstructured meshes using a discontinuous Galerkin method. J. Geophys. Res. 114, B10302 (2009)CrossRefGoogle Scholar
  14. 14.
    Dumbser, M., et al.: An arbitrary high-order discontinuous Galerkin method for elastic waves on unstructured meshes-II. The three-dimensional isotropic case. Geophys. J. Int. 167(1), 319–336 (2006)MathSciNetCrossRefGoogle Scholar
  15. 15.
    Harris, R.A., et al.: The SCEC/USGS dynamic earthquake rupture code verification exercise. Seismol. Res. Lett. 80(1), 119–126 (2009)CrossRefGoogle Scholar
  16. 16.
    Heinecke, A., et al.: Petascale high order dynamic rupture earthquake simulations on heterogeneous supercomputers. In: Proceedings of SC 2014. Gordon Bell Finalist (2014)Google Scholar
  17. 17.
    Ichimura, T., et al.: Implicit nonlinear wave simulation with 1.08 T DOF and 0.270 T unstructured finite elements to enhance comprehensive earthquake simulation. In: Proceedings of SC 2015 (2015)Google Scholar
  18. 18.
    Ichimura, I., et al.: Physics-based urban earthquake simulation enhanced by 10.7 BLNDOF \(\times \, 30\) K time-step unstructured fe non-linear seismic wave simulation. In: Proceedings of SC 2014 (2014)Google Scholar
  19. 19.
    Intel Corporation: Intel(R) 64 and IA-32 Architectures Optimization Reference Manual, January 2016Google Scholar
  20. 20.
    Käser, M., et al.: An arbitrary high-order discontinuous Galerkin method for elastic waves on unstructured meshes-III. Viscoelastic attenuation. Geophys. J. Int. 168(1), 224–242 (2007)CrossRefGoogle Scholar
  21. 21.
    Komatitsch, D., et al.: High-order finite-element seismic wave propagation modeling with MPI on a large GPU cluster. J. Comput. Phys. 229(20), 7692–7714 (2010)MathSciNetCrossRefMATHGoogle Scholar
  22. 22.
    Pelties, C., et al.: Three-dimensional dynamic rupture simulation with a high-order discontinuous Galerkin method on unstructured tetrahedral meshes. J. Geophys. Res. 117, B02309 (2012)CrossRefGoogle Scholar
  23. 23.
    Pelties, C., et al.: Verification of an ADER-DG method for complex dynamic rupture problems. Geosci. Model Dev. Discuss. 6, 5981–6034 (2013)CrossRefGoogle Scholar
  24. 24.
    Shin, J., et al.: Speeding up Nek5000 with autotuning and specialization. In: Proceedings of the 24th ACM International Conference on Supercomputing (ICS 2010), pp. 253–262. ACM, New York (2010)Google Scholar
  25. 25.
    Sodani, A.: Knights Landing (KNL): 2nd generation Intel(R) Xeon Phi(TM) processor. In: Hotchips-2015 (2015)Google Scholar
  26. 26.
    Sodani, A., et al.: Knights Landing (KNL): 2nd generation Intel(R) Xeon Phi(TM) processor. IEEE Micro, Hot Chips Special Issue, (2016, to appear)Google Scholar
  27. 27.
    Tago, J., et al.: A 3D hp-adaptive discontinuous Galerkin method for modeling earthquake dynamics. J. Geophys. Res. 117, B09312 (2012)CrossRefGoogle Scholar
  28. 28.
    Tu, T., et al.: From mesh generation to scientific visualization: an end-to-end approach to parallel supercomputing. In: Proceedings of SC 2006 (2006)Google Scholar
  29. 29.
    Wilcox, L.C., et al.: A high-order discontinuous Galerkin method for wave propagation through coupled elastic-acoustic media. J. Comput. Phys. 229(24), 9373–9396 (2010)MathSciNetCrossRefMATHGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Alexander Heinecke
    • 1
  • Alexander Breuer
    • 2
  • Michael Bader
    • 3
  • Pradeep Dubey
    • 1
  1. 1.Intel CorporationSanta ClaraUSA
  2. 2.University of California, San DiegoLa JollaUSA
  3. 3.Technische Universität MünchenGarchingGermany

Personalised recommendations