Hybrid CPU-GPU Generation of the Hamiltonian and Overlap Matrices in FLAPW Methods

  • Diego Fabregat-Traver
  • Davor Davidović
  • Markus Höhnerbach
  • Edoardo Di Napoli
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10164)


In this paper we focus on the integration of high-performance numerical libraries in ab initio codes and the portability of performance and scalability. The target of our work is FLEUR, a software for electronic structure calculations developed in the Forschungszentrum Jülich over the course of two decades. The presented work follows up on a previous effort to modernize legacy code by re-engineering and rewriting it in terms of highly optimized libraries. We illustrate how this initial effort to get efficient and portable shared-memory code enables fast porting of the code to emerging heterogeneous architectures. More specifically, we port the code to nodes equipped with multiple GPUs. We divide our study in two parts. First, we show considerable speedups attained by minor and relatively straightforward code changes to off-load parts of the computation to the GPUs. Then, we identify further possible improvements to achieve even higher performance and scalability. On a system consisting of 16-cores and 2 GPUs, we observe speedups of up to 5\(\times \) with respect to our optimized shared-memory code, which in turn means between 7.5\(\times \) and 12.5\(\times \) speedup with respect to the original FLEUR code.


DFT High-performance computing Performance portability Heterogeneous architectures FLAPW method FLEUR 



This work was partially funded by the Ministry of Science and Education of the Republic of Croatia and the Deutsche Akademische Austauschdienst (DAAD) from funds of the Bundesministeriums für Bildung und Forschung (BMBF) through project “PPP Kroatien” ID 57216700. Financial support from the Jülich Aachen Research Alliance-High Performance Computing and the Deutsche Forschungsgemeinschaft (DFG) through grant GSC 111 is also gratefully acknowledged. Furthermore, the authors thank the RWTH IT Center and the Jülich Supercomputing Centre for the computational resources.


  1. 1.
    cuBLAS: The NVIDIA CUDA Basic Linear Algebra Subroutines.
  2. 2.
    Di Napoli, E., Peise, E., Hrywniak, M., Bientinesi, P.: High-performance generation of the hamiltonian and overlap matrices in FLAPW methods. Comput. Phys. Commun. 211, 61–72 (2016). doi: 10.1016/j.cpc.2016.10.003 CrossRefGoogle Scholar
  3. 3.
    FLEUR: The Jülich FLAPW code family, October 2016.
  4. 4.
    Jansen, H.J.F., Freeman, A.J.: Total-energy full-potential linearized augmented-plane-wave method for bulk solids - electronic and structural-properties of tungsten. Phys. Rev. B 30(2), 561–569 (1984)CrossRefGoogle Scholar
  5. 5.
    Kohn, W., Sham, L.J.: Self-consistent equations including exchange and correlation effects. Phys. Rev. 140, A1133–A1138 (1965)MathSciNetCrossRefGoogle Scholar
  6. 6.
    Nogueira, F., Marques, M.A.L., Fiolhais, C.: A Primer in Density Functional Theory. Lecture Notes in Physics. Springer, Heidelberg (2003)zbMATHGoogle Scholar
  7. 7.
    Sholl, D., Steckel, J.A.: Density Functional Theory: A Practical Introduction. Wiley, New York (2011)Google Scholar
  8. 8.
    Tomov, S., Nath, R., Ltaief, H., Dongarra, J.: Dense linear algebra solvers for multicore with GPU accelerators. In: Proceedings of the IEEE IPDPS 2010, pp. 1–8. IEEE Computer Society, Atlanta, GA, 19–23 April 2010. doi: 10.1109/IPDPSW.2010.5470941
  9. 9.
    Wang, L., Wu, W., Xu, Z., Xiao, J., Yang, Y.: BLASX: A high performance level-3 BLAS library for heterogeneous multi-GPU computing. In: Proceedings of the 2016 International Conference on Supercomputing, ICS 2016, pp. 20:1–20:11. ACM, New York (2016)Google Scholar
  10. 10.
    Wimmer, E., Krakauer, H., Weinert, M., Freeman, A.J.: Full-potential self-consistent linearized-augmented-plane-wave method for calculating the electronic-structure of molecules and surfaces - O2 molecule. Phys. Rev. B 24(2), 864–875 (1981)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Diego Fabregat-Traver
    • 1
  • Davor Davidović
    • 2
  • Markus Höhnerbach
    • 1
  • Edoardo Di Napoli
    • 1
    • 3
    • 4
  1. 1.AICESRWTH AachenAachenGermany
  2. 2.RBIZagrebCroatia
  3. 3.JARA-HPCJülichGermany
  4. 4.Jülich Supercomputing Center, Forschungszentrum JülichJülichGermany

Personalised recommendations