Hybrid CPU-GPU Generation of the Hamiltonian and Overlap Matrices in FLAPW Methods
In this paper we focus on the integration of high-performance numerical libraries in ab initio codes and the portability of performance and scalability. The target of our work is FLEUR, a software for electronic structure calculations developed in the Forschungszentrum Jülich over the course of two decades. The presented work follows up on a previous effort to modernize legacy code by re-engineering and rewriting it in terms of highly optimized libraries. We illustrate how this initial effort to get efficient and portable shared-memory code enables fast porting of the code to emerging heterogeneous architectures. More specifically, we port the code to nodes equipped with multiple GPUs. We divide our study in two parts. First, we show considerable speedups attained by minor and relatively straightforward code changes to off-load parts of the computation to the GPUs. Then, we identify further possible improvements to achieve even higher performance and scalability. On a system consisting of 16-cores and 2 GPUs, we observe speedups of up to 5\(\times \) with respect to our optimized shared-memory code, which in turn means between 7.5\(\times \) and 12.5\(\times \) speedup with respect to the original FLEUR code.
KeywordsDFT High-performance computing Performance portability Heterogeneous architectures FLAPW method FLEUR
This work was partially funded by the Ministry of Science and Education of the Republic of Croatia and the Deutsche Akademische Austauschdienst (DAAD) from funds of the Bundesministeriums für Bildung und Forschung (BMBF) through project “PPP Kroatien” ID 57216700. Financial support from the Jülich Aachen Research Alliance-High Performance Computing and the Deutsche Forschungsgemeinschaft (DFG) through grant GSC 111 is also gratefully acknowledged. Furthermore, the authors thank the RWTH IT Center and the Jülich Supercomputing Centre for the computational resources.
- 1.cuBLAS: The NVIDIA CUDA Basic Linear Algebra Subroutines. https://developer.nvidia.com/cublas
- 3.FLEUR: The Jülich FLAPW code family, October 2016. http://www.flapw.de/pm/index.php
- 7.Sholl, D., Steckel, J.A.: Density Functional Theory: A Practical Introduction. Wiley, New York (2011)Google Scholar
- 8.Tomov, S., Nath, R., Ltaief, H., Dongarra, J.: Dense linear algebra solvers for multicore with GPU accelerators. In: Proceedings of the IEEE IPDPS 2010, pp. 1–8. IEEE Computer Society, Atlanta, GA, 19–23 April 2010. doi: 10.1109/IPDPSW.2010.5470941
- 9.Wang, L., Wu, W., Xu, Z., Xiao, J., Yang, Y.: BLASX: A high performance level-3 BLAS library for heterogeneous multi-GPU computing. In: Proceedings of the 2016 International Conference on Supercomputing, ICS 2016, pp. 20:1–20:11. ACM, New York (2016)Google Scholar