Abstract
OpenACC has been highly successful in adapting legacy CPU-only applications for modern heterogeneous computing environments equipped with GPUs, as demonstrated by many projects as well as our previous experience. In this work, OpenACC is leveraged to transform another Computational Fluid Dynamics (CFD) high order solver FINE/FR to be GPU-eligible. On the Summit supercomputer, impressive GPU speedup ranging from 6X to 80X has been achieved using up to 12,288 GPUs. Techniques critical to achieving good speedup include aggressive reduction of data transfers between CPUs and GPUs, and optimizations targeted at improving exposed parallelism to GPUs. We have demonstrated that OpenACC offers an efficient, portable and easily-maintainable approach to achieve fast turnaround time for high-fidelity industrial simulations.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Top 500 list supercomputer statistics in June of 2010, 2015 and 2020. https://www.top500.org/statistics/list/. Accessed 20 Aug 2020
Adhianto, L., et al.: HPCTOOLKIT: tools for performance analysis of optimized parallel programs. Concurr. Comput. Pract. Exp. 22(6), 685–701 (2010)
Ghane, M., Chandrasekaran, S., Cheung, M.S.: Gecko: hierarchical distributed view of heterogeneous shared memory architectures. In: Proceedings of the 10th International Workshop on Programming Models and Applications for Multicores and Manycores, pp. 21–30 (2019)
Gutzwiller, D., Srinivasan, R., Demeulenaere, A.: Acceleration of the FINE/Turbo CFD solver in a heterogeneous environment with OpenACC directives. In: Proceedings of the Second Workshop on Accelerator Programming Using Directives, pp. 1–8 (2015)
Huynh, H.T.: A flux reconstruction approach to high-order schemes including discontinuous Galerkin methods. In: 18th AIAA Computational Fluid Dynamics Conference, p. 4079 (2007)
Karypis, G., Schloegel, K., Kumar, V.: ParMETIS: parallel graph partitioning and sparse matrix ordering library (1997)
Touber, E., Sandham, N.D.: Large-eddy simulation of low-frequency unsteadiness in a turbulent shock-induced separation bubble. Theor. Comput. Fluid Dyn. 23(2), 79–107 (2009). https://doi.org/10.1007/s00162-009-0103-z
Acknowledgments
This research used resources of the Oak Ridge Leadership Computing Facility at the Oak Ridge National Laboratory, which is supported by the Office of Science of the U.S. Department of Energy under Contract No. DEAC05-00OR22725. The authors are grateful for the comments from the reviewers which have refined the presentation.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Zhai, X.M.S., Gutzwiller, D., Puri, K., Hirsch, C. (2021). GPU Acceleration of the FINE/FR CFD Solver in a Heterogeneous Environment with OpenACC Directives. In: Bhalachandra, S., Wienke, S., Chandrasekaran, S., Juckeland, G. (eds) Accelerator Programming Using Directives. WACCPD 2020. Lecture Notes in Computer Science(), vol 12655. Springer, Cham. https://doi.org/10.1007/978-3-030-74224-9_3
Download citation
DOI: https://doi.org/10.1007/978-3-030-74224-9_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-74223-2
Online ISBN: 978-3-030-74224-9
eBook Packages: Computer ScienceComputer Science (R0)