Skip to main content
SpringerLink
Log in

Challenges Porting a C++ Template-Metaprogramming Abstraction Layer to Directive-Based Offloading

  • Conference paper
  • First Online:
Accelerator Programming Using Directives (WACCPD 2021)

Part of the book series: Lecture Notes in Computer Science ((LNPSE,volume 13194))

Included in the following conference series:

Abstract

HPC systems employ a growing variety of compute accelerators with different architectures and from different vendors. Large scientific applications are required to run efficiently across these systems but need to retain a single code-base in order to not stifle development. Directive-based offloading programming models set out to provide the required portability, but, to existing codes, they themselves represent yet another API to port to. Here, we present our approach of porting the GPU-accelerated particle-in-cell code PIConGPU to OpenACC and OpenMP target by adding two new backends to its existing C++-template metaprogramming-based offloading abstraction layer alpaka and avoiding other modifications to the application code. We introduce our approach in the face of conflicts between requirements and available features in the standards as well as practical hurdles posed by immature compiler support.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 44.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 59.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    Alpaka is following the approach of very long instruction word (VLIW) architectures in OpenCL in this aspect.

  2. 2.

    Intel i7-4930K, Ubuntu 18.04.

  3. 3.

    Radeon Vega 64, Ubuntu 18.04.

  4. 4.

    GTX Titan Black, Ubuntu 18.04.

  5. 5.

    In [10]: Sect. 2.10.4, Restrictions, bullet 5.

  6. 6.

    OpenACC does not actually have this restriction, but GCC’s implementation is based on the OpenMP implementation and thus inherited this check.

  7. 7.

    Git commit c20cb5547ddd.

References

  1. alpaka. https://github.com/alpaka-group/alpaka

  2. Alpaka SYCL backend development. https://github.com/alpaka-group/alpaka/pull/789

  3. C++ AMP. https://docs.microsoft.com/en-us/cpp/parallel/amp/cpp-amp-cpp-accelerated-massive-parallelism?view=msvc-160

  4. CUDA. https://developer.nvidia.com/cuda-toolkit-archive

  5. Cupla. https://github.com/alpaka-group/cupla

  6. MallocMC. https://github.com/alpaka-group/mallocMC

  7. OpenACC 3.0 API specification. https://www.openacc.org/sites/default/files/inline-images/Specification/OpenACC.3.0.pdf

  8. OpenACC website. https://www.openacc.org

  9. OpenCL. https://www.khronos.org/registry/OpenCL

  10. OpenMP 5.0 API specification. https://www.openmp.org/spec-html/5.0/openmp.html

  11. OpenMP 5.1 API specification – atomic. https://www.openmp.org/spec-html/5.1/openmpsu105.html

  12. OpenMP website. https://www.openmp.org/

  13. RAJA. https://github.com/LLNL/RAJA

  14. ReadonOpenCompute for of LLVM-project. https://github.com/RadeonOpenCompute/llvm-project/tree/roc-4.3.x

  15. SYCL. https://www.khronos.org/registry/SYCL

  16. Thrust. https://thrust.github.io

  17. Top500 entry: Fugaku, A64FX. https://www.top500.org/system/179807

  18. Bussmann, M., et al.: Radiative signatures of the relativistic Kelvin-Helmholtz instability. In: Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis, pp. 5:1–5:12. SC 2013, ACM, New York, NY, USA (2013). http://doi.acm.org/10.1145/2503210.2504564

  19. Demidov, D., Ahnert, K., Rupp, K., Gottschling, P.: Programming CUDA and OpenCL: a case study using modern C++ libraries. SIAM J. Sci. Comput. 35(5), 1–12 (2013). https://doi.org/10.1137/120903683. https://dblp.org/rec/journals/siamsc/DemidovARG13.bib

  20. Dionne, L.: std::tuple\(<\)\(>\) should be trivially constructible, May 2019. https://cplusplus.github.io/LWG/issue3211

  21. Edwards, H.C., Trott, C.R., Sunderland, D.: Kokkos: enabling manycore performance portability through polymorphic memory access patterns. J. Parallel Distrib. Comput. 74(12), 3202–3216 (2014). https://doi.org/10.1016/j.jpdc.2014.07.003,http://www.sciencedirect.com/science/article/pii/S0743731514001257. (domain-Specific Languages and High-Level Frameworks for High-Performance Computing)

  22. Juckeland, G., et al.: From describing to prescribing parallelism: translating the SPEC ACCEL OpenACC Suite to OpenMP target directives. In: Taufer, M., Mohr, B., Kunkel, J.M. (eds.) ISC High Performance 2016. LNCS, vol. 9945, pp. 470–488. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46079-6_33

    Chapter  Google Scholar 

  23. Matthes, A., Widera, R., Zenker, E., Worpitz, B., Huebl, A., Bussmann, M.: Tuning and optimization for a variety of many-core architectures without changing a single line of implementation code using the Alpaka library. In: Kunkel, J.M., Yokota, R., Taufer, M., Shalf, J. (eds.) ISC High Performance 2017. LNCS, vol. 10524, pp. 496–514. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-67630-2_36

    Chapter  Google Scholar 

  24. Ozen, G., Lopez, G.: Accelerating Fortran DO CONCURRENT with GPUs and the NVIDIA HPC SDK. https://developer.nvidia.com/blog/accelerating-fortran-do-concurrent-with-gpus-and-the-nvidia-hpc-sdk/

  25. Zenker, E., et al.: Alpaka - an abstraction library for parallel kernel acceleration. IEEE Computer Society, May 2016. http://arxiv.org/abs/1602.08477

Download references

Acknowledgments

We want to thank Mathew Colgrove (NVIDIA) and the NVHPC team for help with debugging both compiler and code issues, Ron Lieberman (AMD) for testing PIConGPU with AOMP and advice on Clang in general and the SPEC High Performance Group for testing and support. We acknowledge the IT Center of RWTH Aachen for access to their infrastructure and Jonas Hahnfeld for support.

This material is based upon work supported by the U.S. Department of Energy, Office of science, and this research used resources of the Oak Ridge Leadership Computing Facility at the Oak Ridge National Laboratory, which is supported by the Office of Science of the U.S. Department of Energy under Contract No. DE-AC05-00OR22725.

This work was partially funded by the Center of Advanced Systems Understanding (CASUS) which is financed by Germany’s Federal Ministry of Education and Research (BMBF) and by the Saxon Ministry for Science, Culture and Tourism (SMWK) with tax funds on the basis of the budget approved by the Saxon State Parliament.

Author information

Authors and Affiliations

  1. Department of Information Services and Computing, Helmholtz-Zentrum Dresden-Rossendorf (HZDR), Bautzner Landstr. 400, 01328, Dresden, Germany

    Jeffrey Kelling & Guido Juckeland

  2. Insitute of Radiation Physics, Helmholtz-Zentrum Dresden-Rossendorf (HZDR), Bautzner Landstr. 400, 01328, Dresden, Germany

    Sergei Bastrakov, Alexander Debus, Thomas Kluge, Richard Pausch, Klaus Steiniger & René Widera

  3. Department of CIS, University of Delaware, Newark, DE, 19716, USA

    Matt Leinhauser & Sunita Chandrasekaran

  4. Center for Advance Systems Understanding (CASUS), Am Untermarkt 20, 02826, Görlitz, Germany

    Matt Leinhauser, Jan Stephan, Jeff Young & Michael Bussmann

  5. Georgia Tech, School of Computer Science, Atlanta, GA, 30332, USA

    Jeff Young

Authors
  1. Jeffrey Kelling
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sergei Bastrakov
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Alexander Debus
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Thomas Kluge
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Matt Leinhauser
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Richard Pausch
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Klaus Steiniger
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Jan Stephan
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. René Widera
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Jeff Young
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Michael Bussmann
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Sunita Chandrasekaran
    View author publications

    You can also search for this author in PubMed Google Scholar

  13. Guido Juckeland
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jeffrey Kelling .

Editor information

Editors and Affiliations

  1. Lawrence Berkeley National Laboratory, Berkeley, CA, USA

    Sridutt Bhalachandra

  2. Lawrence Berkeley National Laboratory, Berkeley, CA, USA

    Christopher Daley

  3. Oak Ridge National Laboratory, Oak Ridge, DE, USA

    Verónica Melesse Vergara

Rights and permissions

Reprints and permissions

Copyright information

© 2022 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Kelling, J. et al. (2022). Challenges Porting a C++ Template-Metaprogramming Abstraction Layer to Directive-Based Offloading. In: Bhalachandra, S., Daley, C., Melesse Vergara, V. (eds) Accelerator Programming Using Directives. WACCPD 2021. Lecture Notes in Computer Science(), vol 13194. Springer, Cham. https://doi.org/10.1007/978-3-030-97759-7_5

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-97759-7_5

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-97758-0

  • Online ISBN: 978-3-030-97759-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation