Skip to main content
Book cover

International Conference on High Performance Computing

ISC High Performance 2021: High Performance Computing pp 272–290Cite as

Distributed Sparse Block Grids on GPUs

Part of the Lecture Notes in Computer Science book series (LNTCS,volume 12728)

Abstract

We present a design and implementation of distributed sparse block grids that transparently scale from a single CPU to multi-GPU clusters. We support dynamic sparse grids as, e.g., occur in computer graphics with complex deforming geometries and in multi-resolution numerical simulations. We present the data structures and algorithms of our approach, focusing on the optimizations required to render them computationally efficient on CPUs and GPUs alike. We provide a scalable implementation in the OpenFPM software library for HPC. We benchmark our implementation on up to 16 Nvidia GTX 1080 GPUs and up to 64 Nvidia A100 GPUs showing state-of-the-art scalability (68% to 96% parallel efficiency) on three benchmark problems. On a single GPU, our implementation is 14 to 140-fold faster than on a multi-core CPU.

Keywords

  • Sparse grid
  • Block grid
  • CUDA
  • GPU
  • Distributed data

This is a preview of subscription content, access via your institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-030-78713-4_15
  • Chapter length: 19 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   84.99
Price excludes VAT (USA)
  • ISBN: 978-3-030-78713-4
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   109.99
Price excludes VAT (USA)
Learn about institutional subscriptions
Fig. 1.
Fig. 2.
Fig. 3.
Fig. 4.
Fig. 5.
Fig. 6.

References

  1. Adalsteinsson, D., Sethian, J.A.: The fast construction of extension velocities in level set methods. J. Comput. Phys. 148, 2–22 (1999)

    MathSciNet  CrossRef  Google Scholar 

  2. Sean Baxter. moderngpu 2.0 (2016)

    Google Scholar 

  3. Bayati, B., Chatelain, P., Koumoutsakos, P.: Adaptive mesh refinement for stochastic reaction-diffusion processes. J. Chem. Phys. 230(1), 13–26 (2011)

    MathSciNet  MATH  Google Scholar 

  4. Bergdorf, M., Cottet, G.-H., Koumoutsakos, P.: Multilevel adaptive particle methods for convection-diffusion equations. Multiscale Model. Simul. 4(1), 328–357 (2005)

    MathSciNet  CrossRef  Google Scholar 

  5. Bergdorf, M., Koumoutsakos, P.: A Lagrangian particle-wavelet method. Multiscale Model. Simul. 5(3), 980–995 (2006)

    MathSciNet  CrossRef  Google Scholar 

  6. Bergdorf, M., Sbalzarini, I.F., Koumoutsakos, P.: A Lagrangian particle method for reaction-diffusion systems on deforming surfaces. J. Math. Biol. 61, 649–663 (2010)

    MathSciNet  CrossRef  Google Scholar 

  7. Brun, E., Guittet, A., Gibou, F.: A local level-set method using a hash table data structure. J. Comput. Phys. 231(6), 2528–2536 (2012)

    MathSciNet  CrossRef  Google Scholar 

  8. Gupta, A., Incardona, P., Aydin, A.D., Gumhold, S., Gunther, U., Sbalzarini, I F.: An architecture for interactive in situ visualization and its transparent implementation in OpenFPM. In: In Situ Infrastructures for Enabling Extreme-Scale Analysis and Visualization (ISAV’20), pp. 20–26. ACM, New York (2020)

    Google Scholar 

  9. Hoetzlein. R.K.: GVDB: raytracing sparse voxel database structures on the GPU. In: Eurographics/ACM SIGGRAPH Symposium on High Performance Graphics (2016)

    Google Scholar 

  10. Houston, B., Nielsen, M.B., Batty, C., Nilsson, O., Museth, K.: Hierarchical RLE level set: a compact and versatile deformable surface representation. ACM Trans. Graph. 25(1), 151–175 (2006)

    CrossRef  Google Scholar 

  11. Incardona, P., Leo, A., Zaluzhnyi, Y., Ramaswamy, R., Sbalzarini, I.F.: OpenFPM: a scalable open framework for particle and particle-mesh codes on parallel computers. Comput. Phys. Commun. 241, 155–177 (2019)

    CrossRef  Google Scholar 

  12. Kretz, M., Lindenstruth, V.: Vc: A C++ library for explicit vectorization. Softw. Pract. Exper. 42(11), 1409–1430 (2012)

    CrossRef  Google Scholar 

  13. Merrill, D.: CUDA UnBound (CUB) library (2015)

    Google Scholar 

  14. Museth, K.: VDB: high-resolution sparse volumes with dynamic topology. ACM Trans. Graph. 32(3), 27 (2013)

    Google Scholar 

  15. Setaluri, R., Aanjaneya, M., Bauer, S., Sifakis. E.: SPGrid: a sparse paged grid structure applied to adaptive smoke simulation. ACM Trans. Graph. 33(6), 205 (2014)

    Google Scholar 

  16. Zhang, W., et al.: AMReX: a framework for block-structured adaptive mesh refinement. J. Open Source Softw. 4(37), 1370–1370 (2019)

    CrossRef  Google Scholar 

Download references

Acknowledgments

The authors are grateful to the Centre for Information Services and High Performance Computing (ZIH) of TU Dresden and to the Scientific Computing Facility of MPI-CBG for providing their facilities for the benchmarks. This work was supported by the Federal Ministry of Education and Research (Bundesministerium für Bildung und Forschung, BMBF) under funding codes 01/S18026A-F (competence center for Big Data and AI “ScaDS.AI Dresden/Leipzig”) and 031L0160 (project “SPlaT-DM – computer simulation platform for topology-driven morphogenesis”).

Author information

Authors and Affiliations

  1. Technische Universität Dresden, Faculty of Computer Science, Dresden, Germany

    Pietro Incardona & Ivo F. Sbalzarini

  2. Max Planck Institute of Molecular Cell Biology and Genetics, Dresden, Germany

    Pietro Incardona, Tommaso Bianucci & Ivo F. Sbalzarini

  3. Center for Systems Biology Dresden, Dresden, Germany

    Pietro Incardona, Tommaso Bianucci & Ivo F. Sbalzarini

  4. Cluster of Excellence Physics of Life, TU Dresden, Dresden, Germany

    Ivo F. Sbalzarini

  5. Center for Scalable Data Analytics and Artificial Intelligence, Dresden, Germany

    Ivo F. Sbalzarini

Authors
  1. Pietro Incardona
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tommaso Bianucci
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ivo F. Sbalzarini
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ivo F. Sbalzarini .

Editor information

Editors and Affiliations

  1. Hewlett Packard Enterprise, Seattle, WA, USA

    Bradford L. Chamberlain

  2. University of Amsterdam, Amsterdam, The Netherlands

    Ana-Lucia Varbanescu

  3. Extreme Computing Research Center, Thuwal Jeddah, Saudi Arabia

    Hatem Ltaief

  4. The University of Tennessee, Knoxville, Knoxville, TN, USA

    Piotr Luszczek

Rights and permissions

Reprints and Permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Verify currency and authenticity via CrossMark

Cite this paper

Incardona, P., Bianucci, T., Sbalzarini, I.F. (2021). Distributed Sparse Block Grids on GPUs. In: Chamberlain, B.L., Varbanescu, AL., Ltaief, H., Luszczek, P. (eds) High Performance Computing. ISC High Performance 2021. Lecture Notes in Computer Science(), vol 12728. Springer, Cham. https://doi.org/10.1007/978-3-030-78713-4_15

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-78713-4_15

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-78712-7

  • Online ISBN: 978-3-030-78713-4

  • eBook Packages: Computer ScienceComputer Science (R0)