Cluster Optimization and Parallelization of Simulations with Dynamically Adaptive Grids

  • Martin Schreiber
  • Tobias Weinzierl
  • Hans-Joachim Bungartz
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8097)


The present paper studies solvers for partial differential equations that work on dynamically adaptive grids stemming from spacetrees. Due to the underlying tree formalism, such grids efficiently can be decomposed into connected grid regions (clusters) on-the-fly. A graph on those clusters classified according to their grid invariancy, workload, multi-core affinity, and further meta data represents the inter-cluster communication. While stationary clusters already can be handled more efficiently than their dynamic counterparts, we propose to treat them as atomic grid entities and introduce a skip mechanism that allows the grid traversal to omit those regions completely. The communication graph ensures that the cluster data nevertheless are kept consistent, and several shared memory parallelization strategies are feasible. A hyperbolic benchmark that has to remesh selected mesh regions iteratively to preserve conforming tessellations acts as benchmark for the present work. We discuss runtime improvements resulting from the skip mechanism and the implications on shared memory performance and load balancing.


dynamic adaptivity cluster skipping shared memory load balancing space-filling curve 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Bader, M., Böck, C., Schwaiger, J., Vigh, C.A.: Dynamically Adaptive Simulations with Minimal Memory Requirement - Solving the Shallow Water Equations Using Sierpinski Curves. SISC 32(1) (2010)Google Scholar
  2. 2.
    Bader, M., Rahnema, K., Vigh, C.: Memory-Efficient Sierpinski-Order Traversals on Dynamically Adaptive, Recursively Structured Triangular Grids. In: Jónasson, K. (ed.) PARA 2010, Part II. LNCS, vol. 7134, pp. 302–312. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  3. 3.
    Bartholdi, J.J., Goldsman, P.: Vertex-labeling algorithms for the hilbert spacefilling curve. Software: Practice and Experience 31(5), 395–408 (2001)zbMATHCrossRefGoogle Scholar
  4. 4.
    Borkar, S., Chien, A.A.: The future of microproc. Commun. ACM 54 (2011)Google Scholar
  5. 5.
    Bungartz, H.-J., Mehl, M., Weinzierl, T.: A Parallel Adaptive Cartesian PDE Solver Using Space–Filling Curves. In: Nagel, W.E., Walter, W.V., Lehner, W. (eds.) Euro-Par 2006. LNCS, vol. 4128, pp. 1064–1074. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  6. 6.
    Burstedde, C., Wilcox, L.C., Ghattas, O.: p4est: Scalable Algorithms for Parallel Adaptive Mesh Refinement on Forests of Octrees. SISC (3) (2011)Google Scholar
  7. 7.
    Dumbser, M., Käser, M., Toro, E.: An Arbitrary High Order Discontinuous Galerkin Method for Elastic Waves on Unstructured Meshes V: Local Time Stepping and p-Adaptivity. Geophysical Journal Int. 171(2), 695–717 (2007)CrossRefGoogle Scholar
  8. 8.
    Eckhardt, W., Weinzierl, T.: A Blocking Strategy on Multicore Architectures for Dynamically Adaptive PDE Solvers. In: Wyrzykowski, R., Dongarra, J., Karczewski, K., Wasniewski, J. (eds.) PPAM 2009, Part I. LNCS, vol. 6067, pp. 567–575. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  9. 9.
    Griebel, M., Zumbusch, G.: Parallel multigrid in an adaptive PDE solver based on hashing and space-filling curves. Parallel Comp. 25(7), 827–843 (1999)MathSciNetzbMATHCrossRefGoogle Scholar
  10. 10.
    Küstner, T., Weidendorfer, J., Weinzierl, T.: Argument controlled profiling. In: Lin, H.-X., Alexander, M., Forsell, M., Knüpfer, A., Prodan, R., Sousa, L., Streit, A. (eds.) Euro-Par 2009 Workshops. LNCS, vol. 6043, pp. 177–184. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  11. 11.
    LeVeque, R.J., George, D.L., Berger, M.J.: Tsunami modelling with adaptively refined finite volume methods. Acta Numerica 20, 211–289 (2011)MathSciNetzbMATHCrossRefGoogle Scholar
  12. 12.
    Bader, M., Bungartz, H.-J., Schreiber, M.: Invasive computing on high performance shared memory systems. In: Keller, R., Kramer, D., Weiss, J.-P. (eds.) Facing the Multicore-Challenge III 2012. LNCS, vol. 7686, pp. 1–12. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  13. 13.
    March, W.B., et al.: Optimizing the comp. of n-point correlations on large-scale astronomical data. In: Proc. of the Int. Conf. on High Perf. Comp., Netw., Stor. and Analysis, SC 2012. IEEE Computer Society Press (2012)Google Scholar
  14. 14.
    Meister, O., Rahnema, K., Bader, M.: A Software Concept for Cache-Efficient Simulation on Dynamically Adaptive Structured Triangular Grids. In: De Boschhere, K., D’Hollander, E.H., Joubert, G.R., Padua, D., Peters, F. (eds.) Applications, Tools and Techniques on the Road to Exascale Computing. Advances in Parallel Computing, ParCo 2012, Gent, vol. 22, pp. 251–260. IOS Press (May 2012) ISSN: 0927-5452 Google Scholar
  15. 15.
    Rahimian, A., Lashuk, I., Veerapaneni, S., Chandramowlishwaran, A., Malhotra, D., Moon, L., Sampath, R., Shringarpure, A., Vetter, J., Vuduc, R., Zorin, D., Biros, G.: Petascale direct numerical simulation of blood flow on 200k cores and heterog. arch. In: Proc. of the 2010 ACM/IEEE Int. Conf. for HPC, Networking, Storage and Analysis, SC 2010, pp. 1–11. IEEE Computer Society (2010)Google Scholar
  16. 16.
    Rüde, U.: Mathematical and computational techniques for multilevel adaptive methods. Frontiers in Applied Mathematics, vol. 13. SIAM (1993)Google Scholar
  17. 17.
    Sampath, R.S., Biros, G.: A parallel geometric multigrid method for finite elements on octree meshes. SISC 32(3), 1361–1392 (2010)MathSciNetzbMATHGoogle Scholar
  18. 18.
    Schreiber, M., Bungartz, H.-J., Bader, M.: Shared memory parallelization of fully-adaptive simulations using a dynamic tree-split and -join approach. In: IEEE Int. Conf. on High Performance Comp., HiPC (2012)Google Scholar
  19. 19.
    Schreiber, M., et al.: Generation of parameter-optimised algorithms for recursive mesh traversal algorithms (to be published, 2013)Google Scholar
  20. 20.
    Unterweger, K., Weinzierl, T., Ketcheson, D., Ahmadia, A.: Peanoclaw—a functionally-decomposed approach to adaptive mesh refinement with local time stepping for hyperb. conservation law solvers. Technical report, Technische Universität München (2013)Google Scholar
  21. 21.
    Weinzierl, T.: A Framework for Parallel PDE Solvers on Multiscale Adaptive Cartesian Grids. Verlag Dr. Hut (2009)Google Scholar
  22. 22.
    Weinzierl, T., Mehl, M.: Peano – A Traversal and Storage Scheme for Octree-Like Adaptive Cartesian Multiscale Grids. SIAM Journal on Scientific Comp. 33(5), 2732–2760 (2011)MathSciNetzbMATHCrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Martin Schreiber
    • 1
  • Tobias Weinzierl
    • 1
  • Hans-Joachim Bungartz
    • 1
  1. 1.Technische Universität MünchenGarchingGermany

Personalised recommendations