Parallel Structured Adaptive Mesh Refinement
Parallel structured adaptive mesh refinement is a technique for efficient utilization of computational resources. It reduces the computational effort and memory requirements needed for numerical simulation of complex phenomena, described by partial differential equations. Structured adaptive mesh refinement (SAMR) is applied in simulations where the domain is divided into logically rectangular patches, where each patch is discretized with a structured mesh. The purpose of adaptive mesh refinement is to automatically adapt the mesh to the resolution required to epresent important features of the simulated phenomenon in different subdomains. In a parallel computing context, an important consequence of the adaptation is that the dynamically changing resolution leads to a dynamically changing work load, data volume, and communication pattern at run-time. This calls for dynamic load balancing and has implications for data placement as well as parallelization granularity.
This chapter gives an overview of structured adaptive mesh refinement approaches. After a brief introductory survey of SAMR techniques and software packages, the main part of the chapter addresses various issues related to implementation of SAMR on parallel computers. In particular programming models, data placement and load balancing are discussed, for shared memory as well as distributed memory platforms. Various approaches and algorithms are presented. The appropriate choice of dynamic load balancing algorithm, data placement strategy, programming model, etc., depends on both the application state and the computer platform. There is no single best alternative under all circumstances. Consequently, the chapter ends with an account of ongoing research where the objective is to equip SAMR-based simulation software with additional adaptivity, e.g., automatic selection of load balancing algorithms and automatic decision about level of parallelization granularity using a hybrid MPI/OpenMP programming model.
KeywordsLoad Balance Mesh Point Adaptive Mesh Base Grid Dynamic Load Balance
Unable to display preview. Download preview PDF.
- 1.E. Steinthorsson, D. Modiano, Advanced methodology for simulation of complex flows using structured grid systems, Tech. Rep. 95-28, ICOMP, NASA Lewis Research Center, Cleveland, OH (1995).Google Scholar
- 8.P. Lötstedt, S. Söderberg, Parallel solution of hyperbolic pdes with space-time adaptivity, in: D. H. R. Vilsmeier, F. Benkhaldour (Ed.), Finite Volumes for Complex Applications II, Hermes Science, Paris, 1999, pp. 769–776.Google Scholar
- 12.R. A. Trompert, Local uniform grid refinement for time-dependent partial differential equations, Ph.D. thesis, University of Amsterdam (1994).Google Scholar
- 14.P. Colella, D. T. Graves, N. D. Keen, T. J. Ligocki, D. F.Martin, P.W. McCorquodale, D. Modiano, P. O. Schwartz, T. D. Sternberg, B. V. Straalen, Chombo software package for AMR applications: Design document, Available at the Chombo website: http://seesar.lbl.gov/ANAG/chombo/ (September 2008).
- 15.R. Deiterding, Parallel adaptive simulation of multi-dimensional detonation structures, Ph.D. thesis, Brandenburgische Technische Universität Cottbus (2003).Google Scholar
- 16.R. Deiterding, Detonation structure simulation with AMROC, in: L. Y. et. al. (Ed.), High Performance Computing and Communications, No. 3726 in Lecture Notes in Computer Science, Springer, Berlin Heidelberg, (2005), pp. 916–927.Google Scholar
- 17.M. Parashar, J. Browne, System engineering for high performance computing software: The HDDA/DAGH infrastructure for implementation of parallel structured adaptive mesh refinement, in: Structured Adaptive Mesh Refinement Grid Methods, Volume 117 of IMA Volumes in Mathematics and its Applications, Springer-Verlag, Berlin (2000), pp. 1–18.Google Scholar
- 19.A. Wissink, R. Hornung, S. Kohn, S. Smith, N. Elliott, Large scale parallel structured AMR calculations using the SAMRAI framework, in: Proceedings of Supercomputing 2001, Denver, USA, (2001).Google Scholar
- 28.R. D. Hornung, S. Kohn, The SAMRAI homepage, structured adaptive mesh refinement applications infrastructure, http://www.llnl.gov/CASC/SAMRAI/.
- 30.M. Parashar, J. C. Browne, On partitioning dynamic adaptive grid hierarchies, presented at HICSS-29 (1996).Google Scholar
- 31.J. Steensland, Efficient partitioning of dynamic structured grid hierarchies, Ph.D. thesis, Uppsala University (2002).Google Scholar
- 32.Z. Lan, V. Taylor, G. Bryan, Dynamic load balancing of SAMR applications on distributed systems, in: Proceedings of Supercomputing 2001, (2001).Google Scholar
- 34.H. Johansson, A. Vakili, A patch-based partitioner for parallel SAMR applications, accepted for publication in the proceedings of the IASTED International Conference on Parallel and Distributed Computing and Systems, November 2008.Google Scholar
- 36.S. H. Bokhari, T. W. Crockett, D. M. Nicol, Binary dissection: Variants & applications, Tech. Rep. ICASE Report No. 97-29, NASA Langley Research Center, Hampton, VA (1997).Google Scholar
- 40.J. Steensland, Dynamic structured grid hierarchy partitioners using inverse space-filling curves, Tech. Rep. 2001-002, Uppsala University, Department of Information Technology, Uppsala, Sweden (2001).Google Scholar
- 41.J. Steensland, M. Thuné, S. Chandra, M. Parashar, Towards an adaptive meta-partitioner for parallel SAMR applications, in: Proceedings of the IASTED International Conference on Parallel and Distributed Computing Systems, Las Vegas, (2000), pp. 425–430.Google Scholar
- 42.J. Steensland, S. Söderberg, M. Thuné, Comparison of dynamic load balancing techniques for a parallel SAMR algorithm, in: T. Sørevik, F. Manne, R. Moe, A. H. Gebremedhin (Eds.), Applied Parallel Computing—New Paradigms for HPC in Industry and Academia, Springer-Verlag, Heidelberg, (2001), pp. 160–169, (Lecture Notes in Computer Science, Vol. 1947).CrossRefGoogle Scholar
- 45.S. Chandra, Armada: A framework for adaptive application-sensitive runtime management of dynamic applications, Master’s Thesis, Graduate School, Rutgers University, NJ (2002).Google Scholar
- 46.J. Rantakokko, Comparison of parallelization models for structured adaptive mesh refinement, in: M. Danelutto, D. Laforcena, M. Vanneschi (Eds.), Lecture Notes in Computer Science 3149, Springer-Verlag, Heidelberg (2004), pp. 615–623.Google Scholar
- 48.J. Rantakokko, An integrated decomposition and partitioning approach for irregular blockstructured applications, in: J. Romlin et al. (Ed.), Proceedings of the IEEE International Parallel and Distributed Processing Symposium, IPDPS 2000, Springer–Verlag, Berlin, (2000), pp. 485–496, Lecture Notes in Computer Science, Vol. 1800.Google Scholar
- 49.T. Wilhelmsson et al., Increasing resolution and forecast length with a parallel ocean model, in: Proceedings of the Second EuroGOOS International Conference, (1999).Google Scholar
- 50.T. Wilhelmsson, J. Schüle, Running an operational baltic sea model on the T3E, in: Proceedings of the Fifth European SGI/Cray MPP Workshop, Cineca, Bologna, (1999).Google Scholar
- 52.H. Johansson, Performance characterization and evaluation of parallel PDE solvers, Licentiate Thesis 2006-010, Department of Information Technology, Uppsala University (2006).Google Scholar
- 53.H. Johansson, Design and implementation of a dynamic and adaptive meta-partitioner for parallel SAMR grid hierarchies;, Technical Report 2008-017, Department of Information Technology, Uppsala University (2008).Google Scholar
- 54.The Common Component Architechture, http://www.cca-forum.org/.
- 55.L. Li, B. Norris, H. Johansson, L. C. McInnes, J. Ray, Component infrastructure for managing performance data and runtime adaptation of parallel applications, accepted for publication in the Proceedings of PARA2008, Trondheim, Norway, 2008.Google Scholar
- 56.J. O. Kephart, D.M. Chess, The vision of autonomic computing, IEEE Computer 36 (1) (2003) 41–50.Google Scholar