Petascale Block-Structured AMR Applications without Distributed Meta-data

  • Brian Van Straalen
  • Phil Colella
  • Daniel T. Graves
  • Noel Keen
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6853)


Adaptive mesh refinement (AMR) applications to solve partial differential equations (PDE) are very challenging to scale efficiently to the petascale regime.

We describe optimizations to the Chombo AMR framework that enable it to scale efficiently to petascale on the Cray XT5. We describe an example of a hyperbolic solver (inviscid gas dynamics) and an matrix-free geometric multigrid elliptic solver. Both show good weak scaling to 131K processors without any thread-level or SIMD vector parallelism.

This paper describes the algorithms used to compress the Chombo metadata and the optimizations of the Chombo infrastructure that are necessary for this scaling result. That we are able to achieve petascale performance without distribution of the metadata is a significant advance which allows for much simpler and faster AMR codes.


Weak Scaling Multigrid Iteration Grid Hierarchy Processor Assignment High Concurrency 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Almgren, A., Bell, J., Kasen, D., Lijewski, M., Nonaka, A., Nugent, P., Rendleman, C., Thomas, R., Zingale, M.: Maestro, castro and sedona – petascale codes for astrophysical applications. In: SciDAC 2010. J. of Physics: Conference Series (2010)Google Scholar
  2. 2.
    Almgren, A.S., Buttke, T., Colella, P.: A fast adaptive vortex method in three dimensions. J. Comput. Phys. 113(2), 177–200 (1994)CrossRefMathSciNetzbMATHGoogle Scholar
  3. 3.
    Berger, M., Collela, P.: Local adaptive mesh refinement for shock hydrodynamics. J. Computational Physics 82, 64–84 (1989)CrossRefzbMATHGoogle Scholar
  4. 4.
    Berger, M., Oliger, J.: Adaptive mesh refinement for hyperbolic partial differential equations. Journal of Computational Physics 53, 484–512 (1984)CrossRefMathSciNetzbMATHGoogle Scholar
  5. 5.
  6. 6.
    Colella, P., Graves, D., Ligocki, T., Martin, D., Modiano, D., Serafini, D., Straalen, B.V.: Chombo software package for AMR applications: design document,
  7. 7.
    MacNeice, P., Olson, K.M., Mobarry, C., deFainchtein, R., Packer, C.: Paramesh: A parallel adaptive mesh refinement community toolkit. Computer Physics Communications 126, 330–354 (2000)CrossRefzbMATHGoogle Scholar
  8. 8.
    Martin, D., Colella, P., Graves, D.T.: A cell-centered adaptive projection method for the incompressible Navier-Stokes equations in three dimensions. Journal of Computational Physics 227, 1863–1886 (2008)CrossRefMathSciNetzbMATHGoogle Scholar
  9. 9.
    Miller, G., Colella, P.: A conservative three-dimensional Eulerian method for coupled solid-fluid shock capturing. Journal of Computational Physics 183, 26–82 (2002)CrossRefMathSciNetzbMATHGoogle Scholar
  10. 10.
    Straalen, B.V., Shalf, J., Ligock, T., Keen, N., Yang, W.-S.: Parallelization of structured, hierarchical adaptive mesh refinement algorithms. In: IPDPS:Interational Conference on Parallel and Distributed Computing Systems (2009)Google Scholar
  11. 11.
    Thompson, M.C., Ferziger, J.H.: An adaptive multigrid technique for the incompressible Navier-Stokes equations. Journal of Computational Physics 82(1), 94–121 (1989)CrossRefzbMATHGoogle Scholar
  12. 12.
    Wissink, A.M., Hornung, R.D., Kohn, S.R., Smith, S.S., Elliot, N.: Large scale parallel structured amr calculations using the samrai framework. In: SC 2001 Conference on High Perfomrance Computing (2001)Google Scholar
  13. 13.
    Woodward, P.R., Colella, P.: The numerical simulation of two-dimensional fluid flow with strong shocks. Journal of Computational Physics 54, 115–173 (1984)CrossRefMathSciNetzbMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Brian Van Straalen
    • 1
  • Phil Colella
    • 1
  • Daniel T. Graves
    • 1
  • Noel Keen
    • 1
  1. 1.Applied Numerical Algorithms GroupLawrence Berkeley National LaboratoryBerkeleyUSA

Personalised recommendations