Skip to main content
SpringerLink
Log in

Scalability of an Unstructured Grid Continuous Galerkin Based Hurricane Storm Surge Model

  • Published:
Journal of Scientific Computing Aims and scope Submit manuscript

Abstract

This paper evaluates the parallel performance and scalability of an unstructured grid Shallow Water Equation (SWE) hurricane storm surge model. We use the ADCIRC model, which is based on the generalized wave continuity equation continuous Galerkin method, within a parallel computational framework based on domain decomposition and the MPI (Message Passing Interface) library. We measure the performance of the model run implicitly and explicitly on various grids. We analyze the performance as well as accuracy with various spatial and temporal discretizations. We improve the output writing performance by introducing sets of dedicated writer cores. Performance is measured on the Texas Advanced Computing Center Ranger machine. A high resolution 9,314,706 finite element node grid with 1 s time steps can complete a day of real time hurricane storm surge simulation in less than 20 min of computer wall clock time, using 16,384 cores with sets of dedicated writer cores.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Resio, D.T., Westerink, J.J.: Hurricanes and the physics of surges. Phys. Today 61(9), 33–38 (2008)

    Article  Google Scholar 

  2. Kolar, R.L., Westerink, J.J., Cantekin, M.E., Blain, C.A.: Aspects of nonlinear simulations using shallow-water models based on the wave continuity equation. Comput. Fluids 23, 523–528 (1994)

    Article  MATH  Google Scholar 

  3. Luettich, R.A., Hu, S., Westerink, J.J.: Development of the direct stress solution technique for three dimensional hydrodynamic models using finite elements. Int. J. Numer. Methods Fluids 19, 295–319 (1994)

    Article  MATH  Google Scholar 

  4. Grenier, R.R., Luettich, R.A., Westerink, J.J.: A comparison of the nonlinear frictional characteristics of two-dimensional and three-dimensional models of a shallow tidal embayment. J. Geophys. Res. 100(C7), 13719–13735 (1995)

    Article  Google Scholar 

  5. Kolar, R.L., Gray, W.G., Westerink, J.J.: Boundary conditions in shallow water models—an alternative implementation for finite element codes. Int. J. Numer. Methods Fluids 22, 603–618 (1996)

    Article  MATH  MathSciNet  Google Scholar 

  6. Atkinson, J.H., Westerink, J.J., Hervouet, J.M.: Similarities between the quasi-bubble and the generalized wave continuity equation solutions to the shallow water equations. Int. J. Numer. Methods Fluids 45, 689–714 (2004)

    Article  MATH  Google Scholar 

  7. Atkinson, J.H., Westerink, J.J., Luettich, R.A.: Two-dimensional dispersion analyses of finite element approximations to the shallow water equations. Int. J. Numer. Methods Fluids 45, 715–749 (2004)

    Article  MATH  Google Scholar 

  8. Luettich, R.A., Westerink, J.J.: Formulation and numerical implementation of the 2D/3D ADCIRC finite element model version 44.XX. http://adcirc.org/adcirc_theory_2004_12_08.pdf

  9. Dawson, C., Westerink, J.J., Feyen, J.C., Pothina, D.: Continuous, discontinuous and coupled discontinuous-continuous Galerkin finite element methods for the shallow water equations. Int. J. Numer. Methods Fluids 52, 63–88 (2006)

    Article  MATH  MathSciNet  Google Scholar 

  10. Kubatko, E.J., Westerink, J.J., Dawson, C.: hp discontinuous Galerkin methods for advection dominated problems in shallow water flow. Comput. Methods Appl. Mech. Eng. 196, 437–451 (2006)

    Article  MATH  Google Scholar 

  11. Kubatko, E.J., Westerink, J.J., Dawson, C.: Semi-discrete discontinuous Galerkin methods and stage exceeding order strong stability preserving Runge-Kutta time discretizations. J. Comput. Phys. 222, 832–848 (2007)

    Article  MATH  MathSciNet  Google Scholar 

  12. Kubatko, E.J., Dawson, C., Westerink, J.J.: Time step restrictions for Runge-Kutta discontinuous Galerkin methods on triangular grids. J. Comput. Phys. 227, 9697–9710 (2008)

    Article  MATH  MathSciNet  Google Scholar 

  13. Kubatko, E.J., Bunya, S., Dawson, C., Westerink, J.J., Mirabito, C.: A performance comparison of continuous and discontinuous finite element shallow water models. J. Sci. Comput. 40, 315–339 (2009)

    Article  MATH  MathSciNet  Google Scholar 

  14. Bunya, S., Kubatko, E.J., Westerink, J.J., Dawson, C.: A wetting and drying treatment for the Runge-Kutta discontinuous Galerkin solution to the shallow water equations. Comput. Methods Appl. Mech. Eng. 198, 1548–1562 (2009)

    Article  MathSciNet  Google Scholar 

  15. Kubatko, E.J., Bunya, S., Dawson, C., Westerink, J.J.: Dynamic p-adaptive Runge-Kutta discontinuous Galerkin methods for the shallow water equations. Comput. Methods Appl. Mech. Eng. 198, 1766–1774 (2009)

    Article  MathSciNet  Google Scholar 

  16. Westerink, J.J., Luettich, R.A., Muccino, J.C.: Modeling tides in the western North Atlantic using unstructured graded grids. Tellus 46A, 178–199 (1994)

    Google Scholar 

  17. Blain, C.A., Westerink, J.J., Luettich, R.A.: The influence of domain size on the response characteristics of a hurricane storm surge model. J. Geophys. Res. 99(C9), 18467–18479 (1994)

    Article  Google Scholar 

  18. Blain, C.A., Westerink, J.J., Luettich, R.A.: Grid convergence studies for the prediction of hurricane storm surge. Int. J. Numer. Methods Fluids 26, 369–401 (1998)

    Article  MATH  Google Scholar 

  19. Mukai, A., Westerink, J.J., Luettich, R.A., Mark, D.: Eastcoast 2001: A tidal constituent database for the western North Atlantic, Gulf of Mexico, and Caribbean Sea. Tec. Rep. ERDC/CHL TR-02-24, U.S. Army Corps of Engineers, 25 (2002)

  20. Westerink, J.J., Luettich, R.A., Feyen, J.C., Atkinson, J.H., Dawson, C., Roberts, H.J., Powell, M.D., Dunion, J.P., Kubatko, E.J., Pourtaheri, H.: A basin to channel scale unstructured grid hurricane storm surge model applied to Southern Louisiana. Mon. Weather Rev. 136(3), 833–864 (2008)

    Article  Google Scholar 

  21. Bunya, S., Dietrich, J.C., Westerink, J.J., Ebersole, B.A., Smith, J.M., Atkinson, J.H., Jensen, R., Resio, D.T., Luettich, R.A., Dawson, C., Cardone, V.J., Cox, A.T., Powell, M.D., Westerink, H.J., Roberts, H.J.: A high resolution coupled riverine flow tide, wind, wind wave and storm surge model for Southern Louisiana and Mississippi: part I—model development and validation. Mon. Weather Rev. (2009). doi:10.1175/2009MWR2907.1

    MATH  Google Scholar 

  22. Dietrich, J.C., Bunya, S., Westerink, J.J., Ebersole, B.A., Smith, J.M., Atkinson, J.H., Jensen, R., Resio, D.T., Luettich, R.A., Dawson, C., Cardone, V.J., Cox, A.T., Powell, M.D., Westerink, H.J., Roberts, H.J.: A high resolution coupled riverine flow tide, wind, wind wave and storm surge model for Southern Louisiana and Mississippi: part II—synoptic description and analyses of hurricane Katrina and Rita. Mon. Weather Rev. (2009). doi:10.1175/2009MWR2906.1

    Google Scholar 

  23. http://www.tacc.utexas.edu/

  24. Kincaid, D.R., Respess, J.R., Young, D.M., Grimes, R.G.: ITPACK 2C: A Fortran package for solving large sparse linear systems by adaptive accelerated iterative methods. ACM Trans. Math. Softw. 8(3) (1982)

  25. Karypis, G., Kumar, V.: Multilevel k-way partitioning scheme for irregular graphs. J. Parallel Distrib. Comput. 48(1), 96–129 (1998)

    Article  MathSciNet  Google Scholar 

  26. http://www.erdc.hpc.mil/systemNews/Cray_XT3/home

  27. http://www.erdc.hpc.mil/systemNews/Cray_XT4/home

  28. URS, Final coastal and riverine high-water marks collection for Hurricane Katrina in Louisiana. FEMA-1603-DR-LA, Task Orders 412 and 419, Federal Emergency Management Agency, Washington DC, 76 pp. (2006)

  29. Ebersole, B.A., Westerink, J.J., Resio, D.T., Dean, R.G.: Performance Evaluation of the New Orleans and Southeast Louisiana hurricane protection system, vol. IV—The storm. Final Report of the Interagency Performance Evaluation Task Force, U.S. Army Corps of Engineers, Washington, D.C., 263 pp. (2007)

Download references

Author information

Authors and Affiliations

  1. Environmental Fluid Dynamics Laboratories, Department of Civil Engineering and Geological Sciences, University of Notre Dame, Notre Dame, IN, USA

    S. Tanaka & J. J. Westerink

  2. Science and Safety Policy Research Division, Mitsubishi Research Institute, Inc., Chiyoda-ku, Tokyo, Japan

    S. Bunya

  3. Institute for Computational Engineering and Sciences, The University of Texas at Austin, Austin, TX, USA

    C. Dawson

  4. Institute of Marine Sciences, University of North Carolina at Chapel Hill, Morehead City, NC, USA

    R. A. Luettich Jr.

Authors
  1. S. Tanaka
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. S. Bunya
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. J. J. Westerink
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. C. Dawson
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. R. A. Luettich Jr.
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to S. Tanaka.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Tanaka, S., Bunya, S., Westerink, J.J. et al. Scalability of an Unstructured Grid Continuous Galerkin Based Hurricane Storm Surge Model. J Sci Comput 46, 329–358 (2011). https://doi.org/10.1007/s10915-010-9402-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10915-010-9402-1

Keywords

Search

Navigation