Skip to main content
SpringerLink
Log in

Dynamic Load Balancing for Coupled Simulation Methods

  • Conference paper
  • First Online:
Sustained Simulation Performance 2019 and 2020

  • 237 Accesses

Abstract

A dynamic load balancing technique for simulation methods based on hierarchical Cartesian meshes is presented for two applications in this paper. The first method is a hybrid CFD/CAA solver for the prediction of aeroacoustic noise. In this application, a finite-volume method for the large eddy simulation of the turbulent flow field is coupled to a discontinuous Galerkin method for the solution of the acoustic perturbation equations to predict the generation and propagation of the sound field. The second simulation method predicts a combustion process of a premixed fuel. The turbulent flow field is predicted again by large eddy simulation using the finite-volume method, which is coupled to a level-set solver used for the prediction of the flame surface. In both applications, a joint Cartesian mesh is used for the involved solvers, which allows to efficiently redistribute the computational load using a space filling curve. The results show that the dynamic load balancing can enhance the parallel efficiency even for static meshes. The simulation of the combustion process with a solution adaptvie mesh technique demonstrates the necessity of a dynamic load balancing technique.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 149.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 199.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 199.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    The geometry of the burner has been provided by EM2C.

References

  1. Ewert, R., Schröder, W.: Acoustic perturbation equations based on flow decomposition via source filtering. J. Comput. Phys. 188(2), 365–398 (2003). http://dx.doi.org/10.1016/S0021-9991(03)00168-2, https://doi.org/10.1016/S0021-9991(03)00168-2

  2. Bridges, J., Brown, C.: Parametric testing of chevrons on single flow hot jets. In: AIAA Paper 2004–2824. http://dx.doi.org/10.2514/6.2004-2824, https://doi.org/10.2514/6.2004-2824

  3. Niemöller, A., Meinke, M., Schröder, W., Albring, T., Gauger, N.: Noise reduction using a Direct-Hybrid CFD/CAA method, Paper 2019-2579. In: AIAA (2019). http://dx.doi.org/10.2514/6.2019-2579, https://doi.org/10.2514/6.2019-2579

  4. Schlottke-Lakemper, M., Yu, H., Berger, S., Meinke, M., Schröder, W.: A fully coupled hybrid computational aeroacoustics method on hierarchical Cartesian meshes. Comput. Fluids 144, 137–153 (2017). http://dx.doi.org/10.1016/j.compfluid.2016.12.001, https://doi.org/10.1016/j.compfluid.2016.12.001

  5. Bailly, C., Juve, D.: Numerical solution of acoustic propagation problems using linearized Euler equations. AIAA J. 38(1), 22–29 (2000). http://dx.doi.org/10.2514/2.949, https://doi.org/10.2514/2.949

  6. Pitsch, H.: A consistent level set formulation for large-eddy simulation of premixed turbulent combustion. Combust. Flame 143(4), 587–598 (2005)

    Article  Google Scholar 

  7. Schlimpert, S., Feldhusen, A., Grimmen, J.H., Roidl, B., Meinke, M., Schröder, W.: Hydrodynamic instability and shear layer effects in turbulent premixed combustion. Phys. Fluids 28(1), 017104 (2016)

    Google Scholar 

  8. Herff, S., Pausch, K., Schlimpert, S., Nawroth, H., Paschereit, C., Schröder, W.: Impact of burner plenum acoustics on the sound emission of a turbulent lean premixed open flame. Int. J. Spray Combust. (2020) (accepted for publication)

    Google Scholar 

  9. Pausch, K., Herff, S., Schröder, W.: Noise sources of an unconfined and a confined swirl burner. J. Sound. Vib. 475(9), 115293 (2020)

    Article  Google Scholar 

  10. Hartmann, D., Meinke, M., Schröder, W.: A strictly conservative Cartesian cut-cell method for compressible viscous flows on adaptive grids. Comp. Meth. Appl. Mech. Eng. 200(9–12), 1038–1052 (2011). http://dx.doi.org/10.1016/j.cma.2010.05.015, https://doi.org/10.1016/j.cma.2010.05.015

  11. Meinke, M., Schröder, W., Krause, E., Rister, T.: A comparison of second and sixth-order methods for large-eddy simulations. Comput. Fluids 31(4–7), 695–718 (2002). http://dx.doi.org/10.1016/S0045-7930(01)00073-1, https://doi.org/10.1016/S0045-7930(01)00073-1

  12. Boris, J.P., Grinstein, F.F., Oran, E.S., Kolbe, R.L.: New insights into large eddy simulation. Fluid Dyn. Res. 10(4–6), 199–228 (1992). http://dx.doi.org/10.1016/0169-5983(92)90023-P, https://doi.org/10.1016/0169-5983(92)90023-P

  13. Schneiders, L., Günther, C., Meinke, M., Schröder, W.: An efficient conservative cut-cell method for rigid bodies interacting with viscous compressible flows. J. Comput. Phys. 311, 62–86 (2016). http://dx.doi.org/10.1016/j.jcp.2016.01.026, https://doi.org/10.1016/j.jcp.2016.01.026

  14. Hartmann, D., Meinke, M., Schröder, W.: An adaptive multilevel multigrid formulation for Cartesian hierarchical grid methods. Comput. Fluids 37(9), 1103–1125 (2008). http://dx.doi.org/10.1016/j.compfluid.2007.06.007, https://doi.org/10.1016/j.compfluid.2007.06.007

  15. Günther, C., Meinke, M., Schröder, W.: A flexible level-set approach for tracking multiple interacting interfaces in embedded boundary methods, Comput. Fluids 102, 182–202 (2014). http://dx.doi.org/10.1016/j.compfluid.2014.06.023, https://doi.org/10.1016/j.compfluid.2014.06.023

  16. Kopriva, D., Woodruff, S., Hussaini, M.: Discontinuous spectral element approximation of Maxwell’s equations. In: Discontinuous Galerkin Methods, Vol. 11 of LNCSE. Springer (2000). http://dx.doi.org/10.1007/978-3-642-59721-3_33, https://doi.org/10.1007/978-3-642-59721-3_33

  17. Kopriva, D.A., Woodruff, S.L., Hussaini, M.: Computation of electromagnetic scattering with a non-conforming discontinuous spectral element method. Int. J. Numer. Meth. Eng. 53(1), 105–222 (2002). http://dx.doi.org/10.1002/nme.394, https://doi.org/10.1002/nme.394

  18. Carpenter, M.H., Kennedy, C.: Fourth-order 2N-storage Runge-Kutta schemes. NASA Report TM 109112, NASA Langley Research Center (1994)

    Google Scholar 

  19. Nourgaliev, R., Theofanous, T.: High-fidelity interface tracking in compressible flows: Unlimited anchored adaptive level set. J. Comput. Phys. 224, 836–866 (2007)

    Article  Google Scholar 

  20. Shu, C.-W., Osher, S.: efficient implementation of essentially non-oscillatory shock capturing schemes. J. Comput. Phys. 77, 439–471 (1988)

    Article  MathSciNet  Google Scholar 

  21. Hartmann, D., Meinke, M., Schröder, W.: The constrained reinitialization equation for level set methods. J. Comput. Phys. 229, 1514–1535 (2010)

    Article  MathSciNet  Google Scholar 

  22. Lintermann, A., Schlimpert, S., Grimmen, J.H., Günther, C., Meinke, M., Schröder, W.: Massively parallel grid generation on HPC systems. Comp. Meth. Appl. Mech. Eng. 277, 131–153 (2014). http://dx.doi.org/10.1016/j.cma.2014.04.009, https://doi.org/10.1016/j.cma.2014.04.009

  23. Pinar, A., Aykanat, C.: Fast optimal load balancing algorithms for 1D partitioning, J. Parallel Distrib. Comput. 64(8), 974–996 (2004). http://dx.doi.org/10.1016/j.jpdc.2004.05.003, https://doi.org/10.1016/j.jpdc.2004.05.003

  24. Miguet, S., Pierson, J.M.: Heuristics for 1D rectilinear partitioning as a low cost and high quality answer to dynamic load balancing. In: High-Performance Computing and Networking, Vol. 1225 of LNCS, pp. 550–564. Springer (1997). http://dx.doi.org/10.1007/BFb0031628, https://doi.org/10.1007/BFb0031628

  25. Schlottke-Lakemper, , M., Niemöller, A., Meinke, M., Schröder, W.: Efficient parallelization for volume-coupled multiphysics simulations. Comp. Meth. Appl. Mech. Eng. 352 461–487 (2019). http://dx.doi.org/10.1016/j.cma.2019.04.032, https://doi.org/10.1016/j.cma.2019.04.032

  26. Hendrickson, B., Devine, K.: Dynamic load balancing in computational mechanics. Comp. Meth. Appl. Mech. Eng. 184(2–4), 485–500 (2000). http://dx.doi.org/10.1016/S0045-7825(99)00241-8, https://doi.org/10.1016/S0045-7825(99)00241-8

  27. DeRose, L., Homer, B., Johnson, D.: Detecting application load imbalance on high end massively parallel systems. In: Parallel Processing, vol. 4641 of LNCS, pp. 150–159. Springer (2007). http://dx.doi.org/10.1007/978-3-540-74466-5-17, https://doi.org/10.1007/978-3-540-74466-5-17

  28. Böhme, D.: Characterizing Load and Communication Imbalance in Parallel Applications, vol. 23 of IAS, Forschungszentrum Jülich (2014). http://dx.doi.org/10.1109/IPDPSW.2012.321, https://doi.org/10.1109/IPDPSW.2012.321

  29. Teresco, J.D., Devine, K.D., Flaherty, J.E.: Partitioning and dynamic load balancing for the numerical solution of partial differential equations. In: Numerical Solution of Partial Differential Equations on Parallel Computers, vol. 51 of LNCSE, pp. 55–88. Springer (2006). http://dx.doi.org/10.1007/3-540-31619-1-2, https://doi.org/10.1007/3-540-31619-1-2

  30. Hendrickson, B.: Load balancing fictions, falsehoods and fallacies. Appl. Math. Model. 25(2), 99–108 (2000). http://dx.doi.org/10.1016/S0307-904X(00)00042-1, https://doi.org/10.1016/S0307-904X(00)00042-1

  31. Watts, J., Taylor, S.: A practical approach to dynamic load balancing. IEEE Trans. Parallel Distrib. Syst. 9(3), 235–248 (1998). http://dx.doi.org/10.1109/71.674316, https://doi.org/10.1109/71.674316

  32. Jetley, P., Gioachin, F., Mendes, C., Kalé, L., Quinn, T.: Massively parallel cosmological simulations with ChaNGa. In: IPDPS, pp. 1–12. IEEE (2008). http://dx.doi.org/10.1109/IPDPS.2008.4536319, https://doi.org/10.1109/IPDPS.2008.4536319

  33. Dongarra, J., et al.: The international exascale software project roadmap. Int. J. High Perform. Comput. Appl. 25(1), 3–60 (2011). http://dx.doi.org/10.1177/1094342010391989, https://doi.org/10.1177/1094342010391989

  34. Menon, H., Jain, N., Zheng, G., Kalé, L.: Automated load balancing invocation based on application characteristics. in: Cluster Comput. 373–381. IEEE (2012). http://dx.doi.org/10.1109/CLUSTER.2012.61, https://doi.org/10.1109/CLUSTER.2012.61

  35. Anderson, E., Bai, Z., Bischof, C., Blackford, S., Demmel, J., Dongarra, J., Du Croz, J., Greenbaum, A., Hammarling, S., McKenney, A., Sorensen, D.: LAPACK Users’ Guide, 3rd edn. SIAM (1999)

    Google Scholar 

  36. Tanna, H.K.: An experimental study of jet noise Part I: Turbulent mixing noise. J. Sound Vibr. 50(3), 405–428 (1977). http://dx.doi.org/10.1016/0022-460X(77)90493-X, https://doi.org/10.1016/0022-460X(77)90493-X

  37. Pauz, V., Niemöller, A., Meinke, M., Schröder, W.: Numerical analysis of chevron nozzle noise. In: AIAA Paper 2017–3853. http://dx.doi.org/10.2514/6.2017-3853, https://doi.org/10.2514/6.2017-3853

  38. Moeck, J.P., Bourgouin, J.-F., Durox, D., Schuller, T., Candel, S.: Nonlinear interaction between a processing vortex core and acoustic oscillations in a turbulent swirling flame. Combust. Flame 159(8), 2650–2668 (2012)

    Article  Google Scholar 

  39. Gupta, A., Lilley, D., Syred, N.: Swirl Flows. Abacus Press, Tunbridge Wells, England (1984)

    Google Scholar 

  40. Poinsot, T., Lele, S.K.: Boundary conditions for direct simulations of compressible viscous flows. J. Comput. Phys. 101(1), 104–129 (1992)

    Article  MathSciNet  Google Scholar 

  41. Rudy, D.H., Strikwerda, J.C.: A nonreflecting outflow boundary condition for subsonic Navier-stokes calculations. J. Comput. Phys. 36, 55–70 (1980)

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgements

This study was funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation), DFG project number 335858412 and 247310774. The authors gratefully acknowledge the Gauss Centre for Supercomputing (GCS) for providing computing time for a GCS Large-Scale Project on the GCS share of the supercomputer “Hazel Hen” at HLRS Stuttgart. GCS is the alliance of the three national supercomputing centres HLRS (Universität Stuttgart), JSC (Forschungszentrum Jülich), and LRZ (Bayerische Akademie der Wissenschaften), funded by the German Federal Ministry of Education and Research (BMBF) and the German State Ministries for Research of Baden-Württemberg (MWK), Bayern (StMWFK), and Nordrhein-Westfalen (MIWF).

Author information

Authors and Affiliations

  1. Institute of Aerodynamics, RWTH Aachen University, Wüllnerstrasse 5a, 52062, Aachen, Germany

    Matthias Meinke, Ansgar Niemöller, Sohel Herff & Wolfgang Schröder

  2. JARA Center for Simulation and Data Science, RWTH Aachen University, Seffenter Weg 23, 52074, Aachen, Germany

    Wolfgang Schröder

Authors
  1. Matthias Meinke
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ansgar Niemöller
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sohel Herff
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Wolfgang Schröder
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Matthias Meinke .

Editor information

Editors and Affiliations

  1. High Performance Computing Center (HLRS), University of Stuttgart, Stuttgart, Germany

    Michael M. Resch

  2. High Performance Computing Center (HLRS), University of Stuttgart, Stuttgart, Germany

    Manuela Wossough

  3. Europe GmbH, NEC High Performance Computing, Düsseldorf, Germany

    Wolfgang Bez

  4. Europe GmbH, NEC High Performance Computing, Düsseldorf, Germany

    Erich Focht

  5. Cyberscience Center, Tohoku University, Sendai, Japan

    Hiroaki Kobayashi

Rights and permissions

Reprints and permissions

Copyright information

© 2021 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Meinke, M., Niemöller, A., Herff, S., Schröder, W. (2021). Dynamic Load Balancing for Coupled Simulation Methods. In: Resch, M.M., Wossough, M., Bez, W., Focht, E., Kobayashi, H. (eds) Sustained Simulation Performance 2019 and 2020. Springer, Cham. https://doi.org/10.1007/978-3-030-68049-7_5

Download citation

Publish with us

Policies and ethics

Search

Navigation