Skip to main content

Large-Scale Parallelization Based on CPU and GPU Cluster for Cosmological Fluid Simulations

  • Conference paper

Part of the Communications in Computer and Information Science book series (CCIS,volume 405)

Abstract

In this study, we present our parallel implementation for large-scale cosmological simulations of 3D supersonic fluids based on CPU and GPU clusters. Our developments are based on an OpenMP parallelized CPU code named WIGEON. It is shown that a speedup of 13~31 (depending on the specific GPU card) can be achieved compared to the sequential Fortran code by using the GPU as the accelerator. Further more, our results show that the pure MPI parallelization scales very well up to ten thousand CPU cores. In addition, a hybrid CPU/GPU parallelization scheme is introduced and a detailed analysis of the speedup and the scaling on the different number of CPU and GPU cards are presented (up to 256 GPU cards due to computing resource limitation). The efficiency of our scaling and high speedup relies on domain decomposition approach, optimization of the WENO algorithm and a series of techniques to optimize the CUDA implementation, especially in the memory access pattern. We believe this hybrid MPI+CUDA code can be an excellent candidate for 10 Peta-scale computing and beyond.

Keywords

  • Cosmological hydrodynamics
  • heterogeneous
  • WENO
  • GPU
  • large-scale cluster

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-642-53962-6_18
  • Chapter length: 14 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   84.99
Price excludes VAT (USA)
  • ISBN: 978-3-642-53962-6
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   109.99
Price excludes VAT (USA)

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Feng, L.-L., Shu, C.-W., Zhang, M.: A hybrid cosmological hydrodynamic/N-body code based on a weighted essentially non-oscillatory scheme. The Astrophysical Journal (September 2004)

    Google Scholar 

  2. Anderson Jr., J.D.: Fundamentals of Aerodynamics, 3rd edn. (January 2001)

    Google Scholar 

  3. Robert, W.F., Alan, T.M.: Introduction To Fluid Mechanics, 4th edn.

    Google Scholar 

  4. Juan-Chen, H., Herng, L., Tsang-Jen, H., Tse-Yang, H.: Parallel preconditioned WENO scheme for three-dimensional flow simulation of NREL Phase VI Rotor. Computers & Fluids, 276-282 (2011)

    Google Scholar 

  5. Laurent, T., Andres, E.T., Thomas, B.G., Gilmar, M.: A massively parallel hybrid scheme for direct numerical simulation of turbulent viscoelastic channel flow. Computers & Fluids, 134–142 (2011)

    Google Scholar 

  6. http://www.top500.org/

  7. Kestener, P., Château, F., Teyssier, R.: Accelerating euler equations numerical solver on graphics processing units. In: Hsu, C.-H., Yang, L.T., Park, J.H., Yeo, S.-S. (eds.) ICA3PP 2010, Part I. LNCS, vol. 6082, pp. 281–288. Springer, Heidelberg (2010)

    CrossRef  Google Scholar 

  8. Tölke, J., Krafczyk, M.: TeraFLOP computing on a desktop PC with GPUs for 3D CFD. International Journal of Computational Fluid Dynamics, 443–456 (2008)

    Google Scholar 

  9. Athanasios, S.A., Konstantinos, I.K., Eleftherios, D.P., John, A.E.: Acceleration of a Finite-Difference WENO Scheme for Large-Scale Simulations on Many-Core Architectures. The American Institute of Aeronautics and Astronautics (2010)

    Google Scholar 

  10. Appleyard, J., Drikakis, D.: Higher-order CFD and interface tracking methods on highly-Parallel MPI and GPU systems. Computers & Fluids, 101–105 (2011)

    Google Scholar 

  11. Michael, G., Peter, Z.: A multi-GPU accelerated solver for the three-dimensional two-phase incompressible Navier-Stokes equations. Computer Science-Research and Development, 65–73 (2010)

    Google Scholar 

  12. Paulius, M.: 3D finite difference computation on GPUs using CUDA. Architectual Support for Programming Languages and Operating Systems, 79–84 (2009)

    Google Scholar 

  13. Jiang, G.S., Shu, C.W.: Efficient Implementation of Weighted ENO Schemes. J. Computational Physics, 202–208 (1996)

    Google Scholar 

  14. Balsara, D.S., Shu, C.W.: Monotonicity Preserving Weighted Essentially Non-oscillatory Schemes with Increasingly High Order of Accuracy. J. Computational Physics, 405–452 (2000)

    Google Scholar 

  15. Chi-Wang, S.: Total Variation Diminishing Time Discretizations. Siam Journal on Scientific and Statistical Computing (1988)

    Google Scholar 

  16. Dana, A.J., Julien, C.T., Inanc, S.: An MPI-CUDA Implementation for Massively Parallel Incompressible Flow Computaions on multi-CPU clusters. The American Institute of Aeronautics and Astronautics (2010)

    Google Scholar 

  17. John, L.H., David, A.P.: Computer Architecture: A Quantitative Approach, 5th edn.

    Google Scholar 

  18. Paulius, M.: Analysis-Driven Optimization. In: SC 2010. ACM (2010)

    Google Scholar 

  19. NVIDIA’s Next Generation CUDA Compute Architecture: Kepler GK110 (v1.0, 2012)

    Google Scholar 

  20. Compute Command Line Profiler User Guide. DU-05982-001_v03 (November 2011)

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2014 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Meng, C., Wang, L., Cao, Z., Feng, Ll., Zhu, W. (2014). Large-Scale Parallelization Based on CPU and GPU Cluster for Cosmological Fluid Simulations . In: Li, K., Xiao, Z., Wang, Y., Du, J., Li, K. (eds) Parallel Computational Fluid Dynamics. ParCFD 2013. Communications in Computer and Information Science, vol 405. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-53962-6_18

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-53962-6_18

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-53961-9

  • Online ISBN: 978-3-642-53962-6

  • eBook Packages: Computer ScienceComputer Science (R0)