Vector Computers in a World of Commodity Clusters, Massively Parallel Systems and Many-Core Many-Threaded CPUs: Recent Experience Based on an Advanced Lattice Boltzmann Flow Solver

  • Thomas Zeiser
  • Georg Hager
  • Gerhard Wellein


This report summarizes experience gained during the last year using the NEC SX-8 at HLRS and its wide range of competitors: commodity clusters with Infiniband interconnect, massively parallel systems (Cray XT4, IBM BlueGene L/P) and emerging many-core many-threaded CPUs (SUN Niagara2 processor). The observations are based on low-level benchmarks and the use of an advanced lattice Boltzmann flow solver developed in the framework of an international development consortium (ILBDC).


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    L. Axner, J. Bernsdorf, T. Zeiser, P. Lammers, J. Linxweiler, and A.G. Hoekstra. Performance evaluation of a parallel sparse lattice Boltzmann solver. J. Comput. Phys., 227(10):4895–4911, 2008. MATHCrossRefMathSciNetGoogle Scholar
  2. 2.
    P. Bhatnagar, E.P. Gross, and M.K. Krook. A model for collision processes in gases. I. small amplitude processes in charged and neutral one-component systems. Phys. Rev., 94(3):511–525, 1954. MATHCrossRefGoogle Scholar
  3. 3.
    M. Bouzidi, M. Firdaouss, and P. Lallemand. Momentum transfer of a Boltzmann-lattice fluid with boundaries. Phys. Fluids, 13(11):3452–3459, 2001. CrossRefGoogle Scholar
  4. 4.
    S. Chen and G.D. Doolen. Lattice Boltzmann method for fluid flows. Annu. Rev. Fluid Mech., 30:329–364, 1998. CrossRefMathSciNetGoogle Scholar
  5. 5.
    D. d‘Humières, I. Ginzburg, M. Krafczyk, P. Lallemand, and L.-S. Luo. Multiple-relaxation-time lattice Boltzmann models in three dimensions. Phil. Trans. R. Soc. Lond. A, 360(1792):437–452, 2002. MATHCrossRefGoogle Scholar
  6. 6.
    B. Ferreol and D.H. Rothman. Lattice-Boltzmann simulations of flow through Fontainebleau sandstone. Transport in Porous Media, 20:3–20, 1995. CrossRefGoogle Scholar
  7. 7.
    I. Ginzburg. Equilibrium-type and link-type lattice Boltzmann models for generic advection and anisotropic-dispersion equation. Advances in Water Resources, 28(11):1171–1195, 2005. CrossRefGoogle Scholar
  8. 8.
    I. Ginzburg and D. d’Humières. Multireflection boundary conditions for lattice Boltzmann models. Phys. Rev. E, 68(6):066614_30, 2003. CrossRefMathSciNetGoogle Scholar
  9. 9.
    J. Götz. Numerical simulation of bloodflow in aneurysms using the lattice Boltzmann method. Master thesis, Lehrstuhl für Informatik 10 (Systemsimulation), Universität Erlangen-Nürnberg, 2006. Google Scholar
  10. 10.
    G. Hager, H. Stengel, T. Zeiser, and G. Wellein. RZBENCH: Performance evaluation of current HPC architectures using low-level and application benchmarks. In HLRB/KONWIHR Results and Review Workshop, LRZ-Munich, Garching, Dec. 3-4, 2007,, Berlin, Heidelberg, in press, 2008. Springer-Verlag.
  11. 11.
    G. Hager and G. Wellein. Architecture and performance characteristics of modern high performance computers. In H. Fehske, R. Schneider, and A. Weiße, editors, Computational Many-Particle Physics, volume 739 of Lecture notes in Physics. pages 681–730, Berlin, Heidelberg, 2008. Springer-Verlag. CrossRefGoogle Scholar
  12. 12.
    G. Hager and G. Wellein. Optimization techniques for modern high performance computers. In H. Fehske, R. Schneider, and A. Weiße, editors, Computational Many-Particle Physics, volume 739 of Lecture notes in Physics, pages 731–767, Berlin, Heidelberg, 2008. Springer-Verlag. CrossRefGoogle Scholar
  13. 13.
    G. Hager, T. Zeiser, J. Treibig, and G. Wellein, Optimizing performance on modern HPC systems: Learning from simple kernel benchmarks. In E. Krause, Y. Shokin, M. Resch, and N. Shokina, editors, Computational Science and High Performance Computing II: The 2nd Russian-German Advanced Research Workshop, Stuttgart, Germany, March 14 to 16, 2005, volume 91 of Notes on Numerical Fluid Mechanics and Multidisciplinary Design (NNFM), pages 273–287, Berlin, Heidelberg, 2006. Springer-Verlag. CrossRefGoogle Scholar
  14. 14.
    G. Hager, T. Zeiser, and G. Wellein. Data access optimizations for highly threaded multi-core CPUs with multiple memory controllers. In Workshop on Large-Scale Parallel Processing 2008 (IPDPS2008), Miami, FL, April 18, 2008,, 2008.
  15. 15.
    X. He and L.-S. Luo. Lattice Boltzmann model for the incompressible Navier-Stokes equation. J. Stat. Phys., 88(3/4):927–944, 1997. MATHCrossRefMathSciNetGoogle Scholar
  16. 16.
    X. He and L.-S. Luo. Theory of the lattice Boltzmann method: From the Boltzmann equation to the lattice Boltzmann equation. Phys. Rev. E, 56(6):6811–6817, 1997. CrossRefGoogle Scholar
  17. 17.
    V. Heuveline, M.J. Krause, and J. Latt. Towards a hybrid parallelization of lattice Boltzmann methods. Proceedings of ICMMES2007, 2008. Google Scholar
  18. 18.
    G. Karypis and V. Kumar. METIS: Serial graph partitioning and fill-reducing matrix ordering, 1998. Google Scholar
  19. 19.
    J.D. McCalpin. STREAM: Sustainable memory bandwidth in high performance computers, 1991-2007. Google Scholar
  20. 20.
    S. Succi. The Lattice Boltzmann Equation – For Fluid Dynamics and Beyond. Clarendon Press, 2001. Google Scholar
  21. 21.
    Sun Microsystems. OpenSPARC T2 core microarchitecture specification. Technical report, 2007. Google Scholar
  22. 22.
    Sun Microsystems. private communication, 2008. Google Scholar
  23. 23.
    G. Wellein, T. Zeiser, S. Donath, and G. Hager. On the single processor performance of simple lattice Boltzmann kernels. Computers & Fluids, 35(8-9):910–919, 2006. CrossRefGoogle Scholar
  24. 24.
    D. Yu, R. Mei, L.-S. Luo, and W. Shyy. Viscous flow computations with the method of lattice Boltzmann equation. Progr. Aero. Sci., 39:329–367, 2003. CrossRefGoogle Scholar
  25. 25.
    T. Zeiser. Combination of detailed CFD simulations using the lattice Boltzmann method and experimental measurements using the NMR/MRI technique. In E. Krause, W. Jäger, and M. Resch, editors, High Performance Computing in Science and Engineering ’04, Transactions of the High Performance Computing Center, Stuttgart (HLRS) 2004, pages 277–292, Berlin, Heidelberg, 2005. Springer-Verlag. CrossRefGoogle Scholar
  26. 26.
    T. Zeiser. Simulation von durchströmten Schüttungen auf Hochleistungsrechnern. PhD thesis, Technische Fakultät, Universität Erlangen-Nürnberg, 2008. Google Scholar
  27. 27.
    T. Zeiser, M. Bashoor-Zadeh, A. Darabi, and G. Baroud. Pore-scale analysis of Newtonian flow in the explixit geometry of vertebral trabecular bone using lattice Boltzmann simulation. J. Eng. Med., Proc. Inst. Mech. Eng. Part H, 222(2):185–194, 2008. CrossRefGoogle Scholar
  28. 28.
    T. Zeiser, J. Götz, and M. Stürmer. On performance and accuracy of lattice Boltzmann approaches for single phase flow in porous media. In Computational Science and High Performance Computing – Russian-German Adwanced Research Workshop, Novosibirk, Russia, July 2007, Berlin, Heidelberg, in press. Springer-Verlag. Google Scholar
  29. 29.
    T. Zeiser, G. Wellein, G. Hager, S. Donath, F. Deserno, P. Lammers, and M. Wierse. Optimized lattice Boltzmann kernels as testbeds for processor performance. Technical report, Regionales Rechenzentrum Erlangen, Universität Erlangen-Nürnberg, May 2004. Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Thomas Zeiser
    • 1
  • Georg Hager
    • 1
  • Gerhard Wellein
    • 1
  1. 1.Regionales Rechenzentrum Erlangen (RRZE)Universität Erlangen-NürnbergErlangenGermany

Personalised recommendations