Computing and Visualization in Science

, Volume 13, Issue 7, pp 299–314 | Cite as

Highly interactive computational steering for coupled 3D flow problems utilizing multiple GPUs

Towards intuitive desktop environments for interactive 3D fluid structure interaction
  • Jan LinxweilerEmail author
  • Manfred Krafczyk
  • Jonas Tölke
Regular Article


Most computational fluid dynamics (CFD) simulations require massive computational power which is usually provided by traditional High Performance Computing (HPC) environments. Although interactivity of the simulation process is highly appreciated by scientists and engineers, due to limitations of typical HPC environments, present CFD simulations are usually executed non interactively. A recent trend is to harness the parallel computational power of graphics processing units (GPUs) for general purpose applications. As an alternative to traditional massively parallel computing, GPU computing has also gained popularity in the CFD community, especially for its application to the lattice Boltzmann method (LBM). For instance, Tölke and others presented very efficient implementations of the LBM for 2D as well as 3D space (Toelke J, in Comput Visual Sci. (2008); Toelke J and Krafczk M, in Int J Comput Fluid Dyn 22(7): 443–456 (2008)). In this work we motivate the use of GPU computing to facilitate interactive CFD simulations. In our approach, the simulation is executed on multiple GPUs instead of traditional HPC environments, which allows the integration of the complete simulation process into a single desktop application. To demonstrate the feasibility of our approach, we show a fully bidirectional fluid-structure-interaction for self induced membrane oscillations in a turbulent flow. The efficiency of the approach allows a 3D simulation close to realtime.


Computational steering Computational fluid dynamics Lattice Boltzmann method Graphical processing unit High performance computing Design patterns 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Advanced Visual Systems: AVS/Express. Visited on 2010-02-19
  2. 2.
    ANSYS: ANSYS FLUENT Flow Modeling Software. Visited on 2010-02-19
  3. 3.
    ANSYS: ANSYS CFX (2010). Visited on 2010-02-19
  4. 4.
    Belleman R., Sloot P.: Dynamic exploration environments. In: Meij, J. (ed.) Dealing with the Data Flood (Mining Data, Text and Multimedia), pp. 771–787. STT, Beweton (2002)Google Scholar
  5. 5.
    Bindick, S., Ahrenholz, B., Krafczyk, M.: Transiente 3D-simulation thermisch induzierter Strömungen in und an Gebäuden mit kinetischen Methoden. In: Peil, U. (ed.) WtG-Berichte Nr. 11—Windingenieurwesen in Forschung und Praxis, pp. 215–220 (2009)Google Scholar
  6. 6.
    Bower, A., McGlashan, B.: Twisting the triad: the evolution of the dolphin smalltalk mvp application framework. In: European Smalltalk User Group (ESUG) (2000)Google Scholar
  7. 7.
    Brodlie, K., Wood, J.: Computational steering in visualization dataflow environments. In: Oxley, L., Kulasiri, D. (eds.) MODSIM 2007 International Congress on Modelling and Simulation, pp. 3077–3083 (2007)Google Scholar
  8. 8.
    Brodlie, K.W., Duce, D.A., Gallop, J.R., Walton, J.P.R.B., Wood, J.D.: Distributed and collaborative visualization. Comput. Graph. Forum 23(2), 223+ (2004)Google Scholar
  9. 9.
    Bryson S.: Virtual reality in scientific visualization. Commun. ACM 39(5), 62–71 (1996)CrossRefGoogle Scholar
  10. 10.
    Chatzinikos, F., Wright, H.: Computational steering by direct image manipulation. In: VMV ’01: Proceedings of the Vision Modeling and Visualization Conference 2001, pp. 455–462. Aka GmbH (2001)Google Scholar
  11. 11.
    Chen S., Doolen G.D.: Lattice boltzmann method for fluid flows. Ann. Rev. Fluid Mech. 30(1), 329–364 (1998)MathSciNetCrossRefGoogle Scholar
  12. 12.
    Computational Engineering International (CEI): EnSight. Visited on 2010-02-25
  13. 13.
    Conner, B.D., Snibbe, S.S., Herndon, K.P., Robbins, D.C., Zeleznik, R.C., van Dam, A.: Three-dimensional widgets. In: SI3D ’92: Proceedings of the 1992 Symposium on Interactive 3D Graphics, pp. 183–188. ACM, New York, NY, USA (1992)Google Scholar
  14. 14.
    Crane, K., Llamas, I., Tariq, S.: Real-time simulation and rendering of 3d fluids. In: Nguyen, H. (ed.) GPU Gems 3, Chap. 30. Addison Wesley Professional (2007)Google Scholar
  15. 15.
    van Dam A., Forsberg A.S., Laidlaw D.H., LaViola J.J., Simpson R.M.: Immersive vr for scientific visualization: a progress report. IEEE Comput. Graph. Appl. 20, 26–52 (2000)Google Scholar
  16. 16.
    d’Humières D., Bouzidi M., Lallemand P.: Thirteen-velocity three-dimensional lattice boltzmann model. Phys. Rev. E 63(6), 066,702 (2001)Google Scholar
  17. 17.
    d’Humières D., Ginzburg I., Krafczyk M., Lallemand P., Luo L.S.: Multiple-relaxation-time lattice Boltzmann models in three dimensions. Royal Soc. London Philos. Trans. Ser. A 360, 437–451 (2002)zbMATHCrossRefGoogle Scholar
  18. 18.
    d’Humières D., Lallemand P., Frisch U.: Lattice gas model for 3D hydrodynamics. Europhys. Lett. 2, 291–297 (1986)CrossRefGoogle Scholar
  19. 19.
    Eickermann, T., Frings, W.: Visit—a visualization interface toolkit version 1.0. Internal report ib-2000, Forschungszentrum Jülich (2000)Google Scholar
  20. 20.
    Esnard, A., Richart, N., Coulaud, O.: A steering environment for online parallel visualization of legacy parallel simulations. In: DS-RT ’06: Proceedings of the 10th IEEE International Symposium on Distributed Simulation and Real-Time Applications, pp. 7–14. IEEE Computer Society, Washington, DC, USA (2006)Google Scholar
  21. 21.
    Exa: PowerFLOW. Visited on 2010-02-19
  22. 22.
    Fan, Z., Qiu, F., Kaufman, A., Yoakum-Stover, S.: Gpu cluster for high performance computing. In: SC ’04: Proceedings of the 2004 ACM/IEEE Conference on Supercomputing, p. 47. IEEE Computer Society (2004)Google Scholar
  23. 23.
    Fatica, M.: Accelerating linpack with cuda on heterogenous clusters. In: GPGPU-2: Proceedings of 2nd Workshop on General Purpose Processing on Graphics Processing Units, pp. 46–51. ACM, New York, NY, USA (2009)Google Scholar
  24. 24.
    Foster I., Kesselman C.: The Grid: Blueprint for a New Computing Infrastructure. Morgan Kaufmann Publishers, Los Altos (1998)Google Scholar
  25. 25.
    Fowler, M.: Development of further patterns of enterprise application architecture. “Website” (2010). Available online at, visited on 2010-02-15
  26. 26.
    Frisch U., d’Humières D., Hasslacher B., Lallemand P., Pomeau Y., Rivet J.P.: Lattice gas hydrodynamics in two and three dimensions. Complex Syst. 1, 75–136 (1987)Google Scholar
  27. 27.
    Frisch U., Hasslacher B., Pomeau Y.: Lattice-Gas automata for the Navier-Stokes equation. Phys. Rev. Lett. 56, 1505–1508 (1986)CrossRefGoogle Scholar
  28. 28.
    Gamma E., Helm R., Johnson R.E., Vlissides J.: Design Patterns: Elements of Reusable Object-Oriented Software. Addison-Wesley, Reading, MA (1995)Google Scholar
  29. 29.
    Geist G.A. II, Arthur J., Kohl J.A., Papadopoulos P.M.: Cumulvs: Providing fault-tolerance, visualization and steering of parallel applications. Int. J. High Perform. Comput. Appl. 11, 224–236 (1996)CrossRefGoogle Scholar
  30. 30.
    Geller, S., Tölke, J., Krafczyk, M.: Lattice Boltzmann methods on quadree-type grids for fluid-structure interaction. In: Bungartz, H., Schäfer, M. (eds.) Fluid-Structure Interaction, Modelling, Simulation and Optimisation, Lecture Notes in Computational Science and Engineering, vol. 53, pp. 270–293. Springer (2006)Google Scholar
  31. 31.
    Globus, A.: A software model for visualization of large unsteady 3-d cfd results. In: AIAA 95-0115, 33rd Aerospace Sciences Meeting and Exhibit, January 912, pp. 92–031 (1991)Google Scholar
  32. 32.
    Group, K.: OpenCL—the open standard for parallel programming of heterogeneous systems. Visited on 2010-02-22
  33. 33.
    Group, K.: Opengl—the industry standard for high performance graphics. Visited on 2010-02-20
  34. 34.
    Haber R., McNabb D.: Visualization idioms: a conceptual model for scientific visualization systems. In: Nielson, G., Shriver, B., Rosenblum, L. (eds) Visualization in Scientific Computing, pp. 74–93. IEEE Computer Society Press, Silver Spring (1990)Google Scholar
  35. 35.
    Hart D., Kraemer E.: Consistency considerations in the interactive steering of computations. Int. J. Parallel Distrib. Syst. Netw. 2, 171–179 (1999)Google Scholar
  36. 36.
    He X., Luo L.S.: Theory of the lattice Boltzmann method: from the Boltzmann equation to the lattice Boltzmann equation. Phys. Rev. E 56, 6811–6817 (1997)CrossRefGoogle Scholar
  37. 37.
    Holmes, I.R., Kawalsky, R.S.: The realitygrid pda and smartphone clients: developing effective handheld user interfaces for e-science. In: Cox, S.J. (ed.) Proceedings of UK e-Science All Hands Meeting, pp. 502–509. EPSRC (2006)Google Scholar
  38. 38.
    Hou, S.: A Lattice Boltzmann method for incompressible, viscous flow. Ph.D. thesis, Kansas State University (1995)Google Scholar
  39. 39.
    Janssen, C., Krafczyk, M.: Free surface flow simulations on gpus using the lbm (2010). Submitted to: Computers & Mathematics with ApplicationsGoogle Scholar
  40. 40.
    Johnson C., Parker S.G., Hansen C., Kindlmann G.L., Livnat Y.: Interactive simulation and visualization. Computer 32(12), 59–65 (1999)CrossRefGoogle Scholar
  41. 41.
    Kohl J.A., Wilde T., Bernholdt D.E.: Cumulvs: interacting with high-performance scientific simulations, for visualization, steering and fault tolerance. Int. J. High Perform. Comput. Appl. 20(2), 255–285 (2006)CrossRefGoogle Scholar
  42. 42.
    Kok A.J.F., van Liere R.: A multimodal virtual reality interface for 3d interaction with vtk. Knowl. Inf. Syst. 13(2), 197–219 (2007)CrossRefGoogle Scholar
  43. 43.
    Krafczyk M., Tölke J., Luo L.S.: Large-Eddy simulations with a multiple-relaxation LBE model. Int. J. Modern Phys. B 17, 33–39 (2003)CrossRefGoogle Scholar
  44. 44.
    Lallemand P., Luo L.S.: Theory of the lattice Boltzmann method: dispersion, dissipation, isotropy, Galilean invariance, and stability. Phys. Rev. E 61(6), 6546–6562 (2000)MathSciNetCrossRefGoogle Scholar
  45. 45.
    Linxweiler, J., Tölke, J., Krafczyk, M.: Applying modern soft- and hardware technologies for computational steering approaches in computational fluid dynamics. In: Cyberworlds, 2007. CW ’07. International Conference on, pp. 41–45 (2007)Google Scholar
  46. 46.
    Luksch, P., Rathmayer, S., Sunderam, V.: Internet-based collaborative simulation in computational prototyping and scientific research. In: The 2000 International Conference on Parallel and Distributed Processing Techniques and Applications (PDPTA’2000). Las Vegas, Nevada, USA (2000)Google Scholar
  47. 47.
    Marshall R., Kempf J., Dyer S., Yen C.C.: Visualization methods and simulation steering for a 3d turbulence model of lake erie. SIGGRAPH Comput. Graph. 24(2), 89–97 (1990)CrossRefGoogle Scholar
  48. 48.
    McCormick, B., Fanti, T.D., Brown, M.: Visualization in scientific computing: Report of the nsf advisory panel on graphics, image processing and workstations. ACM Comput. Graph. 21(6) (1987)Google Scholar
  49. 49.
    Meyer B.: Object-Oriented Software Construction, 2nd edn. Prentice Hall PTR, Englewood Cliffs (2000)Google Scholar
  50. 50.
    Microsoft: DirectX. Visited on 2010-02-20
  51. 51.
    Moore, G.E.: Cramming more components onto integrated circuits. Electronics (1965)Google Scholar
  52. 52.
    Mulder, J.D., van Wijk, J.: 3d Computational steering with parametrized geometric objects. In: Visualization ’95 Proceedings of the 1995 Visualization Conference, pp. 304–311 (1995)Google Scholar
  53. 53.
    Mulder J.D., van Wijk J.J., van Liere R.: A survey of computational steering environments. Future Gener. Comput. Syst. 15(1), 119–129 (1999)CrossRefGoogle Scholar
  54. 54.
    Munzner T., Johnson C., Moorhead R., Pfister H., Rheingans P., Yoo T.S.: Nih-nsf visualization research challenges report summary. IEEE Comput. Graph. Appl. 26, 20–24 (2006)CrossRefGoogle Scholar
  55. 55.
    Norman D.A.: The Psychology of Everyday Things. Doubleday, Garden City (1988)Google Scholar
  56. 56.
    NVIDIA: NVIDIA CUDA Programming Guide 2.3.1 (2009)Google Scholar
  57. 57.
    Parker, S.G., Johnson, C.R.: Scirun: a scientific programming environment for computational steering. In: Supercomputing ’95: Proceedings of the 1995 ACM/IEEE Conference on Supercomputing (CDROM), p. 52. ACM, New York, NY, USA (1995)Google Scholar
  58. 58.
    Parker S.G., Weinstein D.W., Johnson C.R.: The scirun computational steering software system. In: Arge, E., Bruaset, A.M., Langtangen, H.P. (eds) Modern Software Tools for Scientific Computing, pp. 5–44. Birkhauser Boston Inc., Cambridge, MA, USA (1997)Google Scholar
  59. 59.
    Pickles S.M., Haines R., Pinning R.L., Porter A.R.: A practical toolkit for computational steering. Philos. Trans.: Math. Phys. Eng. Sci. 363(1833), 1843–1853 (2005)CrossRefGoogle Scholar
  60. 60.
    Potel, M.: Mvp: Model-view-presenter, the taligent programming model for c++ and java. Tech. rep., Taligent (1996)Google Scholar
  61. 61.
    Qian Y.H., d’Humières D., Lallemand P.: Lattice BGK models for Navier-Stokes equation. Europhys. Lett. (EPL) 17(6), 479–484 (1992)zbMATHCrossRefGoogle Scholar
  62. 62.
    Reenskaug, T.: Models—views—controllers. Tech. rep., Technical Note, Xerox Parc (1979)Google Scholar
  63. 63.
    Schroeder W., Martin K., Lorensen B.: The Visualization Toolkit, 3rd edn. Kitware Inc., USA (2007)Google Scholar
  64. 64.
    Schroeder, W.J., Martin, K.M., Lorensen, W.E.: The design and implementation of an object-oriented toolkit for 3d graphics and visualization. In: IEEE Visualization, pp. 93–100 (1996)Google Scholar
  65. 65.
    Smith, M.H., van Dalsem, W.R., Dougherty, R.C., Buning, P.G.: Analysis and visualization of complex unsteady three-dimensional flows. In: 27th AIAA Aerospace Sciences Meeting and Exhibit (1989)Google Scholar
  66. 66.
    Song, D., Golin, E., Norman, M.: A fine-grain dataflow model for scientific visualization systems. In: Workshop Papers of the Fourth Eurographics Workshop on Visualization in Scientific Computing, Abingdon, United Kingdom, pp. 21–23 (1993)Google Scholar
  67. 67.
    Sterling J.D., Chen S.: Stability analysis of lattice Boltzmann methods. J. Comput. Phys. 123(1), 196–206 (1996)zbMATHCrossRefGoogle Scholar
  68. 68.
    Succi S.: The Lattice Boltzmann Equation for Fluid Dynamics and Beyond (Numerical Mathematics and Scientific Computation). Oxford University Press, USA (2001)Google Scholar
  69. 69.
    Tecplot, I.: Tecplot 360. Visited on 2010-02-19
  70. 70.
    The HDF Group: HDF5 User’s Guide. The HDF Group (2008)Google Scholar
  71. 71.
    Tölke, J.: Implementation of a lattice boltzmann kernel using the compute unified device architecture developed by nvidia. Comput. Visual. Sci. (2008)Google Scholar
  72. 72.
    Tölke J., Krafczyk M.: Teraflop computing on a desktop pc with gpus for 3d cfd. Int. J. Comput. Fluid Dyn. 22(7), 443–456 (2008)zbMATHCrossRefGoogle Scholar
  73. 73.
    Upson C., Faulhaber T. Jr, Kamins D., Laidlaw D.H., Schlegel D., Vroom J., Gurwitz R., van Dam A.: The application visualization system: a computational environment for scientific visualization. IEEE Comput. Graph. Appl. 9(4), 30–42 (1989)CrossRefGoogle Scholar
  74. 74.
    van Liere, R., Mulder, J.D.: Ubiquitous computational steering. In: IEEE Visualization ’97 (1997)Google Scholar
  75. 75.
    van Liere R., Mulder J.D., van Wijk J.J.: Computational steering. Future Gener. Comput. Syst. 12(5), 441–450 (1997)CrossRefGoogle Scholar
  76. 76.
    Wellein, G., Zeiser, T., Hager, G., Donath, S.: On the single processor performance of simple lattice boltzmann kernels. Comput. Fluids 35(8–9), 910–919 (2006). In: Proceedings of the First International Conference for Mesoscopic Methods in Engineering and ScienceGoogle Scholar
  77. 77.
    Wenisch, P.: Computational steering of cfd simulations on teraflop-supercomputers. Ph.D. thesis, Technische Universität München (2008)Google Scholar
  78. 78.
    Wenisch, P., Wenisch, O., Rank, E.: Harnessing high-performance computers for computational steering. Recent advances in parallel virtual machine and message passing interface, pp. 536–543 (2005)Google Scholar
  79. 79.
    Wijk, J.J.V., Liere, R.V., Mulder, J.D., Wijk, J.J.V., Liere, R.V., Mulder, J.D.: Bringing computational steering to the user. In: Presented at the Dagstuhl Seminar on Scientific Visualization, pp. 304–313 (1997)Google Scholar
  80. 80.
    Wolfram S.: Cellular automaton fluids: basic theory. J. Stat. Phys. 45, 471–526 (1986)MathSciNetzbMATHCrossRefGoogle Scholar
  81. 81.
    Wood, J., Brodlie, K., Walton, J.: gViz—visualization and steering for the grid. In: Proceedings of e-Science All Hands Meeting (2003)Google Scholar
  82. 82.
    Wright, H.: Putting visualization first in computational steering. In: Proceedings of UK e-Science All Hands Meeting, pp. 326–331 (2004)Google Scholar
  83. 83.
    Wright H., Crompton R., Kharche S., Wenisch P.: Steering and visualization: enabling technologies for computational science. Future Gener. Comput. Syst. 26(3), 506–513 (2010)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag 2011

Authors and Affiliations

  • Jan Linxweiler
    • 1
    Email author
  • Manfred Krafczyk
    • 1
  • Jonas Tölke
    • 2
  1. 1.Institute for Computational Modelling in Civil EngineeringBraunschweig University of TechnologyBraunschweigGermany
  2. 2.Ingrain, Inc.HoustonUSA

Personalised recommendations