The Visual Computer

, Volume 33, Issue 9, pp 1211–1224 | Cite as

Video-based fluid reconstruction and its coupling with SPH simulation

  • Chao Wang
  • Changbo WangEmail author
  • Hong Qin
  • Tai-you Zhang
Original Article


Conventional methods to create fluid animation primarily resort to physically based simulation via numerical integration, whose performance is dominantly hindered by large amount of numerical calculation and low efficiency. Alternatively, video-based methods could easily reconstruct fluid surfaces from videos, yet they are not able to realize two-way dynamic interaction with their surrounding environment in a physically correct manner. In this paper, we propose a hybrid method that combines video-based fluid surface reconstruction and popular fluid animation models to compute and re-animate fluid surface. First, the fluid surface’s height field corresponding to each video frame is estimated by using the shape-from-shading method. After denoising, hole-filling, and smoothing operations, the height field is utilized to calculate the velocity field, where the shallow water model is adopted. Then we treat the height field and velocity field as real data to drive the simulation. Still, only one layer of surface particles is not capable of driving the smoothed particle hydrodynamics (SPH) system. The surface particles (including 3D position and its velocity) are then employed to guide the spatial sampling of the entire volume underneath. Second, the volume particles corresponding to each video frame are imported into the SPH system to couple with other possible types of particles (used to define interacting objects), whose movement is dictated by the direct forcing method, and fluid particles’ geometry information is then corrected by both physical models and real video data. The resulting animation approximates the reconstruction surface from the input video, and new physically based coupling behaviors are also appended. We document our system’s detailed implementation and showcase visual performance across a wide range of scenes.


Video-based reconstruction Height field Shallow water SPH Two-way coupling Physically based simulation 



This paper is partially supported by Natural Science Foundation of China under Grant Nos. 61532002, 61272199, National High-tech R&D Program of China (863 Program) under Grant 2015AA016404, the Specialized Research Fund for Doctoral Program of Higher Education under Grant 20130076110008, and Open Funding Project of State Key Laboratory of Virtual Reality Technology and Systems of Beihang University under Grant BUAA-VR-15KF-14. The authors would like to thank all reviewers for their very helpful and constructive comments and suggestions. We also thank Dyntex Dataset for the support of rich fluid videos for our study.

Supplementary material

371_2016_1284_MOESM1_ESM.mp4 (201.7 mb)
Supplementary material 1 (mp4 206580 KB)


  1. 1.
    Akinci, N., Ihmsen, M., Akinci, G., Solenthaler, B., Teschner, M.: Versatile rigid-fluid coupling for incompressible sph. ACM Trans. Graph. 31(4), 62:1–62:8 (2012)CrossRefGoogle Scholar
  2. 2.
    Becker, M., Teschner, M.: Weakly compressible sph for free surface flows. In: Proceedings of the 2007 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, pp. 209–217 (2007)Google Scholar
  3. 3.
    Besl, P.J., McKay, N.D.: Method for registration of 3-d shapes. IEEE Trans. Pattern Anal. Mach. Intell. 14(2), 239–256 (1992)CrossRefGoogle Scholar
  4. 4.
    Chao, J., Mei, R., Singh, R., Shyy, W.: A filter-based, mass-conserving lattice boltzmann method for immiscible multiphase flows. Int. J. Numer. Methods Fluids 66(5), 622–647 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
  5. 5.
    Chen, S., Doolen, G.D.: Lattice Boltzmann method for fluid flows. Annu. Rev. Fluid Mech. 30(1), 329–364 (1998)MathSciNetCrossRefGoogle Scholar
  6. 6.
    Chentanez, N., Müller, M.: Real-time eulerian water simulation using a restricted tall cell grid. ACM Trans. Graph. 30(4), 82:1–82:10 (2011)CrossRefGoogle Scholar
  7. 7.
    Foster, N., Metaxas, D.: Realistic animation of liquids. Graph. Models Image Process. 58(5), 471–483 (1996)CrossRefGoogle Scholar
  8. 8.
    Gregson, J., Ihrke, I., Thuerey, N., Heidrich, W.: From capture to simulation-connecting forward and inverse problems in fluids. ACM Trans. Graph. 33(4), 139:1–139:11 (2014)CrossRefGoogle Scholar
  9. 9.
    He, X., Wang, H., Zhang, F., Wang, H., Wang, G., Zhou, K.: Robust simulation of sparsely sampled thin features in sph-based free surface flows. ACM Trans. Graph. 34(1), 7:1–7:9 (2014)CrossRefGoogle Scholar
  10. 10.
    Ihmsen, M., Akinci, N., Becker, M., Teschner, M.: A parallel sph implementation on multi-core cpus. Comput. Graph. Forum 30(1), 99–112 (2011)CrossRefGoogle Scholar
  11. 11.
    Ihmsen, M., Orthmann, J., Solenthaler, B., Kolb, A., Teschner, M.: Sph fluids in computer graphics. In: Eurographics, pp. 21–42 (2014)Google Scholar
  12. 12.
    Kutulakos, K., Steger, E.: A theory of refractive and specular 3d shape by light-path triangulation. Int. J. Comput. Vis. 76(1), 13–29 (2007)CrossRefGoogle Scholar
  13. 13.
    Kwatra, V., Mordohai, P., Narain, R., Penta, S.K., Carlson, M., Pollefeys, M., Lin, M.C.: Fluid in video: augmenting real video with simulated fluids. Comput. Graph. Forum 27(2), 487–496 (2008)CrossRefGoogle Scholar
  14. 14.
    Ladický, L., Jeong, S., Solenthaler, B., Pollefeys, M., Gross, M.: Data-driven fluid simulations using regression forests. ACM Trans. Graph. 34(6), 199:1–199:9 (2015)CrossRefGoogle Scholar
  15. 15.
    Lentine, M., Zheng, W., Fedkiw, R.: A novel algorithm for incompressible flow using only a coarse grid projection. ACM Trans. Graph. 29(4), 114:1–114:9 (2010)CrossRefGoogle Scholar
  16. 16.
    Li, C., Pickup, D., Saunders, T., Cosker, D., Marshall, D., Hall, P., Willis, P.: Water surface modeling from a single viewpoint video. IEEE Trans. Vis. Comput. Graphics 19(7), 1242–1251 (2013)CrossRefGoogle Scholar
  17. 17.
    Low, K.L.: Linear Least-squares Optimization for Point-to-plane ICP Surface Registration. University of North Carolina, Chapel Hill (2004)Google Scholar
  18. 18.
    Monaghan, J.J.: Smoothed particle hydrodynamics. Annu. Rev. Astron. Astrophys. 30, 543–574 (1992)CrossRefGoogle Scholar
  19. 19.
    Müller, M., Charypar, D., Gross, M.: Particle-based fluid simulation for interactive applications. In: Proceedings of the 2003 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, pp. 154–159 (2003)Google Scholar
  20. 20.
    Ojeda, J., Susín, A.: Enhanced lattice boltzmann shallow waters for real-time fluid simulations. Eurographics 2013, 25–28 (2013)Google Scholar
  21. 21.
    Pan, Z., Huang, J., Tong, Y., Bao, H.: Wake synthesis for shallow water equation. Comput. Graph. Forum 31, 2029–2036 (2012)CrossRefGoogle Scholar
  22. 22.
    Péteri, R., Fazekas, S., Huiskes, M.J.: Dyntex: a comprehensive database of dynamic textures. Pattern Recognit. Lett. 31(12), 1627–1632 (2010)CrossRefGoogle Scholar
  23. 23.
    Pickup, D., Li, C., Cosker, D., Hall, P., Willis, P.: Reconstructing mass-conserved water surfaces using shape from shading and optical flow. In: Proceedings of the 10th Asian Conference on Computer Vision, Volume Part IV, pp. 189–201 (2011)Google Scholar
  24. 24.
    Ping-Sing, T., Shah, M.: Shape from shading using linear approximation. Image Vis. Comput. 12(8), 487–498 (1994)CrossRefGoogle Scholar
  25. 25.
    Quan, H., Wang, C., Song, Y.: Fluid re-simulation based on physically driven model from video. The Visual Computer, pp. 1–14 (2015)Google Scholar
  26. 26.
    Robert Bridson, A.K.P.: Fluid simulation for computer animation (2008)Google Scholar
  27. 27.
    Shao, X., Zhou, Z., Magnenat-Thalmann, N., Wu, W.: Stable and fast fluid-solid coupling for incompressible sph. Comput. Graph. Forum 34(1), 191–204 (2015)CrossRefGoogle Scholar
  28. 28.
    Solenthaler, B., Mller, M., Gross, M.H., Bucher, P., Chentanez, N.: Sph based shallow water simulation. Virtual Reality Interactions and Physical Simulations, pp. 39–46 (2011)Google Scholar
  29. 29.
    Stam, J.: Stable fluids. In: Proceedings of the 26th Annual Conference on Computer Graphics and Interactive Techniques, pp. 121–128 (1999)Google Scholar
  30. 30.
    Tan, P., Zeng, G., Wang, J., Kang, S.B., Quan, L.: Image-based tree modeling. ACM Trans. Graph. 26(3) (2007)Google Scholar
  31. 31.
    Thurey, N., Müller-Fischer, M., Schirm, S., Gross, M.: Real-time breaking waves for shallow water simulations. In: Proceedings of the 15th Pacific Conference on Computer Graphics and Applications, pp. 39–46 (2007)Google Scholar
  32. 32.
    Wang, H., Liao, M., Zhang, Q., Yang, R., Turk, G.: Physically guided liquid surface modeling from videos. ACM Trans. Graph. 28(3), 341–352 (2009)Google Scholar
  33. 33.
    Zhang, R., Tsai, P.S., Cryer, J.E., Shah, M.: Shape-from-shading: a survey. IEEE Trans. Pattern Anal. Mach. Intell. 21(8), 690–706 (1999)CrossRefzbMATHGoogle Scholar
  34. 34.
    Zhu, B., Lu, W., Cong, M., Kim, B., Fedkiw, R.: A new grid structure for domain extension. ACM Trans. Graph. 32(4), 63:1–63:12 (2013)CrossRefzbMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2016

Authors and Affiliations

  1. 1.School of Computer Science and Software EngineeringEast China Normal UniversityShanghaiChina
  2. 2.MOE International Joint Lab of Trustworthy SoftwareEast China Normal UniversityShanghaiChina
  3. 3.Department of Computer ScienceState University of New York at Stony BrookStony BrookUSA

Personalised recommendations