Physics-preserving fluid reconstruction from monocular video coupling with SFS and SPH

  • 82 Accesses


We propose a joint method to reconstruct dynamic fluid volume sequences from a monocular video. Compared with previous methods, sophisticated equipment or careful experimental setups are not required. In order to recover the surface detail and maintain its physical property, a joint reconstruction method coupled with shape from shading (SFS) and smoothed particle hydrodynamics (SPH) algorithms is proposed. SFS is the first used to recover the height field for each frame of the video, and then the height field of the first frame is further extended to a volume as the initial state of the SPH simulation. Based on the above initial data, the key idea of our method is to optimize a SPH model to simulate the fluid volume sequences conforming to the laws of physical motion, and correct the fluid volumes to refine the surface details conforming to the recovered height field by SFS. Our experimental results compare favorably to the state of the art in terms of global motion features and fluid surface details and demonstrate the performance of our approach.

This is a preview of subscription content, log in to check access.

Access options

Buy single article

Instant unlimited access to the full article PDF.

US$ 39.95

Price includes VAT for USA

Subscribe to journal

Immediate online access to all issues from 2019. Subscription will auto renew annually.

US$ 199

This is the net price. Taxes to be calculated in checkout.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Change history

  • 12 August 2019

    The Acknowledgements section is missing in the original article. It is given below.


  1. 1.

    Akinci, N., Ihmsen, M., Akinci, G., Solenthaler, B., Teschner, M.: Versatile rigid-fluid coupling for incompressible SPH. ACM Trans. Gr. 31(4), 62 (2012)

  2. 2.

    Akinci, N., Akinci, G., Teschner, M.: Versatile surface tension and adhesion for SPH fluids. ACM Trans. Gr. 32(6), 182 (2013)

  3. 3.

    Alduán, I., Tena, A., Otaduy, M.A.: Dyverso: a versatile multi-phase position-based fluids solution for VFX. Comput. Gr. Forum 36, 32–44 (2017)

  4. 4.

    Balschbach, G., Klinke, J., Jähne, B.: Multichannel shape from shading techniques for moving specular surfaces. In: European Conference on Computer Vision, Springer, pp 170–184 (1998)

  5. 5.

    Band, S., Gissler, C., Peer, A., Teschner, M.: Mls pressure extrapolation for the boundary handling in divergence-free SPH. In: Proceedings of the 14th Workshop on Virtual Reality Interactions and Physical Simulations, Eurographics Association, pp 55–63 (2018)

  6. 6.

    Becker, M., Teschner, M.: Weakly compressible SPH for free surface flows. In: Proceedings of the 2007 ACM SIGGRAPH/Eurographics symposium on Computer animation, Eurographics Association, pp 209–217 (2007)

  7. 7.

    Bender, J., Koschier, D.: Divergence-free smoothed particle hydrodynamics. In: Proceedings of the 14th ACM SIGGRAPH/Eurographics Symposium on Computer Animation, ACM, pp 147–155 (2015)

  8. 8.

    Bender, J., Koschier, D.: Divergence-free SPH for incompressible and viscous fluids. IEEE Trans. Visual Comput. Gr. 23(3), 1193–1206 (2017)

  9. 9.

    Besl, P.J., McKay, N.D.: Method for registration of 3-d shapes. Sensor fusion IV: control paradigms and data structures. Int. Soc. Opt. Photonics 1611, 586–607 (1992)

  10. 10.

    Bridson, R.: Fluid Simulation for Computer Graphics. CRC Press, Boca Raton (2015)

  11. 11.

    Camassa, R., Holm, D.D.: An integrable shallow water equation with peaked solitons. Phys. Rev. Lett. 71(11), 1661 (1993)

  12. 12.

    Chentanez, N., Müller, M.: Real-time eulerian water simulation using a restricted tall cell grid. ACM Trans. Gr. 30, 82 (2011)

  13. 13.

    Eckert, M.L., Heidrich, W., Thuerey, N.: Coupled fluid density and motion from single views. Comput. Gr. Forum 37, 47–58 (2018)

  14. 14.

    Enright, D., Marschner, S., Fedkiw, R.: Animation and rendering of complex water surfaces. ACM Trans. Gr. 21, 736–744 (2002)

  15. 15.

    Gingold, R.A., Monaghan, J.J.: Smoothed particle hydrodynamics: theory and application to non-spherical stars. Mon. Not. R. Astron. Soc. 181(3), 375–389 (1977)

  16. 16.

    Gissler, C., Peer, A., Band, S., Bender, J., Teschner, M.: Interlinked SPH pressure solvers for strong fluid-rigid coupling. ACM Trans. Gr. 38(1), 5 (2019)

  17. 17.

    Gregson, J., Ihrke, I., Thuerey, N., Heidrich, W.: From capture to simulation: connecting forward and inverse problems in fluids. ACM Trans. Gr. 33(4), 139 (2014)

  18. 18.

    He, X., Wang, H., Zhang, F., Wang, H., Wang, G., Zhou, K.: Robust simulation of sparsely sampled thin features in SPH-based free surface flows. ACM Trans. Gr. 34(1), 7 (2014)

  19. 19.

    Hilsenstein, V.: Surface reconstruction of water waves using thermographic stereo imaging. In: Image and Vision Computing New Zealand, Citeseer, vol 2 (2005)

  20. 20.

    Ihmsen, M., Cornelis, J., Solenthaler, B., Horvath, C., Teschner, M.: Implicit incompressible SPH. IEEE Trans. Vis. Comput. Gr. 20(3), 426–435 (2014a)

  21. 21.

    Ihmsen, M., Orthmann, J., Solenthaler, B., Kolb, A., Teschner, M.: SPH Fluids in Computer Graphics. The Eurographics Association, Aire-la-Ville (2014b)

  22. 22.

    Ihrke, I., Goidluecke, B., Magnor, M.: Reconstructing the geometry of flowing water. In: Tenth IEEE International Conference on Computer Vision, 2005. ICCV 2005, IEEE, vol 2, pp 1055–1060 (2005)

  23. 23.

    Kutulakos, K.N., Steger, E.: A theory of refractive and specular 3d shape by light-path triangulation. Int. J. Comput. Vis. 76(1), 13–29 (2008)

  24. 24.

    Li, C., Shaw, M., Pickup, D., Cosker, D., Willis, P., Hall, P.: Realtime video based water surface approximation. In: 2011 Conference for Visual Media Production (CVMP), IEEE, pp 109–117 (2011)

  25. 25.

    Li, C., Pickup, D., Saunders, T., Cosker, D., Marshall, D., Hall, P., Willis, P.: Water surface modeling from a single viewpoint video. IEEE Trans. Vis. Comput. Gr. 19(7), 1242–1251 (2013)

  26. 26.

    Limtrakul, S., Hantanong, W., Kanongchaiyos, P., Nishita, T.: Reviews on physically based controllable fluid animation. Eng. J. 14(2), 41–52 (2010)

  27. 27.

    Lucy, L.B.: A numerical approach to the testing of the fission hypothesis. Astron. J. 82, 1013–1024 (1977)

  28. 28.

    Monaghan, J.J.: Smoothed particle hydrodynamics. Ann. Rev. Astron. Astrophys. 30(1), 543–574 (1992)

  29. 29.

    Morris, N.J., Kutulakos, K.N.: Dynamic refraction stereo. IEEE Trans. Pattern Anal. Mach. Intell. 33(8), 1518–1531 (2011)

  30. 30.

    Müller, M., Charypar, D., Gross, M.: Particle-based fluid simulation for interactive applications. In: Proceedings of the 2003 ACM SIGGRAPH/Eurographics symposium on Computer animation, Eurographics Association, pp 154–159 (2003)

  31. 31.

    Murase, H.: Surface shape reconstruction of a nonrigid transport object using refraction and motion. IEEE Trans. Pattern Anal. Mach. Intell. 10, 1045–1052 (1992)

  32. 32.

    Okabe, M., Dobashi, Y., Anjyo, K., Onai, R.: Fluid volume modeling from sparse multi-view images by appearance transfer. ACM Trans. Gr. 34(4), 93 (2015)

  33. 33.

    Péteri, R., Fazekas, S., Huiskes, M.J.: Dyntex: A comprehensive database of dynamic textures. Pattern Recogn. Lett. 31(12), 1627–1632 (2010)

  34. 34.

    Pickup, D., Li, C., Cosker, D., Hall, P., Willis, P.: Reconstructing mass-conserved water surfaces using shape from shading and optical flow. In: Asian Conference on Computer Vision, Springer, pp 189–201 (2010)

  35. 35.

    Ping-Sing, T., Shah, M.: Shape from shading using linear approximation. Image Vis. Comput. 12(8), 487–498 (1994)

  36. 36.

    Qian, Y., Gong, M., Yang, Y.H.: Stereo-based 3d reconstruction of dynamic fluid surfaces by global optimization. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 1269–1278 (2017)

  37. 37.

    Qian, Y., Zheng, Y., Gong, M., Yang, Y.H.: Simultaneous 3d reconstruction for water surface and underwater scene. In: Proceedings of the European Conference on Computer Vision (ECCV), pp 754–770 (2018)

  38. 38.

    Quan, H., Yu, M., Song, X., Gao, Y.: Real time reconstruction of fluid in video. Int. J. Model. Simul. Sci. Comput. 4(04), 1342001 (2013)

  39. 39.

    Quan, H., Song, X., Yu, M., Song, Y.: 3d fluid scene synthesis and animation. In: Proceedings of the 13th ACM SIGGRAPH International Conference on Virtual-Reality Continuum and its Applications in Industry, ACM, pp 219–222 (2014)

  40. 40.

    Quan, H., Song, Y., Zhou, X., Zhang, L.: Physically-based enhancement in shallow water resimulation. Int. J. Model. Simul. Sci. Comput. 7(03), 1650029 (2016)

  41. 41.

    Quan, H., Wang, C., Song, Y.: Fluid re-simulation based on physically driven model from video. Vis. Comput. 33(1), 85–98 (2017)

  42. 42.

    Rusinkiewicz, S., Levoy, M.: Efficient variants of the ICP algorithm. In: 3-D Digital Imaging and Modeling, 2001. Proceedings. Third International Conference on, IEEE, pp 145–152 (2001)

  43. 43.

    Schechter, H., Bridson, R.: Ghost SPH for animating water. ACM Trans. Gr. 31(4), 61 (2012)

  44. 44.

    Sinha, S.N., Steedly, D., Szeliski, R., Agrawala, M., Pollefeys, M.: Interactive 3d architectural modeling from unordered photo collections. ACM Trans. Gr. 27, 159 (2008)

  45. 45.

    Solenthaler, B., Pajarola, R.: Predictive-corrective incompressible SPH. ACM Trans. Gr. 28, 40 (2009)

  46. 46.

    Song, Y., Quan, H., Huang, Y.: Enhancement in realistic fluid re-simulation. In: 2016 International Conference on Artificial Intelligence: Technologies and Applications, Atlantis Press (2016)

  47. 47.

    Takahashi, T., Dobashi, Y., Fujishiro, I., Nishita, T., Lin, M.C.: Implicit formulation for SPH-based viscous fluids. Comput. Gr. Forum 34, 493–502 (2015)

  48. 48.

    Takahashi, T., Dobashi, Y., Nishita, T., Lin, M.C.: An efficient hybrid incompressible SPH solver with interface handling for boundary conditions. Comput. Gr. Forum 37, 313–324 (2018)

  49. 49.

    Tan, P., Zeng, G., Wang, J., Kang, S.B., Quan, L.: Image-based tree modeling. ACM Trans. Gr. 26, 87 (2007)

  50. 50.

    Tsai, J.C.P.S., Cryer, J., Shah, M.: Integration of shape from shading and stereo. Pattern Recogn. 28(7), 1033–1043 (1995)

  51. 51.

    van der Laan, W.J., Green, S., Sainz, M.: Screen space fluid rendering with curvature flow. In: Proceedings of the 2009 symposium on Interactive 3D graphics and games, ACM, pp 91–98 (2009)

  52. 52.

    Wang, C., Wang, C., Qin, H.: Ty, Zhang: video-based fluid reconstruction and its coupling with SPH simulation. Vis. Comput. 33(9), 1211–1224 (2017)

  53. 53.

    Wang, H., Liao, M., Zhang, Q., Yang, R., Turk, G.: Physically guided liquid surface modeling from videos. ACM Trans. Gr. 28(3), 90 (2009)

  54. 54.

    Wei, Y., Ofek, E., Quan, L., Shum, H.Y.: Modeling hair from multiple views. ACM Trans. Gr. 24, 816–820 (2005)

  55. 55.

    White, R., Crane, K., Forsyth, D.A.: Capturing and animating occluded cloth. ACM Trans. Gr. 26, 34 (2007)

  56. 56.

    Williamson, D.L., Drake, J.B., Hack, J.J., Jakob, R., Swarztrauber, P.N.: A standard test set for numerical approximations to the shallow water equations in spherical geometry. J. Comput. Phys. 102(1), 211–224 (1992)

  57. 57.

    Wu, W., Li, H., Su, T., Liu, H., Lv, Z.: Gpu-accelerated SPH fluids surface reconstruction using two-level spatial uniform grids. Vis. Comput. 33(11), 1429–1442 (2017)

  58. 58.

    Xu, W., Chatterjee, A., Zollhöfer, M., Rhodin, H., Mehta, D., Seidel, H.P., Theobalt, C.: Monoperfcap: human performance capture from monocular video. ACM Trans. Gr. 37(2), 27 (2018)

  59. 59.

    Yan, X., Jiang, Y.T., Li, C.F., Martin, R.R., Hu, S.M.: Multiphase SPH simulation for interactive fluids and solids. ACM Trans. Gr. 35(4), 79 (2016)

  60. 60.

    Yang, M., Li, X., Liu, Y., Yang, G., Wu, E.: A novel surface tension formulation for SPH fluid simulation. Vis. Comput. 33(5), 597–606 (2017)

  61. 61.

    Yu, J., Turk, G.: Reconstructing surfaces of particle-based fluids using anisotropic kernels. ACM Trans. Gr. 32(1), 5 (2013)

  62. 62.

    Yu, M., Quan, H.: Fluid surface reconstruction based on specular reflection model. Comput. Anim. Virtual Worlds 24(5), 497–510 (2013)

  63. 63.

    Zhang, R., Tsai, P.S., Cryer, J.E., Shah, M.: Shape-from-shading: a survey. IEEE Trans. Pattern Anal. Mach. Intell. 21(8), 690–706 (1999)

Download references

Author information

Correspondence to Xukun Shen.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (mp4 204441 KB)

Supplementary material 1 (mp4 204441 KB)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Nie, X., Hu, Y. & Shen, X. Physics-preserving fluid reconstruction from monocular video coupling with SFS and SPH. Vis Comput (2019).

Download citation


  • Fluid reconstruction
  • SPH
  • Shape from shading
  • Video-based reconstruction
  • Physically based simulation