Skip to main content
SpringerLink
Log in

Reconstructing and editing fluids using the adaptive multilayer external force guiding model

  • Research Paper
  • Published:
Science China Information Sciences Aims and scope Submit manuscript

Abstract

Reconstructing monocular fluids usually involves a tedious trial-and-error process, and editing desired fluid behaviors is notoriously difficult to predict and control. To address these problems, we propose an adaptive multilayer external force guiding model that alleviates the challenging parameter tuning and satisfies user-defined requirements. External forces cause the effect of target particles on fluid particles. The adaptive multilayer scheme makes the whole 3D fluid volume subject to the shape and motion of the water captured by the input video or designed by users. Therefore, we can avoid the tedious and laborious parameter tuning and easily balance the smoothness of fluid volume and the details of the water surface. Simultaneously, to vividly reproduce the inflow and outflow of the video scene, we construct a generation and extinction model to add or delete fluid particles according to the three-dimensional velocity field of target particles calculated using a hybrid model coupling shape-from-shading with optical flow. Besides, we edit fluids by subtly extracting features selected by users using the off-screen rendering method and seamlessly integrating them using the dynamic weight approach. Experiments show that our approach is comparable to the state-of-the-art reconstruction quality and is remarkably convenient in authoring flows by editing fluids. Furthermore, our results can be effectively applied to any desired new scenario.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Wang H, Liao M, Zhang Q, et al. Physically guided liquid surface modeling from videos. ACM Trans Graph, 2009, 28: 1–11

    Article  Google Scholar 

  2. Tewari A, Zollhöfer M, Garrido P, et al. Self-supervised multi-level face model learning for monocular reconstruction at over 250 Hz. In: Proceedings of 2018 IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, 2018. 2549–2559

  3. Thapa S, Li N, Ye J. Dynamic fluid surface reconstruction using deep neural network. In: Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, 2020. 21–30

  4. Müller M, Charypar D, Gross M H. Particle-based fluid simulation for interactive applications. In: Proceedings of the 2003 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, San Diego, 2003. 154–159

  5. Foster N, Metaxas D. Realistic animation of liquids. Graph Model Image Process, 1996, 58: 471–483

    Article  Google Scholar 

  6. Stam J. Stable fluids. In: Proceedings of the 26th Annual Conference on Computer Graphics and Interactive Techniques, Los Angeles, 1999. 121–128

  7. Takahashi T, Lin M C. Video-guided real-to-virtual parameter transfer for viscous fluids. ACM Trans Graph, 2019, 38: 1–12

    Article  Google Scholar 

  8. Zhang R, Tsai P-S, Cryer J E, et al. Shape-from-shading: a survey. IEEE Trans Pattern Anal Machine Intell, 1999, 21: 690–706

    Article  MATH  Google Scholar 

  9. Tsai P-S, Shah M. Shape from shading using linear approximation. Image Vision Comput, 1994, 12: 487–498

    Article  Google Scholar 

  10. Dou P, Wu Y, Shah S K, et al. Monocular 3D facial shape reconstruction from a single 2D image with coupled-dictionary learning and sparse coding. Pattern Recogn, 2018, 81: 515–527

    Article  Google Scholar 

  11. Nie X, Hu Y, Su Z, et al. External forces guided fluid surface and volume reconstruction from monocular video. In: Proceedings of Pacific Graphics Short Papers, Seoul, 2019

  12. Eckert M L, Heidrich W, Thürey N. Coupled fluid density and motion from single views. Comput Graph Forum, 2018, 37: 47–58

    Article  Google Scholar 

  13. Foster N, Fedkiw R. Practical animation of liquids. In: Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques, Los Angeles, 2001. 23–30

  14. Rasmussen N, Enright D, Nguyen D Q, et al. Directable photorealistic liquids. In: Proceedings of the 2004 ACM SIG-GRAPH/Eurographics Symposium on Computer Animation, Grenoble, 2004. 193–202

  15. Angelidis A, Neyret F, Singh K, et al. A controllable, fast and stable basis for vortex based smoke simulation. In: Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation, Vienna, 2006. 25–32

  16. Kim Y, Machiraju R, Thompson D S. Path-based control of smoke simulations. In: Proceedings of the ACM SIG-GRAPH/Eurographics Symposium on Computer Animation, Vienna, 2006. 33–42

  17. Treuille A, McNamara A, Popović Z, et al. Keyframe control of smoke simulations. ACM Trans Graph, 2003, 22: 716–723

    Article  Google Scholar 

  18. Pan Z, Huang J, Tong Y, et al. Interactive localized liquid motion editing. ACM Trans Graph, 2013, 32: 1–10

    Article  Google Scholar 

  19. Thürey N, Keiser R, Pauly M, et al. Detail-preserving fluid control. Graph Model, 2009, 71: 221–228

    Article  Google Scholar 

  20. Zhang X, Liu S. SPH fluid control with self-adaptive turbulent details. Comp Anim Virtual Worlds, 2015, 26: 357–366

    Article  Google Scholar 

  21. Zhang X, Liu S. Parallel SPH fluid control with dynamic details. Comput Anim Virtual Worlds, 2018, 29: e1801

    Article  Google Scholar 

  22. Nielsen M B, Christensen B B, Zafar N B, et al. Guiding of smoke animations through variational coupling of simulations at different resolutions. In: Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation, New Orleans, 2009. 217–226

  23. Nielsen M B, Bridson R. Guide shapes for high resolution naturalistic liquid simulation. ACM Trans Graph, 2011, 30: 1–8

    Article  Google Scholar 

  24. Huang R, Melek Z, Keyser J. Preview-based sampling for controlling gaseous simulations. In: Proceedings of the Eurographics/ACM SIGGRAPH Symposium on Computer Animation, Vancouver, 2011. 177–186

  25. Feng G, Liu S. Detail-preserving SPH fluid control with deformation constraints. Comput Anim Virtual Worlds, 2018, 29: e1781

    Article  Google Scholar 

  26. Zhang G, Lu D, Zhu D, et al. Rigid-motion-inspired liquid character animation. Comput Anim Virtual Worlds, 2013, 24: 205–213

    Article  Google Scholar 

  27. Lu J M, Chen X S, Yan X, et al. A rigging-skinning scheme to control fluid simulation. Comput Graph Forum, 2019, 38: 501–512

    Article  Google Scholar 

  28. Ma P, Tian Y, Pan Z, et al. Fluid directed rigid body control using deep reinforcement learning. ACM Trans Graph, 2018, 37: 1–11

    Google Scholar 

  29. Zhang G, Zhu D, Qiu X, et al. Skeleton-based control of fluid animation. Vis Comput, 2011, 27: 199–210

    Article  Google Scholar 

  30. Ihmsen M, Orthmann J, Solenthaler B, et al. SPH fluids in computer graphics. In: Proceedings of the 35th Annual Conference of the European Association for Computer Graphics, Strasbourg, 2014

  31. Yang T, Martin R R, Lin M C, et al. Pairwise force SPH model for real-time multi-interaction applications. IEEE Trans Visual Comput Graph, 2017, 23: 2235–2247

    Article  Google Scholar 

  32. Snape P, Zafeiriou S. Kernel-PCA analysis of surface normals for shape-from-shading. In: Proceedings of 2014 IEEE Conference on Computer Vision and Pattern Recognition, Columbus, 2014. 1059–1066

  33. Murase H. Surface shape reconstruction of an undulating transparent object. In: Proceedings of Third International Conference on Computer Vision, Osaka, 1990. 313–317

  34. Pickup D, Li C, Cosker D, et al. Reconstructing mass-conserved water surfaces using shape from shading and optical flow. In: Proceedings of the 10th Asian Conference on Computer Vision, Queenstown, 2010. 189–201

  35. Li C, Pickup D, Saunders T, et al. Water surface modeling from a single viewpoint video. IEEE Trans Visual Comput Graph, 2013, 19: 1242–1251

    Article  Google Scholar 

  36. Péteri R, Fazekas S, Huiskes M J. DynTex: a comprehensive database of dynamic textures. Pattern Recogn Lett, 2010, 31: 1627–1632

    Article  Google Scholar 

  37. Madill J, Mould D. Target particle control of smoke simulation. In: Proceedings of Graphics Interface 2013, Regina, 2013. 125–132

  38. Becker M, Teschner M. Weakly compressible SPH for free surface flows. In: Proceedings of the 2007 ACM SIG-GRAPH/Eurographics Symposium on Computer Animation, San Diego, 2007. 209–217

  39. Raveendran K, Wojtan C, Thürey N, et al. Blending liquids. ACM Trans Graph, 2014, 33: 1–10

    Article  Google Scholar 

  40. Manteaux P, Vimont U, Wojtan C, et al. Space-time sculpting of liquid animation. In: Proceedings of the 9th International Conference on Motion in Games, Burlingame, 2016. 61–71

  41. Jian M, Guo F, Yin C, et al. Automatic correction of non-uniform illumination for 3D surface heightmap reconstruction. In: Proceedings of the IEEE International Conference on Multimedia and Expo, New York City, 2009. 1402–1405

  42. Liu Z C, Wang D W, Liu Y, et al. Adaptive adjustment algorithm for non-uniform illumination images based on 2D Gamma function (in Chinese). Trans Beijing Inst Tech, 2016, 36: 191–196, 214

    Google Scholar 

  43. Rahman S, Rahman M M, Abdullah-Al-Wadud M, et al. An adaptive gamma correction for image enhancement. J Image Video Proc, 2016, 2016: 35

    Article  Google Scholar 

  44. Wang D, Yan W, Zhu T, et al. An adaptive correction algorithm for non-uniform illumination panoramic images based on the improved bilateral gamma function. In: Proceedings of 2017 International Conference on Digital Image Computing: Techniques and Applications (DICTA), 2017. 1–6

  45. Han P, Wang D, Yang X, et al. An improved adaptive correction algorithm for non-uniform illumination panoramic image. In: Proceedings of IEEE 2nd International Conference on Electronic Information and Communication Technology (ICEICT), 2019. 258–262

  46. Karmakar S, Ghosh K, Sarkar S, et al. Design of a low-pass filter by multi-scale even order Gaussian derivatives. Signal Process, 2006, 86: 3923–3933

    Article  MATH  Google Scholar 

  47. Ruoho-Airola T, Salmi T, Amnel T. Visualization of air quality time series by low-pass Gaussian filtering. In: Proceedings of the 20th International Conference for Environmental Protection, Graz, 2006. 291–294

  48. Qu F, Ren D, Liu X, et al. A face image illumination quality evaluation method based on Gaussian low-pass filter. In: Proceedings of the 2nd IEEE International Conference on Cloud Computing and Intelligence Systems, Hangzhou, 2012. 176–180

  49. Guo X, Li Y, Ma J, et al. Mutually guided image filtering. IEEE Trans Pattern Anal Mach Intell, 2020, 42: 694–707

    Article  Google Scholar 

  50. Nie X, Hu Y, Shen X. Physics-preserving fluid reconstruction from monocular video coupling with SFS and SPH. Vis Comput, 2020, 36: 1247–1257

    Article  Google Scholar 

  51. Wang C, Wang C, Qin H, et al. Video-based fluid reconstruction and its coupling with SPH simulation. Vis Comput, 2017, 33: 1211–1224

    Article  Google Scholar 

  52. Li Z, Murez Z, Kriegman D J, et al. Learning to see through turbulent water. In: Proceedings of IEEE Winter Conference on Applications of Computer Vision, Lake Tahoe, 2018. 512–520

  53. Stets J D, Li Z, Frisvad J R, et al. Single-shot analysis of refractive shape using convolutional neural networks. In: Proceedings of IEEE Winter Conference on Applications of Computer Vision, Waikoloa Village, 2019. 995–1003

Download references

Acknowledgements

This work was supported in part by National Key R&D Program of China (Grant No. 2019YFF0301305) and National Natural Science Foundation of China (Grant No. U19A2063). The authors sincerely thank the anonymous reviewers for their kind suggestions.

Author information

Authors and Affiliations

  1. State Key Laboratory of Virtual Reality Technology and Systems, Beihang University, Beijing, 100191, China

    Xiaoying Nie, Yong Hu, Xukun Shen & Zhiyuan Su

  2. School of New Media Art and Design, Beihang University, Beijing, 100191, China

    Yong Hu & Xukun Shen

Authors
  1. Xiaoying Nie
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yong Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xukun Shen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zhiyuan Su
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xukun Shen.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Nie, X., Hu, Y., Shen, X. et al. Reconstructing and editing fluids using the adaptive multilayer external force guiding model. Sci. China Inf. Sci. 65, 212102 (2022). https://doi.org/10.1007/s11432-020-3322-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11432-020-3322-x

Keywords

Search

Navigation