Skip to main content

A Level Set Theory for Neural Implicit Evolution Under Explicit Flows

  • Conference paper
  • First Online:
Computer Vision – ECCV 2022 (ECCV 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13662))

Included in the following conference series:

Abstract

Coordinate-based neural networks parameterizing implicit surfaces have emerged as efficient representations of geometry. They effectively act as parametric level sets with the zero-level set defining the surface of interest. We present a framework that allows applying deformation operations defined for triangle meshes onto such implicit surfaces. Several of these operations can be viewed as energy-minimization problems that induce an instantaneous flow field on the explicit surface. Our method uses the flow field to deform parametric implicit surfaces by extending the classical theory of level sets. We also derive a consolidated view for existing methods on differentiable surface extraction and rendering, by formalizing connections to the level-set theory. We show that these methods drift from the theory and that our approach exhibits improvements for applications like surface smoothing, mean-curvature flow, inverse rendering and user-defined editing on implicit geometry.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    This might be due to the unconstrained Lipschitz constants of MLPs [28].

References

  1. Aliev, K.A., Sevastopolsky, A., Kolos, M., Ulyanov, D., Lempitsky, V.: Neural point-based graphics (2020)

    Google Scholar 

  2. Alldieck, T., Xu, H., Sminchisescu, C.: imghum: implicit generative models of 3D human shape and articulated pose. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 5461–5470 (2021)

    Google Scholar 

  3. Ambrosio, L., Soner, H.M.: Level set approach to mean curvature flow in arbitrary codimension. J. Differ. Geom. 43(4), 693–737 (1996)

    Article  MathSciNet  MATH  Google Scholar 

  4. Azinović, D., Li, T.M., Kaplanyan, A., Nießner, M.: Inverse path tracing for joint material and lighting estimation. In: Proceedings of the Computer Vision and Pattern Recognition (CVPR). IEEE (2019)

    Google Scholar 

  5. Azinović, D., Martin-Brualla, R., Goldman, D.B., Nießner, M., Thies, J.: Neural RGB-D surface reconstruction. arXiv preprint arXiv:2104.04532 (2021)

  6. Bangaru, S., Li, T.M., Durand, F.: Unbiased warped-area sampling for differentiable rendering. ACM Trans. Graph. 39(6), 245:1-245:18 (2020)

    Article  Google Scholar 

  7. Batchelor, C.K., Batchelor, G.: An Introduction to Fluid Dynamics. Cambridge University Press, Cambridge (2000)

    Book  MATH  Google Scholar 

  8. Botsch, M., Kobbelt, L., Pauly, M., Alliez, P., Lévy, B.: Polygon Mesh Processing. CRC Press, New York (2010)

    Book  Google Scholar 

  9. Botsch, M., Sorkine, O.: On linear variational surface deformation methods. IEEE Trans. Visual. Comput. Graphics 14(1), 213–230 (2007)

    Article  Google Scholar 

  10. Botsch, M., Sumner, R., Pauly, M., Gross, M.: Deformation transfer for detail-preserving surface editing. In: Vision, Modeling & Visualization, pp. 357–364. Citeseer (2006)

    Google Scholar 

  11. Chabra, R., et al.: Deep local shapes: learning local SDF priors for detailed 3D reconstruction. arXiv preprint arXiv:2003.10983 (2020)

  12. Chan, E.R., Monteiro, M., Kellnhofer, P., Wu, J., Wetzstein, G.: pi-GAN: periodic implicit generative adversarial networks for 3d-aware image synthesis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5799–5809 (2021)

    Google Scholar 

  13. Chen, Z., Zhang, H.: Learning implicit fields for generative shape modeling. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2019

    Google Scholar 

  14. Chong, E.K., Zak, S.H.: An Introduction to Optimization. Wiley, Hoboken (2004)

    MATH  Google Scholar 

  15. Cole, F., Genova, K., Sud, A., Vlasic, D., Zhang, Z.: Differentiable surface rendering via non-differentiable sampling. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 6088–6097 (2021)

    Google Scholar 

  16. Davies, T., Nowrouzezahrai, D., Jacobson, A.: On the effectiveness of weight-encoded neural implicit 3D shapes. arXiv preprint arXiv:2009.09808 (2020)

  17. Desbrun, M., Meyer, M., Schröder, P., Barr, A.H.: Implicit fairing of irregular meshes using diffusion and curvature flow. In: Proceedings of the 26th Annual Conference on Computer Graphics and Interactive Techniques, pp. 317–324 (1999)

    Google Scholar 

  18. Goel, P., Cohen, L., Guesman, J., Thamizharasan, V., Tompkin, J., Ritchie, D.: Shape from tracing: towards reconstructing 3D object geometry and svbrdf material from images via differentiable path tracing. In: 2020 International Conference on 3D Vision (3DV), pp. 1186–1195. IEEE (2020)

    Google Scholar 

  19. Gropp, A., Yariv, L., Haim, N., Atzmon, M., Lipman, Y.: Implicit geometric regularization for learning shapes. In: Proceedings of Machine Learning and Systems 2020 (2020)

    Google Scholar 

  20. Gupta, K., Chandraker, M.: Neural mesh flow: 3D manifold mesh generation via diffeomorphic flows. In: Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M.F., Lin, H. (eds.) Advances in Neural Information Processing Systems, vol. 33, pp. 1747–1758. Curran Associates, Inc. (2020). https://proceedings.neurips.cc/paper/2020/file/1349b36b01e0e804a6c2909a6d0ec72a-Paper.pdf

  21. Hart, J.C.: Sphere tracing: a geometric method for the antialiased ray tracing of implicit surfaces. Visual Comput. 12(10), 527–545 (1996)

    Article  Google Scholar 

  22. Hasselgren, J., Hofmann, N., Munkberg, J.: Shape, light & material decomposition from images using monte carlo rendering and denoising. arXiv preprint arXiv:2206.03380 (2022)

  23. Jiang, Y., Ji, D., Han, Z., Zwicker, M.: SDFDIFF: differentiable rendering of signed distance fields for 3d shape optimization. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1251–1261 (2020)

    Google Scholar 

  24. Jiang, Y., Ji, D., Han, Z., Zwicker, M.: SDFDIFF: differentiable rendering of signed distance fields for 3D shape optimization. In: The IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) June 2020

    Google Scholar 

  25. Kellnhofer, P., Jebe, L.C., Jones, A., Spicer, R., Pulli, K., Wetzstein, G.: Neural lumigraph rendering. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4287–4297 (2021)

    Google Scholar 

  26. Laine, S., Hellsten, J., Karras, T., Seol, Y., Lehtinen, J., Aila, T.: Modular primitives for high-performance differentiable rendering. ACM Trans. Graphics 39(6), 1–14 (2020)

    Article  Google Scholar 

  27. Li, T.M., Aittala, M., Durand, F., Lehtinen, J.: Differentiable monte carlo ray tracing through edge sampling. ACM Trans. Graph. (Proc. SIGGRAPH Asia) 37(6), 222:1–222:11 (2018)

    Google Scholar 

  28. Liu, H.T.D., Williams, F., Jacobson, A., Fidler, S., Litany, O.: Learning smooth neural functions via lipschitz regularization. arXiv preprint arXiv:2202.08345 (2022)

  29. Liu, S., Zhang, Y., Peng, S., Shi, B., Pollefeys, M., Cui, Z.: Dist: Rendering deep implicit signed distance function with differentiable sphere tracing. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (June 2020)

    Google Scholar 

  30. Liu, S., Li, T., Chen, W., Li, H.: Soft rasterizer: a differentiable renderer for image-based 3D reasoning. In: The IEEE International Conference on Computer Vision (ICCV), October 2019

    Google Scholar 

  31. Lorensen, W.E., Cline, H.E.: Marching cubes: a high resolution 3D surface construction algorithm. ACM SIGGRAPH Comput. Graphics 21(4), 163–169 (1987)

    Article  Google Scholar 

  32. Luan, F., Zhao, S., Bala, K., Dong, Z.: Unified shape and SVBRDF recovery using differentiable monte Carlo rendering. ArXiv (2021)

    Google Scholar 

  33. Markstein, G.H.: Nonsteady flame propagation: AGARDograph. Elsevier (2014)

    Google Scholar 

  34. Martel, J.N.P., Lindell, D.B., Lin, C.Z., Chan, E.R., Monteiro, M., Wetzstein, G.: Acorn: adaptive coordinate networks for neural scene representation. ACM Trans. Graph. (SIGGRAPH) 40(4) (2021)

    Google Scholar 

  35. Mehta, I., Gharbi, M., Barnes, C., Shechtman, E., Ramamoorthi, R., Chandraker, M.: Modulated periodic activations for generalizable local functional representations. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 14214–14223, October 2021

    Google Scholar 

  36. Mescheder, L., Oechsle, M., Niemeyer, M., Nowozin, S., Geiger, A.: Occupancy networks: learning 3D reconstruction in function space. In: Proceedings IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2019)

    Google Scholar 

  37. Mu, J., Qiu, W., Kortylewski, A., Yuille, A., Vasconcelos, N., Wang, X.: A-SDF: learning disentangled signed distance functions for articulated shape representation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 13001–13011 (2021)

    Google Scholar 

  38. Munkberg, J., et al.: Extracting Triangular 3D Models, Materials, and Lighting From Images. arXiv:2111.12503 (2021)

  39. Museth, K., Breen, D.E., Whitaker, R.T., Barr, A.H.: Level set surface editing operators. In: Proceedings of the 29th Annual Conference on Computer Graphics and Interactive Techniques, pp. 330–338 (2002)

    Google Scholar 

  40. Nicolet, B., Jacobson, A., Jakob, W.: Large steps in inverse rendering of geometry. ACM Trans. Graphics (Proc. SIGGRAPH Asia) 40(6), December 2021. https://doi.org/10.1145/3478513.3480501, https://rgl.epfl.ch/publications/Nicolet2021Large

  41. Niemeyer, M., Mescheder, L., Oechsle, M., Geiger, A.: Differentiable volumetric rendering: learning implicit 3D representations without 3d supervision. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2020

    Google Scholar 

  42. Nimier-David, M., Vicini, D., Zeltner, T., Jakob, W.: Mitsuba 2: a retargetable forward and inverse renderer. Trans. Graphics (Proc. SIGGRAPH Asia) 38(6), December 2019. https://doi.org/10.1145/3355089.3356498

  43. Oechsle, M., Peng, S., Geiger, A.: UNISURF: unifying neural implicit surfaces and radiance fields for multi-view reconstruction. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 5589–5599 (2021)

    Google Scholar 

  44. Osher, S., Fedkiw, R.P.: Level set methods: an overview and some recent results. J. Comput. Phys. 169(2), 463–502 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  45. Osher, S., Sethian, J.A.: Fronts propagating with curvature-dependent speed: algorithms based on hamilton-jacobi formulations. Journal of Computational Physics 79, 12–49 (1988)

    Article  MathSciNet  MATH  Google Scholar 

  46. Park, J.J., Florence, P., Straub, J., Newcombe, R., Lovegrove, S.: DeepSDF: learning continuous signed distance functions for shape representation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2019

    Google Scholar 

  47. Quilez, I.: https://iquilezles.org/www/articles/distfunctions/distfunctions.htm

  48. Ravi, N., et al.: Accelerating 3d deep learning with pytorch3d. http://arxiv.org/abs/2007.08501 (2020)

  49. Remelli, E., et al.: MeshSDF: differentiable ISO-surface extraction. In: Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M.F., Lin, H. (eds.) Advances in Neural Information Processing Systems, vol. 33, pp. 22468–22478. Curran Associates, Inc. (2020). https://proceedings.neurips.cc/paper/2020/file/fe40fb944ee700392ed51bfe84dd4e3d-Paper.pdf

  50. Rückert, D., Franke, L., Stamminger, M.: ADOP: approximate differentiable one-pixel point rendering. arXiv preprint arXiv:2110.06635 (2021)

  51. Shen, T., Gao, J., Yin, K., Liu, M.Y., Fidler, S.: Deep marching tetrahedra: a hybrid representation for high-resolution 3d shape synthesis. In: Advances in Neural Information Processing Systems (NeurIPS) (2021)

    Google Scholar 

  52. Sitzmann, V., Chan, E.R., Tucker, R., Snavely, N., Wetzstein, G.: MetaSDF: meta-learning signed distance functions. In: Proceedings of the NeurIPS (2020)

    Google Scholar 

  53. Sitzmann, V., Martel, J.N., Bergman, A.W., Lindell, D.B., Wetzstein, G.: Implicit neural representations with periodic activation functions. In: arXiv (2020)

    Google Scholar 

  54. Sorkine, O., Alexa, M.: As-rigid-as-possible surface modeling. In: Symposium on Geometry Processing, vol. 4, pp. 109–116 (2007)

    Google Scholar 

  55. Sorkine, O., Cohen-Or, D., Lipman, Y., Alexa, M., Rössl, C., Seidel, H.P.: Laplacian surface editing. In: Proceedings of the 2004 Eurographics/ACM SIGGRAPH symposium on Geometry Processing, pp. 175–184 (2004)

    Google Scholar 

  56. Stanley, K.O.: Compositional pattern producing networks: a novel abstraction of development. Genetic Program. Evol. Mach. 8(2), 131–162 (2007)

    Article  Google Scholar 

  57. Sucar, E., Liu, S., Ortiz, J., Davison, A.J.: imap: implicit mapping and positioning in real-time. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 6229–6238 (2021)

    Google Scholar 

  58. Süli, E., Mayers, D.: An Introduction to Numerical Analysis. An Introduction to Numerical Analysis, Cambridge University Press (2003). https://books.google.com/books?id=hj9weaqJTbQC

  59. Takikawa, T., et al.: Neural geometric level of detail: real-time rendering with implicit 3D shapes (2021)

    Google Scholar 

  60. Tancik, M., et al.: Fourier features let networks learn high frequency functions in low dimensional domains. arXiv preprint arXiv:2006.10739 (2020)

  61. Taubin, G.: A signal processing approach to fair surface design. In: Proceedings of the 22nd Annual Conference on Computer Graphics and Interactive Techniques, pp. 351–358 (1995)

    Google Scholar 

  62. Terzopoulos, D., Platt, J., Barr, A., Fleischer, K.: Elastically deformable models. In: Proceedings of the 14th Annual Conference on Computer Graphics and Interactive Techniques, pp. 205–214 (1987)

    Google Scholar 

  63. Tewari, A., et al.: Advances in neural rendering. arXiv preprint arXiv:2111.05849 (2021)

  64. Whitaker, R.T.: A level-set approach to 3D reconstruction from range data. Int. J. Comput. Vision 29(3), 203–231 (1998)

    Article  Google Scholar 

  65. Xie, Y., et al.: Neural fields in visual computing and beyond. arXiv preprint arXiv:2111.11426 (2021)

  66. Xu, Y., Peng, S., Yang, C., Shen, Y., Zhou, B.: 3D-aware image synthesis via learning structural and textural representations (2021)

    Google Scholar 

  67. Yang, G., Belongie, S., Hariharan, B., Koltun, V.: Geometry processing with neural fields. In: Thirty-Fifth Conference on Neural Information Processing Systems (2021)

    Google Scholar 

  68. Yariv, L., Gu, J., Kasten, Y., Lipman, Y.: Volume rendering of neural implicit surfaces. In: Advances in Neural Information Processing Systems 34 (2021)

    Google Scholar 

  69. Yariv, L., et al.: Multiview neural surface reconstruction by disentangling geometry and appearance. In: Advances in Neural Information Processing Systems, vol. 33 (2020)

    Google Scholar 

  70. Zayer, R., Rössl, C., Karni, Z., Seidel, H.P.: Harmonic guidance for surface deformation. In: Computer Graphics Forum, vol. 24, pp. 601–609. Citeseer (2005)

    Google Scholar 

  71. Zhang, C., Miller, B., Yan, K., Gkioulekas, I., Zhao, S.: Path-space differentiable rendering. ACM Trans. Graph. 39(4), 143:1–143:19 (2020)

    Google Scholar 

  72. Zhu, Z., et al.: Nice-slam: Neural implicit scalable encoding for slam. arXiv preprint arXiv:2112.12130 (2021)

Download references

Acknowledgements

This work was supported in part by NSF CAREER 1751365, NSF IIS 2110409, ONR grant N000142012529, NSF Chase-CI grant 1730158, Adobe, Google, an Amazon Research Award, the Ronald L. Graham Chair and UC San Diego Center for Visual Computing. We thank Ceh Jan and 3dmixers users roman_hegglin and PhormaSolutions for the 3D models. We thank the anonymous reviewers for helpful comments and discussions.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Ishit Mehta .

Editor information

Editors and Affiliations

1 Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 8759 KB)

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Mehta, I., Chandraker, M., Ramamoorthi, R. (2022). A Level Set Theory for Neural Implicit Evolution Under Explicit Flows. In: Avidan, S., Brostow, G., Cissé, M., Farinella, G.M., Hassner, T. (eds) Computer Vision – ECCV 2022. ECCV 2022. Lecture Notes in Computer Science, vol 13662. Springer, Cham. https://doi.org/10.1007/978-3-031-20086-1_41

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-20086-1_41

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-20085-4

  • Online ISBN: 978-3-031-20086-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics