Skip to main content
SpringerLink
Log in

Gradient Flows for Optimizing Triangular Mesh-based Surfaces: Applications to 3D Reconstruction Problems Dealing with Visibility

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

This article tackles the problem of using variational methods for evolving 3D deformable surfaces. We give an overview of gradient descent flows when the shape is represented by a triangular mesh-based surface, and we detail the gradients of two generic energy functionals which embody a number of energies used in mesh processing and computer vision. In particular, we show how to rigorously account for visibility in the surface optimization process. We present different applications including 3D reconstruction from multiple views for which the visibility is fundamental. The gradient correctly takes into account the visibility changes that occur when a surface moves; this forces the contours generated by the reconstructed surface to match with the apparent contours in the input images.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Bertalmio, M., Sapiro, G., Cheng, L.-T., & Osher, S. (2001). Variational problems and pdes on implicit surfaces. Variational and Level Set Methods in Computer Vision.

  • Birkbeck, N., Cobzas, D., Sturm, P., & Jägersand, M. (2006). Variational shape and reflectance estimation under changing light and viewpoints. In Proceedings of European conference on computer vision (Vol. 1, pp. 536–549).

    Google Scholar 

  • Chang, J. Y., Lee, K. M., & Lee, S. U. (2007). Multiview normal field integration using level set methods. In IEEE conference in computer vision and pattern recognition. Washington: IEEE Computer Society.

    Google Scholar 

  • Charpiat, G., Maurel, P., Pons, J.-P., Keriven, R., & Faugeras, O. (2007). Generalized gradients: Priors on minimization flows. International Journal of Computer Vision, 73(3), 325–344.

    Article  Google Scholar 

  • Debreuve, É., Gastaud, M., Barlaud, M., & Aubert, G. (2007). Using the shape gradient for active contour segmentation: from the continuous to the discrete formulation. Journal of Mathematical Imaging and Vision.

  • Delaunoy, A., Prados, E., Gargallo, P., Pons, J.-P., & Sturm, P. (2008). Minimizing the multi-view stereo reprojection error for triangular surface meshes. In British machine and vision conference, Leeds, UK, 2008.

    Google Scholar 

  • Desbrun, M., Meyerm, M., Schröder, P., & Barr, A. (1999). Implicit fairing of irregular meshes using diffusion and curvature flow. In SIGGRAPH ’99 (pp. 317–324).

    Chapter  Google Scholar 

  • Duan, Y., Yang, L., Qin, H., & Samaras, D. (2004). Shape reconstruction from 3d and 2d data using pde-based deformable surfaces. In European conference on computer vision, Prague, Czech Republic (pp. 238–251).

    Google Scholar 

  • Dziuk, G., & Elliott, C. M. (2007). Finite elements on evolving surfaces. IMA Journal of Numerical Analysis, 27(2), 262–292.

    Article  MathSciNet  MATH  Google Scholar 

  • Eckstein, I., Pons, J.-P., Tong, Y., Kuo, C.-C. J., & Desbrun, M. (2007). Generalized surface flows for mesh processing. In Eurographics symposium on geometry processing.

    Google Scholar 

  • Faugeras, O. D., & Keriven, R. (1998). Variational-principles, surface evolution, pdes, level set methods, and the stereo problem. IEEE Transactions on Image Processing, 7(3), 336–344.

    Article  MathSciNet  MATH  Google Scholar 

  • Gargallo, P. (2008). Contributions to the Bayesian approach to Multi-view Stereo. PhD thesis, Institut National Polytechique de Grenoble, France, February.

  • Gargallo, P., Prados, E., & Sturm, P. (2007). Minimizing the reprojection error in surface reconstruction from images. In Proceedings of the international conference on computer vision, Rio de Janeiro, Brazil. Los Alamitos: IEEE Computer Society Press.

    Google Scholar 

  • Goldlucke, B., Ihrke, I., Linz, C., & Magnor, M. (2007). Weighted minimal hypersurface reconstruction. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29(7), 1194–1208.

    Article  Google Scholar 

  • Goldlücke, B., & Magnor, M. A. (2004). Weighted minimal hypersurfaces and their applications in computer vision. In T. Pajdla & J. Matas (Eds.), Lecture notes in computer science: Vol. 3022. European conference on computer vision (pp. 366–378). Berlin: Springer.

    Google Scholar 

  • Gupta, R., & Hartley, R. I. (1997). Linear pushbroom cameras. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(9), 963–975.

    Article  Google Scholar 

  • Hernandez, C., & Schmitt, F. (2004). Silhouette and stereo fusion for 3d object modeling. Computer Vision and Image Understanding, 96(3), 367–392.

    Article  Google Scholar 

  • Hernandez, C., Vogiatzis, G., & Cipolla, R. (2008). Multiview photometric stereo. IEEE Transactions on Pattern Analysis and Machine Intelligence, 30(3), 548–554.

    Article  Google Scholar 

  • Jin, H., Yezzi, A. J., & Soatto, S. (2002). Variational multiframe stereo in the presence of specular reflections. In 3DPVT (pp. 626–630).

    Google Scholar 

  • Jin, H., Soatto, S., & Yezzi, A. J. (2005). Multi-view stereo reconstruction of dense shape and complex appearance. International Journal of Computer Vision, 63(3), 175–189.

    Article  Google Scholar 

  • Jin, H., Cremers, D., Wang, D., Prados, E., Yezzi, A., & Soatto, S. (2008). 3-d reconstruction of shaded objects from multiple images under unknown illumination. International Journal of Computer Vision, 76(3).

  • Kolev, K., & Cremers, D. (2008). Integration of multiview stereo and silhouettes via convex functionals on convex domains. In European conference on computer vision, Oct 2008.

    Google Scholar 

  • Kolev, K., & Cremers, D. (2009). Continuous ratio optimization via convex relaxation with applications 3D multiview reconstruction. In IEEE conference on computer vision and pattern recognition.

    Google Scholar 

  • Kolev, K., Klodt, M., Brox, T., & Cremers, D. (2009). Continuous global optimization in multiview 3D reconstruction. In International Journal of Computer Vision.

  • Kolev, K., Pock, T., & Cremers, D. (2010) Anisotropic minimal surfaces integrating photoconsistency and normal information for multiview stereo. In European conference on computer vision 2010.

    Google Scholar 

  • Kolmogorov, & Zabih (2004). What energy functions can be minimized via graph cuts. IEEETPAMI: IEEE Transactions on Pattern Analysis and Machine Intelligence, 26.

  • Labatut, P., Keriven, R., & Pons, J.-P. (2006). Fast level set multi-view stereo on graphics hardware. In 3DPVT ’06: Proceedings of the third international symposium on 3d data processing, visualization, and transmission (3DPVT’06), Washington, DC, USA (pp. 774–781). Washington: IEEE Computer Society.

    Chapter  Google Scholar 

  • Meyer, M., Desbrun, M., Schröder, P., & Barr, A. H. (2002). Discrete differential-geometry operators for triangulated 2-manifolds.

  • Nehab, D., Rusinkiewicz, S., Davis, J., & Ramamoorthi, R. (2005). Efficiently combining positions and normals for precise 3D geometry. In SIGGRAPH (pp. 536–543).

    Google Scholar 

  • Osher, S., & Sethian, J. A. (1988). Fronts propagating with curvature-dependent speed: Algorithms based on Hamilton-Jacobi formulations. Journal of Computational Physics, 79, 12–49.

    Article  MathSciNet  MATH  Google Scholar 

  • Pons, J.-P., & Boissonnat, J.-D. (2007). A Lagrangian approach to dynamic interfaces through kinetic triangulation of the ambient space. Computer Graphics Forum, 26(2), 227–239.

    Article  Google Scholar 

  • Pons, J.-P., Keriven, R., & Faugeras, O. (2005). Modelling dynamic scenes by registering multi-view image sequences. In IEEE conference on computer vision and pattern recognition, San Diego, USA (pp. 822–827).

    Google Scholar 

  • Pons, J.-P., Keriven, R., & Faugeras, O. (2007). Multi-view stereo reconstruction and scene flow estimation with a global image-based matching score. The International Journal of Computer Vision, 72(2), 179–193.

    Article  Google Scholar 

  • Seitz, S. M., Curless, B., Diebel, J., Scharstein, D., & Szeliski, R. (2006). A comparison and evaluation of multi-view stereo reconstruction algorithms. In IEEE conference on computer vision and pattern recognition (pp. 519–528).

    Google Scholar 

  • Sinha, S. N., & Pollefeys, M. (2005). Multi-view reconstruction using photo-consistency and exact silhouette constraints: a maximum-flow formulation. In IEEE international conference on computer vision (pp. 349–356).

    Google Scholar 

  • Slabaugh, G., & Unal, G. (2005). Active polyhedron: surface evolution theory applied to deformable meshes. In IEEE conference on computer vision and pattern recognition (Vol. 2, pp. 84–91).

    Google Scholar 

  • Soatto, S., Yezzi, A. J., & Jin, H. (2003). Tales of shape and radiance in multi-view stereo. In IEEE international conference on computer vision (pp. 974–981).

    Chapter  Google Scholar 

  • Solem, J. E., & Heyden, A. (2006). Reconstructing open surfaces from image data. Int. J. Comput. Vision, 69(3), 267–275.

    Article  Google Scholar 

  • Solem, J. E., & Overgaard, N. Chr. (2005). A geometric formulation of gradient descent for variational problems with moving surfaces. In R. Kimmel, N. Sochen, & J. Weickert (Eds.), LNCS: Vol. 3459. Scale-Space 2005 (pp. 419–430). Berlin: Springer.

    Google Scholar 

  • Solem, J. E., Aanaes, H., & Heyden, A. (2004). A variational analysis of shape from specularities using sparse data. In 3DPVT ’04: Proceedings of the 3D data processing, visualization, and transmission, 2nd international symposium, Washington, DC, USA, 2004 (pp. 26–33). Washington: IEEE Computer Society.

    Chapter  Google Scholar 

  • Vlasic, D., Peers, P., Baran, I., Debevec, P., Popović, J., Rusinkiewicz, S., & Matusik, W. (2009). Dynamic shape capture using multi-view photometric stereo. ACM Transactions on Graphics, 28(5), 174.

    Article  Google Scholar 

  • Vogiatzis, G., Hernández Esteban, C., Torr, P. H. S., & Cipolla, R. (2007). Multiview stereo via volumetric graph-cuts and occlusion robust photo-consistency. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29(12), 2241–2246.

    Article  Google Scholar 

  • Vu, H., Keriven, R., Labatut, P., & Pons, J.-P. (2009). Towards high-resolution large-scale multi-view stereo. In IEEE conference on computer vision and pattern recognition, Jun 2009.

    Google Scholar 

  • Whitaker, R. T. (1998). A level-set approach to 3d reconstruction from range data. International Journal of Computer Vision, 29(3), 203–231.

    Article  Google Scholar 

  • Yezzi, A. J., & Soatto, S. (2001). Stereoscopic segmentation. In IEEE international conference on computer vision.

    Google Scholar 

  • Yezzi, A., & Soatto, S. (2003). Stereoscopic segmentation. International Journal of Computer Vision, 53(1), 31–43.

    Article  MathSciNet  Google Scholar 

  • Yoon, K.-J., Prados, E., & Sturm, P. (2009, to appear). Joint estimation of shape and reflectance using multiple images with known illumination conditions. International Journal of Computer Vision.

  • Yu, T., Xu, N., & Ahuja, N. (2007). Shape and view independent reflectance map from multiple views. International Journal of Computer Vision, 73(2), 123–138.

    Article  Google Scholar 

  • Zaharescu, A., Boyer, E., & Horaud, R. P. (2007). Transformesh: a topology-adaptive mesh-based approach to surface evolution. In Proceedings Asian conference on computer vision, Tokyo, Japan, November 2007.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. INRIA Rhône-Alpes, Perception team, LJK, Grenoble, France

    Amaël Delaunoy & Emmanuel Prados

Authors
  1. Amaël Delaunoy
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Emmanuel Prados
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Amaël Delaunoy.

Additional information

Research was supported by the Agence Nationale pour la Recherche within the Flamenco project (Grant ANR-06-MDCA-007).

Rights and permissions

Reprints and permissions

About this article

Cite this article

Delaunoy, A., Prados, E. Gradient Flows for Optimizing Triangular Mesh-based Surfaces: Applications to 3D Reconstruction Problems Dealing with Visibility. Int J Comput Vis 95, 100–123 (2011). https://doi.org/10.1007/s11263-010-0408-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-010-0408-9

Keywords

Search

Navigation