Advertisement

International Journal of Computer Vision

, Volume 95, Issue 2, pp 100–123 | Cite as

Gradient Flows for Optimizing Triangular Mesh-based Surfaces: Applications to 3D Reconstruction Problems Dealing with Visibility

  • Amaël Delaunoy
  • Emmanuel Prados
Article

Abstract

This article tackles the problem of using variational methods for evolving 3D deformable surfaces. We give an overview of gradient descent flows when the shape is represented by a triangular mesh-based surface, and we detail the gradients of two generic energy functionals which embody a number of energies used in mesh processing and computer vision. In particular, we show how to rigorously account for visibility in the surface optimization process. We present different applications including 3D reconstruction from multiple views for which the visibility is fundamental. The gradient correctly takes into account the visibility changes that occur when a surface moves; this forces the contours generated by the reconstructed surface to match with the apparent contours in the input images.

Keywords

Triangle mesh-based surface Gradient descent flow Surface evolution Variational methods Shape gradient Visibility 3D reconstruction Multi-view stereovision 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Bertalmio, M., Sapiro, G., Cheng, L.-T., & Osher, S. (2001). Variational problems and pdes on implicit surfaces. Variational and Level Set Methods in Computer Vision. Google Scholar
  2. Birkbeck, N., Cobzas, D., Sturm, P., & Jägersand, M. (2006). Variational shape and reflectance estimation under changing light and viewpoints. In Proceedings of European conference on computer vision (Vol. 1, pp. 536–549). Google Scholar
  3. Chang, J. Y., Lee, K. M., & Lee, S. U. (2007). Multiview normal field integration using level set methods. In IEEE conference in computer vision and pattern recognition. Washington: IEEE Computer Society. Google Scholar
  4. Charpiat, G., Maurel, P., Pons, J.-P., Keriven, R., & Faugeras, O. (2007). Generalized gradients: Priors on minimization flows. International Journal of Computer Vision, 73(3), 325–344. CrossRefGoogle Scholar
  5. Debreuve, É., Gastaud, M., Barlaud, M., & Aubert, G. (2007). Using the shape gradient for active contour segmentation: from the continuous to the discrete formulation. Journal of Mathematical Imaging and Vision. Google Scholar
  6. Delaunoy, A., Prados, E., Gargallo, P., Pons, J.-P., & Sturm, P. (2008). Minimizing the multi-view stereo reprojection error for triangular surface meshes. In British machine and vision conference, Leeds, UK, 2008. Google Scholar
  7. Desbrun, M., Meyerm, M., Schröder, P., & Barr, A. (1999). Implicit fairing of irregular meshes using diffusion and curvature flow. In SIGGRAPH ’99 (pp. 317–324). CrossRefGoogle Scholar
  8. Duan, Y., Yang, L., Qin, H., & Samaras, D. (2004). Shape reconstruction from 3d and 2d data using pde-based deformable surfaces. In European conference on computer vision, Prague, Czech Republic (pp. 238–251). Google Scholar
  9. Dziuk, G., & Elliott, C. M. (2007). Finite elements on evolving surfaces. IMA Journal of Numerical Analysis, 27(2), 262–292. MathSciNetzbMATHCrossRefGoogle Scholar
  10. Eckstein, I., Pons, J.-P., Tong, Y., Kuo, C.-C. J., & Desbrun, M. (2007). Generalized surface flows for mesh processing. In Eurographics symposium on geometry processing. Google Scholar
  11. Faugeras, O. D., & Keriven, R. (1998). Variational-principles, surface evolution, pdes, level set methods, and the stereo problem. IEEE Transactions on Image Processing, 7(3), 336–344. MathSciNetzbMATHCrossRefGoogle Scholar
  12. Gargallo, P. (2008). Contributions to the Bayesian approach to Multi-view Stereo. PhD thesis, Institut National Polytechique de Grenoble, France, February. Google Scholar
  13. Gargallo, P., Prados, E., & Sturm, P. (2007). Minimizing the reprojection error in surface reconstruction from images. In Proceedings of the international conference on computer vision, Rio de Janeiro, Brazil. Los Alamitos: IEEE Computer Society Press. Google Scholar
  14. Goldlucke, B., Ihrke, I., Linz, C., & Magnor, M. (2007). Weighted minimal hypersurface reconstruction. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29(7), 1194–1208. CrossRefGoogle Scholar
  15. Goldlücke, B., & Magnor, M. A. (2004). Weighted minimal hypersurfaces and their applications in computer vision. In T. Pajdla & J. Matas (Eds.), Lecture notes in computer science: Vol. 3022. European conference on computer vision (pp. 366–378). Berlin: Springer. Google Scholar
  16. Gupta, R., & Hartley, R. I. (1997). Linear pushbroom cameras. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(9), 963–975. CrossRefGoogle Scholar
  17. Hernandez, C., & Schmitt, F. (2004). Silhouette and stereo fusion for 3d object modeling. Computer Vision and Image Understanding, 96(3), 367–392. CrossRefGoogle Scholar
  18. Hernandez, C., Vogiatzis, G., & Cipolla, R. (2008). Multiview photometric stereo. IEEE Transactions on Pattern Analysis and Machine Intelligence, 30(3), 548–554. CrossRefGoogle Scholar
  19. Jin, H., Yezzi, A. J., & Soatto, S. (2002). Variational multiframe stereo in the presence of specular reflections. In 3DPVT (pp. 626–630). Google Scholar
  20. Jin, H., Soatto, S., & Yezzi, A. J. (2005). Multi-view stereo reconstruction of dense shape and complex appearance. International Journal of Computer Vision, 63(3), 175–189. CrossRefGoogle Scholar
  21. Jin, H., Cremers, D., Wang, D., Prados, E., Yezzi, A., & Soatto, S. (2008). 3-d reconstruction of shaded objects from multiple images under unknown illumination. International Journal of Computer Vision, 76(3). Google Scholar
  22. Kolev, K., & Cremers, D. (2008). Integration of multiview stereo and silhouettes via convex functionals on convex domains. In European conference on computer vision, Oct 2008. Google Scholar
  23. Kolev, K., & Cremers, D. (2009). Continuous ratio optimization via convex relaxation with applications 3D multiview reconstruction. In IEEE conference on computer vision and pattern recognition. Google Scholar
  24. Kolev, K., Klodt, M., Brox, T., & Cremers, D. (2009). Continuous global optimization in multiview 3D reconstruction. In International Journal of Computer Vision. Google Scholar
  25. Kolev, K., Pock, T., & Cremers, D. (2010) Anisotropic minimal surfaces integrating photoconsistency and normal information for multiview stereo. In European conference on computer vision 2010. Google Scholar
  26. Kolmogorov, & Zabih (2004). What energy functions can be minimized via graph cuts. IEEETPAMI: IEEE Transactions on Pattern Analysis and Machine Intelligence, 26. Google Scholar
  27. Labatut, P., Keriven, R., & Pons, J.-P. (2006). Fast level set multi-view stereo on graphics hardware. In 3DPVT ’06: Proceedings of the third international symposium on 3d data processing, visualization, and transmission (3DPVT’06), Washington, DC, USA (pp. 774–781). Washington: IEEE Computer Society. CrossRefGoogle Scholar
  28. Meyer, M., Desbrun, M., Schröder, P., & Barr, A. H. (2002). Discrete differential-geometry operators for triangulated 2-manifolds. Google Scholar
  29. Nehab, D., Rusinkiewicz, S., Davis, J., & Ramamoorthi, R. (2005). Efficiently combining positions and normals for precise 3D geometry. In SIGGRAPH (pp. 536–543). Google Scholar
  30. Osher, S., & Sethian, J. A. (1988). Fronts propagating with curvature-dependent speed: Algorithms based on Hamilton-Jacobi formulations. Journal of Computational Physics, 79, 12–49. MathSciNetzbMATHCrossRefGoogle Scholar
  31. Pons, J.-P., & Boissonnat, J.-D. (2007). A Lagrangian approach to dynamic interfaces through kinetic triangulation of the ambient space. Computer Graphics Forum, 26(2), 227–239. CrossRefGoogle Scholar
  32. Pons, J.-P., Keriven, R., & Faugeras, O. (2005). Modelling dynamic scenes by registering multi-view image sequences. In IEEE conference on computer vision and pattern recognition, San Diego, USA (pp. 822–827). Google Scholar
  33. Pons, J.-P., Keriven, R., & Faugeras, O. (2007). Multi-view stereo reconstruction and scene flow estimation with a global image-based matching score. The International Journal of Computer Vision, 72(2), 179–193. CrossRefGoogle Scholar
  34. Seitz, S. M., Curless, B., Diebel, J., Scharstein, D., & Szeliski, R. (2006). A comparison and evaluation of multi-view stereo reconstruction algorithms. In IEEE conference on computer vision and pattern recognition (pp. 519–528). Google Scholar
  35. Sinha, S. N., & Pollefeys, M. (2005). Multi-view reconstruction using photo-consistency and exact silhouette constraints: a maximum-flow formulation. In IEEE international conference on computer vision (pp. 349–356). Google Scholar
  36. Slabaugh, G., & Unal, G. (2005). Active polyhedron: surface evolution theory applied to deformable meshes. In IEEE conference on computer vision and pattern recognition (Vol. 2, pp. 84–91). Google Scholar
  37. Soatto, S., Yezzi, A. J., & Jin, H. (2003). Tales of shape and radiance in multi-view stereo. In IEEE international conference on computer vision (pp. 974–981). CrossRefGoogle Scholar
  38. Solem, J. E., & Heyden, A. (2006). Reconstructing open surfaces from image data. Int. J. Comput. Vision, 69(3), 267–275. CrossRefGoogle Scholar
  39. Solem, J. E., & Overgaard, N. Chr. (2005). A geometric formulation of gradient descent for variational problems with moving surfaces. In R. Kimmel, N. Sochen, & J. Weickert (Eds.), LNCS: Vol. 3459. Scale-Space 2005 (pp. 419–430). Berlin: Springer. Google Scholar
  40. Solem, J. E., Aanaes, H., & Heyden, A. (2004). A variational analysis of shape from specularities using sparse data. In 3DPVT ’04: Proceedings of the 3D data processing, visualization, and transmission, 2nd international symposium, Washington, DC, USA, 2004 (pp. 26–33). Washington: IEEE Computer Society. CrossRefGoogle Scholar
  41. Vlasic, D., Peers, P., Baran, I., Debevec, P., Popović, J., Rusinkiewicz, S., & Matusik, W. (2009). Dynamic shape capture using multi-view photometric stereo. ACM Transactions on Graphics, 28(5), 174. CrossRefGoogle Scholar
  42. Vogiatzis, G., Hernández Esteban, C., Torr, P. H. S., & Cipolla, R. (2007). Multiview stereo via volumetric graph-cuts and occlusion robust photo-consistency. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29(12), 2241–2246. CrossRefGoogle Scholar
  43. Vu, H., Keriven, R., Labatut, P., & Pons, J.-P. (2009). Towards high-resolution large-scale multi-view stereo. In IEEE conference on computer vision and pattern recognition, Jun 2009. Google Scholar
  44. Whitaker, R. T. (1998). A level-set approach to 3d reconstruction from range data. International Journal of Computer Vision, 29(3), 203–231. CrossRefGoogle Scholar
  45. Yezzi, A. J., & Soatto, S. (2001). Stereoscopic segmentation. In IEEE international conference on computer vision. Google Scholar
  46. Yezzi, A., & Soatto, S. (2003). Stereoscopic segmentation. International Journal of Computer Vision, 53(1), 31–43. MathSciNetCrossRefGoogle Scholar
  47. Yoon, K.-J., Prados, E., & Sturm, P. (2009, to appear). Joint estimation of shape and reflectance using multiple images with known illumination conditions. International Journal of Computer Vision. Google Scholar
  48. Yu, T., Xu, N., & Ahuja, N. (2007). Shape and view independent reflectance map from multiple views. International Journal of Computer Vision, 73(2), 123–138. CrossRefGoogle Scholar
  49. Zaharescu, A., Boyer, E., & Horaud, R. P. (2007). Transformesh: a topology-adaptive mesh-based approach to surface evolution. In Proceedings Asian conference on computer vision, Tokyo, Japan, November 2007. Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2010

Authors and Affiliations

  1. 1.INRIA Rhône-AlpesPerception team, LJKGrenobleFrance

Personalised recommendations