Skip to main content
SpringerLink
Log in

Multi-View Stereo Reconstruction of Dense Shape and Complex Appearance

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

We address the problem of estimating the three-dimensional shape and complex appearance of a scene from a calibrated set of views under fixed illumination. Our approach relies on a rank condition that must be satisfied when the scene exhibits “specular + diffuse” reflectance characteristics. This constraint is used to define a cost functional for the discrepancy between the measured images and those generated by the estimate of the scene, rather than attempting to match image-to-image directly. Minimizing such a functional yields the optimal estimate of the shape of the scene, represented by a dense surface, as well as its radiance, represented by four functions defined on such a surface. These can be used to generate novel views that capture the non-Lambertian appearance of the scene.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Bhat, D.N. and Nayar, S.K. 1995. Stereo in the presence of specular reflection. In Proc. Int. Conf. on Computer Vision, pp. 1086–1092.

  • Blake, A. 1985. Specular stereo. In Proc. Int. J. Conf. on Artificial Intell., pp. 973–976.

  • Brelstaff, G. and Blake, A. 1988. Detecting specular reflections using Lambertian constraints. In Proc. Int. Conf. on Computer Vision, pp. 297–302.

  • Chan, T.F. and Vese, L.A. 2001. Active contours without edges. IEEE Trans. on Image Processing, 10(2):266–277.

    Google Scholar 

  • Chen, W., Bouguet, J.-Y., Chu, M., and Grzeszczuk, R. 2002. Light field mapping: Efficient representation and hardware rendering of surface light fields. In Proc. ACM SIGGRAPH.

  • Epstein, C.L. and Gage, M. 1987. The curve shortening flow. Wave motion: Theory, modelling, and computation (Berkeley, Calif., 1986), Math. Sci. Res. Inst. Publ., 7:15–59.

    Google Scholar 

  • Faugeras, O. 1993. Three-Dimensional Computer Vision. MIT Press.

  • Faugeras, O. and Keriven, R. 1998. Variational principles, surface evolution, pde’s, level set methods and the stereo problem. IEEE Trans. on Image Processing, 7(3):336–344.

    Google Scholar 

  • Gortler, S., Grzeszczuk, R., Szeliski, R., and Cohen, M. 1996. The lumigraph. In Proc. ACM SIGGRAPH, pp. 43–54.

  • Hartley, R. and Zisserman, A. 2000. Multiple View Geometry in Computer Vision. Cambridge University Press.

  • Hertzmann, A. and Seitz, S.M. 2003. Shape and materials by example: A photometric stereo approach. In Proc. IEEE Conf on Comp Vision and Pattern Recogn.

  • Horn, B.K.P. 1986. Robot Vision. MIT Press.

  • Ikeuchi, K. 1981. Determining surface orientations of specular surfaces by using the photometric stereo method. IEEE Trans. on Pattern Analysis and Machine Intell., 3(6):661–669.

    Google Scholar 

  • Jin, H. 2003. Variational methods for shape reconstruction in computer vision. Ph.D. thesis, Washington University, Saint Louis, Missouri.

  • Jin, H., Soatto, S., and Yezzi, A.J. 2004. Multi-view stereo reconstruction of Dense shape and complex Appearance. UCLA CAM Report 04-53, Department of Mathematics, University of California at Los Angeles.

  • Jin, H., Yezzi, A.J., and Soatto, S. 2002. Variational multiframe stereo in the presence of specular reflections. In Proc. of the First Intl. Symp. on 3D Data Processing Visual. and Trans.

  • Jin, H., Yezzi, A.J., and Soatto, S. 2004. Region-based Segmentation on Evolving Surfaces with Application to 3D Reconstruction of Shape and Piecewise Constant Radiance. In Proc. of Euro. Conf. on Computer Vision.

  • Jin, H., Yezzi, A.J., Tsai, Y.-H., Cheng, L.-T., and Soatto, S. 2003. Estimation of 3D surface shape and smooth radiance from 2D images: A level set approach. J. of Scientific Computing, 19(1–3): 267–292.

    Google Scholar 

  • Keriven, R. 1997. Equations aux Dérivées Partielles, Evolutions de Courbes et de Surfaces et Espaces d’Echelle: Applications à la Vision par Ordinateur. Ph.D. thesis. ENPC, France.

  • Kim, J., Kolmogorov, V., and Zabih, R. 2003. Visual correspondence using energy minimization and mutual information. In Proc. of Intl. Conf. on Computer Vision.

  • Kutulakos, K.N. and Seitz, S.M. 2000. A theory of shape by space carving. Int. J. of Computer Vision, 38(3):199–218.

    Google Scholar 

  • Ma, Y., Soatto, S., Kosecka, J., and Sastry, S. 2003. An Invitation to 3-D Vision. Springer-Verlag.

  • Magda, S., Zickler, T., Kriegman, D.J., and Belhumeur, P.N. 2001. Beyond Lambert: Reconstructing Surfaces with Arbitrary BRDFs. In Proc. Intl. Conf. on Computer Vision.

  • Nayar, S.K., Fang, X.S., and Boult, T. 1993. Removal of specularities using color and polarization. In Proc. IEEE Conf. on Comp. Vision and Pattern Recogn, pp. 585–590.

  • Nishino, K., Sato, Y., and Ikeuchi, K. 1999. Eigen-texture method: Appearance compression based on 3D model In Proc. IEEE Conf on Comp Vision and Pattern Recogn.

  • Okutomi, M. and Kanade, T. 1993. A multiple baseline stereo. IEEE Trans. on Pattern Analysis and Machine Intell., 15(4):353–363.

    Google Scholar 

  • Osher, S.J. and Sethian, J.A. 1988. Fronts propagating with curvature-dependent speed: algorithms based on hamilton-jacobi equations. J. of Comp. Physics, 79:12–49.

    Google Scholar 

  • Soatto, S., Yezzi, A.J., and Jin, H. 2003. Tales of shape and radiance in multi-view stereo. In Proc. Intl. Conf. on Computer Vision, pp. 171–178.

  • Tsin, Y., Kang, S.B., and Szeliski, R. 2003. Stereo Matching with Reflections and Translucency. In Proc. IEEE Conf. on Comp. Vis. and Patt. Recog.

  • Ward, G. 1992. Measuring and modeling anisotropic reflection. In Proc. ACM SIGGRAPH, pp. 265–272.

  • Yezzi, A.J., and Soatto, S. 2003. Stereoscopic segmentation. Intl. J. of Computer Vision, 53(1):31-43.

    Google Scholar 

  • Yu, Y., Debevec, P., Malik, J., and Hawkins, T. 1999. Inverse global illumination: Recovering reflectance models of real scenes from photographs. In Proc. ACM SIGGRAPH.

  • Zickler, T., Belhumeur, P.N., and Kriegman, D.J. 2002. Helmholtz stereopsis: Exploiting reciprocity for surface reconstruction. Intl. J. of Computer Vision, 49(2/3):215–227.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Office of Technology, Adobe Systems Incorporated, 345 Park Avenue, San Jose, CA, 95110

    Hailin Jin

  2. Computer Science Department, University of California, Los Angeles, CA, 90095

    Stefano Soatto

  3. School of Electrical and Computer Engineering, Georgia Institute of Technology, Atlanta, GA, 30332

    Anthony J. Yezzi

Authors
  1. Hailin Jin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Stefano Soatto
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Anthony J. Yezzi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hailin Jin.

Additional information

This research was performed while Hailin Jin was with Computer Science Department, University of California at Los Angeles.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Jin, H., Soatto, S. & Yezzi, A.J. Multi-View Stereo Reconstruction of Dense Shape and Complex Appearance. Int J Comput Vision 63, 175–189 (2005). https://doi.org/10.1007/s11263-005-6876-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-005-6876-7

Keywords

Search

Navigation