Variational Shape and Reflectance Estimation Under Changing Light and Viewpoints

  • Neil Birkbeck
  • Dana Cobzas
  • Peter Sturm
  • Martin Jagersand
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3951)


Fitting parameterized 3D shape and general reflectance models to 2D image data is challenging due to the high dimensionality of the problem. The proposed method combines the capabilities of classical and photometric stereo, allowing for accurate reconstruction of both textured and non-textured surfaces. In particular, we present a variational method implemented as a PDE-driven surface evolution interleaved with reflectance estimation. The surface is represented on an adaptive mesh allowing topological change. To provide the input data, we have designed a capture setup that simultaneously acquires both viewpoint and light variation while minimizing self-shadowing. Our capture method is feasible for real-world application as it requires a moderate amount of input data and processing time. In experiments, models of people and everyday objects were captured from a few dozen images taken with a consumer digital camera. The capture process recovers a photo-consistent model of spatially varying Lambertian and specular reflectance and a highly accurate geometry.


Light Variation Photometric Stereo Specular Surface Chess Game Deformable Mesh 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Scharstein, D., Szeliski, R.: A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. Int. J. Comput. Vision 47, 7–42 (2002)CrossRefMATHGoogle Scholar
  2. 2.
    Faugeras, O., Keriven, R.: Variational principles, surface evolution, pde’s, level set methods and the stereo problem. IEEE Trans. Image Processing 7, 336–344 (1998)MathSciNetCrossRefMATHGoogle Scholar
  3. 3.
    Robert, L., Deriche, R.: Dense depth map reconstruction: A minimization and regularization approach which preserves discontinuities. In: ECCV 1996, pp. 439–451 (1996)Google Scholar
  4. 4.
    Yang, R., Pollefeys, M., Welch, G.: Dealing with textureless regions and specular highlights - a pregressive space carving scheme using a novel photo-consistency measure. In: ICCV (2003)Google Scholar
  5. 5.
    Esteban, C.H., Schmitt, F.: Silhouette and stereo fusion for 3d object modeling. Computer Vision and Image Understanding 96, 367–392 (2004)CrossRefGoogle Scholar
  6. 6.
    Fua, P., Leclerc, Y.: Object-centered surface reconstruction: combining multi-image stereo shading. In: Image Understanding Workshop, pp. 1097–1120 (1993)Google Scholar
  7. 7.
    Jin, H., Yezzi, A., Soatto, S.: Stereoscopic shading: Integrating shape cues in a variational framework. In: CVPR, pp. 169–176 (2000)Google Scholar
  8. 8.
    Zhang, L., Curless, B., Hertzmann, A., Seitz, S.M.: Shape and motion under varying illumination: Unifying structure from motion, photometric stereo, and multi-view stereo. In: ICCV (2003)Google Scholar
  9. 9.
    Weber, M., Blake, A., Cipolla, R.: Towards a complete dense geometric and photometric reconstruction under varying pose and illumination. In: BMVC (2002)Google Scholar
  10. 10.
    Yu, T.-L., Xu, N., Ahuja, N.: Shape and view independent reflectance map from multiple views. In: Pajdla, T., Matas, J(G.) (eds.) ECCV 2004. LNCS, vol. 3024, pp. 602–615. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  11. 11.
    Jin, H., Soatto, S., Yezzi, A.: Multi-view stereo reconstruction of dense shape and complex appearance. IJCV 63, 175–189 (2005)CrossRefGoogle Scholar
  12. 12.
    Yu, T., Xu, N., Ahuja, N.: Recovering shape and reflectance model of non-lambertian objects from multiple views. In: CVPR (2004)Google Scholar
  13. 13.
    Debevec, P.E., Malik, J.: Recovering high dynamic range radiance maps from photographs. In: Siggraph (1997)Google Scholar
  14. 14.
    Sethian, J.: Level Set Methods. Cambridge University Press, Cambridge (1996)MATHGoogle Scholar
  15. 15.
    Duan, Y., Yang, L., Qin, H., Samaras, D.: Shape reconstruction from 3D and 2D data using PDE-based deformable surfaces. In: Pajdla, T., Matas, J(G.) (eds.) ECCV 2004. LNCS, vol. 3023, pp. 238–251. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  16. 16.
    Caselles, V., Kimmel, R., Sapiro, G., Sbert, C.: Minimal surfaces based object segmentation. PAMI 19, 394–398 (1997)CrossRefMATHGoogle Scholar
  17. 17.
    Zhang, L., Seitz, S.: Image-based multiresolution modeling by surface deformation. Technical Report CMU-RI-TR-00-07, Carnegie Mellon University (2000)Google Scholar
  18. 18.
    Zach, C., Klaus, A., Hadwiger, M., Karner, K.: Accurate dense stereo reconstruction using graphics hardware. In: Eurographics 2003, pp. 227–234 (2003)Google Scholar
  19. 19.
    Ikeuchi, K., Sato, K.: Determining reflectance properties of an object using range and brightness images. PAMI 13, 1139–1153 (1991)CrossRefGoogle Scholar
  20. 20.
    Lafortune, E.P., Foo, S.C., Torrance, K.E., Greenberg, D.P.: Non-linear approximation of reflectance functions. In: SIGGRAPH (1997)Google Scholar
  21. 21.
    Marschner, S.R.: Inverse rendering for computer graphics. PhD thesis, Cornell University (1998)Google Scholar
  22. 22.
    Baumberg, A., Lyons, A., Taylor, R.: 3D S.O.M. - a commercial software solution to 3d scanning. In: Vision, Video, and Graphics (VVG 2003), pp. 41–48 (2003)Google Scholar
  23. 23.
    Lensch, H.P.A., Goesele, M., Kautz, J., Heidrich, W., Seidel, H.P.: Image-based reconstruction of spatially varying materials. In: Eurographics Workshop on Rendering Techniques, pp. 103–114 (2001)Google Scholar
  24. 24.
    Lévy, B., Petitjean, S., Ray, N., Maillot, J.: Least squares conformal maps for automatic texture atlas generation. In: SIGGRAPH 2002, pp. 62–371 (2002)Google Scholar
  25. 25.
    Surazhsky, T., Magid, E., Soldea, O., Elber, G., Rivlin, E.: A comparison of gaussian and mean curvatures triangular meshes. In: ICRA 2003, pp. 1021–1026 (2003)Google Scholar
  26. 26.
    Lachaud, J.O., Montanvert, A.: Deformable meshes with automated topology changes for coarse-to-fine 3D surface extraction. Medical Image Analysis 3, 187–207 (1999)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Neil Birkbeck
    • 1
  • Dana Cobzas
    • 1
  • Peter Sturm
    • 2
  • Martin Jagersand
    • 1
  1. 1.Computer ScienceUniversity of AlbertaCanada
  2. 2.INRIA Rhone-AlpesFrance

Personalised recommendations