International Journal of Computer Vision

, Volume 76, Issue 3, pp 245–256 | Cite as

3-D Reconstruction of Shaded Objects from Multiple Images Under Unknown Illumination

  • Hailin Jin
  • Daniel Cremers
  • Dejun Wang
  • Emmanuel Prados
  • Anthony Yezzi
  • Stefano Soatto
Article

Abstract

We propose a variational algorithm to jointly estimate the shape, albedo, and light configuration of a Lambertian scene from a collection of images taken from different vantage points. Our work can be thought of as extending classical multi-view stereo to cases where point correspondence cannot be established, or extending classical shape from shading to the case of multiple views with unknown light sources. We show that a first naive formalization of this problem yields algorithms that are numerically unstable, no matter how close the initialization is to the true geometry. We then propose a computational scheme to overcome this problem, resulting in provably stable algorithms that converge to (local) minima of the cost functional. We develop a new model that explicitly enforces positivity in the light sources with the assumption that the object is Lambertian and its albedo is piecewise constant and show that the new model significantly improves the accuracy and robustness relative to existing approaches.

Keywords

Stereoscopic segmentation Shape from shading Multi-view stereo Variational 3D reconstruction Level set methods Lighting and appearance reconstruction 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Abraham, R., Marsden, J. E., & Ratiu, T. (1993). Applied Mathematical Sciences: Vol. 75. Manifolds, tensor analysis, and applications (2nd ed.). New York: Springer. Google Scholar
  2. Belhumeur, P., Kriegman, D., & Yuille, A. L. (1999). The generalized bas relief ambiguity. International Journal of Computer Vision, 35, 33–44. CrossRefGoogle Scholar
  3. Brox, T., & Weickert, J. (2006). Level set segmentation with multiple regions. IEEE Transactions on Image Processing, 15(10). Google Scholar
  4. Chefd’hotel, C., Tschumperlé, D., Deriche, R., & Faugeras, O. (2004). Regularizing flows for constrained matrix-valued images. Journal of Mathematical Imaging and Vision, 20(1–2), 147–162. CrossRefMathSciNetGoogle Scholar
  5. Chen, H. F., Belhumeur, P. N., & Jacobs, D. W. (2000). In search of illumination invariants. In Proceedings of the IEEE conference on computer vision and pattern recognition Google Scholar
  6. Durou, J.-D., Falcone, M., & Sagona, M. (2004). A survey of numerical methods for shape from shading. Research report 2004-2-R, IRIT, January 2004. Google Scholar
  7. Faugeras, O., & Keriven, R. (1998). Variational principles, surface evolution, pdes, level set methods, and the stereo problem. IEEE Transactions on Image Processing, 7(3), 336–344. MATHCrossRefMathSciNetGoogle Scholar
  8. Horn, B. K. P. (1986). Robot vision. Cambridge: MIT. Google Scholar
  9. Horn, B., & Brooks, M. (Eds.). (1989). Shape from shading. Cambridge: MIT. Google Scholar
  10. Jin, H. (2003). Variational methods for shape reconstruction in computer vision. PhD thesis, Electrical Engineering Department, Washington University, August 2003. Google Scholar
  11. Jin, H., Soatto, S., & Yezzi, A. J. (2003a). Multi-view stereo beyond Lambert. In Proceedings of the IEEE conference on computer vision and pattern recognition (Vol. I, pp. 171–178), June 2003. Google Scholar
  12. Jin, H., Yezzi, A. J., Tsai, Y.-H., Cheng, L.-T., & Soatto, S. (2003b). Estimation of 3D surface shape and smooth radiance from 2D images: a level set approach. Journal of Scientific Computing, 19(1–3), 267–292. MATHCrossRefMathSciNetGoogle Scholar
  13. Jin, H., Cremers, D., Yezzi, A., & Soatto, S. (2004a). Shedding light on stereoscopic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 36–42). Google Scholar
  14. Jin, H., Yezzi, A. J., & Soatto, S. (2004b). Region-based segmentation on evolving surfaces with application to 3d shape and radiance estimation. In Proceedings of the European conference on computer vision (pp. 114–125), May 2004. Google Scholar
  15. Jin, H., Yezzi, A., & Soatto, S. (2007, in press). Mumford-shah on the move: region-based segmentation on deforming manifolds with application to 3-D reconstruction of shape and appearance from multi-view images. Journal of Mathematical Imaging and Vision. Google Scholar
  16. Keriven, R., Pons, J.-P., & Faugeras, O. (2006). Multi-view stereo reconstruction and scene flow estimation with a global image-based matching score. To appear in the international journal of computer vision. International Journal of Computer Vision Google Scholar
  17. Klette, R., Kozera, R., & Schlüns, K. (1998). Shape from shading and photometric stereo methods. Technical Report CITR-TR-20, University of Auckland, New Zealand. Google Scholar
  18. Koenderink, J., & van Doorn, A. (1980). Photometric invariants related to solid shape. Optica Acta, 27(7), 981–996. MathSciNetGoogle Scholar
  19. Kolev, K., Brox, T., & Cremers, D. (2006). Robust variational segmentation of 3D objects from multiple views. In K. Franke et al. (Eds.), Lecture notes in computer science: Vol. 4174. Pattern recognition (proceedings DAGM) (pp. 688–697). Berlin: Springer. CrossRefGoogle Scholar
  20. Kuhn, H. W., & Tucker, A. W. (1951). Nonlinear programming. In J. Neyman (Ed.), Proceedings of the second Berkeley symposium on mathematical statistics and probability (pp. 481–492) Berkeley: University of California Press. Google Scholar
  21. Kutulakos, K. N., & Seitz, S. M. (2000). A theory of shape by space carving. International Journal of Computer Vision, 38(3), 199–218. MATHCrossRefGoogle Scholar
  22. Langer, M. S., & Zucker, S. W. (1994). Shape from shading on a cloudy day. Journal of Optical Society of America, 11, 467–478. CrossRefGoogle Scholar
  23. Ma, Y., Soatto, S., Kosecka, J., & Sastry, S. (2003). An invitation to 3D vision, from images to models. Berlin: Springer. Google Scholar
  24. Nayar, S., Ikeuchi, K., & Kanade, T. (1991). Surface reflection: physical and geometrical perspectives. IEEE Transactions on Pattern Analysis and Machine Intelligence, 13(7), 611–634. CrossRefGoogle Scholar
  25. Oliensis, J., & Dupuis, P. (1993). A global algorithm for shape from shading. In Proceedings of the international conference on computer vision (pp. 692–710). Google Scholar
  26. Osher, S., & Sethian, J. (1988). Fronts propagating with curvature-dependent speed: algorithms based on Hamilton–Jacobi equations. Journal of Computational Physics, 79, 12–49. MATHCrossRefMathSciNetGoogle Scholar
  27. Prados, E. (2004). Application of the theory of the viscosity solutions to the shape from shading problem. PhD thesis, Univ. of Nice-Sophia Antipolis. Google Scholar
  28. Robles-Kelly, A., & Hancock, E. R. (2004). Estimating the surface radiance function from single images. In 3DPVT (pp. 494–501). Google Scholar
  29. Robles-Kelly, A., & Hancock, E. R. (2005). Surface radiance correction for shape-from-shading. Pattern Recognition, 38(10), 1574–1595. CrossRefGoogle Scholar
  30. Samaras, D. (2003). Illumination constraints in deformable models for shape and light direction estimation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(2), 247–264. CrossRefMathSciNetGoogle Scholar
  31. Samaras, D., Metaxas, D., Fua, P. V., & Leclerc, Y. G. (2000). Variable albedo surface reconstruction from stereo and shape from shading. In Proceedings of the IEEE international conference on computer vision and pattern recognition (Vol. I, pp. 480–487). Google Scholar
  32. Seitz, S., Curless, B., Diebel, J., Scharstein, D., & Szeliski, R. (2006). A comparison and evaluation of multi-view stereo reconstruction algorithms. In Proceedings of the IEEE international conference on computer vision and pattern recognition (pp. 519–526). Google Scholar
  33. Stewart, A. J., & Langer, M. S. (1997). Towards accurate recovery of shape from shading under diffuse lighting. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(9), 1020–1025. CrossRefGoogle Scholar
  34. Urakawa, H. (1993). Calculus of variations and harmonic maps. Providence: American Mathematical Society. MATHGoogle Scholar
  35. Vedaldi, A., & Soatto, S. (2006). Viewpoint invariance for non-planar scenes. Technical Report TR050012, UCLA CSD. Google Scholar
  36. Vese, L. A., & Chan, T. F. (2002). A multiphase level set framework for image processing using the Mumford–Shah functional. International Journal of Computer Vision, 50(3), 271–293. MATHCrossRefGoogle Scholar
  37. Yezzi, A., & Soatto, S. (2001). Stereoscopic segmentation. In Proceedings of the international conference on computer vision (pp. 59–66). Google Scholar
  38. Yuille, A. L., Snow, D., Epstein, R., & Belhumeur, P. (1999). Determining generative models of objects under varying illumination: shape and albedo from multiple images using svd and integrability. International Journal of Computer Vision, 35, 203–222. CrossRefGoogle Scholar
  39. Yuille, A., Coughlan, J. M., & Konishi, S. (2003). Kgbr viewpoint-lighting ambiguity. Journal of the Optical Society of America A, 20(1), 24–31. CrossRefGoogle Scholar
  40. Zhang, R., Tsai, P.-S., Cryer, J., & Shah, M. (1999). Shape from shading: a survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 21(8), 690–706. CrossRefGoogle Scholar
  41. Zheng, Q., & Chellappa, R. (1991). Estimation of illuminant direction, albedo and shape from shading. IEEE Transactions on Pattern Analysis and Machine Intelligence, 13(7), 680–702. CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2007

Authors and Affiliations

  • Hailin Jin
    • 1
  • Daniel Cremers
    • 2
  • Dejun Wang
    • 5
  • Emmanuel Prados
    • 3
  • Anthony Yezzi
    • 4
  • Stefano Soatto
    • 5
  1. 1.Office of TechnologyAdobe Systems IncorporatedSan JoseUSA
  2. 2.Dept. of Computer ScienceBonnGermany
  3. 3.INRIA Rhône-AlpesMontbonnotFrance
  4. 4.School of Electrical and Computer EngineeringGeorgia Institute of TechnologyAtlantaUSA
  5. 5.Computer Science DepartmentUniversity of CaliforniaLos AngelesUSA

Personalised recommendations