Reflectance and Natural Illumination from a Single Image

  • Stephen Lombardi
  • Ko Nishino
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7577)

Abstract

Estimating reflectance and natural illumination from a single image of an object of known shape is a challenging task due to the ambiguities between reflectance and illumination. Although there is an inherent limitation in what can be recovered as the reflectance band-limits the illumination, explicitly estimating both is desirable for many computer vision applications. Achieving this estimation requires that we derive and impose strong constraints on both variables. We introduce a probabilistic formulation that seamlessly incorporates such constraints as priors to arrive at the maximum a posteriori estimates of reflectance and natural illumination. We begin by showing that reflectance modulates the natural illumination in a way that increases its entropy. Based on this observation, we impose a prior on the illumination that favors lower entropy while conforming to natural image statistics. We also impose a prior on the reflectance based on the directional statistics BRDF model that constrains the estimate to lie within the bounds and variability of real-world materials. Experimental results on a number of synthetic and real images show that the method is able to achieve accurate joint estimation for different combinations of materials and lighting.

References

  1. 1.
    Ramamoorthi, R., Hanrahan, P.: A signal-processing framework for inverse rendering. In: Proc. of ACM SIGGRAPH, pp. 117–128 (2001)Google Scholar
  2. 2.
    Chandraker, M., Ramamoorthi, R.: What an image reveals about material reflectance. In: ICCV, pp. 1–8 (2011)Google Scholar
  3. 3.
    Lensch, H.P.A., Kautz, J., Goesele, M., Heidrich, W., Seidel, H.P.: Image-based Reconstruction of Spatial Appearance and Geometric Detail. ACM Trans. on Graphics 22(2), 234–257 (2003)CrossRefGoogle Scholar
  4. 4.
    Zheng, Q., Chellappa, R.: Estimation of illuminant direction, albedo, and shape from shading. IEEE Trans. on Pattern Analysis and Machine Intelligence 13, 680–702 (1991)CrossRefGoogle Scholar
  5. 5.
    Zickler, T., Ramamoorthi, R., Enrique, S., Belhumeur, P.N.: Reflectance Sharing: Predicting Appearance from A Sparse Set of Images of a Known Shape. IEEE Trans. on Pattern Analysis and Machine Intelligence 28(8), 1287–1302 (2006)CrossRefGoogle Scholar
  6. 6.
    Hara, K., Nishino, K., Ikeuchi, K.: Mixture of Spherical Distributions for Single-View Relighting. IEEE Trans. on Pattern Analysis and Machine Intelligence 30(1), 25–35 (2008)CrossRefGoogle Scholar
  7. 7.
    Hara, K., Nishino, K.: Variational Estimation of Inhomogeneous Specular Reflectance and Illumination from a Single View. Journal of Optical Society America, A 28(2), 136–146 (2011)CrossRefGoogle Scholar
  8. 8.
    Marschner, S., Greenberg, D.: Inverse lighting for photography. In: IS&T/SID Fifth Color Imaging Conference, pp. 262–265. The Society for Imaging Science and Technology (1997)Google Scholar
  9. 9.
    Nishino, K., Ikeuchi, K., Zhang, Z.: Re-rendering from a sparse set of images. Technical Report DU-CS-05-12, Dept. of Computer Science, Drexel University (2005)Google Scholar
  10. 10.
    Romeiro, F., Zickler, T.: Blind Reflectometry. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part I. LNCS, vol. 6311, pp. 45–58. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  11. 11.
    Nishino, K.: Directional Statistics BRDF Model. In: ICCV, pp. 476–483 (2009)Google Scholar
  12. 12.
    Nishino, K., Lombardi, S.: Directional Statistics-based Reflectance Model for Isotropic Bidirectional Reflectance Distribution Functions. Journal of Optical Society America, A 28(1), 8–18 (2011)CrossRefGoogle Scholar
  13. 13.
    Huang, J., Mumford, D.: Statistics of natural images and models. In: CVPR, pp. 541–547 (1999)Google Scholar
  14. 14.
    Alldrin, N.G., Mallick, S.P., Kriegman, D.J.: Resolving the generalized bas-relief ambiguity by entropy minimization. In: CVPR, pp. 1–7 (June 2007)Google Scholar
  15. 15.
    Finlayson, G.D., Drew, M.S., Lu, C.: Entropy minimization for shadow removal. Int’l Journal of Computer Vision 85, 35–57 (2009)CrossRefGoogle Scholar
  16. 16.
    Rusinkiewicz, S.: A New Change of Variables for Efficient BRDF Representation. In: Eurographics Workshop on Rendering, pp. 11–22 (1998)Google Scholar
  17. 17.
    Lombardi, S., Nishino, K.: Single image multimaterial estimation. In: CVPR, pp. 238–245 (2012)Google Scholar
  18. 18.
    Matusik, W., Pfister, H., Brand, M., McMillan, L.: A Data-Driven Reflectance Nodel. ACM Trans. on Graphics 22(3), 759–769 (2003)CrossRefGoogle Scholar
  19. 19.
    Marquardt, D.W.: An algorithm for least-squares estimation of nonlinear parameters. SIAM Journal on Applied Mathematics 11(2), 431–441 (1963)MathSciNetMATHCrossRefGoogle Scholar
  20. 20.
    Byrd, R.H., Lu, P., Nocedal, J., Zhu, C.: A Limited Memory Algorithm for Bound Constrained Optimization. SIAM Journal on Scientific Computing 16, 1190–1208 (1995)MathSciNetMATHCrossRefGoogle Scholar
  21. 21.
    Buchsbaum, G.: A spatial processor model for object colour perception. Journal of the Franklin Institute 310(1), 1–26 (1980)MathSciNetCrossRefGoogle Scholar
  22. 22.
    Debevec, P.: Rendering synthetic objects into real scenes: bridging traditional and image-based graphics with global illumination and high dynamic range photography. In: Proc. of ACM SIGGRAPH, pp. 189–198 (1998)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Stephen Lombardi
    • 1
  • Ko Nishino
    • 1
  1. 1.Department of Computer ScienceDrexel UniversityPhiladelphiaUSA

Personalised recommendations