Color Constancy, Intrinsic Images, and Shape Estimation

  • Jonathan T. Barron
  • Jitendra Malik
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7575)


We present SIRFS (shape, illumination, and reflectance from shading), the first unified model for recovering shape, chromatic illumination, and reflectance from a single image. Our model is an extension of our previous work [1], which addressed the achromatic version of this problem. Dealing with color requires a modified problem formulation, novel priors on reflectance and illumination, and a new optimization scheme for dealing with the resulting inference problem. Our approach outperforms all previously published algorithms for intrinsic image decomposition and shape-from-shading on the MIT intrinsic images dataset [1, 2] and on our own “naturally” illuminated version of that dataset.


Color Constancy Natural Illumination Laplacian Pyramid Shadow Removal Intrinsic Image 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Barron, J.T., Malik, J.: Shape, albedo, and illumination from a single image of an unknown object. In: CVPR (2012)Google Scholar
  2. 2.
    Grosse, R., Johnson, M.K., Adelson, E.H., Freeman, W.T.: Ground-truth dataset and baseline evaluations for intrinsic image algorithms. In: ICCV (2009)Google Scholar
  3. 3.
    van Helmholtz, H.: Treatise on physiological optics, 2 vols., translated. Optical Society of America, Washington, DC (1924)Google Scholar
  4. 4.
    Land, E.H., McCann, J.J.: Lightness and retinex theory. JOSA (1971)Google Scholar
  5. 5.
    Horn, B.K.P.: Determining lightness from an image. Computer Graphics and Image Processing (1974)Google Scholar
  6. 6.
    Forsyth, D.A.: A novel algorithm for color constancy. IJCV (1990)Google Scholar
  7. 7.
    Maloney, L.T., Wandell, B.A.: Color constancy: a method for recovering surface spectral reflectance. JOSA A (1986)Google Scholar
  8. 8.
    Klinker, G., Shafer, S., Kanade, T.: A physical approach to color image understanding. IJCV (1990)Google Scholar
  9. 9.
    Finlayson, G., Hordley, S., Hubel, P.: Color by correlation: a simple, unifying framework for color constancy. TPAMI (2001)Google Scholar
  10. 10.
    Brainard, D.H., Freeman, W.T.: Bayesian color constancy. JOSA A (1997)Google Scholar
  11. 11.
    Gijsenij, A., Gevers, T., van de Weijer, J.: Generalized gamut mapping using image derivative structures for color constancy. IJCV (2010)Google Scholar
  12. 12.
    Gehler, P., Rother, C., Kiefel, M., Zhang, L., Schoelkopf, B.: Recovering intrinsic images with a global sparsity prior on reflectance. In: NIPS (2011)Google Scholar
  13. 13.
    Barrow, H., Tenenbaum, J.: Recovering intrinsic scene characteristics from images. In: Computer Vision Systems (1978)Google Scholar
  14. 14.
    Tappen, M.F., Freeman, W.T., Adelson, E.H.: Recovering intrinsic images from a single image. TPAMI (2005)Google Scholar
  15. 15.
    Shen, J., Yang, X., Jia, Y., Li, X.: Intrinsic images using optimization. In: CVPR (2011)Google Scholar
  16. 16.
    Horn, B.K.P.: Shape from shading: A method for obtaining the shape of a smooth opaque object from one view. Technical report, MIT (1970)Google Scholar
  17. 17.
    Brooks, M.J., Horn, B.K.P.: Shape from shading. MIT Press (1989)Google Scholar
  18. 18.
    Zhang, R., Tsai, P., Cryer, J., Shah, M.: Shape-from-shading: a survey. TPAMI (1999)Google Scholar
  19. 19.
    Johnson, M.K., Adelson, E.H.: Shape estimation in natural illumination. In: CVPR (2011)Google Scholar
  20. 20.
    Gilchrist, A.: Seeing in Black and White. Oxford University Press (2006)Google Scholar
  21. 21.
    Boyaci, H., Doerschner, K., Snyder, J.L., Maloney, L.T.: Surface color perception in three-dimensional scenes. Visual Neuroscience (2006)Google Scholar
  22. 22.
    Barron, J.T., Malik, J.: High-frequency shape and albedo from shading using natural image statistics. In: CVPR (2011)Google Scholar
  23. 23.
    Shen, L., Yeo, C.: Intrinsic images decomposition using a local and global sparse representation of reflectance. In: CVPR (2011)Google Scholar
  24. 24.
    Alldrin, N., Mallick, S., Kriegman, D.: Resolving the generalized bas-relief ambiguity by entropy minimization. In: CVPR (2007)Google Scholar
  25. 25.
    Finlayson, G.D., Drew, M.S., Lu, C.: Entropy minimization for shadow removal. IJCV (2009)Google Scholar
  26. 26.
    Omer, I., Werman, M.: Color lines: Image specific color representation. In: CVPR (2004)Google Scholar
  27. 27.
    Principe, J.C., Xu, D.: Learning from examples with quadratic mutual information. In: Workshop on Neural Networks for Signal Processing (1998)Google Scholar
  28. 28.
    Chen, J., Paris, S., Durand, F.: Real-time edge-aware image processing with the bilateral grid. In: SIGGRAPH (2007)Google Scholar
  29. 29.
    Terzopoulos, D.: Image analysis using multigrid relaxation methods. TPAMI 8, 129–139 (1986)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Jonathan T. Barron
    • 1
  • Jitendra Malik
    • 1
  1. 1.UC BerkeleyUSA

Personalised recommendations