Advertisement

The Visual Computer

, Volume 33, Issue 3, pp 355–369 | Cite as

Intrinsic image estimation using near-\(L_0\) sparse optimization

  • Shouhong Ding
  • Bin Sheng
  • Zhifeng Xie
  • Lizhuang MaEmail author
Original Article

Abstract

The objective of intrinsic images estimation is to decompose an input image into its intrinsic shading and reflectance components. This is a well-known under-constrained problem that has long been an open challenge. This paper proposes a novel approach for automatic intrinsic images decomposition that uses a new reflectance sparsity prior. On the basis of the observation that the reflectance of natural objects is commonly piecewise constant, we formalize this constraint on the entire reflectance image using the \(L_0\) sparse loss function that enforces the variation in reflectance images to be of high-frequency and sparse. This new sparsity constraint significantly improves the quality of Retinex intrinsic images estimation. It also functions effectively by combining a class of global sparsity priors on reflectance. Experimental results on MIT benchmark dataset as well as various real-world images and synthetic images demonstrate the effectiveness and versatility of our approach.

Keywords

Intrinsic image decomposition Reflectance Shading \(L_0\) sparsity 

Notes

Acknowledgments

The work is supported by the National Natural Science Foundation of China (Nos. 61133009, 61472245, 61572316 and 61303093), National High-tech R&D Program of China (863 Program) (Grant No. 2015AA011604). The authors would also like to thank the reviewers for their helpful suggestions.

Supplementary material

371_2015_1205_MOESM1_ESM.pdf (86 mb)
Supplementary material 1 (pdf 88093 KB)

References

  1. 1.
    Bousseau, Adrien, Paris, Sylvain, Durand, Frédo: User-assisted intrinsic images. ACM Trans. Graph. 28(5), 130:1–130:10 (2009)CrossRefGoogle Scholar
  2. 2.
    Tappen, M.F., Freeman, W.T., Adelson, E.H.: Recovering intrinsic images from a single image. Pattern Anal. Mach. Intell. IEEE Trans. 27(9) (2004)Google Scholar
  3. 3.
    Bell, S., Bala, K., Snavely, N.: Intrinsic images in the wild. ACM Trans. Graphi. (SIGGRAPH), 33(4) (2014)Google Scholar
  4. 4.
    Shen, L., Yeo, C.: Intrinsic images decomposition using a local and global sparse representation of reflectance. In: Computer Vision and Pattern Recognition (CVPR), 2011 IEEE Conference on, pp. 697–704 (2011)Google Scholar
  5. 5.
    Gehler, P., Rother, C., Kiefel, M., Zhang, L., Bernhard, S.: Recovering intrinsic images with a global sparsity prior on reflectance. Adv. Neural Inf. Process. Syst. 24, 765–773 (2011)Google Scholar
  6. 6.
    Shen, L., Yeo, C., Hua, B.-S.: Intrinsic image decomposition using a sparse representation of reflectance. Pattern Anal. Mach. Intell. IEEE Trans. 35(12), 2904–2915 (2013)CrossRefGoogle Scholar
  7. 7.
    Shen, L., Tan, P., Lin, S.: Intrinsic image decomposition with non-local texture cues. In: Computer vision and pattern recognition. CVPR 2008, pp. 1–7. IEEE Conference on (2008)Google Scholar
  8. 8.
    Zhao, Q., Tan, P., Dai, Q., Shen, L., Enhua, W., Lin, S.: A closed-form solution to retinex with nonlocal texture constraints. Pattern Anal. Mach. Intell. IEEE Trans. 34(7), 1437–1444 (2012)CrossRefGoogle Scholar
  9. 9.
    Grosse, R., Johnson, M.K., Adelson, E.H., Freeman, W.T.: Ground truth dataset and baseline evaluations for intrinsic image algorithms. In: Computer Vision, 2009 IEEE 12th International Conference on, pp. 2335–2342 (2009)Google Scholar
  10. 10.
    Beigpour, S., Serra, M., van de Weijer, J., Benavente, R., Vanrell, M., Penacchio, O., Samaras, D.: Intrinsic image evaluation on synthetic complex scenes. In: Image Processing (ICIP), 2013 20th IEEE International Conference on, pp. 285–289 (2013)Google Scholar
  11. 11.
    Barrow, H., Tenenbaum, J.: Recovering intrinsic scene characteristics from images. Comput. Vis. Syst. (1978)Google Scholar
  12. 12.
    Land, E.H., McCann, J.J.: Lightness and retinex theory. J. Opt. Soc. Am. 61, 1 (1971)Google Scholar
  13. 13.
    Funt, B.V., Drew, M.S., Brockington, M.: Recovering shading from color images. In: Computer VisionECCV’92, pp. 124–132. Springer, New York (1992)Google Scholar
  14. 14.
    Tappen, M.F., Adelson, E.H., Freeman, W.T.: Estimating intrinsic component images using non-linear regression. In: Computer Vision and Pattern Recognition, 2006 IEEE Computer Society Conference on, vol. 2, pp. 1992–1999. IEEE (2006)Google Scholar
  15. 15.
    Werman, M., Omer, I.: Image specific color representation. In: Computer vision and pattern recognition (CVPR). IEEE Conference on. IEEE (2004)Google Scholar
  16. 16.
    Serra, M., Penacchio, O., Benavente, R., Vanrell, M.: Names and shades of color for intrinsic image estimation. In: Computer vision and pattern recognition (CVPR), IEEE Conference on, pp. 278–285 (2012)Google Scholar
  17. 17.
    Barron, J.T., Malik, J.: Color constancy, intrinsic images, and shape estimation. In: Proceedings of the 12th European Conference on Computer Vision, vol. part IV, ECCV’12, pp. 57–70. Springer, Berlin, Heidelberg (2012)Google Scholar
  18. 18.
    Weiss, Y.: Deriving intrinsic images from image sequences. ICCV 2, 68–75 (2001)Google Scholar
  19. 19.
    Laffont, P.-Y., Bousseau, A., Paris, S., Durand, F., Drettakis, G.: Coherent intrinsic images from photo collections. ACM Trans. Graph. 31 (2012)Google Scholar
  20. 20.
    Laffont, P., Bousseau, A., Drettakis, G.: Rich intrinsic image decomposition of outdoor scenes from multiple views. Vis. Comput. Graph. IEEE Trans. 19(2), 210–224 (2013)CrossRefGoogle Scholar
  21. 21.
    Chen, Q., Koltun, V.: A simple model for intrinsic image decomposition with depth cues. In: Computer Vision (ICCV), IEEE International Conference on, pp. 241–248 (2013)Google Scholar
  22. 22.
    Barron, J.T., Malik, J.: Intrinsic scene properties from a single rgb-d image. In: Computer vision and pattern recognition (CVPR), IEEE Conference on, pp. 17–24 (2013)Google Scholar
  23. 23.
    Bonneel, N., Sunkavalli, K., Tompkin, J., Sun, D., Paris, S., Pfister, H.: Interactive intrinsic video editing. ACM Trans. Graph 33(6), 197 (2014)Google Scholar
  24. 24.
    Ye, G., Garces, E., Liu, Y., Dai, Q., Gutierrez, D.: Intrinsic video and applications. ACM Trans. Graph. (SIGGRAPH), 33(4), 80 (2014)Google Scholar
  25. 25.
    Garces, E., Munoz, A., Lopez-Moreno, J., Gutierrez, D.: Intrinsic images by clustering. In: Computer graphics forum, vol. 31, pp. 1415–1424. Wiley Online Library (2012)Google Scholar
  26. 26.
    Li, C., Li, F., Kao, C.-Y., Xu, C.: Image segmentation with simultaneous illumination and reflectance estimation: an energy minimization approach. In: Computer vision, IEEE 12th International Conference on, pp. 702–708. IEEE (2009)Google Scholar
  27. 27.
    Horn B.K.P.: Determining lightness from an image. In: Computer graphics and image processing, vol. 3, pp. 277–299. Elsevier, New York (1974)Google Scholar
  28. 28.
    Blake, A.: Boundary conditions for lightness computation in mondrian world. In: Computer vision, graphics, and image processing, vol. 32, pp. 314–327. Elsevier (1985)Google Scholar
  29. 29.
    Barron, J.T., Malik. J.: Shape, illumination, and reflectance from shading. Tech. Rep. UC Berkeley (2013)Google Scholar
  30. 30.
    Finlayson, G.D., Hordley, S.D., Drew, M.S.: Removing shadows from images using retinex. In: Color and imaging conference, vol. 2002, pp. 73–79. Society for Imaging, Science and Technology (2002)Google Scholar
  31. 31.
    Finlayson, G.D., Hordley, S.D., Lu, C., Drew, M.S.: On the removal of shadows from images. Pattern Anal. Mach. Intell. IEEE Trans. 28(1), 59–68 (2006)Google Scholar
  32. 32.
    Li, X., Cewu, L., Yi, X., Jia, J.: Image smoothing via l0 gradient minimization. ACM Trans. Graph. 30(6), 174 (2011)CrossRefGoogle Scholar
  33. 33.
    Xu, L., Zheng, S., Jia, J.: Unnatural l0 sparse representation for natural image deblurring. In: Computer vision and pattern recognition (CVPR), IEEE Conference on, pp. 1107–1114. IEEE (2013)Google Scholar
  34. 34.
    Comaniciu, D., Meer, P.: Mean shift: a robust approach toward feature space analysis. Pattern Anal. Mach. Intell. IEEE Trans. 24(5), 603–619 (2002)CrossRefGoogle Scholar
  35. 35.
  36. 36.
    Krahenbuhl, P., Koltun V.: Parameter learning and convergent inference for dense random fields. In: International Conference on Machine Learning (2013)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2016

Authors and Affiliations

  • Shouhong Ding
    • 1
  • Bin Sheng
    • 1
  • Zhifeng Xie
    • 2
  • Lizhuang Ma
    • 1
    Email author
  1. 1.Department of Computer Science and Engineering (CSE)Shanghai Jiao Tong UniversityShanghaiChina
  2. 2.Department of Film and Television EngineeringShanghai UniversityShanghaiChina

Personalised recommendations