Abstract
The objective of intrinsic images estimation is to decompose an input image into its intrinsic shading and reflectance components. This is a well-known under-constrained problem that has long been an open challenge. This paper proposes a novel approach for automatic intrinsic images decomposition that uses a new reflectance sparsity prior. On the basis of the observation that the reflectance of natural objects is commonly piecewise constant, we formalize this constraint on the entire reflectance image using the \(L_0\) sparse loss function that enforces the variation in reflectance images to be of high-frequency and sparse. This new sparsity constraint significantly improves the quality of Retinex intrinsic images estimation. It also functions effectively by combining a class of global sparsity priors on reflectance. Experimental results on MIT benchmark dataset as well as various real-world images and synthetic images demonstrate the effectiveness and versatility of our approach.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Bousseau, Adrien, Paris, Sylvain, Durand, Frédo: User-assisted intrinsic images. ACM Trans. Graph. 28(5), 130:1–130:10 (2009)
Tappen, M.F., Freeman, W.T., Adelson, E.H.: Recovering intrinsic images from a single image. Pattern Anal. Mach. Intell. IEEE Trans. 27(9) (2004)
Bell, S., Bala, K., Snavely, N.: Intrinsic images in the wild. ACM Trans. Graphi. (SIGGRAPH), 33(4) (2014)
Shen, L., Yeo, C.: Intrinsic images decomposition using a local and global sparse representation of reflectance. In: Computer Vision and Pattern Recognition (CVPR), 2011 IEEE Conference on, pp. 697–704 (2011)
Gehler, P., Rother, C., Kiefel, M., Zhang, L., Bernhard, S.: Recovering intrinsic images with a global sparsity prior on reflectance. Adv. Neural Inf. Process. Syst. 24, 765–773 (2011)
Shen, L., Yeo, C., Hua, B.-S.: Intrinsic image decomposition using a sparse representation of reflectance. Pattern Anal. Mach. Intell. IEEE Trans. 35(12), 2904–2915 (2013)
Shen, L., Tan, P., Lin, S.: Intrinsic image decomposition with non-local texture cues. In: Computer vision and pattern recognition. CVPR 2008, pp. 1–7. IEEE Conference on (2008)
Zhao, Q., Tan, P., Dai, Q., Shen, L., Enhua, W., Lin, S.: A closed-form solution to retinex with nonlocal texture constraints. Pattern Anal. Mach. Intell. IEEE Trans. 34(7), 1437–1444 (2012)
Grosse, R., Johnson, M.K., Adelson, E.H., Freeman, W.T.: Ground truth dataset and baseline evaluations for intrinsic image algorithms. In: Computer Vision, 2009 IEEE 12th International Conference on, pp. 2335–2342 (2009)
Beigpour, S., Serra, M., van de Weijer, J., Benavente, R., Vanrell, M., Penacchio, O., Samaras, D.: Intrinsic image evaluation on synthetic complex scenes. In: Image Processing (ICIP), 2013 20th IEEE International Conference on, pp. 285–289 (2013)
Barrow, H., Tenenbaum, J.: Recovering intrinsic scene characteristics from images. Comput. Vis. Syst. (1978)
Land, E.H., McCann, J.J.: Lightness and retinex theory. J. Opt. Soc. Am. 61, 1 (1971)
Funt, B.V., Drew, M.S., Brockington, M.: Recovering shading from color images. In: Computer VisionECCV’92, pp. 124–132. Springer, New York (1992)
Tappen, M.F., Adelson, E.H., Freeman, W.T.: Estimating intrinsic component images using non-linear regression. In: Computer Vision and Pattern Recognition, 2006 IEEE Computer Society Conference on, vol. 2, pp. 1992–1999. IEEE (2006)
Werman, M., Omer, I.: Image specific color representation. In: Computer vision and pattern recognition (CVPR). IEEE Conference on. IEEE (2004)
Serra, M., Penacchio, O., Benavente, R., Vanrell, M.: Names and shades of color for intrinsic image estimation. In: Computer vision and pattern recognition (CVPR), IEEE Conference on, pp. 278–285 (2012)
Barron, J.T., Malik, J.: Color constancy, intrinsic images, and shape estimation. In: Proceedings of the 12th European Conference on Computer Vision, vol. part IV, ECCV’12, pp. 57–70. Springer, Berlin, Heidelberg (2012)
Weiss, Y.: Deriving intrinsic images from image sequences. ICCV 2, 68–75 (2001)
Laffont, P.-Y., Bousseau, A., Paris, S., Durand, F., Drettakis, G.: Coherent intrinsic images from photo collections. ACM Trans. Graph. 31 (2012)
Laffont, P., Bousseau, A., Drettakis, G.: Rich intrinsic image decomposition of outdoor scenes from multiple views. Vis. Comput. Graph. IEEE Trans. 19(2), 210–224 (2013)
Chen, Q., Koltun, V.: A simple model for intrinsic image decomposition with depth cues. In: Computer Vision (ICCV), IEEE International Conference on, pp. 241–248 (2013)
Barron, J.T., Malik, J.: Intrinsic scene properties from a single rgb-d image. In: Computer vision and pattern recognition (CVPR), IEEE Conference on, pp. 17–24 (2013)
Bonneel, N., Sunkavalli, K., Tompkin, J., Sun, D., Paris, S., Pfister, H.: Interactive intrinsic video editing. ACM Trans. Graph 33(6), 197 (2014)
Ye, G., Garces, E., Liu, Y., Dai, Q., Gutierrez, D.: Intrinsic video and applications. ACM Trans. Graph. (SIGGRAPH), 33(4), 80 (2014)
Garces, E., Munoz, A., Lopez-Moreno, J., Gutierrez, D.: Intrinsic images by clustering. In: Computer graphics forum, vol. 31, pp. 1415–1424. Wiley Online Library (2012)
Li, C., Li, F., Kao, C.-Y., Xu, C.: Image segmentation with simultaneous illumination and reflectance estimation: an energy minimization approach. In: Computer vision, IEEE 12th International Conference on, pp. 702–708. IEEE (2009)
Horn B.K.P.: Determining lightness from an image. In: Computer graphics and image processing, vol. 3, pp. 277–299. Elsevier, New York (1974)
Blake, A.: Boundary conditions for lightness computation in mondrian world. In: Computer vision, graphics, and image processing, vol. 32, pp. 314–327. Elsevier (1985)
Barron, J.T., Malik. J.: Shape, illumination, and reflectance from shading. Tech. Rep. UC Berkeley (2013)
Finlayson, G.D., Hordley, S.D., Drew, M.S.: Removing shadows from images using retinex. In: Color and imaging conference, vol. 2002, pp. 73–79. Society for Imaging, Science and Technology (2002)
Finlayson, G.D., Hordley, S.D., Lu, C., Drew, M.S.: On the removal of shadows from images. Pattern Anal. Mach. Intell. IEEE Trans. 28(1), 59–68 (2006)
Li, X., Cewu, L., Yi, X., Jia, J.: Image smoothing via l0 gradient minimization. ACM Trans. Graph. 30(6), 174 (2011)
Xu, L., Zheng, S., Jia, J.: Unnatural l0 sparse representation for natural image deblurring. In: Computer vision and pattern recognition (CVPR), IEEE Conference on, pp. 1107–1114. IEEE (2013)
Comaniciu, D., Meer, P.: Mean shift: a robust approach toward feature space analysis. Pattern Anal. Mach. Intell. IEEE Trans. 24(5), 603–619 (2002)
http://www.gatsby.ucl.ac.uk/~edward/code/minimize/minimize.m. Accessed 19 Jan 2016
Krahenbuhl, P., Koltun V.: Parameter learning and convergent inference for dense random fields. In: International Conference on Machine Learning (2013)
Acknowledgments
The work is supported by the National Natural Science Foundation of China (Nos. 61133009, 61472245, 61572316 and 61303093), National High-tech R&D Program of China (863 Program) (Grant No. 2015AA011604). The authors would also like to thank the reviewers for their helpful suggestions.
Author information
Authors and Affiliations
Corresponding author
Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Ding, S., Sheng, B., Xie, Z. et al. Intrinsic image estimation using near-\(L_0\) sparse optimization. Vis Comput 33, 355–369 (2017). https://doi.org/10.1007/s00371-015-1205-9
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00371-015-1205-9