Intrinsic Image Decomposition Using Structure-Texture Separation and Surface Normals

  • Junho Jeon
  • Sunghyun Cho
  • Xin Tong
  • Seungyong Lee
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8695)

Abstract

While intrinsic image decomposition has been studied extensively during the past a few decades, it is still a challenging problem. This is partly because commonly used constraints on shading and reflectance are often too restrictive to capture an important property of natural images, i.e., rich textures. In this paper, we propose a novel image model for handling textures in intrinsic image decomposition, which enables us to produce high quality results even with simple constraints. We also propose a novel constraint based on surface normals obtained from an RGB-D image. Assuming Lambertian surfaces, we formulate the constraint based on a locally linear embedding framework to promote local and global consistency on the shading layer. We demonstrate that combining the novel texture-aware image model and the novel surface normal based constraint can produce superior results to existing approaches.

Keywords

intrinsic image decomposition structure-texture separation RGB-D image 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Supplementary material

978-3-319-10584-0_15_MOESM1_ESM.pdf (10.5 mb)
Electronic Supplementary Material (PDF 10,709 KB)

References

  1. 1.
    Yu, Y., Malik, J.: Recovering photometric properties of architectural scenes from photographs. In: Proc. of SIGGRAPH, pp. 207–217. ACM (1998)Google Scholar
  2. 2.
    Laffont, P.Y., Bousseau, A., Paris, S., Durand, F., Drettakis, G., et al.: Coherent intrinsic images from photo collections. ACM Transactions on Graphics 31(6) (2012)Google Scholar
  3. 3.
    Khan, E.A., Reinhard, E., Fleming, R.W., Bülthoff, H.H.: Image-based material editing. ACM Transactions on Graphics 25(3), 654–663 (2006)CrossRefGoogle Scholar
  4. 4.
    Land, E.H., McCann, J.J.: Lightness and retinex theory. Journal of the Optical Society of America 61(1) (1971)Google Scholar
  5. 5.
    Barrow, H.G., Tenenbaum, J.M.: Recovering intrinsic scene characteristics from images. Computer Vision Systems (1978)Google Scholar
  6. 6.
    Funt, B.V., Drew, M.S., Brockington, M.: Recovering shading from color images. In: Sandini, G. (ed.) ECCV 1992. LNCS, vol. 588, pp. 124–132. Springer, Heidelberg (1992)CrossRefGoogle Scholar
  7. 7.
    Kimmel, R., Elad, M., Shaked, D., Keshet, R., Sobel, I.: A variational framework for retinex. International Journal of Computer Vision 52, 7–23 (2003)CrossRefMATHGoogle Scholar
  8. 8.
    Roweis, S.T., Saul, L.K.: Nonlinear dimensionality reduction by locally linear embedding. Science 290(5500), 2323–2326 (2000)CrossRefGoogle Scholar
  9. 9.
    Weiss, Y.: Deriving intrinsic images from image sequences. In: Proc. of ICCV (2001)Google Scholar
  10. 10.
    Matsushita, Y., Lin, S., Kang, S.B., Shum, H.-Y.: Estimating intrinsic images from image sequences with biased illumination. In: Pajdla, T., Matas, J(G.) (eds.) ECCV 2004, Part II. LNCS, vol. 3022, pp. 274–286. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  11. 11.
    Laffont, P.Y., Bousseau, A., Drettakis, G.: Rich intrinsic image decomposition of outdoor scenes from multiple views. IEEE Transactions on Visualization and Computer Graphics 19(2) (2013)Google Scholar
  12. 12.
    Lee, K.J., Zhao, Q., Tong, X., Gong, M., Izadi, S., Lee, S.U., Tan, P., Lin, S.: Estimation of intrinsic image sequences from image+depth video. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part VI. LNCS, vol. 7577, pp. 327–340. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  13. 13.
    Barron, J.T., Malik, J.: Intrinsic scene properties from a single RGB-D image. In: Proc. of CVPR (2013)Google Scholar
  14. 14.
    Chen, Q., Koltun, V.: A simple model for intrinsic image decomposition with depth cues. In: Proc. of ICCV (2013)Google Scholar
  15. 15.
    Bousseau, A., Paris, S., Durand, F.: User-assisted intrinsic images. ACM Transactions on Graphics 28(5) (2009)Google Scholar
  16. 16.
    Kwatra, V., Han, M., Dai, S.: Shadow removal for aerial imagery by information theoretic intrinsic image analysis. In: International Conference on Computational Photography (2012)Google Scholar
  17. 17.
    Perona, P., Malik, J.: Scale-space and edge detection using anisotropic diffusion. IEEE Transactions on Pattern Analysis and Machine Intelligence 12(7), 629–639 (1990)CrossRefGoogle Scholar
  18. 18.
    Rudin, L., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Physica D 60(1-4), 259–268 (1992)CrossRefMATHGoogle Scholar
  19. 19.
    Tomasi, C., Manduchi, R.: Bilateral filtering for gray and color images. In: Proc. of ICCV (1998)Google Scholar
  20. 20.
    Buades, A., Coll, B., Morel, J.M.: A non-local algorithm for image denoising. In: Proc. of CVPR (2005)Google Scholar
  21. 21.
    Farbman, Z., Fattal, R., Lischinski, D., Szeliski, R.: Edge-preserving decompositions for multi-scale tone and detail manipulation. ACM Transactions on Graphics 27(3), 67:1–67:10 (2008)Google Scholar
  22. 22.
    Xu, L., Lu, C., Xu, Y., Jia, J.: Image smoothing via L0 gradient minimization. ACM Transactions on Graphics 30(6), 174:1–174:12 (2011)Google Scholar
  23. 23.
    Subr, K., Soler, C., Durand, F.: Edge-preserving multiscale image decomposition based on local extrema. ACM Transactions on Graphics 28(5), 147:1–147:9 (2009)Google Scholar
  24. 24.
    Xu, L., Yan, Q., Xia, Y., Jia, J.: Structure extraction from texture via relative total variation. ACM Transactions on Graphics 31(6), 139:1–139:10 (2012)Google Scholar
  25. 25.
    Karacan, L., Erdem, E., Erdem, A.: Structure-preserving image smoothing via region covariances. ACM Transactions on Graphics 32(6), 176:1–176:11 (2013)Google Scholar
  26. 26.
    Tuzel, O., Porikli, F., Meer, P.: Region covariance: A fast descriptor for detection and classification. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3952, pp. 589–600. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  27. 27.
    Levin, A., Lischinski, D., Weiss, Y.: A closed-form solution to natural image matting. IEEE Transactions on Pattern Analysis and Machine Intelligence 30(2), 228–242 (2008)CrossRefGoogle Scholar
  28. 28.
    Shen, L., Yeo, C.: Intrinsic images decomposition using a local and global sparse representation of reflectance. In: Proc. of CVPR (2011)Google Scholar
  29. 29.
    Zhao, Q., Tan, P., Dai, Q., Shen, L., Wu, E., Lin, S.: A closed-form solution to retinex with nonlocal texture constraints. IEEE Transactions on Pattern Analysis and Machine Intelligence 34(7) (2012)Google Scholar
  30. 30.
    Silberman, N., Hoiem, D., Kohli, P., Fergus, R.: Indoor segmentation and support inference from RGBD images. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part V. LNCS, vol. 7576, pp. 746–760. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  31. 31.
    Butler, D.J., Wulff, J., Stanley, G.B., Black, M.J.: A naturalistic open source movie for optical flow evaluation. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part VI. LNCS, vol. 7577, pp. 611–625. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  32. 32.
    Grosse, R., Johnson, M.K., Adelson, E.H., Freeman, W.T.: Ground truth dataset and baseline evaluations for intrinsic image algorithms. In: Proc. of ICCV (2009)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Junho Jeon
    • 1
  • Sunghyun Cho
    • 2
  • Xin Tong
    • 3
  • Seungyong Lee
    • 1
  1. 1.POSTECHKorea
  2. 2.Adobe ResearchKorea
  3. 3.Microsoft Research AsiaChina

Personalised recommendations