Skip to main content
SpringerLink
Log in

A Variational Model for Intrinsic Light Field Decomposition

  • Conference paper
  • First Online:
Computer Vision – ACCV 2016 (ACCV 2016)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 10113))

Included in the following conference series:

Abstract

We present a novel variational model for intrinsic light field decomposition, which is performed on four-dimensional ray space instead of a traditional 2D image. As most existing intrinsic image algorithms are designed for Lambertian objects, their performance suffers when considering scenes which exhibit glossy surfaces. In contrast, the rich structure of the light field with many densely sampled views allows us to cope with non-Lambertian objects by introducing an additional decomposition term that models specularity. Regularization along the epipolar plane images further encourages albedo and shading consistency across views. In evaluations of our method on real-world data sets captured with a Lytro Illum plenoptic camera, we demonstrate the advantages of our approach with respect to intrinsic image decomposition and specular removal.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Land, E.H., McCann, J.J.: Lightness and retinex theory. J. Opt. Soc. Am. 61, 1–11 (1978)

    Article  Google Scholar 

  2. Shroff, N., Taguchi, Y., Tuzel, O., Veeraraghavan, A., Ramalingam, S., Okuda, H.: Finding a needle in a specular haystack. In: 2011 IEEE International Conference on Robotics and Automation (ICRA), pp. 5963–5970 (2011)

    Google Scholar 

  3. Beigpour, S., van de Weijer, J.: Object recoloring based on intrinsic image estimation. In: IEEE International Conference on Computer Vision (2016)

    Google Scholar 

  4. Wang, T.C., Chandraker, M., Efros, A., Ramamoorthi, R.: SVBRDF-invariant shape and reflectance estimation from light-field cameras. In: Proceedings of International Conference on Computer Vision and Pattern Recognition (2016)

    Google Scholar 

  5. Finlayson, G.D., Hordley, S.D., Drew, M.S.: Removing shadows from images using retinex. In: Color Imaging Conference: Color Science and Engineering Systems, Technologies, and Applications (2002)

    Google Scholar 

  6. Finlayson, G.D., Hordley, S.D., Lu, C., Drew, M.S.: On the removal of shadows from images. IEEE Trans. Pattern Anal. Mach. Intell. 28, 59–68 (2006)

    Article  Google Scholar 

  7. Shafer, S.: Using color to separate reflection components. Color Res. Appl. 10, 210–218 (1985)

    Article  Google Scholar 

  8. Levoy, M.: Light fields and computational imaging. Computer 39, 46–55 (2006)

    Article  Google Scholar 

  9. Wanner, S., Goldluecke, B.: Variational light field analysis for disparity estimation and super-resolution. IEEE Trans. Pattern Anal. Mach. Intell. 36, 606–619 (2014)

    Article  Google Scholar 

  10. Tao, M., Su, J.C., Wang, T.C., Malik, J., Ramamoorthi, R.: Depth estimation and specular removal for glossy surfaces using point and line consistency with light-field cameras. IEEE Trans. Pattern Anal. Mach. Intell. 38, 1155–1169 (2015)

    Article  Google Scholar 

  11. Tao, M., Srinivasan, P., Hadap, S., Rusinkiewicz, S., Malik, J., Ramamoorthi, R.: Shape estimation from shading, defocus, and correspondence using light-field angular coherence. IEEE Trans. Pattern Anal. Mach. Intell. 39(3), 546–560 (2016)

    Article  Google Scholar 

  12. Goldluecke, B., Wanner, S.: The variational structure of disparity and regularization of 4D light fields. In: Proceedings of International Conference on Computer Vision and Pattern Recognition (2013)

    Google Scholar 

  13. Chen, Q., Koltun, V.: A simple model for intrinsic image decomposition with depth cues. In: Proceedings of International Conference on Computer Vision (2013)

    Google Scholar 

  14. Tao, M.W., Wang, T.-C., Malik, J., Ramamoorthi, R.: Depth estimation for glossy surfaces with light-field cameras. In: Agapito, L., Bronstein, M.M., Rother, C. (eds.) ECCV 2014. LNCS, vol. 8926, pp. 533–547. Springer, Cham (2015). doi:10.1007/978-3-319-16181-5_41

    Google Scholar 

  15. Barrow, H.G., Tenenbaum, J.M.: Recovering intrinsic scene characteristics from images. Comput. Vis. Syst. 23, 3–26 (1978)

    Google Scholar 

  16. Tappen, M.F., Freeman, W.T., Adelson, E.H.: Recovering intrinsic images from a single image. IEEE Trans. Pattern Anal. Mach. Intell. 27, 1459–1472 (2005)

    Article  Google Scholar 

  17. Chung, Y., Cherng, S., Bailey, R.R., Chen, S.W.: Intrinsic image extraction from a single image. J. Inf. Sci. Eng. 25, 1939–1953 (2009)

    Google Scholar 

  18. Grosse, R., Johnson, M.K., Adelson, E.H., Freeman, W.T.: Ground truth dataset and baseline evaluations for intrinsic image algorithm. In: Proceedings of International Conference on Computer Vision (2009)

    Google Scholar 

  19. Barron, J.T., Malik, J.: High-frequency shape and albedo from shading using natural image statistics. In: Proceedings of International Conference on Computer Vision and Pattern Recognition (2011)

    Google Scholar 

  20. Barron, J.T., Malik, J.: Color constancy, intrinsic images, and shape estimation. In: Proceedings of European Conference on Computer Vision (2012)

    Google Scholar 

  21. Barron, J.T., Malik, J.: Intrinsic scene properties from a single RGB-D image. IEEE Trans. Pattern Anal. Mach. Intell. 38, 690–703 (2015)

    Article  Google Scholar 

  22. Shen, L., Tan, P., Lin, S.: Intrinsic image decomposition with non-local texture cues. In: Proceedings of International Conference on Computer Vision and Pattern Recognition (2008)

    Google Scholar 

  23. Finlayson, G.D., Drew, M.S., Lu, C.: Intrinsic images by entropy minimization. In: Pajdla, T., Matas, J. (eds.) ECCV 2004. LNCS, vol. 3023, pp. 582–595. Springer, Heidelberg (2004). doi:10.1007/978-3-540-24672-5_46

    Chapter  Google Scholar 

  24. Gehler, P.V., Rother, C., Kiefel, M., Zhang, L., Schölkopf, B.: Recovering intrinsic images with a global sparsity prior on reflectance. In: NIPS (2011)

    Google Scholar 

  25. Bell, S., Bala, K., Snavely, N.: Intrinsic images in the wild. ACM Trans. Graph. (SIGGRAPH) 33, 159:1–159:12 (2014)

    Article  Google Scholar 

  26. Lee, K.J., Zhao, Q., Tong, X., Gong, M., Izadi, S., Lee, S.U., Tan, P., Lin, S.: Estimation of intrinsic image sequences from image+depth video. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, vol. 7577, pp. 327–340. Springer, Heidelberg (2012). doi:10.1007/978-3-642-33783-3_24

    Chapter  Google Scholar 

  27. Jeon, J., Cho, S., Tong, X., Lee, S.: Intrinsic image decomposition using structure-texture separation and surface normals. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8695, pp. 218–233. Springer, Cham (2014). doi:10.1007/978-3-319-10584-0_15

    Google Scholar 

  28. Blake, A., Bülthoff, H.: Shape from specularities: computation and psychophysics. Phil. Trans. R. Soc. Lond. B 331, 237–252 (1991)

    Article  Google Scholar 

  29. Swaminathan, R., Kang, S.B., Szeliski, R., Criminisi, A., Nayar, S.K.: On the motion and appearance of specularities in image sequences. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002. LNCS, vol. 2350, pp. 508–523. Springer, Heidelberg (2002). doi:10.1007/3-540-47969-4_34

    Chapter  Google Scholar 

  30. Adato, Y., Vasilyev, Y., Ben-Shahar, O., Zickler, T.: Toward a theory of shape from specular flow. In: Proceedings of International Conference on Computer Vision (2007)

    Google Scholar 

  31. Bolles, R.C., Baker, H.H., Marimont, D.H.: Epipolar-plane image analysis: an approach to determining structure from motion. Int. J. Comput. Vision 1, 7–55 (1987)

    Article  Google Scholar 

  32. Yang, K., Gao, S., Li, Y.: Efficient illuminant estimation for color constancy using grey pixels. In: Proceedings of International Conference on Computer Vision and Pattern Recognition (2015)

    Google Scholar 

  33. Weiss, Y.: Deriving intrinsic images from image sequences. In: Proceedings of International Conference on Computer Vision (2001)

    Google Scholar 

  34. Finlayson, G.D., Drew, M.S., Lu, C.: Entropy minimization for shadow removal. IJCV 85(1), 35–57 (2009)

    Article  Google Scholar 

  35. Tao, M., Srinivasan, P., Malik, J., Rusinkiewicz, S., Ramamoorthi, R.: Depth from shading, defocus, and correspondence using light-field angular coherence. In: Proceedings of International Conference on Computer Vision and Pattern Recognition (2015)

    Google Scholar 

  36. Tian, Q., Clark, J.J.: Real-time specularity detection using unnormalized wiener entropy. In: Computer and Robot Vision (CRV), pp. 356–363 (2013)

    Google Scholar 

  37. Wanner, S., Meister, S., Goldluecke, B.: Datasets and benchmarks for densely sampled 4D light fields. In: Vision, Modelling and Visualization (VMV) (2013)

    Google Scholar 

  38. Butler, D.J., Wulff, J., Stanley, G.B., Black, M.J.: A naturalistic open source movie for optical flow evaluation. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, vol. 7577, pp. 611–625. Springer, Heidelberg (2012). doi:10.1007/978-3-642-33783-3_44

    Chapter  Google Scholar 

Download references

Acknowledgements

This work was supported by the ERC Starting Grant “Light Field Imaging and Analysis” (LIA 336978, FP7-2014).

Author information

Authors and Affiliations

  1. University of Konstanz, Konstanz, Germany

    Anna Alperovich & Bastian Goldluecke

Authors
  1. Anna Alperovich
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bastian Goldluecke
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Anna Alperovich .

Editor information

Editors and Affiliations

  1. National Tsing Hua University, Hsinchu, Taiwan

    Shang-Hong Lai

  2. Graz University of Technology, Graz, Austria

    Vincent Lepetit

  3. Drexel University, Philadelphia, Pennsylvania, USA

    Ko Nishino

  4. The University of Tokyo , Tokyo, Japan

    Yoichi Sato

1 Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 12066 KB)

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

Alperovich, A., Goldluecke, B. (2017). A Variational Model for Intrinsic Light Field Decomposition. In: Lai, SH., Lepetit, V., Nishino, K., Sato, Y. (eds) Computer Vision – ACCV 2016. ACCV 2016. Lecture Notes in Computer Science(), vol 10113. Springer, Cham. https://doi.org/10.1007/978-3-319-54187-7_5

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-54187-7_5

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-54186-0

  • Online ISBN: 978-3-319-54187-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation