Advertisement

A Variational Model for Intrinsic Light Field Decomposition

  • Anna AlperovichEmail author
  • Bastian Goldluecke
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10113)

Abstract

We present a novel variational model for intrinsic light field decomposition, which is performed on four-dimensional ray space instead of a traditional 2D image. As most existing intrinsic image algorithms are designed for Lambertian objects, their performance suffers when considering scenes which exhibit glossy surfaces. In contrast, the rich structure of the light field with many densely sampled views allows us to cope with non-Lambertian objects by introducing an additional decomposition term that models specularity. Regularization along the epipolar plane images further encourages albedo and shading consistency across views. In evaluations of our method on real-world data sets captured with a Lytro Illum plenoptic camera, we demonstrate the advantages of our approach with respect to intrinsic image decomposition and specular removal.

Keywords

Light Field Specular Surface Specular Component Intrinsic Image Occlusion Boundary 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Notes

Acknowledgements

This work was supported by the ERC Starting Grant “Light Field Imaging and Analysis” (LIA 336978, FP7-2014).

Supplementary material

416261_1_En_5_MOESM1_ESM.pdf (11.8 mb)
Supplementary material 1 (pdf 12066 KB)

References

  1. 1.
    Land, E.H., McCann, J.J.: Lightness and retinex theory. J. Opt. Soc. Am. 61, 1–11 (1978)CrossRefGoogle Scholar
  2. 2.
    Shroff, N., Taguchi, Y., Tuzel, O., Veeraraghavan, A., Ramalingam, S., Okuda, H.: Finding a needle in a specular haystack. In: 2011 IEEE International Conference on Robotics and Automation (ICRA), pp. 5963–5970 (2011)Google Scholar
  3. 3.
    Beigpour, S., van de Weijer, J.: Object recoloring based on intrinsic image estimation. In: IEEE International Conference on Computer Vision (2016)Google Scholar
  4. 4.
    Wang, T.C., Chandraker, M., Efros, A., Ramamoorthi, R.: SVBRDF-invariant shape and reflectance estimation from light-field cameras. In: Proceedings of International Conference on Computer Vision and Pattern Recognition (2016)Google Scholar
  5. 5.
    Finlayson, G.D., Hordley, S.D., Drew, M.S.: Removing shadows from images using retinex. In: Color Imaging Conference: Color Science and Engineering Systems, Technologies, and Applications (2002)Google Scholar
  6. 6.
    Finlayson, G.D., Hordley, S.D., Lu, C., Drew, M.S.: On the removal of shadows from images. IEEE Trans. Pattern Anal. Mach. Intell. 28, 59–68 (2006)CrossRefGoogle Scholar
  7. 7.
    Shafer, S.: Using color to separate reflection components. Color Res. Appl. 10, 210–218 (1985)CrossRefGoogle Scholar
  8. 8.
    Levoy, M.: Light fields and computational imaging. Computer 39, 46–55 (2006)CrossRefGoogle Scholar
  9. 9.
    Wanner, S., Goldluecke, B.: Variational light field analysis for disparity estimation and super-resolution. IEEE Trans. Pattern Anal. Mach. Intell. 36, 606–619 (2014)CrossRefGoogle Scholar
  10. 10.
    Tao, M., Su, J.C., Wang, T.C., Malik, J., Ramamoorthi, R.: Depth estimation and specular removal for glossy surfaces using point and line consistency with light-field cameras. IEEE Trans. Pattern Anal. Mach. Intell. 38, 1155–1169 (2015)CrossRefGoogle Scholar
  11. 11.
    Tao, M., Srinivasan, P., Hadap, S., Rusinkiewicz, S., Malik, J., Ramamoorthi, R.: Shape estimation from shading, defocus, and correspondence using light-field angular coherence. IEEE Trans. Pattern Anal. Mach. Intell. 39(3), 546–560 (2016)CrossRefGoogle Scholar
  12. 12.
    Goldluecke, B., Wanner, S.: The variational structure of disparity and regularization of 4D light fields. In: Proceedings of International Conference on Computer Vision and Pattern Recognition (2013)Google Scholar
  13. 13.
    Chen, Q., Koltun, V.: A simple model for intrinsic image decomposition with depth cues. In: Proceedings of International Conference on Computer Vision (2013)Google Scholar
  14. 14.
    Tao, M.W., Wang, T.-C., Malik, J., Ramamoorthi, R.: Depth estimation for glossy surfaces with light-field cameras. In: Agapito, L., Bronstein, M.M., Rother, C. (eds.) ECCV 2014. LNCS, vol. 8926, pp. 533–547. Springer, Cham (2015). doi: 10.1007/978-3-319-16181-5_41 Google Scholar
  15. 15.
    Barrow, H.G., Tenenbaum, J.M.: Recovering intrinsic scene characteristics from images. Comput. Vis. Syst. 23, 3–26 (1978)Google Scholar
  16. 16.
    Tappen, M.F., Freeman, W.T., Adelson, E.H.: Recovering intrinsic images from a single image. IEEE Trans. Pattern Anal. Mach. Intell. 27, 1459–1472 (2005)CrossRefGoogle Scholar
  17. 17.
    Chung, Y., Cherng, S., Bailey, R.R., Chen, S.W.: Intrinsic image extraction from a single image. J. Inf. Sci. Eng. 25, 1939–1953 (2009)Google Scholar
  18. 18.
    Grosse, R., Johnson, M.K., Adelson, E.H., Freeman, W.T.: Ground truth dataset and baseline evaluations for intrinsic image algorithm. In: Proceedings of International Conference on Computer Vision (2009)Google Scholar
  19. 19.
    Barron, J.T., Malik, J.: High-frequency shape and albedo from shading using natural image statistics. In: Proceedings of International Conference on Computer Vision and Pattern Recognition (2011)Google Scholar
  20. 20.
    Barron, J.T., Malik, J.: Color constancy, intrinsic images, and shape estimation. In: Proceedings of European Conference on Computer Vision (2012)Google Scholar
  21. 21.
    Barron, J.T., Malik, J.: Intrinsic scene properties from a single RGB-D image. IEEE Trans. Pattern Anal. Mach. Intell. 38, 690–703 (2015)CrossRefGoogle Scholar
  22. 22.
    Shen, L., Tan, P., Lin, S.: Intrinsic image decomposition with non-local texture cues. In: Proceedings of International Conference on Computer Vision and Pattern Recognition (2008)Google Scholar
  23. 23.
    Finlayson, G.D., Drew, M.S., Lu, C.: Intrinsic images by entropy minimization. In: Pajdla, T., Matas, J. (eds.) ECCV 2004. LNCS, vol. 3023, pp. 582–595. Springer, Heidelberg (2004). doi: 10.1007/978-3-540-24672-5_46 CrossRefGoogle Scholar
  24. 24.
    Gehler, P.V., Rother, C., Kiefel, M., Zhang, L., Schölkopf, B.: Recovering intrinsic images with a global sparsity prior on reflectance. In: NIPS (2011)Google Scholar
  25. 25.
    Bell, S., Bala, K., Snavely, N.: Intrinsic images in the wild. ACM Trans. Graph. (SIGGRAPH) 33, 159:1–159:12 (2014)CrossRefGoogle Scholar
  26. 26.
    Lee, K.J., Zhao, Q., Tong, X., Gong, M., Izadi, S., Lee, S.U., Tan, P., Lin, S.: Estimation of intrinsic image sequences from image+depth video. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, vol. 7577, pp. 327–340. Springer, Heidelberg (2012). doi: 10.1007/978-3-642-33783-3_24 CrossRefGoogle Scholar
  27. 27.
    Jeon, J., Cho, S., Tong, X., Lee, S.: Intrinsic image decomposition using structure-texture separation and surface normals. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8695, pp. 218–233. Springer, Cham (2014). doi: 10.1007/978-3-319-10584-0_15 Google Scholar
  28. 28.
    Blake, A., Bülthoff, H.: Shape from specularities: computation and psychophysics. Phil. Trans. R. Soc. Lond. B 331, 237–252 (1991)CrossRefGoogle Scholar
  29. 29.
    Swaminathan, R., Kang, S.B., Szeliski, R., Criminisi, A., Nayar, S.K.: On the motion and appearance of specularities in image sequences. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002. LNCS, vol. 2350, pp. 508–523. Springer, Heidelberg (2002). doi: 10.1007/3-540-47969-4_34 CrossRefGoogle Scholar
  30. 30.
    Adato, Y., Vasilyev, Y., Ben-Shahar, O., Zickler, T.: Toward a theory of shape from specular flow. In: Proceedings of International Conference on Computer Vision (2007)Google Scholar
  31. 31.
    Bolles, R.C., Baker, H.H., Marimont, D.H.: Epipolar-plane image analysis: an approach to determining structure from motion. Int. J. Comput. Vision 1, 7–55 (1987)CrossRefGoogle Scholar
  32. 32.
    Yang, K., Gao, S., Li, Y.: Efficient illuminant estimation for color constancy using grey pixels. In: Proceedings of International Conference on Computer Vision and Pattern Recognition (2015)Google Scholar
  33. 33.
    Weiss, Y.: Deriving intrinsic images from image sequences. In: Proceedings of International Conference on Computer Vision (2001)Google Scholar
  34. 34.
    Finlayson, G.D., Drew, M.S., Lu, C.: Entropy minimization for shadow removal. IJCV 85(1), 35–57 (2009)CrossRefGoogle Scholar
  35. 35.
    Tao, M., Srinivasan, P., Malik, J., Rusinkiewicz, S., Ramamoorthi, R.: Depth from shading, defocus, and correspondence using light-field angular coherence. In: Proceedings of International Conference on Computer Vision and Pattern Recognition (2015)Google Scholar
  36. 36.
    Tian, Q., Clark, J.J.: Real-time specularity detection using unnormalized wiener entropy. In: Computer and Robot Vision (CRV), pp. 356–363 (2013)Google Scholar
  37. 37.
    Wanner, S., Meister, S., Goldluecke, B.: Datasets and benchmarks for densely sampled 4D light fields. In: Vision, Modelling and Visualization (VMV) (2013)Google Scholar
  38. 38.
    Butler, D.J., Wulff, J., Stanley, G.B., Black, M.J.: A naturalistic open source movie for optical flow evaluation. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, vol. 7577, pp. 611–625. Springer, Heidelberg (2012). doi: 10.1007/978-3-642-33783-3_44 CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.University of KonstanzKonstanzGermany

Personalised recommendations