Advertisement

Layered Scene Reconstruction from Multiple Light Field Camera Views

  • Ole Johannsen
  • Antonin SulcEmail author
  • Nico Marniok
  • Bastian Goldluecke
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10113)

Abstract

We propose a framework to infer complete geometry of a scene with strong reflections or hidden by partially transparent occluders from a set of 4D light fields captured with a hand-held light field camera. For this, we first introduce a variant of bundle adjustment specifically tailored to 4D light fields to obtain improved pose parameters. Geometry is recovered in a global framework based on convex optimization for a weighted minimal surface. To allow for non-Lambertian materials and semi-transparent occluders, the point-wise costs are not based on the principle of photo-consistency. Instead, we perform a layer analysis of the light field obtained by finding superimposed oriented patterns in epipolar plane image space to obtain a set of depth hypotheses and confidence scores, which are integrated into a single functional.

Keywords

Light Field Confidence Score Bundle Adjustment Visual Hull Photometric Stereo 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Notes

Acknowledgement

This work was supported by the ERC Starting Grant “Light Field Imaging and Analysis” (LIA 336978, FP7-2014).

References

  1. 1.
    Gortler, S., Grzeszczuk, R., Szeliski, R., Cohen, M.: The lumigraph. In: Proceedings of SIGGRAPH, pp. 43–54 (1996)Google Scholar
  2. 2.
    Bolles, R., Baker, H., Marimont, D.: Epipolar-plane image analysis: an approach to determining structure from motion. Int. J. Comput. Vis. 1, 7–55 (1987)CrossRefGoogle Scholar
  3. 3.
    Johannsen, O., Sulc, A., Goldluecke, B.: What sparse light field coding reveals about scene structure. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition (2016)Google Scholar
  4. 4.
    Wanner, S., Goldluecke, B.: Reconstructing reflective and transparent surfaces from epipolar plane images. In: Weickert, J., Hein, M., Schiele, B. (eds.) GCPR 2013. LNCS, vol. 8142, pp. 1–10. Springer, Heidelberg (2013). doi: 10.1007/978-3-642-40602-7_1 CrossRefGoogle Scholar
  5. 5.
    Johannsen, O., Sulc, A., Goldluecke, B.: On linear structure from motion for light field cameras. In: Proceedings of the International Conference on Computer Vision (2015)Google Scholar
  6. 6.
    Johannsen, O., Sulc, A., Goldluecke, B.: Variational separation of light field layers. In: Vision, Modelling and Visualization (VMV) (2015)Google Scholar
  7. 7.
    Wanner, S., Goldluecke, B.: Variational light field analysis for disparity estimation and super-resolution. IEEE Trans. Pattern Anal. Mach. Intell. 36, 606–619 (2014)CrossRefGoogle Scholar
  8. 8.
    Chandraker, M., Reddy, D., Wang, Y., Ramamoorthi, R.: What object motion reveals about shape with unknown BRDF and lighting. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition (2013)Google Scholar
  9. 9.
    Tao, M., Wang, T.C., Malik, J., Ramamoorthi, R.: Depth estimation for glossy surfaces with light-field cameras. In: Proceedings of the European Conference on Computer Vision Workshops (2014)Google Scholar
  10. 10.
    Chen, C., Lin, H., Yu, Z., Kang, S.B., J., Y.: Light field stereo matching using bilateral statistics of surface cameras. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition (2014)Google Scholar
  11. 11.
    Stich, T., Tevs, A., Magnor, M.: Global depth from epipolar volumes - a general framework for reconstructing non-Lambertian surfaces. In: 3DPVT (2006)Google Scholar
  12. 12.
    Laurentini, A.: The visual hull concept for visual-based image understanding. IEEE Trans. Pattern Anal. Mach. Intell. 16, 150–162 (1994)CrossRefGoogle Scholar
  13. 13.
    Zuo, X., Du, C., Wang, S., Zheng, J., Yang, R.: Interactive visual hull refinement for specular and transparent object surface reconstruction. In: Proceedings of the International Conference on Computer Vision (2015)Google Scholar
  14. 14.
    Kutulakos, K.N., Seitz, S.M.: A theory of shape by space carving. Int. J. Comput. Vis. 38, 199–218 (2000)CrossRefzbMATHGoogle Scholar
  15. 15.
    Yang, Y., Pollefeys, M., Welch, G.: Dealing with textureless regions and specular highlights - a progressive space carving scheme using a novel photo-consistency measure. In: Proceedings of the International Conference on Computer Vision (2003)Google Scholar
  16. 16.
    Faugeras, O., Keriven, R.: Variational principles, surface evolution, PDE’s, level set methods, and the stereo problem. IEEE Trans. Image Process. 7, 336–344 (1998)MathSciNetCrossRefzbMATHGoogle Scholar
  17. 17.
    Jin, H., Soatto, S., Yezzi, A.: Multi-view stereo reconstruction of dense shape and complex appearance. Int. J. Comput. Vis. 63, 175–189 (2005)CrossRefGoogle Scholar
  18. 18.
    Yu, T., Narendra, A., Chen, W.C.: SDG cut: 3D reconstruction of non-Lambertian objects using graph cuts on surface distance grid. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition (2006)Google Scholar
  19. 19.
    Ihrke, I., Goldluecke, B., Magnor, M.: Reconstructing the geometry of flowing water. In: Proceedings of the International Conference on Computer Vision, pp. 1055–1060 (2005)Google Scholar
  20. 20.
    Goldman, D., Curless, B., Hertzmann, A., Seitz, S.: Shape and spatially-varying BRDFs from photometric stereo. IEEE Trans. Pattern Anal. Mach. Intell. 32, 1060–1071 (2010)CrossRefGoogle Scholar
  21. 21.
    Vogiatzis, G., Hernandez, C., Cipolla, R.: Reconstruction in the round using photometric normals and silhouettes. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition (2006)Google Scholar
  22. 22.
    Kolev, K., Pock, T., Cremers, D.: Anisotropic minimal surfaces integrating photoconsistency and normal information for multiview stereo. In: Proceedings of the European Conference on Computer Vision (2010)Google Scholar
  23. 23.
    Cremers, D., Kolev, K.: Multiview stereo and silhouette consistency via convex functionals over convex domains. IEEE Trans. Pattern Anal. Mach. Intell. 33, 1161–1174 (2011)CrossRefGoogle Scholar
  24. 24.
    Federer, H.: Geometric Measure Theory. Springer, Heidelberg (1969)zbMATHGoogle Scholar
  25. 25.
    Zach, C., Pock, T., Bischof, H.: A globally optimal algorithm for robust TV-L1 range image integration. In: Proceedings of the International Conference on Computer Vision (2007)Google Scholar
  26. 26.
    Graber, G., Pock, T., Bischof, H.: Online 3D reconstruction using convex optimization. In: Proceedings of the International Conference on Computer Vision Workshops (2011)Google Scholar
  27. 27.
    Zickler, T., Belhumeur, P., Kriegman, D., Stereopsis, H.: Exploiting reciprocity for surface reconstruction. Int. J. Comput. Vis. 49, 215–227 (2002)CrossRefzbMATHGoogle Scholar
  28. 28.
    Dansereau, D., Pizarro, O., Williams, S.: Decoding, calibration and rectification for lenselet-based plenoptic cameras. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition, pp. 1027–1034 (2013)Google Scholar
  29. 29.
    Pock, T., Chambolle, A.: Diagonal preconditioning for first order primal-dual algorithms in convex optimization. In: International Conference on Computer Vision (ICCV 2011) (2011)Google Scholar
  30. 30.
    Chan, T., Esedoglu, S., Nikolova, M.: Algorithms for finding global minimizers of image segmentation and denoising models. SIAM J. Appl. Math. 66, 1632–1648 (2006)MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Ole Johannsen
    • 1
  • Antonin Sulc
    • 1
    Email author
  • Nico Marniok
    • 1
  • Bastian Goldluecke
    • 1
  1. 1.University of KonstanzKonstanzGermany

Personalised recommendations