Spatial and Angular Variational Super-Resolution of 4D Light Fields

  • Sven Wanner
  • Bastian Goldluecke
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7576)

Abstract

We present a variational framework to generate super-resolved novel views from 4D light field data sampled at low resolution, for example by a plenoptic camera. In contrast to previous work, we formulate the problem of view synthesis as a continuous inverse problem, which allows us to correctly take into account foreshortening effects caused by scene geometry transformations. High-accuracy depth maps for the input views are locally estimated using epipolar plane image analysis, which yields floating point depth precision without the need for expensive matching cost minimization. The disparity maps are further improved by increasing angular resolution with synthesized intermediate views. Minimization of the super-resolution model energy is performed with state of the art convex optimization algorithms within seconds.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Bolles, R., Baker, H., Marimont, D.: Epipolar-plane image analysis: An approach to determining structure from motion. International Journal of Computer Vision 1, 7–55 (1987) 1, 6CrossRefGoogle Scholar
  2. 2.
    Authors: -. In: Additional Material, reference_2.pdf (2012, under review) 1, 3, 6, 8, 9, 10, 11, 12 Google Scholar
  3. 3.
    Vaish, V., Wilburn, B., Joshi, N., Levoy, M.: Using plane + parallax for calibrating dense camera arrays. In: Proc. International Conference on Computer Vision and Pattern Recognition (2004) 1Google Scholar
  4. 4.
    Levoy, M., Hanrahan, P.: Light field rendering. In: Proc. SIGGRAPH, pp. 31–42 (1996) 1, 2, 8Google Scholar
  5. 5.
    Perwass, C., Wietzke, L.: The next generation of photography (2010) 1, www.raytrix.de
  6. 6.
    Ng, R., Levoy, M., Brédif, M., Duval, G., Horowitz, M., Hanrahan, P.: Light field photography with a hand-held plenoptic camera. Technical Report CSTR 2005-02, Stanford University (2005) 1Google Scholar
  7. 7.
    Bishop, T., Favaro, P.: Full-Resolution Depth Map Estimation from an Aliased Plenoptic Light Field. In: Kimmel, R., Klette, R., Sugimoto, A. (eds.) ACCV 2010, Part II. LNCS, vol. 6493, pp. 186–200. Springer, Heidelberg (2011) 1, 2CrossRefGoogle Scholar
  8. 8.
    Georgiev, T., Lumsdaine, A.: Focused plenoptic camera and rendering. Journal of Electronic Imaging 19, 021106 (2010) 1CrossRefGoogle Scholar
  9. 9.
    McMillan, L., Bishop, G.: Plenoptic modeling: An image-based rendering system. In: Proc. SIGGRAPH, pp. 39–46 (1995) 1, 2Google Scholar
  10. 10.
    Protter, M., Elad, M.: Super-resolution with probabilistic motion estimation. IEEE Trans. Image Processing 18, 1899–1904 (2009) 1, 2, 8MathSciNetCrossRefGoogle Scholar
  11. 11.
    Goldluecke, B., Cremers, D.: Superresolution texture maps for multiview reconstruction. In: Proc. ICCV (2009) 1, 2, 3, 5Google Scholar
  12. 12.
    Adelson, E., Bergen, J.: The plenoptic function and the elements of early vision. Computational Models of Visual Processing 1 (1991) 2Google Scholar
  13. 13.
    Gortler, S., Grzeszczuk, R., Szeliski, R., Cohen, M.: The Lumigraph. In: Proc. SIGGRAPH, pp. 43–54 (1996) 2Google Scholar
  14. 14.
    Shum, H., Chan, S., Kang, S.: Image-based rendering. Springer, New York (2007) 2Google Scholar
  15. 15.
    Kubota, A., Aizawa, K., Chen, T.: Reconstructing dense light field from array of multifocus images for novel view synthesis. IEEE Trans. Image Processing 16, 269–279 (2007) 2, 8MathSciNetCrossRefGoogle Scholar
  16. 16.
    Chai, J.X., Tong, X., Chany, S.C., Shum, H.Y.: Plenoptic sampling. In: Proc. SIGGRAPH, pp. 307–318 (2000) 2, 4Google Scholar
  17. 17.
    Siu, A., Lau, E.: Image registration for image-based rendering. IEEE Trans. Image Processing 14, 241–252 (2005) 2CrossRefGoogle Scholar
  18. 18.
    Geys, I., Koninckx, T.P., Gool, L.V.: Fast interpolated cameras by combining a GPU based plane sweep with a max-flow regularisation algorithm. In: 3DPVT, pp. 534–541 (2004) 2, 8Google Scholar
  19. 19.
    Baker, S., Kanade, T.: Limits on super-resolution and how to break them. IEEE Trans. on Pattern Analysis and Machine Intelligence 24, 1167–1183 (2002) 3, 4CrossRefGoogle Scholar
  20. 20.
    Beck, A., Teboulle, M.: Fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J. Imaging Sciences 2, 183–202 (2009) 8, 9MathSciNetMATHCrossRefGoogle Scholar
  21. 21.
    Bermùdez, A., Moreno, C.: Duality methods for solving variational inequalities. Comp. and Maths. with Appls. 7, 43–58 (1981) 9MATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Sven Wanner
    • 1
  • Bastian Goldluecke
    • 1
  1. 1.Heidelberg Collaboratory for Image ProcessingGermany

Personalised recommendations