Advertisement

Journal of Mathematical Imaging and Vision

, Volume 56, Issue 3, pp 573–590 | Cite as

Lightfield Recovery from Its Focal Stack

  • F. Pérez
  • A. Pérez
  • M. Rodríguez
  • E. Magdaleno
Article

Abstract

The Focal Stack Transform integrates a 4D lightfield over a set of appropriately chosen 2D planes. The result of such integration is an image focused on a determined depth in 3D space. The set of such images is the Focal Stack of the lightfield. This paper studies the existence of an inverse for this transform. Such inverse could be used to obtain a 4D lightfield from a set of images focused on several depths of the scene. In this paper, we show that this inversion cannot be obtained for a general lightfield and introduce a subset of lightfields where this inversion can be computed exactly. We examine the numerical properties of such inversion process for general lightfields and examine several regularization approaches to stabilize the transform. Experimental results are provided for focal stacks obtained from several plenoptic cameras. From a practical point of view, results show how this inversion procedure can be used to recover, compress, and denoise the original 4D lightfield.

Keywords

Lightfield Focal Stack Plenoptic Inverse problems Regularization 

Notes

Acknowledgments

The authors would like to thank R. Ng and Heidelberg University for lightfields that were used in the experimental results. This work has been partially supported by “Ayudas al Fomento de Nuevos Proyectos de Investigación” (Project 2013/0001339) of the University of La Laguna.

Supplementary material

10851_2016_658_MOESM1_ESM.docx (34 kb)
Supplementary material 1 (docx 34 KB)

References

  1. 1.
    Levoy, M.: Light fields and computational imaging. Computer 39(8), 46–55 (2006)CrossRefGoogle Scholar
  2. 2.
    Adelson, E.H., Bergen, J.R.: The Plenoptic Function and the Elements of Early Vision. MIT Press, Cambridge (1991)Google Scholar
  3. 3.
    Levoy, M., Hanrahan, P.: Light field rendering. In: Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques, New York, pp. 31–42 (1996)Google Scholar
  4. 4.
    Gortler, S.J., Grzeszczuk, R., Szeliski, R., Cohen, M.F.: The Lumigraph. In: Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques, New York, pp. 43–54 (1996)Google Scholar
  5. 5.
    Ng, R.: Fourier slice photography. In: ACM SIGGRAPH: Papers, New York, 2005, pp. 735–744 (2005)Google Scholar
  6. 6.
    Nava, F.P., Marichal-Hernández, J.G., Rodríguez-Ramos, J.M.: The discrete focal stack transform. In: Proceedings of European Signal Processing Conference, pp. 1–5 (2008)Google Scholar
  7. 7.
    Lumsdaine, A., Georgiev, T.: The focused plenoptic camera. In: IEEE International Conference on Computational Photography (ICCP), 2009, pp. 1–8 (2009)Google Scholar
  8. 8.
    Lüke, J.P., Pérez Nava, F., Marichal-Hernández, J.G., Rodríguez-Ramos, J.M., Rosa, F.: Near real-time estimation of super-resolved depth and all-in-focus images from a plenoptic camera using graphics processing units. Int. J. Digit. Multimed. Broadcast. 2010, e942037 (2009)Google Scholar
  9. 9.
    Uliyar, M., Putraya, G., Basavaraja, S.V.: Fast EPI based depth for plenoptic cameras. In: 20th IEEE International Conference on Image Processing, pp. 1–4 (2013)Google Scholar
  10. 10.
    Tosic, I., Berkner, K.: Light field scale-depth space transform for dense depth estimation. In: Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition, pp. 441–448 (2014)Google Scholar
  11. 11.
    Pérez, F., Pérez, A., Rodríguez, M., Magdaleno, E.: Super-resolved Fourier-slice refocusing in plenoptic cameras. J. Math. Imaging Vis. 52(2), 200–217 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
  12. 12.
    Georgiev, T., Chunev, G., Lumsdaine, A.: Superresolution with the focused plenoptic camera. Presented at the Proceedings of SPIE 7873, Computational Imaging IX, vol. 7873, p. 78730X–78730X-13 (2011)Google Scholar
  13. 13.
    Perez Nava, F.: Super-resolution in plenoptic cameras by the integration of depth from focus and stereo. In: Proceedings of 19th International Conference on Computer Communications and Networks, pp. 1–6 (2010)Google Scholar
  14. 14.
    Pérez Nava, F., Pérez Nava, A., Rodríguez Valido, M., Magdaleno Castellò, E.: Plenoptic cameras. In: Cristóbal, G., Perrinet, L., Keil, M.S. (eds.) Biologically Inspired Computer Vision, pp. 175–200. Wiley-VCH Verlag GmbH & Co, KGaA, Weinheim (2015)CrossRefGoogle Scholar
  15. 15.
    “Lytro.” https://www.lytro.com (2014). Accessed 04 Jul 2014
  16. 16.
    “Raytrix.” http://www.raytrix.de/ (2014). Accessed 04 Jul 2014
  17. 17.
    Pérez, F., Pérez, A., Rodríguez, M., Magdaleno, E.: A fast and memory-efficient Discrete Focal Stack Transform for plenoptic sensors. Digit. Signal Process. 38, 95–105 (2015)CrossRefGoogle Scholar
  18. 18.
    Stroebel, L.D. (ed.): Basic Photographic Materials and Processes, 2nd edn. Focal Press, Boston (2000)Google Scholar
  19. 19.
    Ferreira, P.J.S.G., Superresolution, the recovery of missing samples, and Vandermonde matrices on the unit circle. In: Proceedings of Workshop Sampling Theory Applications (SAMPTA’99) (1999)Google Scholar
  20. 20.
    Hansen, P.C.: Rank-deficient and discrete ill-posed problems: numerical aspects of linear inversion. SIAM, Philadelphia (1998)CrossRefGoogle Scholar
  21. 21.
    Bertero, M., Boccacci, P.: Introduction to inverse problems in imaging. Bristol, UK?. Institute of Physics Publication, Philadelphia (1998)CrossRefzbMATHGoogle Scholar
  22. 22.
    Reichel, L., Rodriguez, G.: Old and new parameter choice rules for discrete ill-posed problems. Numer. Algorithms 63(1), 65–87 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  23. 23.
    Rudin, L.I., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Phys. Nonlinear Phenom. 60(1–4), 259–268 (1992)MathSciNetCrossRefzbMATHGoogle Scholar
  24. 24.
    Afonso, M.V., Bioucas-Dias, J.M., Figueiredo, M.A.T.: Fast image recovery using variable splitting and constrained optimization. IEEE Trans. Image Process. 19(9), 2345–2356 (2010)MathSciNetCrossRefGoogle Scholar
  25. 25.
    Goldstein, T., Osher, S.: The split Bregman method for L1-regularized problems. SIAM J. Imaging Sci. 2(2), 323–343 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  26. 26.
    Yang, J., Yin, W., Zhang, Y., Wang, Y.: A fast algorithm for edge-preserving variational multichannel image restoration. SIAM J. Imaging Sci. 2(2), 569–592 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  27. 27.
    Dansereau, D.G., Pizarro, O., Williams, S.B.: Decoding, calibration and rectification for lenselet-based plenoptic cameras. Presented at the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2013, pp. 1027–1034 (2013)Google Scholar
  28. 28.
    Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)CrossRefGoogle Scholar
  29. 29.
    Shafer, S.A.: Using color to separate reflection components. Color Res. Appl. 10(4), 210–218 (1985)CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2016

Authors and Affiliations

  • F. Pérez
    • 1
  • A. Pérez
    • 1
  • M. Rodríguez
    • 2
  • E. Magdaleno
    • 2
  1. 1.Departamento de Estadística, Investigación Operativa y ComputaciónUniversity of La LagunaSan Cristóbal de La LagunaSpain
  2. 2.Departamento de Física Fundamental, Experimental, Electrónica y SistemasUniversity of La LagunaSan Cristóbal de La LagunaSpain

Personalised recommendations