Skip to main content
SpringerLink
Log in

Light Field Vision for Artificial Intelligence

  • Chapter
  • First Online:
Artificial Intelligence and Computer Vision

Part of the book series: Studies in Computational Intelligence ((SCI,volume 672 ))

Abstract

Light field camera has been available on the market, and the its capability of capturing both spatial and angular information makes it more powerful for solving computer vision problems. A newly developed Light Field Vision technique shows a big advantage over conventional computer vision techniques. We review the recent progress in Light Field Vision.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 119.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 159.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 159.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. D. Crevier, AI: The Tumultuous History of the Search for Artificial Intelligence (Basic Books, 1993)

    Google Scholar 

  2. http://www.mobvis.org/index.htm

  3. http://www.nvidia.com/object/drivepx.html

  4. E.H. Adelson, J.R. Bergen, The Plenoptic Function and the Elements of Early Vision (Vision and Modeling Group, Media Laboratory, Massachusetts Institute of Technology, 1991)

    Google Scholar 

  5. T. Georgiev, A. Lumsdaine, Focused plenoptic camera and rendering. J. Electron. Imaging 19(2), 021 106–021 106 (2010)

    Google Scholar 

  6. M. Levoy, P. Hanrahan, Light field rendering, in Proceedings of the ACM Conference on Computer Graphics (1996), pp. 31–42

    Google Scholar 

  7. I. Ihm, S. Park, R.K. Lee, Rendering of spherical light fields, in Proceedings of the Fifth Pacific Conference On Computer Graphics And Applications, ser. Pacific Graphics ’97 (1997), pp. 59–68

    Google Scholar 

  8. http://www.viewplus.co.jp/product/camera/profusion25.html

  9. http://www.lytro.com/

  10. http://www.raytrix.de/

  11. A. Shimada, H. Nagahara, R. ichiro Taniguchi, Object detection based on spatio-temporal light field sensing. IPSJ Trans. Comput. Vis. Appl. 5, 129–133 (2013)

    Google Scholar 

  12. S. Wanner, B. Goldluecke, Globally consistent depth labeling of 4d light fields, in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (IEEE, 2012), pp. 41–48

    Google Scholar 

  13. N. Li, J. Ye, Y. Ji, H. Ling, J. Yu, Saliency detection on light field, in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (IEEE, 2014), pp. 2806–2813

    Google Scholar 

  14. Y. Xu, H. Nagahara, A. Shimada, R. Taniguchi, Transcut: transparent object segmentation from a light-field image, in International Conference on Computer Vision (ICCV), 12 (2015)

    Google Scholar 

  15. http://cyberware.com/products/scanners/sphere.html

  16. T.A. Harvey, K.S. Bostwick, S. Marschner, Measuring spatially-and directionally-varying light scattering from biological material. J. Visualized Exp. JoVE (75) (2013)

    Google Scholar 

  17. M. Holroyd, J. Lawrence, T. Zickler, A coaxial optical scanner for synchronous acquisition of 3d geometry and surface reflectance. ACM Trans. Graph. 29(4), 99:1–99:12 (2010). http://doi.acm.org/10.1145/1778765.1778836

  18. http://lightfield.stanford.edu/acq.html

  19. A. Isaksen, L. McMillan, S.J. Gortler, Dynamically reparameterized light fields, in Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques (ACM Press/Addison-Wesley Publishing Co., 2000), pp. 297–306

    Google Scholar 

  20. B. Wilburn, N. Joshi, V. Vaish, E.-V.E. Talvala, E. Antunez, A. Barth, A. Adams, M. Levoy, M. Horowitz, High performance imaging using large camera arrays. ACM Trans. Graph. 24(3), 765–776 (2005)

    Article  Google Scholar 

  21. K. Venkataraman, D. Lelescu, J. Duparré, A. McMahon, G. Molina, P. Chatterjee, R. Mullis, S. Nayar, Picam: an ultra-thin high performance monolithic camera array. ACM Trans. Graph. 32(6), 166:1–166:13 (2013). http://doi.acm.org/10.1145/2508363.2508390

  22. C.-K. Liang, G. Liu, H.H. Chen, Light field acquisition using programmable aperture camera, in IEEE International Conference on Image Processing, vol. 5 (IEEE, 2007), pp. V–233

    Google Scholar 

  23. H. Nagahara, C. Zhou, T. Watanabe, H. Ishiguro, S.K. Nayar, Programmable aperture camera using lcos, in Computer Vision-ECCV 2010 (Springer, 2010), pp. 337–350

    Google Scholar 

  24. C.-K. Liang, T.-H. Lin, B.-Y. Wong, C. Liu, H.H. Chen, Programmable aperture photography: multiplexed light field acquisition. ACM Trans. Graph. (TOG) 27(3), 55 (2008)

    Article  Google Scholar 

  25. A. Veeraraghavan, R. Raskar, A. Agrawal, A. Mohan, J. Tumblin, Dappled photography: mask enhanced cameras for heterodyned light fields and coded aperture refocusing. ACM Trans. Graph. 26(3), 69 (2007). http://doi.acm.org/10.1145/1276377.1276463

  26. R. Ng, M. Levoy, M. Brédif, G. Duval, M. Horowitz, P. Hanrahan, Light field photography with a hand-held plenoptic camera. Comput. Sci. Tech. Rep. CSTR (2005)

    Google Scholar 

  27. A. Lumsdaine, T. Georgiev, The focused plenoptic camera, in IEEE International Conference on Computational Photography (ICCP) (2009), pp. 1–8

    Google Scholar 

  28. R.Y. Tsai, A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses. IEEE J. Robot. Autom. 3, 323–344 (1987)

    Article  Google Scholar 

  29. Z. Zhang, A flexible new technique for camera calibration. IEEE Trans. Pattern Anal. Mach. Intell. 22(11), 1330–1334 (2000)

    Article  Google Scholar 

  30. R. Horaud, G. Csurka, D. Demirdijian, Stereo calibration from rigid motions. IEEE Trans. Pattern Anal. Mach. Intell. 22(12), 1446–1452 (2000)

    Article  Google Scholar 

  31. H. Malm, A. Heyden, Stereo head calibration from a planar object, in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), vol. 2 (2001), pp. II–657

    Google Scholar 

  32. V. Vaish, B. Wilburn, N. Joshi, M. Levoy, Using plane + parallax for calibrating dense camera arrays, in CVPR (1) (2004), pp. 2–9

    Google Scholar 

  33. T. Svoboda, D. Martinec, T. Pajdla, A convenient multi-camera self-calibration for virtual environments. PRESENCE: Teleoperators Virtual Environ. 14(4), 407–422 (2005)

    Article  Google Scholar 

  34. T. Ueshiba, F. Tomita, Plane-based calibration algorithm for multi-camera systems via factorization of homography matrices, in ICCV (2003), pp. 966–973

    Google Scholar 

  35. Y. Xu, K. Maeno, H. Nagahara, R. Taniguchi, Mobile camera array calibration for light field acquisition, in International Conference on Quality Control by Artificial Vision (QCAV), vol. 5 (2013), pp. 283–290

    Google Scholar 

  36. R. Hartley, A. Zisserman, Multiple View Geometry in Computer Vision (Cambridge University Press, Cambridge, 2004)

    Book  MATH  Google Scholar 

  37. N. Snavely, S.M. Seitz, R. Szeliski, Modeling the world from internet photo collections. Int. J. Comput. Vis. 80, 189–210 (2008)

    Article  Google Scholar 

  38. D.G. Dansereau, O. Pizarro, S.B. Williams, Decoding, calibration and rectification for lenselet-based plenoptic cameras, in CVPR (2013), pp. 1027–1034

    Google Scholar 

  39. D. Cho, M. Lee, S. Kim, Y.-W. Tai, Modeling the calibration pipeline of the lytro camera for high quality light-field image reconstruction, in ICCV (2013)

    Google Scholar 

  40. O. Johannsen, C. Heinze, B. Goldluecke, C. Perwaß, On the calibration of focused plenoptic cameras, in Time-of-Flight and Depth Imaging. Sensors, Algorithms, and Applications (Springer, 2013), pp. 302–317

    Google Scholar 

  41. N. Zeller, F. Quint, U. Stilla, Calibration and accuracy analysis of a focused plenoptic camera. ISPRS Ann. Photogrammetry, Remote Sens. Spat. Inf. Sci. 2(3), 205 (2014)

    Article  Google Scholar 

  42. Y. Bok, H.-G. Jeon, I.S. Kweon, Geometric calibration of micro-lens-based light-field cameras using line features, in Proceedings of European Conference on Computer Vision (ECCV) (2014)

    Google Scholar 

  43. K.H. Strobl, M. Lingenauber, Stepwise calibration of focused plenoptic cameras, Comput. Vis. Image Underst. 145(C), 140–147 (2016). http://dx.doi.org/10.1016/j.cviu.2015.12.010

  44. N. Joshi, Color calibration for arrays of inexpensive image sensors. Tech. Rep. CSTR 2004–02 (2004)

    Google Scholar 

  45. A. Ilie, G. Welch, Ensuring color consistency across multiple cameras, in IEEE International Conference on Computer Vision (ICCV’05), vol. 2 (IEEE, 2005), pp. 1268–1275

    Google Scholar 

  46. Y. Chen, C. Cai, J. Liu, Yuv correction for multi-view video compression, in 18th International Conference on Pattern Recognition (ICPR’06), vol. 3 (IEEE, 2006), pp. 734–737

    Google Scholar 

  47. S.A. Fezza, M.-C. Larabi, K.M. Faraoun, Feature-based color correction of multiview video for coding and rendering enhancement. IEEE Trans. Circuits Syst. Video Technol. 24(9), 1486–1498 (2014)

    Article  Google Scholar 

  48. D.G. Lowe, Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)

    Article  Google Scholar 

  49. U. Fecker, M. Barkowsky, A. Kaup, Histogram-based prefiltering for luminance and chrominance compensation of multiview video. IEEE Trans. Circuits Syst. Video Technol. 18(9), 1258–1267 (2008)

    Article  Google Scholar 

  50. S.A. Fezza, M.-C. Larabi, Color correction for stereo and multi-view coding, in Color Image and Video Enhancement (Springer, 2015), pp. 291–314

    Google Scholar 

  51. R. Ng, Fourier slice photography. ACM Trans. Graph. (TOG) 24(3), 735–744 (2005)

    Article  Google Scholar 

  52. V. Vaish, M. Levoy, R. Szeliski, C.L. Zitnick, S.B. Kang, Reconstructing occluded surfaces using synthetic apertures: Stereo, focus and robust measures, in CVPR (2006), pp. 2331–2338

    Google Scholar 

  53. H. Lin, C. Chen, S. Bing Kang, J. Yu, Depth recovery from light field using focal stack symmetry, in Proceedings of the IEEE International Conference on Computer Vision (2015), pp. 3451–3459

    Google Scholar 

  54. T.E. Bishop, S. Zanetti, P. Favaro, Light field superresolution, in IEEE International Conference on Computational Photography (ICCP) (IEEE, 2009), pp. 1–9

    Google Scholar 

  55. T. Bishop, P. Favaro, The light field camera: extended depth of field, aliasing, and superresolution. IEEE Trans. Pattern Anal. Mach. Intell. 34(5), 972–986, 5 (2012). iNSPEC Accession Number: 12617601

    Google Scholar 

  56. T. Georgiev, G. Chunev, A. Lumsdaine, Superresolution with the focused plenoptic camera, in IS&T/SPIE Electronic Imaging (International Society for Optics and Photonics, 2011), pp. 78 730X–78 730X

    Google Scholar 

  57. Z. Yu, J. Yu, A. Lumsdaine, T. Georgiev, An analysis of color demosaicing in plenoptic cameras, in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (IEEE, 2012), pp. 901–908

    Google Scholar 

  58. S. Wanner, B. Goldluecke, Variational light field analysis for disparity estimation and super-resolution. IEEE Trans. Pattern Anal. Mach. Intell. (2014)

    Google Scholar 

  59. A. Levin, F. Durand, Linear view synthesis using a dimensionality gap light field prior, in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (IEEE, 2010), pp. 1831–1838

    Google Scholar 

  60. B.M. Smith, L. Zhang, H. Jin, A. Agarwala, Light field video stabilization, in 12th international conference on computer vision (IEEE, 2009), pp. 341–348

    Google Scholar 

  61. Y. Yoon, H.-G. Jeon, D. Yoo, J.-Y. Lee, I. So Kweon, Learning a deep convolutional network for light-field image super-resolution, in Proceedings of the IEEE International Conference on Computer Vision Workshops (2015), pp. 24–32

    Google Scholar 

  62. D.G. Dansereau, D.L. Bongiorno, O. Pizarro, S.B. Williams, Light field image denoising using a linear 4d frequency-hyperfan all-in-focus filter, in Proceedings of the SPIE Conference on Computational Imaging (SPIE), vol. 8657 (2013)

    Google Scholar 

  63. D.G. Dansereau, O. Pizarro, S.B. Williams, Linear volumetric focus for light field cameras. ACM Trans. Graph. (TOG) 34(2), 15 (2015)

    Article  Google Scholar 

  64. M.W. Tao, S. Hadap, J. Malik, R. Ramamoorthi, Depth from combining defocus and correspondence using light-field cameras, in Proceedings of the IEEE International Conference on Computer Vision (2013), pp. 673–680

    Google Scholar 

  65. M.W. Tao, T.-C. Wang, J. Malik, R. Ramamoorthi, Depth estimation for glossy surfaces with light-field cameras, in Workshop on Light Fields for Computer Vision, ECCV (2014)

    Google Scholar 

  66. M.W. Tao, P.P. Srinivasan, J. Malik, S. Rusinkiewicz, R. Ramamoorthi, Depth from shading, defocus, and correspondence using light-field angular coherence, in 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (IEEE, 2015), pp. 1940–1948

    Google Scholar 

  67. H.-G. Jeon, J. Park, G. Choe, J. Park, Y. Bok, Y.-W. Tai, I.S. Kweon, Accurate depth map estimation from a lenslet light field camera, in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (IEEE, 2015), pp. 1547–1555

    Google Scholar 

  68. T.-C. Wang, A.A. Efros, R. Ramamoorthi, Occlusion-aware depth estimation using light-field cameras, in Proceedings of the IEEE International Conference on Computer Vision (2015), pp. 3487–3495

    Google Scholar 

  69. W. Williem, I. Kyu Park, Robust light field depth estimation for noisy scene with occlusion, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2016), pp. 4396–4404

    Google Scholar 

  70. T.-C. Wang, M. Chandraker, A. Efros, R. Ramamoorthi, Svbrdf-invariant shape and reflectance estimation from light-field cameras, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016)

    Google Scholar 

  71. S. Heber, T. Pock, Shape from light field meets robust pca, in European Conference on Computer Vision (Springer, 2014), pp. 751–767

    Google Scholar 

  72. S. Heber, T. Pock, Convolutional networks for shape from light field, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2016), pp. 3746–3754

    Google Scholar 

  73. J. Zhang, M. Wang, J. Gao, Y. Wang, X. Zhang, X. Wu, Saliency detection with a deeper investigation of light field, in Proceedings of the 24th International Joint Conference on Artificial Intelligence (2015), pp. 2212–2218

    Google Scholar 

  74. X. Zhang, Y. Wang, J. Zhang, L. Hu, M. Wang, Light field saliency vs. 2d saliency: a comparative study. Neurocomputing 166, 389–396 (2015)

    Article  Google Scholar 

  75. A. Shimada, H. Nagahara, R.-I. Taniguchi, Background light ray modeling for change detection. J. Vis. Commun. Image Representation 38, 55–64 (2016)

    Article  Google Scholar 

  76. A. Shimada, H. Nagahara, R.-I. Taniguchi, Change detection on light field for active video surveillance, in 12th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS) (IEEE, 2015), pp. 1–6

    Google Scholar 

  77. D.G. Dansereau, S.B. Williams, P.I. Corke, Simple change detection from mobile light field cameras. Comput. Vis. Image Underst. 145, 160–171 (2016)

    Article  Google Scholar 

  78. D.G. Dansereau, I. Mahon, O. Pizarro, S.B. Williams, Plenoptic flow: closed-form visual odometry for light field cameras, in 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems (IEEE, 2011), pp. 4455–4462

    Google Scholar 

  79. Y. Xu, K. Maeno, H. Nagahara, A. Shimada, R. Taniguchi, Light field distortion feature for transparent object classification. Comput. Vis. Image Underst. (2015)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Creative Media, City University of Hong Kong, Kowloon Tong, Hong Kong

    Yichao Xu & Miu-ling Lam

Authors
  1. Yichao Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Miu-ling Lam
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yichao Xu .

Editor information

Editors and Affiliations

  1. Department of Electrical Engineering and Electronics, Kyushu Institute of Technology, Fukuoka, Japan

    Huimin Lu

  2. School of Information Engineering, Yangzhou University, Yangzhou, China

    Yujie Li

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Xu, Y., Lam, Ml. (2017). Light Field Vision for Artificial Intelligence. In: Lu, H., Li, Y. (eds) Artificial Intelligence and Computer Vision. Studies in Computational Intelligence, vol 672 . Springer, Cham. https://doi.org/10.1007/978-3-319-46245-5_11

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-46245-5_11

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-46244-8

  • Online ISBN: 978-3-319-46245-5

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation