Skip to main content
SpringerLink
Log in

Diffraction Line Imaging

  • Conference paper
  • First Online:
Book cover Computer Vision – ECCV 2020 (ECCV 2020)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 12347))

Included in the following conference series:

Abstract

We present a novel computational imaging principle that combines diffractive optics with line (1D) sensing. When light passes through a diffraction grating, it disperses as a function of wavelength. We exploit this principle to recover 2D and even 3D positions from only line images. We derive a detailed image formation model and a learning-based algorithm for 2D position estimation. We show several extensions of our system to improve the accuracy of the 2D positioning and expand the effective field of view. We demonstrate our approach in two applications: (a) fast passive imaging of sparse light sources like street lamps, headlights at night and LED-based motion capture, and (b) structured light 3D scanning with line illumination and line sensing. Line imaging has several advantages over 2D sensors: high frame rate, high dynamic range, high fill-factor with additional on-chip computation, low cost beyond the visible spectrum, and high energy efficiency when used with line illumination. Thus, our system is able to achieve high-speed and high-accuracy 2D positioning of light sources and 3D scanning of scenes.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Event-based cameras known as dynamic vision sensors [8, 14] output changes in intensity on a per-pixel basis, but the prototype sensors have limited spatial resolution.

  2. 2.

    Eq. (1) assumes a collimated incident beam and an identical index of refraction for the medium on both sides of the grating.

  3. 3.

    Equation (1) strictly holds when the grating groves are perpendicular to the incidence plane. The incidence plane is the plane containing the beam direction and the normal to diffraction grating plane.

  4. 4.

    Normalizing a \(8 \times 9\) consists of dividing all RGB pixel values by a scalar. For example, for \(L^{\infty }\), \({\bar{\mathbf{I}}} [\varOmega (y_m)] = \mathbf{I}[\varOmega (y_m)]/\mathrm{max_{RGB}}\), where \(\mathrm{max_{RGB}}\) is the maximum value across all patch pixels and color channels.

References

  1. Alfano, R.R.: The Supercontinuum Laser Source. Springer, Heidelberg (1989). https://doi.org/10.1007/b106776

    Book  Google Scholar 

  2. Antipa, N.: Diffusercam: lensless single-exposure 3D imaging. Optica 5(1), 1–9 (2018)

    Article  Google Scholar 

  3. Antipa, N., Oare, P., Bostan, E., Ng, R., Waller, L.: Video from stills: lensless imaging with rolling shutter. In: Proceedings of IEEE ICCP, pp. 1–8 (2019)

    Google Scholar 

  4. Chen, Y.L., Wu, B.F., Huang, H.Y., Fan, C.J.: A real-time vision system for nighttime vehicle detection and traffic surveillance. IEEE Trans. Ind. Elect. 58(5), 2030–2044 (2010)

    Article  Google Scholar 

  5. Loadscan: load management solutions (2020). https://www.loadscan.com/

  6. Curless, B., Levoy, M.: Better optical triangulation through spacetime analysis. In: Proceedings of IEEE ICCV, pp. 987–994 (1995)

    Google Scholar 

  7. Dlis2k: ultra configurable digital output (2020). http://dynamax-imaging.com/products/line-scan-product/dlis2k-2/

  8. Gallego, G., et al.: Event-based vision: a survey (2019). arXiv preprint arXiv:1904.08405

  9. Hartley, R., Zisserman, A.: Multiple View Geometry in Computer Vision. Cambridge University Press, Cambridge (2003)

    MATH  Google Scholar 

  10. Harvey, J.E., Vernold, C.L.: Description of diffraction grating behavior in direction cosine space. Appl. Opt. 37(34), 8158–8159 (1998)

    Article  Google Scholar 

  11. Hossain, F., PK, M.K., Yousuf, M.A.: Hardware design and implementation of adaptive canny edge detection algorithm. Int. J. Comput. Appl. 124(9), 31–38 (2015)

    Google Scholar 

  12. Huber, P.J.: Robust estimation of a location parameter. In: Kotz, S., Johnson, N.L. (eds.) Breakthroughs in statistics. Springer Series in Statistics (Perspectives in Statistics), pp. 492–518. Springer, Heidelberg (1992). https://doi.org/10.1007/978-1-4612-4380-9_35

    Chapter  Google Scholar 

  13. Jeon, D.S., et al.: Compact snapshot hyperspectral imaging with diffracted rotation. ACM TOG 38(4), 117 (2019)

    Article  Google Scholar 

  14. Kim, H., Leutenegger, S., Davison, A.J.: Real-time 3D reconstruction and 6-DoF tracking with an event camera. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9910, pp. 349–364. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46466-4_21

    Chapter  Google Scholar 

  15. Kim, J., Han, G., Lim, H., Izadi, S., Ghosh, A.: Thirdlight: Low-cost and high-speed 3D interaction using photosensor markers. In: Proceedingd of CVMP, p. 4. ACM (2017)

    Google Scholar 

  16. Liu, D., Geng, H., Liu, T., Klette, R.: Star-effect simulation for photography. Comput. Graph. 61, 19–28 (2016)

    Article  Google Scholar 

  17. Loewen, E.G., Popov, E.: Diffraction Gratings and Applications. CRC Press, Boca Raton (2018)

    Book  Google Scholar 

  18. Nagaoka, H., Mishima, T.: A combination of a concave grating with a Lummer-Gehrcke plate or an echelon grating for examining fine structure of spectral lines. Astrophys. J. 57, 92 (1923)

    Article  Google Scholar 

  19. Narasimhan, S.G., Nayar, S.K.: Shedding light on the weather. In: Proceedings of IEEE CVPR, pp. 665–672 (2003)

    Google Scholar 

  20. Nayar, S.K., Krishnan, G., Grossberg, M.D., Raskar, R.: Fast separation of direct and global components of a scene using high frequency illumination. In: ACM SIGGRAPH, pp. 935–944 (2006)

    Google Scholar 

  21. Nelson, P., Churchill, W., Posner, I., Newman, P.: From dusk till dawn: localisation at night using artificial light sources. In: Proceedings of IEEE ICRA (2015)

    Google Scholar 

  22. Optotrak certus (2020). https://www.ndigital.com/msci/products/optotrak-certus/

  23. O’Toole, M., Achar, S., Narasimhan, S.G., Kutulakos, K.N.: Homogeneous codes for energy-efficient illumination and imaging. ACM TOG 34(4), 1–13 (2015)

    Article  Google Scholar 

  24. Phase space inc. (2020). http://www.phasespace.com/

  25. Raskar, R., et al.: Prakash: lighting aware motion capture using photosensing markers and multiplexed illuminators. ACM TOG 26(3), 36 (2007)

    Article  Google Scholar 

  26. Saragadam, V., Sankaranarayanan, A.C.: KRISM: Krylov subspace-based optical computing of hyperspectral images. ACM TOG 38(5), 1–14 (2019)

    Article  Google Scholar 

  27. Sheinin, M., Schechner, Y.Y., Kutulakos, K.N.: Computational imaging on the electric grid. In: Proceedings of IEEE CVPR, pp. 2363–2372 (2017)

    Google Scholar 

  28. Sheinin, M., Schechner, Y.Y., Kutulakos, K.N.: Rolling shutter imaging on the electric grid. In: Proceedings of IEEE ICCP, pp. 1–12 (2018)

    Google Scholar 

  29. Tamburo, R., et al.: Programmable automotive headlights. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8692, pp. 750–765. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10593-2_49

    Chapter  Google Scholar 

  30. Vasilyev, A.: The optoelectronic swept-frequency laser and its applications in ranging, three-dimensional imaging, and coherent beam combining of chirped-seed amplifiers. Ph.D. thesis, Caltech (2013)

    Google Scholar 

  31. Vogt, S.S., et al.: HIRES: the high-resolution echelle spectrometer on the Keck 10-m Telescope. In: Instrumentation in Astronomy VIII, vol. 2198, pp. 362–375. International Society for Optics and Photonics (1994)

    Google Scholar 

  32. Wang, J., Gupta, M., Sankaranarayanan, A.C.: Lisens-a scalable architecture for video compressive sensing. In: Proceedings of IEEE ICCP, pp. 1–9. IEEE (2015)

    Google Scholar 

  33. Wang, J., Sankaranarayanan, A.C., Gupta, M., Narasimhan, S.G.: Dual structured light 3D using a 1D sensor. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9910, pp. 383–398. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46466-4_23

    Chapter  Google Scholar 

  34. Weinberg, G., Katz, O.: 100,000 frames-per-second compressive imaging with a conventional rolling-shutter camera by random point-spread-function engineering. arXiv preprint arXiv:2004.09614 (2020)

  35. Xiong, J., et al.: Rainbow particle imaging velocimetry for dense 3D fluid velocity imaging. ACM TOG 36(4), 36 (2017)

    Article  Google Scholar 

  36. Zhi, T., Pires, B.R., Hebert, M., Narasimhan, S.G.: Deep material-aware cross-spectral stereo matching. In: Proceedings of IEEE CVPR, pp. 1916–1925 (2018)

    Google Scholar 

Download references

Acknowledgments

We thank A. Sankaranarayanan and V. Saragadam for help with building the hardware prototype and S. Panev and F. Moreno for neural network-related advice. We were supported in parts by NSF Grants IIS-1900821 and CCF-1730147 and DARPA REVEAL Contract HR0011-16-C-0025.

Author information

Authors and Affiliations

  1. Carnegie Mellon University, Pittsburgh, PA, 15213, USA

    Mark Sheinin, Dinesh N. Reddy, Matthew O’Toole & Srinivasa G. Narasimhan

Authors
  1. Mark Sheinin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dinesh N. Reddy
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Matthew O’Toole
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Srinivasa G. Narasimhan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mark Sheinin .

Editor information

Editors and Affiliations

  1. University of Oxford, Oxford, UK

    Andrea Vedaldi

  2. Graz University of Technology, Graz, Austria

    Horst Bischof

  3. University of Freiburg, Freiburg im Breisgau, Germany

    Thomas Brox

  4. University of North Carolina at Chapel Hill, Chapel Hill, NC, USA

    Jan-Michael Frahm

1 Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 2 (mp4 87691 KB)

Supplementary material 1 (pdf 2577 KB)

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Sheinin, M., Reddy, D.N., O’Toole, M., Narasimhan, S.G. (2020). Diffraction Line Imaging. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, JM. (eds) Computer Vision – ECCV 2020. ECCV 2020. Lecture Notes in Computer Science(), vol 12347. Springer, Cham. https://doi.org/10.1007/978-3-030-58536-5_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-58536-5_1

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-58535-8

  • Online ISBN: 978-3-030-58536-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation