Skip to main content

Efficient Non-Line-of-Sight Imaging from Transient Sinograms

  • Conference paper
  • First Online:
Computer Vision – ECCV 2020 (ECCV 2020)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 12352))

Included in the following conference series:

Abstract

Non-line-of-sight (NLOS) imaging techniques use light that diffusely reflects off of visible surfaces (e.g., walls) to see around corners. One approach involves using pulsed lasers and ultrafast sensors to measure the travel time of multiply scattered light. Unlike existing NLOS techniques that generally require densely raster scanning points across the entirety of a relay wall, we explore a more efficient form of NLOS scanning that reduces both acquisition times and computational requirements. We propose a circular and confocal non-line-of-sight (\(\text {C}^2\text {NLOS}\)) scan that involves illuminating and imaging a common point, and scanning this point in a circular path along a wall. We observe that (1) these \(\text {C}^2\text {NLOS}\) measurements consist of a superposition of sinusoids, which we refer to as a transient sinogram, (2) there exists computationally efficient reconstruction procedures that transform these sinusoidal measurements into 3D positions of hidden scatterers or NLOS images of hidden objects, and (3) despite operating on an order of magnitude fewer measurements than previous approaches, these \(\text {C}^2\text {NLOS}\) scans provide sufficient information about the hidden scene to solve these different NLOS imaging tasks. We show results from both simulated and real \(\text {C}^2\text {NLOS}\) scans (Project page: https://marikoisogawa.github.io/project/c2nlos).

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Adib, F., Hsu, C.Y., Mao, H., Katabi, D., Durand, F.: Capturing the human figure through a wall. ACM Trans. Graph. (TOG) 34(6), 1–13 (2015)

    Article  Google Scholar 

  2. Ahn, B., Dave, A., Veeraraghavan, A., Gkioulekas, I., Sankaranarayanan, A.C.: Convolutional approximations to the general non-line-of-sight imaging operator. In: IEEE International Conference on Computer Vision (ICCV), pp. 7888–7898 (2019)

    Google Scholar 

  3. Ballard, D.H.: Generalizing the Hough transform to fetect arbitrary shapes. In: Readings in Computer Vision, pp. 714–725 (1987)

    Google Scholar 

  4. Bouman, K.L., et al.: Turning corners into cameras: principles and methods. In: IEEE International Conference on Computer Vision (ICCV), pp. 2289–2297 (2017)

    Google Scholar 

  5. Boyd, S., Parikh, N., Chu, E., Peleato, B., Eckstein, J.: Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends Mach. Learn. 3(1), 1–122 (2011)

    Article  Google Scholar 

  6. Buttafava, M., Zeman, J., Tosi, A., Eliceiri, K., Velten, A.: Non-line-of-sight imaging using a time-gated single photon avalanche diode. Opt. Express 23(16), 20997–21011 (2015)

    Article  Google Scholar 

  7. Chan, S., Warburton, R.E., Gariepy, G., Leach, J., Faccio, D.: Non-line-of-sight tracking of people at long range. Opt. Express 25(9), 10109–10117 (2017)

    Article  Google Scholar 

  8. Chandran, S., Jayasuriya, S.: Adaptive lighting for data-driven non-line-of-sight 3D localization and object identification. In: The British Machine Vision Conference (BMVC) (2019)

    Google Scholar 

  9. Galindo, M., Marco, J., O’Toole, M., Wetzstein, G., Gutierrez, D., Jarabo, A.: A dataset for benchmarking time-resolved non-line-of-sight imaging. In: ACM SIGGRAPH 2019 Posters, pp. 1–2 (2019)

    Google Scholar 

  10. Gupta, O., Willwacher, T., Velten, A., Veeraraghavan, A., Raskar, R.: Reconstruction of hidden 3D shapes using diffuse reflections. Opt. Express 20(17), 19096–19108 (2012)

    Article  Google Scholar 

  11. Heide, F., Heidrich, W., Hullin, M.B.: Diffuse mirrors: 3D reconstruction from diffuse indirect illumination using inexpensive time-of-flight sens. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3222–3229 (2014)

    Google Scholar 

  12. Heide, F., O’Toole, M., Zang, K., Lindell, D.B., Diamond, S., Wetzstein, G.: Non-line-of-sight imaging with partial occluders and surface normals. ACM Trans. Graph. 38(3), 1–10 (2019)

    Article  Google Scholar 

  13. Isogawa, M., Yuan, Y., O’Toole, M., Kitani, K.M.: Optical non-line-of-sight physics-based 3D human pose estimation. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 7013–7022 (2020)

    Google Scholar 

  14. Jarabo, A., Marco, J., Muñoz, A., Buisan, R., Jarosz, W., Gutierrez, D.: A framework for transient rendering. ACM Trans. Graph. (ToG) 33(6), 1–10 (2014)

    Article  Google Scholar 

  15. Kak, A.C., Slaney, M., Wang, G.: Principles of computerized tomographic imaging. Med. Phys. 29(1), 107–107 (2002)

    Article  Google Scholar 

  16. Kirmani, A., Hutchison, T., Davis, J., Raskar, R.: Looking around the corner using transient imaging. In: IEEE International Conference on Computer Vision (ICCV), pp. 159–166 (2009)

    Google Scholar 

  17. Klein, J., Peters, C., Laurenzis, M., Hullin, M.: Tracking objects outside the line of sight using 2D intensity images. Sci. Rep. 6(32491), 1–9 (2016)

    Google Scholar 

  18. Klein, J., Peters, C., Laurenzis, M., Hullin, M.: Non-line-of-sight MoCap. In: ACM SIGGRAPH Emerging Technologies, pp. 18:1–18:2 (2017)

    Google Scholar 

  19. La Manna, M., Kine, F., Breitbach, E., Jackson, J., Sultan, T., Velten, A.: Error backprojection algorithms for non-line-of-sight imaging. IEEE Trans. Pattern Anal. Mach. Intell. 41(7), 1615–1626 (2019)

    Article  Google Scholar 

  20. Li, T., Fan, L., Zhao, M., Liu, Y., Katabi, D.: Making the invisible visible: action recognition through walls and occlusions. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 872–881 (2019)

    Google Scholar 

  21. Lindell, D.B., Wetzstein, G., Koltun, V.: Acoustic non-line-of-sight imaging. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6780–6789 (2019)

    Google Scholar 

  22. Lindell, D.B., Wetzstein, G., O’Toole, M.: Wave-based non-line-of-sight imaging using fast F-K migration. ACM Trans. Graph. (TOG) 38(4), 116 (2019)

    Article  Google Scholar 

  23. Maeda, T., Satat, G., Swedish, T., Sinha, L., Raskar, R.: Recent advances in imaging around corners. In: arXiv (2019)

    Google Scholar 

  24. Maeda, T., Wang, Y., Raskar, R., Kadambi, A.: Thermal non-line-of-sight imaging. In: IEEE International Conference on Computational Photography (ICCP), pp. 1–11 (2019)

    Google Scholar 

  25. Metzler, C.A., Lindell, D.B., Wetzstein, G.: Keyhole imaging: non-line-of-sight imaging and tracking of moving objects along a single optical path at long standoff distances. In: arXiv (2019)

    Google Scholar 

  26. O’Toole, M., Heide, F., Lindell, D.B., Zang, K., Diamond, S., Wetzstein, G.: Reconstructing transient images from single-photon sensors. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2289–2297 (2017)

    Google Scholar 

  27. O’Toole, M., Lindell, D.B., Wetzstein, G.: Confocal non-line-of-sight imaging based on the light-cone transform. Nature 555(7696), 338 (2018)

    Article  Google Scholar 

  28. Pediredla, A., Dave, A., Veeraraghavan, A.: SNLOS: Non-line-of-sight scanning through temporal focusing. In: IEEE International Conference on Computational Photography (ICCP), pp. 1–13 (2019)

    Google Scholar 

  29. Redo-Sanchez, A., et al.: Terahertz time-gated spectral imaging for content extraction through layered structures. Nat. Commun. 7, 12665 (2016)

    Article  Google Scholar 

  30. Saunders, C., Murray-Bruce, J., Goyal, V.K.: Computational periscopy with an ordinary digital camera. Nature 565(7740), 472–475 (2019)

    Article  Google Scholar 

  31. Tancik, M., Satat, G., Raskar, R.: Flash photography for data-driven hidden scene recovery. In: arXiv (2018)

    Google Scholar 

  32. Tsai, C.Y., Sankaranarayanan, A.C., Gkioulekas, I.: Beyond volumetric albedo - a surface optimization framework for non-line-of-sight imaging. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2019)

    Google Scholar 

  33. Velten, A., Willwacher, T., Gupta, O., Veeraraghavan, A., Bawendi, M.G., Raskar, R.: Recovering three-dimensional shape around a corner using ultrafast time-of-flight imaging. Nat. Commun. 3, 745 (2012)

    Article  Google Scholar 

  34. Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)

    Article  Google Scholar 

  35. Xin, S., Nousias, S., Kutulakos, K.N., Sankaranarayanan, A.C., Narasimhan, S.G., Gkioulekas, I.: A theory of Fermat paths for non-line-of-sight shape reconstruction. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2019)

    Google Scholar 

  36. Zou, C.C., Ge, S.: A Hough transform-based method for fast detection of fixed period sinusoidal curves in images. In: International Conference on Signal Processing (ICSP), vol. 1, pp. 909–912 (2002)

    Google Scholar 

Download references

Acknowledgements

We thank Ioannis Gkioulekas for helpful discussions and feedback on this work. M. Isogawa is supported by NTT Corporation. M. O’Toole is supported by the DARPA REVEAL program.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Mariko Isogawa .

Editor information

Editors and Affiliations

1 Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (mp4 42570 KB)

Supplementary material 2 (pdf 7782 KB)

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Isogawa, M., Chan, D., Yuan, Y., Kitani, K., O’Toole, M. (2020). Efficient Non-Line-of-Sight Imaging from Transient Sinograms. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, JM. (eds) Computer Vision – ECCV 2020. ECCV 2020. Lecture Notes in Computer Science(), vol 12352. Springer, Cham. https://doi.org/10.1007/978-3-030-58571-6_12

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-58571-6_12

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-58570-9

  • Online ISBN: 978-3-030-58571-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics