Advertisement

Efficient Non-Line-of-Sight Imaging from Transient Sinograms

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 12352)

Abstract

Non-line-of-sight (NLOS) imaging techniques use light that diffusely reflects off of visible surfaces (e.g., walls) to see around corners. One approach involves using pulsed lasers and ultrafast sensors to measure the travel time of multiply scattered light. Unlike existing NLOS techniques that generally require densely raster scanning points across the entirety of a relay wall, we explore a more efficient form of NLOS scanning that reduces both acquisition times and computational requirements. We propose a circular and confocal non-line-of-sight (\(\text {C}^2\text {NLOS}\)) scan that involves illuminating and imaging a common point, and scanning this point in a circular path along a wall. We observe that (1) these \(\text {C}^2\text {NLOS}\) measurements consist of a superposition of sinusoids, which we refer to as a transient sinogram, (2) there exists computationally efficient reconstruction procedures that transform these sinusoidal measurements into 3D positions of hidden scatterers or NLOS images of hidden objects, and (3) despite operating on an order of magnitude fewer measurements than previous approaches, these \(\text {C}^2\text {NLOS}\) scans provide sufficient information about the hidden scene to solve these different NLOS imaging tasks. We show results from both simulated and real \(\text {C}^2\text {NLOS}\) scans (Project page: https://marikoisogawa.github.io/project/c2nlos).

Keywords

Computational imaging Non-line-of-sight imaging 

Notes

Acknowledgements

We thank Ioannis Gkioulekas for helpful discussions and feedback on this work. M. Isogawa is supported by NTT Corporation. M. O’Toole is supported by the DARPA REVEAL program.

Supplementary material

Supplementary material 1 (mp4 42570 KB)

504444_1_En_12_MOESM2_ESM.pdf (7.6 mb)
Supplementary material 2 (pdf 7782 KB)

References

  1. 1.
    Adib, F., Hsu, C.Y., Mao, H., Katabi, D., Durand, F.: Capturing the human figure through a wall. ACM Trans. Graph. (TOG) 34(6), 1–13 (2015)CrossRefGoogle Scholar
  2. 2.
    Ahn, B., Dave, A., Veeraraghavan, A., Gkioulekas, I., Sankaranarayanan, A.C.: Convolutional approximations to the general non-line-of-sight imaging operator. In: IEEE International Conference on Computer Vision (ICCV), pp. 7888–7898 (2019)Google Scholar
  3. 3.
    Ballard, D.H.: Generalizing the Hough transform to fetect arbitrary shapes. In: Readings in Computer Vision, pp. 714–725 (1987)Google Scholar
  4. 4.
    Bouman, K.L., et al.: Turning corners into cameras: principles and methods. In: IEEE International Conference on Computer Vision (ICCV), pp. 2289–2297 (2017)Google Scholar
  5. 5.
    Boyd, S., Parikh, N., Chu, E., Peleato, B., Eckstein, J.: Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends Mach. Learn. 3(1), 1–122 (2011)CrossRefGoogle Scholar
  6. 6.
    Buttafava, M., Zeman, J., Tosi, A., Eliceiri, K., Velten, A.: Non-line-of-sight imaging using a time-gated single photon avalanche diode. Opt. Express 23(16), 20997–21011 (2015)CrossRefGoogle Scholar
  7. 7.
    Chan, S., Warburton, R.E., Gariepy, G., Leach, J., Faccio, D.: Non-line-of-sight tracking of people at long range. Opt. Express 25(9), 10109–10117 (2017)CrossRefGoogle Scholar
  8. 8.
    Chandran, S., Jayasuriya, S.: Adaptive lighting for data-driven non-line-of-sight 3D localization and object identification. In: The British Machine Vision Conference (BMVC) (2019)Google Scholar
  9. 9.
    Galindo, M., Marco, J., O’Toole, M., Wetzstein, G., Gutierrez, D., Jarabo, A.: A dataset for benchmarking time-resolved non-line-of-sight imaging. In: ACM SIGGRAPH 2019 Posters, pp. 1–2 (2019)Google Scholar
  10. 10.
    Gupta, O., Willwacher, T., Velten, A., Veeraraghavan, A., Raskar, R.: Reconstruction of hidden 3D shapes using diffuse reflections. Opt. Express 20(17), 19096–19108 (2012)CrossRefGoogle Scholar
  11. 11.
    Heide, F., Heidrich, W., Hullin, M.B.: Diffuse mirrors: 3D reconstruction from diffuse indirect illumination using inexpensive time-of-flight sens. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3222–3229 (2014)Google Scholar
  12. 12.
    Heide, F., O’Toole, M., Zang, K., Lindell, D.B., Diamond, S., Wetzstein, G.: Non-line-of-sight imaging with partial occluders and surface normals. ACM Trans. Graph. 38(3), 1–10 (2019)CrossRefGoogle Scholar
  13. 13.
    Isogawa, M., Yuan, Y., O’Toole, M., Kitani, K.M.: Optical non-line-of-sight physics-based 3D human pose estimation. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 7013–7022 (2020)Google Scholar
  14. 14.
    Jarabo, A., Marco, J., Muñoz, A., Buisan, R., Jarosz, W., Gutierrez, D.: A framework for transient rendering. ACM Trans. Graph. (ToG) 33(6), 1–10 (2014)CrossRefGoogle Scholar
  15. 15.
    Kak, A.C., Slaney, M., Wang, G.: Principles of computerized tomographic imaging. Med. Phys. 29(1), 107–107 (2002)CrossRefGoogle Scholar
  16. 16.
    Kirmani, A., Hutchison, T., Davis, J., Raskar, R.: Looking around the corner using transient imaging. In: IEEE International Conference on Computer Vision (ICCV), pp. 159–166 (2009)Google Scholar
  17. 17.
    Klein, J., Peters, C., Laurenzis, M., Hullin, M.: Tracking objects outside the line of sight using 2D intensity images. Sci. Rep. 6(32491), 1–9 (2016)Google Scholar
  18. 18.
    Klein, J., Peters, C., Laurenzis, M., Hullin, M.: Non-line-of-sight MoCap. In: ACM SIGGRAPH Emerging Technologies, pp. 18:1–18:2 (2017)Google Scholar
  19. 19.
    La Manna, M., Kine, F., Breitbach, E., Jackson, J., Sultan, T., Velten, A.: Error backprojection algorithms for non-line-of-sight imaging. IEEE Trans. Pattern Anal. Mach. Intell. 41(7), 1615–1626 (2019)CrossRefGoogle Scholar
  20. 20.
    Li, T., Fan, L., Zhao, M., Liu, Y., Katabi, D.: Making the invisible visible: action recognition through walls and occlusions. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 872–881 (2019)Google Scholar
  21. 21.
    Lindell, D.B., Wetzstein, G., Koltun, V.: Acoustic non-line-of-sight imaging. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6780–6789 (2019)Google Scholar
  22. 22.
    Lindell, D.B., Wetzstein, G., O’Toole, M.: Wave-based non-line-of-sight imaging using fast F-K migration. ACM Trans. Graph. (TOG) 38(4), 116 (2019)CrossRefGoogle Scholar
  23. 23.
    Maeda, T., Satat, G., Swedish, T., Sinha, L., Raskar, R.: Recent advances in imaging around corners. In: arXiv (2019)Google Scholar
  24. 24.
    Maeda, T., Wang, Y., Raskar, R., Kadambi, A.: Thermal non-line-of-sight imaging. In: IEEE International Conference on Computational Photography (ICCP), pp. 1–11 (2019)Google Scholar
  25. 25.
    Metzler, C.A., Lindell, D.B., Wetzstein, G.: Keyhole imaging: non-line-of-sight imaging and tracking of moving objects along a single optical path at long standoff distances. In: arXiv (2019)Google Scholar
  26. 26.
    O’Toole, M., Heide, F., Lindell, D.B., Zang, K., Diamond, S., Wetzstein, G.: Reconstructing transient images from single-photon sensors. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2289–2297 (2017)Google Scholar
  27. 27.
    O’Toole, M., Lindell, D.B., Wetzstein, G.: Confocal non-line-of-sight imaging based on the light-cone transform. Nature 555(7696), 338 (2018)CrossRefGoogle Scholar
  28. 28.
    Pediredla, A., Dave, A., Veeraraghavan, A.: SNLOS: Non-line-of-sight scanning through temporal focusing. In: IEEE International Conference on Computational Photography (ICCP), pp. 1–13 (2019)Google Scholar
  29. 29.
    Redo-Sanchez, A., et al.: Terahertz time-gated spectral imaging for content extraction through layered structures. Nat. Commun. 7, 12665 (2016)CrossRefGoogle Scholar
  30. 30.
    Saunders, C., Murray-Bruce, J., Goyal, V.K.: Computational periscopy with an ordinary digital camera. Nature 565(7740), 472–475 (2019)CrossRefGoogle Scholar
  31. 31.
    Tancik, M., Satat, G., Raskar, R.: Flash photography for data-driven hidden scene recovery. In: arXiv (2018)Google Scholar
  32. 32.
    Tsai, C.Y., Sankaranarayanan, A.C., Gkioulekas, I.: Beyond volumetric albedo - a surface optimization framework for non-line-of-sight imaging. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2019)Google Scholar
  33. 33.
    Velten, A., Willwacher, T., Gupta, O., Veeraraghavan, A., Bawendi, M.G., Raskar, R.: Recovering three-dimensional shape around a corner using ultrafast time-of-flight imaging. Nat. Commun. 3, 745 (2012)CrossRefGoogle Scholar
  34. 34.
    Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)CrossRefGoogle Scholar
  35. 35.
    Xin, S., Nousias, S., Kutulakos, K.N., Sankaranarayanan, A.C., Narasimhan, S.G., Gkioulekas, I.: A theory of Fermat paths for non-line-of-sight shape reconstruction. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2019)Google Scholar
  36. 36.
    Zou, C.C., Ge, S.: A Hough transform-based method for fast detection of fixed period sinusoidal curves in images. In: International Conference on Signal Processing (ICSP), vol. 1, pp. 909–912 (2002)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.Carnegie Mellon UniversityPittsburghUSA

Personalised recommendations