Advertisement

International Journal of Computer Vision

, Volume 95, Issue 1, pp 13–28 | Cite as

Looking Around the Corner using Ultrafast Transient Imaging

  • Ahmed KirmaniEmail author
  • Tyler Hutchison
  • James Davis
  • Ramesh Raskar
Article

Abstract

We propose a novel framework called transient imaging for image formation and scene understanding through impulse illumination and time images. Using time-of-flight cameras and multi-path analysis of global light transport, we pioneer new algorithms and systems for scene understanding through time images. We demonstrate that our proposed transient imaging framework allows us to accomplish tasks that are well beyond the reach of existing imaging technology. For example, one can infer the geometry of not only the visible but also the hidden parts of a scene, enabling us to look around corners. Traditional cameras estimate intensity per pixel I(x,y). Our transient imaging camera captures a 3D time-image I(x,y,t) for each pixel and uses an ultra-short pulse laser for illumination. Emerging technologies are supporting cameras with a temporal-profile per pixel at picosecond resolution, allowing us to capture an ultra-high speed time-image. This time-image contains the time profile of irradiance incident at a sensor pixel. We experimentally corroborated our theory with free space hardware experiments using a femtosecond laser and a picosecond accurate sensing device. The ability to infer the structure of hidden scene elements, unobservable by both the camera and illumination source, will create a range of new computer vision opportunities.

Keywords

Light transport Global illumination Multi-path analysis Inverse problems Inverse rendering Computational Imaging 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Supplementary material

11263_2011_470_MOESM1_ESM.pdf (5.4 mb)
Looking Around the Corner using Ultrafast Transient Imaging. (5.44MB)

References

  1. Arvo, J. (1993). Transfer equations in global illumination. In Global Illumination, SIGGRAPH ’93 Course Notes. Google Scholar
  2. Campillo, A., & Shapiro, S. (1983). Picosecond streak camera fluorometry—a review. IEEE Journal of Quantum Electronics. Google Scholar
  3. Dattorro, J. (2006). Convex optimization & Euclidean distance geometry. Morrisville: Lulu.com. Google Scholar
  4. Denk, W., Strickler, J.H., & Webb, W.W. (1990). Two-photon laser scanning fluorescence microscopy. Science, 248, 73–76. CrossRefGoogle Scholar
  5. Garren, D., Goldstein, J., Obuchon, D., Greene, R., & North, J. (2004). SAR image formation algorithm with multipath reflectivity estimation. In Proceedings of the IEEE radar conference, 2004 (pp. 323–328). Google Scholar
  6. Garren, D., Sullivan, D., North, J., & Goldstein, J. (2005). Image preconditioning for a SAR image reconstruction algorithm for multipath scattering. In Proc. of IEEE int. radar conference Google Scholar
  7. Gonzalez-Banos, H., & Davis, J. (2004). Computing depth under ambient illumination using multi-shuttered light. Computer Vision and Pattern Recognition. Google Scholar
  8. Iddan, G. J., & Yahav, G. (2001). 3D imaging in the studio (and elsewhere…). In SPIE. Google Scholar
  9. Immel, D. S., Cohen, M. F., & Greenberg, D. P. (1986). A radiosity method for non-diffuse environments. In ACM SIGGRAPH. Google Scholar
  10. Itatani, J., Quéré, F., Yudin, G., Ivanov, M., Krausz, F., & Corkum, P. (2002). Attosecond streak camera. Physical Review Letters. Google Scholar
  11. Kajiya, J. T. (1986). The rendering equation. In ACM SIGGRAPH. Google Scholar
  12. Kamerman, G. (1993). Active electro-optical system. In The infrared and electro-optical system handbook: Vol. 6. Laser radar [M]. Chapter 1. Google Scholar
  13. Kutulakos, K. N., & Steger, E. (2007). A theory of refractive and specular 3d shape by light-path triangulation. International Journal of Computer Vision. Google Scholar
  14. Lange, R., & Seitz, P. (2001). Solid-state time-of-flight range camera. IEEE Journal of Quantum Electronics. Google Scholar
  15. Morris, N. J. W., & Kutulakos, K. N. (2007). Reconstructing the surface of inhomogeneous transparent scenes by scatter trace photography. International Conference on Computer Vision. Google Scholar
  16. Nayar, S. K., Ikeuchi, K., & Kanade, T. (1990). Shape from interreflections. International Conference on Computer Vision. Google Scholar
  17. Nayar, S. K., Krishnan, G., Grossberg, M. D., & Raskar, R. (2006). Fast separation of direct and global components of a scene using high frequency illumination. In ACM SIGGRAPH. Google Scholar
  18. Ng, R., Marc, L., Mathieu, B., Gene, D., Mark, H., & Pat, H. (2005). Light field photography with a hand-held plenoptic camera. Stanford University Computer Science Tech Report. Google Scholar
  19. Patow, G., & Pueyo, X. (2003). A survey of inverse rendering problems. Computer Graphics Forum. Google Scholar
  20. Ramamoorthi, R., & Hanrahan, P. (2001). A signal-processing framework for inverse rendering. Computer Graphics and Interactive Techniques. Google Scholar
  21. Sarunic, P., White, K., & Rutten, M. (2001). Over-the-horizon radar multipath and multisensor track fusion algorithm development. Google Scholar
  22. Schmitt, J. M. (1999). Optical coherence tomography (oct): a review. IEEE Quantum Electronics. Google Scholar
  23. Seitz, S. M., Matsushita, Y., & Kutulakos, K. N. (2005). A theory of inverse light transport. IEEE International Conference on Computer Vision. Google Scholar
  24. Sen, P., Chen, B., Garg, G., Marschner, S. R., Horowitz, M., Levoy, M., & Lensch, H. P. A. (2005). Dual photography. In ACM SIGGRAPH. Google Scholar
  25. Vandapel, N., Amidi, O., & Miller, J. (2004). Toward laser pulse waveform analysis for scene interpretation. IEEE International Conference on Robotics and Automation. Google Scholar
  26. Veeraraghavan, A., Raskar, R., Agrawal, A., Mohan, A., & Tumblin, J. (2007). Dappled photography: Mask enhanced cameras for heterodyned light fields and coded aperture refocussing. In ACM SIGGRAPH. Google Scholar
  27. Raskar, R., & Davis, J. 5d time-light transport matrix: What can we reason about scene properties, Int. Memo 2007. Google Scholar
  28. Smith, A., Skorupski, J., & Davis, J. Transient rendering, UC Santa Cruz TR UCSC-SOE-08-26, Feb 2008. 2. Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2011

Authors and Affiliations

  • Ahmed Kirmani
    • 1
    Email author
  • Tyler Hutchison
    • 1
  • James Davis
    • 2
  • Ramesh Raskar
    • 1
  1. 1.MIT Media LabCambridgeUSA
  2. 2.Department of Computer ScienceUC Santa CruzSanta CruzUSA

Personalised recommendations