Decomposing Global Light Transport Using Time of Flight Imaging

Abstract

Global light transport is composed of direct and indirect components. In this paper, we take the first steps toward analyzing light transport using the high temporal resolution information of time of flight (ToF) images. With pulsed scene illumination, the time profile at each pixel of these images separates different illumination components by their finite travel time and encodes complex interactions between the incident light and the scene geometry with spatially-varying material properties. We exploit the time profile to decompose light transport into its constituent direct, subsurface scattering, and interreflection components. We show that the time profile is well modelled using a Gaussian function for the direct and interreflection components, and a decaying exponential function for the subsurface scattering component. We use our direct, subsurface scattering, and interreflection separation algorithm for five computer vision applications: recovering projective depth maps, identifying subsurface scattering objects, measuring parameters of analytical subsurface scattering models, performing edge detection using ToF images and rendering novel images of the captured scene with adjusted amounts of subsurface scattering.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21

References

  1. Abramson, N. (1978). Light-in-flight recording by holography. Optics Letters, 3(4), 121–123.

    Article  Google Scholar 

  2. Arvo, J. (1993). Transfer equations in global illumination. SIGGRAPH Course Notes.

  3. Bai, J., Chandraker, M., Ng, T., & Ramamoorthi, R. (2010). A dual theory of inverse and forward light transport. European Conference on Computer Vision, 2010, 294–307.

    Google Scholar 

  4. Brooker, G. (2009). Introduction to sensors for ranging and imaging. Raleigh: Scitech.

    Google Scholar 

  5. Busck, J., & Heiselberg, H. (2004). Gated viewing and high-accuracy three-dimensional laser radar. Applied Optics, 43(24), 4705–4710.

    Article  Google Scholar 

  6. Chen, T., Lensch, H., Fuchs, C., & Seidel, H. (2007). Polarization and phase-shifting for 3d scanning of translucent objects. IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 1–8).

  7. Gleckler, A., & Gelbart, A. (2000). Multiple-slit streak tube imaging lidar (ms-stil) applications. Proceedings SPIE (Vol. 4035, pp. 266–278).

  8. Gupta, M., Agrawal, A., Veeraraghavan, A. & Narasimhan, S. (2011). Structured light 3d scanning in the presence of global illumination. IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 713–720).

  9. Gupta, M., Tian, Y., Narasimhan, S., & Zhang, L. (2009). (De) focusing on global light transport for active scene recovery. IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 2969–2976).

  10. Gupta, O., Willwacher, T., Velten, A., Veeraraghavan, A., & Raskar, R. (2012). Reconstruction of hidden 3d shapes using diffuse reflections. Optics Express, 20(17), 19096–19108.

    Article  Google Scholar 

  11. Hamamatsu. (2012). Guide to streak cameras. http://sales.hamamatsu.com/assets/pdf/catsandguides/e_streakh.pdf.

  12. Heide, F., Hullin, M., Gregson, J., & Heidrich, W. (2013). Low-budget transient imaging using photonic mixer devices. Proceedings of ACM SIGGRAPH, ACM Transactions on Graphics, 32(4), 146:1–146:12.

    Google Scholar 

  13. Holroyd, M., Lawrence, J., & Zickler, T. (2010). A coaxial optical scanner for synchronous acquisition of 3d geometry and surface reflectance. ACM Transactions on Graphics, 29(4), 99.

    Article  Google Scholar 

  14. Huang, D., Swanson, E., Lin, C., Schuman, J., Stinson, W., Chang, W., et al. (1991). Optical coherence tomography. Science, 254(5035), 1178–1181.

    Article  Google Scholar 

  15. Jarabo, A., Masia, B., Velten, A., Barsi, C., Raskar, R., & Gutierrez, D. (2013). Rendering relativistic effects in transient imaging. Proceedings of CEIG.

  16. Kajiya, J. (1986). The rendering equation. ACM SIGGRAPH Computer Graphics, 20(4), 143–150.

    Article  Google Scholar 

  17. Liu, S., Ng, T., & Matsushita, Y. (2010). Shape from second-bounce of light transport. European Conference on Computer Vision, 2010, 280–293.

    Google Scholar 

  18. Mukaigawa, Y., Yagi, Y., & Raskar, R. (2010). Analysis of light transport in scattering media. Proceedings of Computer Vision and Pattern Recognition (CVPR) (pp. 153–160).

  19. Naik, N., Zhao, S., Velten, A., Raskar, R., & Bala, K. (2011). Single view reflectance capture using multiplexed scattering and time-of-flight imaging. ACM Transaction on Graphics, 30(6), 171.

    Article  Google Scholar 

  20. Nayar, S., Krishnan, G., Grossberg, M., & Raskar, R. (2006). Fast separation of direct and global components of a scene using high frequency illumination. ACM Transaction on Graphics, 25(3), 935–944.

    Article  Google Scholar 

  21. Ng, T., Pahwa, R., Bai, J., Tan, K., & Ramamoorthi, R. (2012). From the rendering equation to stratified light transport inversion. International Journal of Computer Vision, 96(2), 235–251.

    Google Scholar 

  22. O’Toole, M., Raskar, R., & Kutulakos, K. (2012). Primal-dual coding to probe light transport. ACM Transactions on Graphics, 31(4), 39.

    Google Scholar 

  23. Pandharkar, R., Velten, A., Bardagjy, A., Lawson, E., Bawendi, M., & Raskar, R. (2011). Estimating motion and size of moving non-line-of-sight objects in cluttered environments. IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 265–272).

  24. Reddy, D., Ramamoorthi, R., & Curless, B. (2012). Frequency-space decomposition and acquisition of light transport under spatially varying illumination. European Conference on Computer Vision (pp. 596–610).

  25. Repasi, E., Lutzmann, P., Steinvall, O., Elmqvist, M., Göhler, B., & Anstett, G. (2009). Advanced short-wavelength infrared range-gated imaging for ground applications in monostatic and bistatic configurations. Applied Optics, 48(31), 5956–5969.

    Article  Google Scholar 

  26. Seitz, S., Matsushita, Y., & Kutulakos, K. (2005). A theory of inverse light transport. Tenth IEEE International Conference on Computer Vision, (Vol. 2, pp. 1440–1447).

  27. Sen, P., Chen, B., Garg, G., Marschner, S., Horowitz, M., Levoy, M., et al. (2005). Dual photography. ACM Transactions on Graphics, 24(3), 745–755.

    Article  Google Scholar 

  28. Velten, A., Willwacher, T., Gupta, O., Veeraraghavan, A., Bawendi, M. G., & Raskar, R. (2012). Recovering three-dimensional shape around a corner using ultrafast time-of-flight imaging. Nature Communication, 3, 745–758.

    Article  Google Scholar 

  29. Velten, A., Wu, D., Jarabo, A., Masia, B., Barsi, C., Joshi, C., et al. (2012). Relativistic ultrafast rendering using time-of-flight imaging. In ACM SIGGRAPH Talks, 41(1).

  30. Velten, A., Wu, D., Jarabo, A., Masia, B., Barsi, C., Joshi, C., Lawson, E., Bawendi, M., Gutierrez, D., & Raskar, R. (2013). Femto-photography: Capturing and visualizing the propagation of light. Proceedindgs of SIGGRAPH, ACM Transaction on Graphics 32(4).

  31. Wang, L., & Wu, H. (2007). Biomedical optics: Principles and imaging. Hoboken, NJ: Wiley.

    Google Scholar 

  32. Wetzstein, G., & Bimber, O. (2007). Radiometric compensation through inverse light transport. Proceedings of the 15th Pacific Conference on Computer Graphics and Applications (pp. 391–399).

  33. Wu, D., Wetzstein, G., Barsi, C., Willwacher, T., O’Toole, M., Naik, N., et al. (2012). Frequency analysis of transient light transport with applications in bare sensor imaging. European Conference on Computer Vision, 2012, 542–555.

    Google Scholar 

  34. Wyant, J. (2002). White light interferometry. In AeroSense 2002 (pp. 98–107). Bellingham, WA: International Society for Optics and Photonics.

  35. Xia, H., & Zhang, C. (2009). Ultrafast ranging lidar based on real-time Fourier transformation. Optics Letters, 34, 2108–2110.

    Article  Google Scholar 

  36. Zhang, L., & Nayar, S. (2006). Projection defocus analysis for scene capture and image display. ACM Transactions on Graphics, 25(3), 907–915.

    Article  Google Scholar 

Download references

Acknowledgments

The work of the MIT affiliated coauthors was funded by the Media Lab Consortium Members, DARPA through the DARPA YFA grant, and the Institute for Soldier Nanotechnologies and U.S. Army Research Office under contract W911NF-07-D-0004. The work of the Tsinghua affiliated coauthors was supported by the National Basic Research Project (No.2010CB731800) of China and the Key Project of NSFC (No. 61120106003 and 60932007). O’Toole received the support of the NSERC PGS-D and GRAND NCE programs. Masia was additionally funded by an FPU grant, project TIN2010-21543 and an NVIDIA Graduate Fellowship.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Andreas Velten.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (mp4 4953 KB)

Supplementary material 2 (mp4 3233 KB)

Supplementary material 3 (mp4 3881 KB)

Supplementary material 4 (mp4 12513 KB)

Supplementary material 5 (mp4 9305 KB)

Supplementary material 6 (mp4 3621 KB)

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Wu, D., Velten, A., O’Toole, M. et al. Decomposing Global Light Transport Using Time of Flight Imaging. Int J Comput Vis 107, 123–138 (2014). https://doi.org/10.1007/s11263-013-0668-2

Download citation

Keywords

  • Light transport analysis
  • Direct/global separation
  • Time of flight imaging
  • Transient imaging
  • Femto-photography