Advertisement

International Journal of Computer Vision

, Volume 125, Issue 1–3, pp 52–64 | Cite as

3D Time-Lapse Reconstruction from Internet Photos

  • Ricardo Martin-BruallaEmail author
  • David Gallup
  • Steven M. Seitz
Article
  • 636 Downloads

Abstract

Given an Internet photo collection of a landmark, we compute a 3D time-lapse video sequence where a virtual camera moves continuously in time and space. While previous work assumed a static camera, the addition of camera motion during the time-lapse creates a very compelling impression of parallax. Achieving this goal, however, requires addressing multiple technical challenges, including solving for time-varying depth maps, regularizing 3D point color profiles over time, and reconstructing high quality, hole-free images at every frame from the projected profiles. Our results show photorealistic time-lapses of skylines and natural scenes over many years, with dramatic parallax effects.

Keywords

Computational photography Time-lapse Internet photos 

Notes

Acknowledgements

The research was supported in part by the National Science Foundation (IIS-1250793), the Animation Research Labs, and Google.

References

  1. Agarwal, S., Furukawa, Y., Snavely, N., Simon, I., Curless, B., Seitz, S. M., et al. (2011). Building rome in a day. Communications of the ACM, 54(10), 105–112.CrossRefGoogle Scholar
  2. Agarwal, S., Mierle, K., Others (2012). Ceres Solver. http://ceres-solver.org.
  3. Bennett, E. P., & McMillan, L.(2007). Computational time-lapse video. In: ACM SIGGRAPH 2007 Papers, SIGGRAPH ’07.Google Scholar
  4. Earth Vision Institute. (2007). Extreme Ice Survey. http://extremeicesurvey.org/.
  5. Hauagge, D., Wehrwein, S., Upchurch, P., Bala, K., Snavely, N. (2014). Reasoning about photo collections using models of outdoor illumination. In: Proceedings of BMVC.Google Scholar
  6. Kang, S. B., & Szeliski, R. (2004). Extracting view-dependent depth maps from a collection of images. International Journal of Computer Vision, 58, 139–163.CrossRefGoogle Scholar
  7. Kemelmacher-Shlizerman, I., Shechtman, E., Garg, R., Seitz, S. M. (2011). Exploring photobios. In: ACM SIGGRAPH 2011 Papers, SIGGRAPH ’11, pp 61:1–61:10.Google Scholar
  8. Klose, F., Wang, O., Bazin, J. C., Magnor, M., & Sorkine-Hornung, A. (2015). Sampling based scene-space video processing. ACM Transactions on Graphics, 34(4), 67:1–67:11.CrossRefGoogle Scholar
  9. Kopf, J., Cohen, M. F., & Szeliski, R. (2014). First-person hyper-lapse videos. ACM Transactions on Graphics, 33(4), 78:1–78:10.CrossRefGoogle Scholar
  10. Laffont, P. Y., Bousseau, A., Paris, S., Durand, F., Drettakis, G. (2012). Coherent intrinsic images from photo collections. ACM Transactions on Graphics (SIGGRAPH Asia Conference Proceedings) 31.Google Scholar
  11. Laforet, V. (2013). Time Lapse Intro: Part I. http://blog.vincentlaforet.com/2013/04/27/time-lapse-intro-part-i/.
  12. Larsen, E., Mordohai, P., Pollefeys, M., & Fuchs, H. (2007). Temporally consistent reconstruction from multiple video streams using enhanced belief propagation. In: Computer Vision, 2007. ICCV 2007. IEEE 11th International Conference on, pp 1–8.Google Scholar
  13. Martin-Brualla, R., Gallup, D., & Seitz, S. M. (2015). Time-lapse mining from internet photos. ACM Transactions on Graphics, 34(4), 62:1–62:8.CrossRefGoogle Scholar
  14. Matzen, K., & Snavely, N. (2014). Scene chronology. In: Proc. European Conf. on Computer Vision.Google Scholar
  15. Newcombe, R. A., Lovegrove, S., & Davison, A. (2011). Dtam: Dense tracking and mapping in real-time. In: Computer Vision (ICCV), 2011 IEEE International Conference on, pp 2320–2327.Google Scholar
  16. Rubinstein, M., Liu, C., Sand, P., Durand, F., & Freeman, W. T. (2011). Motion denoising with application to time-lapse photography. In: Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition, CVPR ’11, pp 313–320.Google Scholar
  17. Schindler, G., & Dellaert, F. (2010). Probabilistic temporal inference on reconstructed 3d scenes. In: Computer Vision and Pattern Recognition (CVPR), 2010 IEEE Conference on, IEEE, pp 1410–1417.Google Scholar
  18. Schindler, G., Dellaert, F., & Kang, S. B. (2007). Inferring temporal order of images from 3d structure. In: Computer Vision and Pattern Recognition, 2007. CVPR ’07. IEEE Conference on, pp 1–7.Google Scholar
  19. Seitz, S., Curless, B., Diebel, J., Scharstein, D., & Szeliski, R. (2006). A comparison and evaluation of multi-view stereo reconstruction algorithms. In: Computer Vision and Pattern Recognition, 2006 IEEE Computer Society Conference on, vol 1, pp 519–528.Google Scholar
  20. Shan, Q., Adams, R., Curless, B., Furukawa, Y., & Seitz, S. (2013). The visual turing test for scene reconstruction. In: 3D Vision - 3DV 2013, 2013 International Conference on, pp 25–32.Google Scholar
  21. Simon, I., Snavely, N., & Seitz, S. (2007). Scene summarization for online image collections. In: Computer Vision, 2007. ICCV 2007. IEEE 11th International Conference on, pp 1–8.Google Scholar
  22. Snavely, N., Garg, R., Seitz, S. M., & Szeliski, R. (2008). Finding paths through the world’s photos. ACM Transactions on Graphics (Proceedings of SIGGRAPH 2008), 27(3), 11–21.Google Scholar
  23. Zhang, G., Jia, J., Wong, T. T., & Bao, H. (2009). Consistent depth maps recovery from a video sequence. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 31(6), 974–988.CrossRefGoogle Scholar
  24. Zhang, L., Curless, B., Seitz, S. (2003). Spacetime stereo: shape recovery for dynamic scenes. In: Computer Vision and Pattern Recognition, 2003. Proceedings. 2003 IEEE Computer Society Conference on, vol 2, pp II–367–74 vol.2.Google Scholar
  25. Zheng, K. C., Colburn, A., Agarwala, A., Agrawala, M., Salesin, D., Curless, B., & Cohen, M. F. (2009). Parallax photography: Creating 3d cinematic effects from stills. In: Proceedings of Graphics Interface 2009, GI ’09, pp 111–118.Google Scholar

Copyright information

© Springer Science+Business Media New York 2017

Authors and Affiliations

  1. 1.Google Inc.Mountain ViewUSA
  2. 2.University of WashingtonSeattleUSA

Personalised recommendations