Advertisement

Computational Visual Media

, Volume 1, Issue 1, pp 49–57 | Cite as

Panorama completion for street views

  • Zhe Zhu
  • Ralph R. Martin
  • Shi-Min HuEmail author
Open Access
Research Article

Abstract

This paper considers panorama images used for street views. Their viewing angle of 360° causes pixels at the top and bottom to appear stretched and warped. Although current image completion algorithms work well, they cannot be directly used in the presence of such distortions found in panoramas of street views. We thus propose a novel approach to complete such 360° panoramas using optimization-based projection to deal with distortions. Experimental results show that our approach is efficient and provides an improvement over standard image completion algorithms.

Keywords

image completion panorama street views structure-rectifying warp 

References

  1. [1]
    Brown, M.; Lowe, D. G. Recognising panoramas. In: Proceedings of the Ninth IEEE International Conference on Computer Vision, Vol. 2, 1218–1225, 2003.Google Scholar
  2. [2]
    Brown, M.; Lowe, D. G. Automatic panoramic image stitching using invariant features. International Journal of Computer Vision Vol. 74, No. 1, 59–73, 2007.CrossRefGoogle Scholar
  3. [3]
    Summa, B.; Tierny, J.; Pascucci, V. Panorama weaving: Fast and flexible seam processing. ACM Transactions on Graphics Vol. 31, No. 4, Article No. 83, 2012.CrossRefGoogle Scholar
  4. [4]
    Gao, J.; Brown, M. S. An interactive editing tool for correcting panoramas. In: SIGGRAPH Asia 2012. Technical Briefs, Article No. 31, 2012.Google Scholar
  5. [5]
    Ballester, C.; Bertalmio, M.; Caselles, V.; Sapiro, G.; Verdera, J. Filling-in by joint interpolation of vector fields and gray levels. IEEE Transactions on Image Processing Vol. 10, No. 8, 1200–1211, 2001.MathSciNetCrossRefzbMATHGoogle Scholar
  6. [6]
    Bertalmio, M.; Sapiro, G.; Caselles, V.; Ballester, C. Image inpainting. In: Proceedings of the 27th annual conference on Computer graphics and interactive techniques, 417–424, 2000.Google Scholar
  7. [7]
    Bertalmio, M.; Vese, L.; Sapiro, G.; Osher, S. Simultaneous structure and texture image inpainting. In: Proceedings 2003. IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Vol. 2, II-707-12, 2003.Google Scholar
  8. [8]
    Levin, A.; Zomet, A.; Weiss, Y. Learning how to inpaint from global image statistics. In: Proceedings of the Ninth IEEE International Conference on Computer Vision, Vol. 2, 305, 2003.Google Scholar
  9. [9]
    Roth, S.; Black, M. J. Fields of experts: A framework for learning image priors. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Vol. 2, 860–867, 2005.Google Scholar
  10. [10]
    Wexler, Y.; Shechtman, E.; Irani, M. Space-time completion of video. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 29, No. 3, 463–476, 2007.CrossRefGoogle Scholar
  11. [11]
    Barnes, C.; Shechtman, E.; Finkelstein, A.; Goldman, D. Patchmatch: A randomized correspondence algorithm for structural image editing. ACM Transactions on Graphics Vol. 28, No. 3, Article No. 24, 2009.CrossRefGoogle Scholar
  12. [12]
    Darabi, S.; Shechtman, E.; Barnes, C.; Goldman, D. B.; Sen, P. Image melding: Combining inconsistent images using patch-based synthesis. ACM Transactions on Graphics Vol. 31, No. 4, Article No. 82, 2012.CrossRefGoogle Scholar
  13. [13]
    He, K.; Sun, J. Image completion approaches using the statistics of similar patches. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 36, No. 12, 2423–2435, 2014.CrossRefGoogle Scholar
  14. [14]
    Information on http://www.adobe.com/technology/ projects.html.Google Scholar
  15. [15]
    Sun, J.; Yuan, L.; Jia, J.; Shum, H.-Y. Image completion with structure propagation. ACM Transactions on Graphics Vol. 24, No. 3, 861–868, 2005.CrossRefGoogle Scholar
  16. [16]
    Huang, H.; Yin, K.; Gong, M.; Lischinski, D.; Cohen-Or, D.; Ascher, U.; Chen, B. “Mind the gap”: Tele-registration for structure-driven image completion. ACM Transactions on Graphics Vol. 32, No. 6, Article No. 174, 2013.Google Scholar
  17. [17]
    Huang, J.-B.; Kang, S. B.; Ahuja, N.; Kopf, J. Image completion using planar structure guidance. ACM Transactions on Graphics Vol. 33, No. 4, Article No. 129, 2014.Google Scholar
  18. [18]
    Zontak, M.; Irani, M. Internal statistics of a single natural image. In: Proceedings of the 2011.IEEE Conference on Computer Vision and Pattern Recognition, 977–984, 2011.Google Scholar
  19. [19]
    Carroll, R.; Agrawal, M.; Agarwala, A. Optimizing content-preserving projections for wide-angle images. ACM Transactions on Graphics Vol. 28, No. 3, Article No. 43, 2009.CrossRefGoogle Scholar
  20. [20]
    Von Gioi, R. G.; Jakubowicz, J.; Morel, J.-M.; Randall, G. LSD: A fast line segment detector with a false detection control. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 32, No. 4, 722–732, 2010.CrossRefGoogle Scholar
  21. [21]
    Chang, C.-H.; Chuang, Y.-Y. A line-structurepreserving approach to image resizing. In: 2012.IEEE Conference on Computer Vision and Pattern Recognition, 1075–1082, 2012.CrossRefGoogle Scholar
  22. [22]
    Zhang, G.-X.; Cheng, M.-M.; Hu, S.-M.; Martin, R. R. A shape-preserving approach to image resizing. Computer Graphics Forum Vol. 28, No. 7, 1897–1906, 2009.CrossRefGoogle Scholar
  23. [23]
    Hel-Or, Y.; Hel- Or, H. Real-time pattern matching using projection kernels. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 27, No. 9, 1430–1445, 2005.CrossRefGoogle Scholar
  24. [24]
    Sun, J. Computing nearest-neighbor fields via propagation-assisted KD-trees. In: Proceedings of the 2012.IEEE Conference on Computer Vision and Pattern Recognition, 111–118, 2012.Google Scholar
  25. [25]
    Simakov, D.; Caspi, Y.; Shechtman, E.; Irani, M. Summarizing visual data using bidirectional similarity. In: IEEE Conference on Computer Vision and Pattern Recognition 2008}. Available at http://www.wisdom. weizmann.ac.il/~vision/VisualSummary/bidirectional similarity CVPR2008.pdfGoogle Scholar

Copyright information

© The Author(s) 2015

Authors and Affiliations

  1. 1.TNLIST, Department of Computer Science and TechnologyTsinghua UniversityBeijingChina
  2. 2.School of Computer Science & InformaticsCardiff UniversityCardiffUK

Personalised recommendations