Abstract
This paper considers panorama images used for street views. Their viewing angle of 360° causes pixels at the top and bottom to appear stretched and warped. Although current image completion algorithms work well, they cannot be directly used in the presence of such distortions found in panoramas of street views. We thus propose a novel approach to complete such 360° panoramas using optimization-based projection to deal with distortions. Experimental results show that our approach is efficient and provides an improvement over standard image completion algorithms.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
Brown, M.; Lowe, D. G. Recognising panoramas. In: Proceedings of the Ninth IEEE International Conference on Computer Vision, Vol. 2, 1218–1225, 2003.
Brown, M.; Lowe, D. G. Automatic panoramic image stitching using invariant features. International Journal of Computer Vision Vol. 74, No. 1, 59–73, 2007.
Summa, B.; Tierny, J.; Pascucci, V. Panorama weaving: Fast and flexible seam processing. ACM Transactions on Graphics Vol. 31, No. 4, Article No. 83, 2012.
Gao, J.; Brown, M. S. An interactive editing tool for correcting panoramas. In: SIGGRAPH Asia 2012. Technical Briefs, Article No. 31, 2012.
Ballester, C.; Bertalmio, M.; Caselles, V.; Sapiro, G.; Verdera, J. Filling-in by joint interpolation of vector fields and gray levels. IEEE Transactions on Image Processing Vol. 10, No. 8, 1200–1211, 2001.
Bertalmio, M.; Sapiro, G.; Caselles, V.; Ballester, C. Image inpainting. In: Proceedings of the 27th annual conference on Computer graphics and interactive techniques, 417–424, 2000.
Bertalmio, M.; Vese, L.; Sapiro, G.; Osher, S. Simultaneous structure and texture image inpainting. In: Proceedings 2003. IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Vol. 2, II-707-12, 2003.
Levin, A.; Zomet, A.; Weiss, Y. Learning how to inpaint from global image statistics. In: Proceedings of the Ninth IEEE International Conference on Computer Vision, Vol. 2, 305, 2003.
Roth, S.; Black, M. J. Fields of experts: A framework for learning image priors. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Vol. 2, 860–867, 2005.
Wexler, Y.; Shechtman, E.; Irani, M. Space-time completion of video. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 29, No. 3, 463–476, 2007.
Barnes, C.; Shechtman, E.; Finkelstein, A.; Goldman, D. Patchmatch: A randomized correspondence algorithm for structural image editing. ACM Transactions on Graphics Vol. 28, No. 3, Article No. 24, 2009.
Darabi, S.; Shechtman, E.; Barnes, C.; Goldman, D. B.; Sen, P. Image melding: Combining inconsistent images using patch-based synthesis. ACM Transactions on Graphics Vol. 31, No. 4, Article No. 82, 2012.
He, K.; Sun, J. Image completion approaches using the statistics of similar patches. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 36, No. 12, 2423–2435, 2014.
Information on http://www.adobe.com/technology/ projects.html.
Sun, J.; Yuan, L.; Jia, J.; Shum, H.-Y. Image completion with structure propagation. ACM Transactions on Graphics Vol. 24, No. 3, 861–868, 2005.
Huang, H.; Yin, K.; Gong, M.; Lischinski, D.; Cohen-Or, D.; Ascher, U.; Chen, B. “Mind the gap”: Tele-registration for structure-driven image completion. ACM Transactions on Graphics Vol. 32, No. 6, Article No. 174, 2013.
Huang, J.-B.; Kang, S. B.; Ahuja, N.; Kopf, J. Image completion using planar structure guidance. ACM Transactions on Graphics Vol. 33, No. 4, Article No. 129, 2014.
Zontak, M.; Irani, M. Internal statistics of a single natural image. In: Proceedings of the 2011.IEEE Conference on Computer Vision and Pattern Recognition, 977–984, 2011.
Carroll, R.; Agrawal, M.; Agarwala, A. Optimizing content-preserving projections for wide-angle images. ACM Transactions on Graphics Vol. 28, No. 3, Article No. 43, 2009.
Von Gioi, R. G.; Jakubowicz, J.; Morel, J.-M.; Randall, G. LSD: A fast line segment detector with a false detection control. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 32, No. 4, 722–732, 2010.
Chang, C.-H.; Chuang, Y.-Y. A line-structurepreserving approach to image resizing. In: 2012.IEEE Conference on Computer Vision and Pattern Recognition, 1075–1082, 2012.
Zhang, G.-X.; Cheng, M.-M.; Hu, S.-M.; Martin, R. R. A shape-preserving approach to image resizing. Computer Graphics Forum Vol. 28, No. 7, 1897–1906, 2009.
Hel-Or, Y.; Hel- Or, H. Real-time pattern matching using projection kernels. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 27, No. 9, 1430–1445, 2005.
Sun, J. Computing nearest-neighbor fields via propagation-assisted KD-trees. In: Proceedings of the 2012.IEEE Conference on Computer Vision and Pattern Recognition, 111–118, 2012.
Simakov, D.; Caspi, Y.; Shechtman, E.; Irani, M. Summarizing visual data using bidirectional similarity. In: IEEE Conference on Computer Vision and Pattern Recognition 2008}. Available at http://www.wisdom. weizmann.ac.il/~vision/VisualSummary/bidirectional similarity CVPR2008.pdf
Author information
Authors and Affiliations
Corresponding author
Additional information
This article is published with open access at Springerlink.com
Zhe Zhu is a Ph.D. candidate in the Department of Computer Science and Technology, Tsinghua University. He received his bachelor’s degree in Wuhan University in 2011. His research interests include computer vision and computer graphics.
Ralph R. Martin is currently a professor in Cardiff University. He obtained his Ph.D. degree in 1983 from Cambridge University. He has published more than 250 papers and 14 books, covering such topics as solid and surface modeling, intelligent sketch input, geometric reasoning, reverse engineering, and various aspects of computer graphics. He is a Fellow of the Learned Society of Wales, the Institute of Mathematics and its Applications, and the British Computer Society. He is on the editorial boards of Computer-Aided Design, Computer Aided Geometric Design, Geometric Models, the International Journal of Shape Modeling, CAD and Applications, and the International Journal of CAD/CAM. He was recently awarded a Friendship Award, China’s highest honor for foreigners.
Shi-Min Hu received the Ph.D. degree from Zhejiang University in 1996. He is currently a professor in the Department of Computer Science and Technology, Tsinghua University, Beijing. His research interests include digital geometry processing, video processing, rendering, computer animation, and computer-aided geometric design. He is on the editorial boards of several international journals, including IEEE Transactions on Visualization and Computer Graphics, The Visual Computer, Computer & Graphics, and Computer-Aided Design.
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (https://creativecommons.org/licenses/by/4.0), which permits use, duplication, adaptation, distribution, and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
About this article
Cite this article
Zhu, Z., Martin, R.R. & Hu, SM. Panorama completion for street views. Comp. Visual Media 1, 49–57 (2015). https://doi.org/10.1007/s41095-015-0008-2
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s41095-015-0008-2