Abstract
Panorama creation from unconstrained hand-held videos is a challenging task due to the presence of large parallax, moving objects and motion blur. Alignment of the frames taken from a hand-held video is often very difficult to perform. The method proposed here aims to generate a panorama view of the video shot given as input. The proposed framework for panorama creation consists of four stages: The first stage performs a sparse frame selection based on alignment and blur score. A global order for aligning the selected frames is generated by computing a Minimum Spanning Tree with the most connected frame as the root of the MST. The third stage performs frame alignment using a novel warping model termed as DiffeoMeshes, a demon-based diffeomorphic registration process for mesh deformation, whereas the fourth stage renders the panorama. For evaluating the alignment performance, experiments were first performed on a standard dataset consisting of pairs of images. We have also created and experimented on a dataset of 20 video shots for generating panorama. Our proposed method performs better than the existing state-of-the-art methods in terms of alignment error and panorama rendering quality.
Partially supported by TCS Foundation, India.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Brown, M., Lowe, D.G., et al.: Recognising panoramas. In: ICCV (2003)
Liu, F., Hu, Y.H., Gleicher, M.L.: Discovering panoramas in web videos. In: ACMMM, pp. 329–338 (2008)
Li, Y., Kang, S.B., Joshi, N., Seitz, S.M., Huttenlocher, D.P.: Generating sharp panoramas from motion-blurred videos. In: CVPR (2010)
Hu, J., Zhang, D.Q., Yu, H., Chen, C.W.: Long scene panorama generation for indoor environment. In: ICIP (2014)
Freeman, S.C., Freeman, L.C.: The networkers network: a study of the impact of a new communications medium on sociometric structure (1979)
Liu, S., Yuan, L., Tan, P., Sun, J.: Bundled camera paths for video stabilization. ACM Trans. Graph. (TOG) 32, 78 (2013)
Szeliski, R.: Image alignment and stitching: a tutorial. Found. Trends® Comput. Graph. Vis. 2, 1–104 (2006)
Brown, M., Lowe, D.G.: Automatic panoramic image stitching using invariant features. Int. J. Comput. Vis. 74, 59–73 (2007)
Szeliski, R., Shum, H.Y.: Creating full view panoramic image mosaics and environment maps. In: SIGGRAPH (1997)
Gao, J., Kim, S.J., Brown, M.S.: Constructing image panoramas using dual-homography warping. In: CVPR (2011)
Lin, W.Y., Liu, S., Matsushita, Y., Ng, T.T., Cheong, L.F.: Smoothly varying affine stitching. In: CVPR (2011)
Liu, F., Gleicher, M., Jin, H., Agarwala, A.: Content-preserving warps for 3D video stabilization. ACM Trans. Graph. (TOG) 28, 44 (2009)
Lin, C.C., Pankanti, S.U., Natesan Ramamurthy, K., Aravkin, A.Y.: Adaptive as-natural-as-possible image stitching. In: CVPR (2015)
Zaragoza, J., Chin, T.J., Brown, M.S., Suter, D.: As-projective-as-possible image stitching with moving DLT. In: CVPR (2013)
Chang, C.H., Sato, Y., Chuang, Y.Y.: Shape-preserving half-projective warps for image stitching. In: CVPR (2014)
Chen, Y.-S., Chuang, Y.-Y.: Natural image stitching with the global similarity prior. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9909, pp. 186–201. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46454-1_12
Lin, K., Jiang, N., Liu, S., Cheong, L.F., Lu, J., Do, M.: Direct photometric alignment by mesh deformation. In: CVPR (2017)
Bartoli, A.: Groupwise geometric and photometric direct image registration. IEEE Trans. Pattern Anal. Mach. Intell. 30, 2098–2108 (2008)
Zhang, F., Liu, F.: Parallax-tolerant image stitching. In: CVPR (2014)
Jacob, G.M., Das, S.: Large parallax image stitching using an edge-preserving Diffeomorphic warping process. In: Blanc-Talon, J., Helbert, D., Philips, W., Popescu, D., Scheunders, P. (eds.) ACIVS 2018. LNCS, vol. 11182, pp. 521–533. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01449-0_44
Lin, K., Jiang, N., Cheong, L.-F., Do, M., Lu, J.: SEAGULL: seam-guided local alignment for parallax-tolerant image stitching. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9907, pp. 370–385. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46487-9_23
Davis, J.: Mosaics of scenes with moving objects. In: CVPR (1998)
Kourogi, M., Kurata, T., Hoshino, J., Muraoka, Y.: Real-time image mosaicing from a video sequence. In: ICIP (1999)
Shum, H.Y., Szeliski, R.: Construction and refinement of panoramic mosaics with global and local alignment. In: ICCV (1998)
Hsieh, J.W.: Fast stitching algorithm for moving object detection and mosaic construction. Image Vis. Comput. 22, 291–306 (2004)
Zhi, Q., Cooperstock, J.R.: Toward dynamic image mosaic generation with robustness to parallax. IEEE Trans. Image Process. 21, 366–378 (2012)
Zeng, L., Zhang, S., Zhang, J., Zhang, Y.: Dynamic image mosaic via sift and dynamic programming. Mach. Vis. Appl. 25, 1271–1282 (2014)
Thirion, J.P.: Image matching as a diffusion process: an analogy with Maxwell’s demons. Med. Image Anal. 2, 243–260 (1998)
Vercauteren, T., Pennec, X., Perchant, A., Ayache, N.: Diffeomorphic demons: efficient non-parametric image registration. NeuroImage 45, S61–S72 (2009)
Fadaeieslam, M.J., Soryani, M., Fathy, M.: Efficient key frames selection for panorama generation from video. J. Electron. Imaging 20, 023015 (2011)
Steedly, D., Pal, C., Szeliski, R.: Efficiently registering video into panoramic mosaics. In: ICCV (2005)
Kim, S.H., et al.: Key frame selection algorithms for automatic generation of panoramic images from crowdsourced geo-tagged videos. In: Pfoser, D., Li, K.-J. (eds.) W2GIS 2013. LNCS, vol. 8470, pp. 67–84. Springer, Heidelberg (2014). https://doi.org/10.1007/978-3-642-55334-9_5
Elhamifar, E., Sapiro, G., Sastry, S.S.: Dissimilarity-based sparse subset selection. IEEE Trans. Pattern Anal. Mach. Intell. 38, 2182–2197 (2016)
Crete, F., Dolmiere, T., Ladret, P., Nicolas, M.: The blur effect: perception and estimation with a new no-reference perceptual blur metric. In: Human Vision and Electronic Imaging XII, vol. 6492, p. 64920I (2007)
Supplementary Document. http://www.cse.iitm.ac.in/~vplab/ACCV_18/Supplementary.pdf
Opsahl, T., Agneessens, F., Skvoretz, J.: Node centrality in weighted networks: generalizing degree and shortest paths. Soc. Netw. 32, 245–251 (2010)
Erkan, G., Radev, D.R.: LexPageRank: Prestige in multi-document text summarization. In: Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing (2004)
Santos-Ribeiro, A., Nutt, D.J., McGonigle, J.: Inertial demons: a momentum-based Diffeomorphic registration framework. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) MICCAI 2016. LNCS, vol. 9902, pp. 37–45. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46726-9_5
Chambolle, A.: An algorithm for total variation minimization and applications. J. Math. Imaging Vis. 20, 89–97 (2004)
Goldstein, T., Studer, C., Baraniuk, R.: A field guide to forward-backward splitting with a FASTA implementation. arXiv eprint abs/1411.3406 (2014)
Igarashi, T., Moscovich, T., Hughes, J.F.: As-rigid-as-possible shape manipulation. ACM Trans. Graph. (TOG) 24, 1134–1141 (2005)
Burt, P.J., Adelson, E.H.: A multiresolution spline with application to image mosaics. ACM Trans. Graph. (TOG) 2, 217–236 (1983)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Jacob, G.M., Das, S. (2019). Panorama from Representative Frames of Unconstrained Videos Using DiffeoMeshes. In: Jawahar, C., Li, H., Mori, G., Schindler, K. (eds) Computer Vision – ACCV 2018. ACCV 2018. Lecture Notes in Computer Science(), vol 11363. Springer, Cham. https://doi.org/10.1007/978-3-030-20893-6_11
Download citation
DOI: https://doi.org/10.1007/978-3-030-20893-6_11
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-20892-9
Online ISBN: 978-3-030-20893-6
eBook Packages: Computer ScienceComputer Science (R0)