Abstract
This paper presents a fast image blending approach for combining a set of registered images into a composite mosaic with no visible seams and minimal texture distortion on mobile phones. A unique seam image is generated using two-pass nearest distance transform, which is independent on the order of input images and has good scalability. Each individual mask can be extracted from this seam image quickly. To promote blending speed and reduce memory usage in building high resolution image mosaics on mobile phones, the seam image and mask images are compressed using run-length encoding, and all the following mask operations are built on run-length encoding scheme. Moreover, single instruction multiple data instruction set is used in Gaussian and Laplacian pyramids construction to improve the blending speed further. The use of run-length encoding for masks processing leads to reduced memory requirements and a compact storage of the mask data, and the use of single instruction multiple data instruction set achieves better parallelism and faster execution speed on mobile phones.
Similar content being viewed by others
References
Agarwala A (2007) Efficient gradient-domain compositing using quadtrees. ACM Trans Graphics (TOG) 26(3):94
Agarwala A, Dontcheva M, Agrawala M, Drucker S, Colburn A, Curless B, Salesin D, Cohen M (2004) Interactive digital photomontage. ACM Trans Graphics (TOG) 23(3):294–302
Avidan S, Shamir A (2007) Seam carving for content-aware image resizing. ACM Trans Graph 26 (3):10–19
Bie X, Wang W, Sun H, Huang H, Zhang M (2013) Intent-aware image cloning. Vis Comput 29(6-8):599–608
Brown M, Lowe DG (2007) Automatic panoramic image stitching using invariant features. Int J Comput Vis 74(1):59–73
Burt PJ, Adelson EH (1983) A multiresolution spline with application to image mosaics. ACM Trans Graphics (TOG) 2(4):217–236
Chang C, Chou C, Chang EY (2017) Clkn: Cascaded lucas-kanade networks for image alignment. In: 2017 IEEE Conference on computer vision and pattern recognition (CVPR), pp 3777–3785
DeTone D, Malisiewicz T, Rabinovich A (2016) Deep image homography estimation. arXiv:abs/1606.03798
Ding M, Tong RF (2010) Content-aware copying and pasting in images. Vis Comput 26(6-8):721–729
Efros A, Freeman W (2001) Image quilting for texture synthesis and transfer. In: Proceedings of the 28th annual conference on computer graphics and interactive techniques, SIGGRAPH ’01. ACM, New York, pp 341–346
Farbman Z, Hoffer G, Lipman Y, Cohen-Or D, Lischinski D (2009) Coordinates for instant image cloning. ACM Trans Graph 28(3):67:1–67:9
Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial nets. In: Ghahramani Z, Welling M, Cortes C, Lawrence ND, Weinberger KQ (eds) Advances in neural information processing systems 27. Curran Associates Inc, pp 2672–2680
Gracias N, Mahoor M, Negahdaripour S, Gleason A (2009) Fast image blending using watersheds and graph cuts. Image Vis Comput 27 (5):597–607
Hao C, Chen Y, Wu W, Wu E (2015) An iterated randomized search algorithm for large-scale texture synthesis and manipulations. Vis Comput 31(11):1447–1458
He K, Chang H, Sun J (2013) Rectangling panoramic images via warping. ACM Trans Graphics (TOG) 32(4):1–10
Jia J, Sun J, Tang CK, Shum HY (2006) Drag-and-drop pasting. ACM Trans Graph 25(3):631–637
Kolmogorov V, Zabin R (2004) What energy functions can be minimized via graph cuts?. IEEE Trans Patt Anal and Mach Intel 26(2):147–159
Kwatra V, Schödl A, Essa I, Turk G, Bobick A (2003) Graphcut textures: Image and video synthesis using graph cuts. In: ACM SIGGRAPH 2003 Papers, SIGGRAPH ’03. ACM, New York, pp 277–286
Levin A, Zomet A, Peleg S, Weiss Y (2004) Seamless image stitching in the gradient domain. In: Proceedings of the European conference on computer vision (ECCV ’04). Springer, Prague, pp 377–389
Liu G, Reda FA, Shih KJ, Wang TC, Tao A, Catanzaro B (2018) Image inpainting for irregular holes using partial convolutions. In: The european conference on computer vision (ECCV)
Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J of Comput Vis 60(2):91–110
Muja M, Lowe DG (2014) Scalable nearest neighbor algorithms for high dimensional data. IEEE Trans Patt Anal and Mach Intel 36(11):2227–2240
Pérez P, Gangnet M, Blake A (2003) Poisson image editing. ACM Trans Graphics (TOG) 22(3):313–318
Rocco I, Arandjelovic R, Sivic J (2017) Convolutional neural network architecture for geometric matching. In: 2017 IEEE conference on computer vision and pattern recognition (CVPR), pp 39–48
Summa B, Tierny J, Pascucci V (2012) Panorama weaving: fast and flexible seam processing. ACM Trans Graph 31(4):83:1–83:11
Szeliski R, Uyttendaele M, Steedly D (2011) Fast poisson blending using multi-splines. In: Proceedings of the 2011 IEEE hy (ICCP). IEEE, Pittsburgh, pp 1–8
Wu H, Zheng S, Zhang J, Huang K (2019) GP-GAN: Towards Realistic high-resolution image blending. In: Proceedings of the 27th ACM international conference on multimedia. Association for Computing Machinery, New York, pp 2487–2495
Xie ZF, Shen Y, Ma LZ, Chen ZH (2010) Seamless video composition using optimized mean-value cloning. Vis Comput 26(6-8):1123–1134
Xiong Y, Pulli K (2010) Fast panorama stitching for high-quality panoramic images on mobile phones. IEEE Trans Consum Electron 56(2):298–306
Zhang Y, Tian Y, Kong Y, Zhong B, Fu Y (2018) Residual dense network for image super-resolution. In: The IEEE conference on computer vision and pattern recognition (CVPR)
Acknowledgements
This research was funded by the Natural Science Foundation of China under Grant No. 61662072.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Zhao, Y. Fast image blending for high-quality panoramic images on mobile phones. Multimed Tools Appl 80, 499–516 (2021). https://doi.org/10.1007/s11042-020-09717-5
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-020-09717-5