Skip to main content
SpringerLink
Log in

Misalignment-eliminated warping image stitching method with grid-based motion statistics matching

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Aligning images is one of the main goals of image stitching. Limited by matching accuracy and deformability, the results of current mainstream stitching approaches in large parallax scenes usually contain obvious stitching errors. A misalignment-eliminated warping image stitching based on grid-based motion statistics (GMS) matching is proposed. A matching approach from coarse-to-fine composed of oriented FAST and rotated BRIEF (ORB) and GMS is integrated to provide more accurate and higher number of inliers for subsequent warping. The local homography with global similarity transformation constraint is used to warp the images to achieve initial image alignment. For the projection biases after local warping, a post-processing step based on thin plate spline (TPS) is proposed for further correction. Both qualitative and quantitative comparisons in experiments of challenging cases show that this method can accurately align images while maintaining a natural look at the same time.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Bastug E, Bennis M, Medard M, Debbah M (2017) Toward interconnected virtual reality: opportunities, challenges, and enablers. IEEE Commun Mag 55(6):110–117

    Article  Google Scholar 

  2. Bian JW, Lin WY, Matsushita Y et al (2017) GMS: grid-based motion statistics for fast, ultra-robust feature correspondence. In: Proc IEEE Conf Comput Vis Pattern Recogn, pp 2828–2837

    Google Scholar 

  3. Bookstein FL (1989) Principal warps: thin-plate splines and the decomposition of deformations. IEEE Trans Pattern Anal Mach Intell 11(6):567–585

    Article  Google Scholar 

  4. Brown M, Lowe DG (2007) Automatic panoramic image stitching using invariant features. Int J Comput Vis 74(1):59–73

    Article  Google Scholar 

  5. Caesar H, Bankiti V, Lang AH et al (2020) Nuscenes: a multimodal dataset for autonomous driving. In: Proc IEEE/CVF Conf Comput Vis pattern Recogn, pp 11621–11631

    Google Scholar 

  6. Cao Q, Shi Z, Wang P et al (2020) A seamless image-stitching method based on human visual discrimination and attention. F Appl Sci (Basel) 10(4):1462

    Google Scholar 

  7. Chang CH, Sato Y, Chuang YY (2014)Shape-preserving half projective warps for image stitching. In: Proc IEEE Conf. Comput Vis Pattern Recogn, pp 3254–3261

    Google Scholar 

  8. Chen YS, Chuang YY (2016) Natural image stitching with the global similarity prior. In: Proc Eur Conf Comput Vis, pp 186–201

    Google Scholar 

  9. Debabrata G, Kaabouch N (2016) A survey on image mosaicing techniques. J Vis Commun Image Represent 34:1–11

    Article  Google Scholar 

  10. Edward R, Drummond T (2006) Machine learning for high-speed corner detection. In: Comput Vis ECCV

    Google Scholar 

  11. Fischler MA, Bolles RC (1981) Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun Assoc Comp Mach 24(6):381–395

    MathSciNet  Google Scholar 

  12. Gao J, Kim SJ, Brown MS (2011) Constructing image panoramas using dual-homography warping. In: Proc IEEE Conf Comput Vis Pattern Recogn, pp 49–56

    Google Scholar 

  13. Guo Y, Zhao R, Wu S, Wang C (2018) Image capture pattern optimization for panoramic photography. Multimed Tools Appl 77(17):22299–22318

    Article  Google Scholar 

  14. Han J, Pauwels EJ, De Zeeuw P (2013) Visible and infrared image registration in man-made environments employing hybrid visual features. Pattern Recogn Lett 34(1):42–51

    Article  Google Scholar 

  15. Hardy RL (1990) Theory and applications of the multiquadric-biharmonic method 20 years of discovery 1968–1988. Comput Math Appl 19(8–9):163–208

    Article  MathSciNet  Google Scholar 

  16. Harris C, Stephens M (1988) A combined corner and edge detector. In: Proceedings Alvey vision Conference, pp 147–151

    Google Scholar 

  17. Hejazifar H, Hassan K (2018) Fast and robust seam estimation to seamless image stitching. Signal Image Video Process 12(5):885–893

    Article  Google Scholar 

  18. Krishnakumar K, Gandhi SI (2019) Video stitching using interacting multiple model based feature tracking. Multimed Tools Appl 78(2):1375–1397

    Article  Google Scholar 

  19. Li S, Yuan L, Sun J et al (2015)Dual-feature warping-based motion model estimation. In: ICCV, pp 4283–4291

    Google Scholar 

  20. Li J, Wang Z, Lai S, Zhai Y, Zhang M (2018)Parallax-tolerant image stitching based on robust elastic warping. IEEE Trans Multimedia 20(7):1672–1687

    Article  Google Scholar 

  21. Lin WY, Liu S, Matsushita Y et al (2011) Smoothly varying affine stitching. In: Proc IEEE Conf Comput Vis Pattern Recogn, pp 345–352

    Google Scholar 

  22. Lin W, Cheng M, Lu J et al (2014) Bilateral functions for global motion modeling. In: Comput Vis ECCV, pp 341–356

    Google Scholar 

  23. Lin CC, Pankanti SU, Ramamurthy KN, Aravkin AY (2015) Adaptive as-natural-as-possible image stitching. In: Proc IEEE Conf. Comput Vis Pattern Recogn, pp 1155–1163

    Google Scholar 

  24. Lin K, Jiang N, Cheong LF et al (2016) Seagull: seam-guided local alignment for parallax-tolerant image stitching. In: Proc Eur Conf Comput Vis, pp 370–385

    Google Scholar 

  25. Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60(2):91–110

    Article  Google Scholar 

  26. Nie L, Lin C, Liao K, Liu M, Zhao Y (2020) A view-free image stitching network based on global homography. J Vis Commun Image Represent 73:102950

    Article  Google Scholar 

  27. Qiu Z, Tang H, Tian D (2009)Non-rigid medical image registration based on the thin-plate spline algorithm. Proc WRI World Congr Comput Sci Inf Eng 2:522–527

    Google Scholar 

  28. Rublee E, Rabaud V, Konolige K, Bradski G (2011) Orb: an efficient alternative to sift or surf. In: Proc of ICCV, pp 2564–2571

    Google Scholar 

  29. Sawhney HS, Kumar R (1999) True multi-image alignment and its application to mosaicing and lens distortion correction. IEEE Trans Pattern Anal Mach Intell 21:235–243

    Article  Google Scholar 

  30. Wu G, Lin Z, Ding G, Ni Q, Han J (2020) On aggregation of unsupervised deep binary descriptor with weak bits. IEEE Trans Image Process 29:9266–9278

    Article  MathSciNet  Google Scholar 

  31. Yan W, Yue G, Fang Y, Chen H, Tang C, Jiang G (2020) Perceptual objective quality assessment of stereoscopic stitched images. Signal Process 172:107541

    Article  Google Scholar 

  32. Zaragoza J, Chin TJ, Tran QH et al (2014)As-projective-as-possible image stitching with moving dlt. IEEE Trans Pattern Anal Mach Intell 36(7):1285–1298

    Article  Google Scholar 

  33. Zhang G, He Y, Chen W, Jia J, Bao H (2016)Multi-viewpoint panorama construction with wide-baseline images. IEEE Trans Image Process 25(7):3099–3111

    Article  MathSciNet  Google Scholar 

  34. Zhao S, Lau T, Luo J et al (2019) Unsupervised 3D end-to-end medical image registration with volume tweening network. IEEE J Biomed Health Inform 24(5):1394–1404

    Article  Google Scholar 

  35. Zhou H, Kuang Y, Yu Z, Ren S, Zhang Y, Lu T, Ma J (2018) Image deformation with vector-field interpolation based on MRLS-TPS. IEEE Access 6:75886–75898

    Article  Google Scholar 

Download references

Acknowledgments

This paper was supported by the National Natural Science Foundation of China (No. 62071326) and the National Natural Science Foundation of China (No. 61674115).

Author information

Authors and Affiliations

  1. School of Microelectronics, Tianjin University, Tianjin, China

    Zaifeng Shi, Pumeng Wang & Cheng Ding

  2. School of Mathematical Sciences, Tianjin Normal University, Tianjin, China

    Qingjie Cao

  3. College of Intelligence and Computing, Tianjin University, Tianjin, China

    Tao Luo

Authors
  1. Zaifeng Shi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Pumeng Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Qingjie Cao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Cheng Ding
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Tao Luo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zaifeng Shi.

Ethics declarations

Conflict of interest

The authors declare no conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Shi, Z., Wang, P., Cao, Q. et al. Misalignment-eliminated warping image stitching method with grid-based motion statistics matching. Multimed Tools Appl 81, 10723–10742 (2022). https://doi.org/10.1007/s11042-022-12064-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-12064-2

Keywords

Search

Navigation