Skip to main content
SpringerLink
Log in

Video stitching based on multi-view spatiotemporal feature points and grid-based matching

  • Original Article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

Video stitching enables creation of videos with a wide field of view from videos captured from ordinary, probably moderately displaced, cameras. In this paper, a novel video stitching algorithm based on multi-view feature detection and registration using region-based statistics has been proposed. While stitching the videos captured using moving cameras, the motion in those cameras will affect the alignment of video frames. Movements in the cameras and in the objects will affect the temporal consistency while stitching the videos. Temporally reliable feature points are considered to maintain both spatial and temporal consistency using multi-view spatiotemporal feature points in this paper. Experiments have been carried out on different data sets, and the results showed the better and effective performance of the proposed approach.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Andersen, D., Popescu, V., Cabrera, M.E., Shanghavi, A., Gomez, G., Marley, S., Mullis, B., Wachs, J.: Virtual annotations of the surgical field through an augmented reality transparent display. Vis. Comput. 32(11), 1481–1498 (2016)

    Article  Google Scholar 

  2. Bay, H., Tuytelaars, T., Van Gool, L.: Surf: speeded up robust features. In: European Conference on Computer Vision, Springer, Berlin, pp 404–417 (2006)

  3. Bian, J., Lin, W.Y., Matsushita, Y., Yeung, S.K., Nguyen, T.D., Cheng, M.M.: Gms: Grid-based motion statistics for fast, ultra-robust feature correspondence. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, pp 2828–2837 (2017)

  4. Brown, M., Lowe, D.G.: Automatic panoramic image stitching using invariant features. Int. J. Comput. Vis. 74(1), 59–73 (2007)

    Article  Google Scholar 

  5. Chakraborty, B., Holte, M.B., Moeslund, T.B., Gonzalez, J.: Selective spatio-temporal interest points. Comput. Vis. Image Underst. 116(3), 396–410 (2012)

    Article  Google Scholar 

  6. Dollár, P., Rabaud, V., Cottrell, G., Belongie, S.: Behavior recognition via sparse spatio-temporal features. In: 2nd Joint IEEE International Workshop on Visual Surveillance and Performance Evaluation of Tracking and Surveillance, 2005, IEEE, pp 65–72 (2005)

  7. Eden, A., Uyttendaele, M., Szeliski, R.: Seamless image stitching of scenes with large motions and exposure differences. In: 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, IEEE, vol 2, pp 2498–2505 (2006)

  8. Guo, H., Liu, S., He, T., Zhu, S., Zeng, B., Gabbouj, M.: Joint video stitching and stabilization from moving cameras. IEEE Trans. Image Process. 25(11), 5491–5503 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  9. Harris, C., Stephens, M.: A combined corner and edge detector. In: Alvey Vision Conference, Citeseer vol 15, pp 10–5244 (1988)

  10. Jia, J., Tang, C.K.: Eliminating structure and intensity misalignment in image stitching. In: Tenth IEEE International Conference on Computer Vision, 2005. ICCV 2005, IEEE, vol 2, pp 1651–1658 (2005)

  11. Jia, J., Tang, C.K.: Image stitching using structure deformation. IEEE Trans. Pattern Anal. Mach. Intell. 30(4), 617–631 (2008)

    Article  Google Scholar 

  12. Jiang, W., Gu, J.: Video stitching with spatial-temporal content-preserving warping. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp 42–48 (2015)

  13. Kim, B.S., Choi, K.A., Park, W.J., Kim, S.W., Ko, S.J.: Content-preserving video stitching method for multi-camera systems. IEEE Trans. Consum. Electron. 63(2), 109–116 (2017)

    Article  Google Scholar 

  14. Laptev, I., Marszalek, M., Schmid, C., Rozenfeld, B.: Learning realistic human actions from movies. In: IEEE Conference on Computer Vision and Pattern Recognition, 2008. CVPR 2008, IEEE, pp 1–8 (2008)

  15. Lei, Y., Luo, W., Wang, Y., Huang, J.: Video sequence matching based on the invariance of color correlation. IEEE Trans. Circuits Syst. Video Technol. 22(9), 1332–1343 (2012)

    Article  Google Scholar 

  16. Levi, G., Hassner, T.: Latch: learned arrangements of three patch codes. In: 2016 IEEE Winter Conference on Applications of Computer Vision (WACV), IEEE, pp 1–9 (2016)

  17. Levin, A., Zomet, A., Peleg, S., Weiss, Y.: Seamless image stitching in the gradient domain. In: European Conference on Computer Vision, Springer, pp 377–389 (2004)

  18. Li, J., Xu, W., Zhang, J., Zhang, M., Wang, Z., Li, X.: Efficient video stitching based on fast structure deformation. IEEE Trans Cybernet 45(12), 2707–2719 (2015)

    Article  Google Scholar 

  19. Liu, Y., Yu, D., Chen, X., Li, Z., Fan, J.: Top-sift: the selected sift descriptor based on dictionary learning. The Visual Computer pp 1–11 (2018)

  20. Lu, S.P., Ceulemans, B., Munteanu, A., Schelkens, P.: Spatio-temporally consistent color and structure optimization for multiview video color correction. IEEE Trans. Multimedia 17(5), 577–590 (2015)

    Article  Google Scholar 

  21. Neubeck, A., Van Gool, L.: Efficient non-maximum suppression. In: 18th International Conference on Pattern Recognition, 2006. ICPR 2006, IEEE, vol 3, pp 850–855 (2006)

  22. Nie, Y., Su, T., Zhang, Z., Sun, H., Li, G.: Dynamic video stitching via shakiness removing. IEEE Trans. Image Process. 27(1), 164–178 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  23. Oikonomopoulos, A., Patras, I., Pantic, M.: Spatiotemporal localization and categorization of human actions in unsegmented image sequences. IEEE Trans. Image Process. 20(4), 1126–1140 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  24. Perazzi, F., Sorkine-Hornung, A., Zimmer, H., Kaufmann, P., Wang, O., Watson, S., Gross, M.: Panoramic video from unstructured camera arrays. Comput. Gr. Forum 34, 57–68 (2015)

    Article  Google Scholar 

  25. Sipiran, I., Bustos, B.: Harris 3d: a robust extension of the harris operator for interest point detection on 3d meshes. Vis. Comput. 27(11), 963 (2011)

    Article  Google Scholar 

  26. Tomasi, C., Kanade, T.: Detection and tracking of point features (1991)

  27. Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)

    Article  Google Scholar 

  28. Willems, G., Tuytelaars, T., Van Gool, L.: An efficient dense and scale-invariant spatio-temporal interest point detector. In: European Conference on Computer Vision, Springer, pp 650–663 (2008)

  29. Wong, S.F., Cipolla, R.: Extracting spatiotemporal interest points using global information. In: IEEE 11th International Conference on Computer Vision, 2007. ICCV 2007, IEEE, pp 1–8 (2007)

  30. Xu, W., Mulligan, J.: Performance evaluation of color correction approaches for automatic multi-view image and video stitching. In: 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, pp 263–270 (2010)

  31. Zaragoza, J., Chin, T.J., Brown, M.S., Suter, D.: As-projective-as-possible image stitching with moving dlt. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 2339–2346 (2013)

  32. Zhang, F., Liu, F.: Parallax-tolerant image stitching. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 3262–3269 (2014)

Download references

Author information

Authors and Affiliations

  1. Department of Electronics Engineering, Madras Institute of Technology, Anna University, Chennai, 600044, India

    K. Krishnakumar & S. Indira Gandhi

Authors
  1. K. Krishnakumar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. S. Indira Gandhi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to K. Krishnakumar.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (avi 56977 KB)

Supplementary material 2 (avi 56977 KB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Krishnakumar, K., Indira Gandhi, S. Video stitching based on multi-view spatiotemporal feature points and grid-based matching. Vis Comput 36, 1837–1846 (2020). https://doi.org/10.1007/s00371-019-01780-w

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-019-01780-w

Keywords

Search

Navigation