Skip to main content
SpringerLink
Log in

Real time video summarizing using image semantic segmentation for CBVR

  • Special Issue Paper
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

Retrieving the exact video of choice in real time requires searching in annotated videos. Manual annotation is impossible for the huge data available nowadays. Hence, an effective model is proposed for summarizing the videos frame wise using stacked generalization to ensemble different machine learning algorithms. Also, the ranks are given to videos on the basis of the time a particular building or monument appears in the video. The videos are queried using KD tree. Semantic segmentation corresponds to the content of the video and hence the content based video retrieval.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Barshandeh, S., Piri, F., Sangani, S. R.: Hmpa: an innovative hybrid multi-population algorithm based on artificial ecosystem-based and harris hawks optimization algorithms for engineering problems. Eng. Comput. 1–45 (2020)

  2. Dhiman, G., Kumar, V.: Spotted hyena optimizer: a novel bio-inspired based metaheuristic technique for engineering applications. Adv. Eng. Softw. 114, 48–70 (2017)

    Article  Google Scholar 

  3. Barshandeh, S., Haghzadeh, M.: A new hybrid chaotic atom search optimization based on tree-seed algorithm and levy flight for solving optimization problems. Eng. Comput. 1–44 (2020)

  4. Kaur, S., Awasthi, L.K., Sangal, A., Dhiman, G.: Tunicate swarm algorithm: a new bio-inspired based metaheuristic paradigm for global optimization. Eng. Appl. Artif. Intell. 90, 103541 (2020)

    Article  Google Scholar 

  5. Fernandez, J.B., Venkatesh, G.M., Zhang, D., Little, S., O’Connor, N.E.: Semi-automatic multi-object video annotation based on tracking, prediction and semantic segmentation. In: 2019 International Conference on Content-Based Multimedia Indexing (CBMI), Sep. 2019, pp. 1–4

  6. Dhiman, G.: Esa: a hybrid bio-inspired metaheuristic optimization approach for engineering problems. Eng. Comput. 1–31 (2019)

  7. Dhiman, G., Kaur, A.: Stoa: a bio-inspired based optimization algorithm for industrial engineering problems. Eng. Appl. Artif. Intell. 82, 148–174 (2019)

    Article  Google Scholar 

  8. Abdollahian, G., Delp, E.J.: User generated video annotation using geo-tagged image databases. In: 2009 IEEE International Conference on Multimedia and Expo, June 2009, pp. 610–613

  9. Kletz, S., Leibetseder, A., Schoeffmann, K.: A comparative study of video annotation tools for scene understanding: yet (not) another annotation tool. In: Proceedings of the 10th ACM Multimedia Systems Conference, pp. 133–144. ACM (2019)

  10. Liu, F., Wang, Y., Wang, F.-C., Zhang, Y.-Z., Lin, J.: Intelligent and secure content-based image retrieval for mobile users. IEEE Access 7, 119209–119222 (2019)

    Article  Google Scholar 

  11. Real, L.G.S., Bueno, R., Ribeiro, M.X.: Evaluating boundary conditions and hierarchical visualization in cbir. In: 2019 23rd International Conference Information Visualisation (IV), pp. 68–73 (2019)

  12. Latif, A., Rasheed, A., Sajid, U., Ahmed, J., Ali, N., Ratyal, N.I., Zafar, B., Dar, S.H., Sajid, M., Khalil, T.: Content-based image retrieval and feature extraction: a comprehensive review. Math. Probl. Eng. 2019 1–21 (2019)

  13. Ye, W., Chen, H., Zhang, Z., Liu, Y., Weng, S., Chang, C.: Hybrid scheme of image’s regional colorization using mask r-cnn and Poisson editing. IEEE Access 7, 115901–115913 (2019)

    Article  Google Scholar 

  14. Safdari, M., Moallem, P., Satari, M.: Sift detector boosted by adaptive contrast threshold to improve matching robustness of remote sensing panchromatic images. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 12(2), 675–684 (2019)

    Article  Google Scholar 

  15. Chiu, L., Chang, T., Chen, J., Chang, N.Y.: Fast sift design for real-time visual feature extraction. IEEE Trans. Image Process. 22(8), 3158–3167 (2013)

    Article  Google Scholar 

  16. Vijayan, V., Pushpalatha, K.: A comparative analysis of rootsift and sift methods for drowsy features extraction. Procedia Comput. Sci. 171, 436–445 (2020)

    Article  Google Scholar 

  17. Li, Y., Yang, C., Zhang, L., Xia, R., Fan, L., Xie, W.: A novel surf based on a unified model of appearance and motion-variation. IEEE Access 6, 31065–31076 (2018)

    Article  Google Scholar 

  18. Fanqing, M., Fucheng, Y.: A tracking algorithm based on orb. In: Proceedings 2013 International Conference on Mechatronic Sciences, Electric Engineering and Computer (MEC). IEEE, pp. 1187–1190 (2013)

  19. Rosten, E., Porter, R., Drummond, T.: Faster and better: a machine learning approach to corner detection. IEEE Trans. Pattern Anal. Mach. Intell. 32(1), 105–119 (2008)

    Article  Google Scholar 

  20. Lam, S.-K., Jiang, G., Wu, M., Cao, B.: Area-time efficient streaming architecture for fast and brief detector. IEEE Trans. Circuits Syst. II Express Br. 66(2), 282–286 (2018)

    Article  Google Scholar 

  21. Li, M.J., Ng, M.K., Cheung, Y., Huang, J.Z.: Agglomerative fuzzy k-means clustering algorithm with selection of number of clusters. IEEE Trans. Knowl. Data Eng. 20(11), 1519–1534 (2008)

    Article  Google Scholar 

  22. Zhang, Q.: The application of improved particle swarm optimization in slab stacking problem. In: IEEE International Conference on Artificial Intelligence and Computer Applications (ICAICA) 2020, pp. 150–154 (2020)

  23. Sigletos, G., Paliouras, G., Spyropoulos, C.D., Hatzopoulos, M.: Combining information extraction systems using voting and stacked generalization. J. Mach. Learn. Res. 6(Nov), 1751–1782 (2005)

    MathSciNet  MATH  Google Scholar 

  24. Alsmadi, M.K.: An efficient similarity measure for content based image retrieval using memetic algorithm. Egypt. J. Basic Appl. Sci. 4(2), 112–122 (2017)

    Google Scholar 

  25. Bozas, K., Izquierdo, E.: Large scale sketch based image retrieval using patch hashing. In: International symposium on visual computing, pp. 210–219. Springer (2012)

Download references

Author information

Authors and Affiliations

  1. Indian Institute of Technology, Delhi, India

    Rahul Jain

  2. Indian Institute of Information Technology, Nagpur, India

    Pooja Jain & Tapan Kumar

  3. Department of Computer Science, Government Bikram College of Commerce, Punjabi University, Patiala, Punjab, India

    Gaurav Dhiman

Authors
  1. Rahul Jain
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Pooja Jain
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tapan Kumar
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Gaurav Dhiman
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Gaurav Dhiman.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jain, R., Jain, P., Kumar, T. et al. Real time video summarizing using image semantic segmentation for CBVR. J Real-Time Image Proc 18, 1827–1836 (2021). https://doi.org/10.1007/s11554-021-01151-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11554-021-01151-6

Keywords

Search

Navigation