Skip to main content
SpringerLink
Log in

Content Based Video Retrieval—Methods, Techniques and Applications

  • Chapter
  • First Online:
Advanced Soft Computing Techniques in Data Science, IoT and Cloud Computing

Part of the book series: Studies in Big Data ((SBD,volume 89))

Abstract

Videos are rich information sources than individual images, they are considered as most influential communication media compared to others. The amount of video data produced and dispensed are growing exponentially day by day with the availability of electronic media such as smart phones, handicams etc. and broadband services at cheaper rates, as well as easy accessibility of those media in the market. Video data storage and access founds its applications in different fields such as digital libraries, video on demand, entertainment etc. and these applications are popular and needs regular access of videos from the libraries. All the above said compound reasons demanded the need of development of efficient video management and retrieval systems which can efficiently retrieve videos similar to the query as well as with a less response time. Video retrieval is made possible by searching of the desired video through a user demanded query. The user inputted query may be in the form of representative keywords or a single image or group of images. The video retrieval systems are classified as text based or content based, according to the query inputted by the user. In a text based video retrieval system query is in the form of representative keywords and the database videos are tagged with appropriate text. An example of concept based search and retrieval system is YouTube. The principal drawback in concept based system is mapping of high level or rich semantics to low level features, which is known as semantic gap. Another drawback in concept based video retrieval systems is intention gap, which denotes gap between query at querying time and intention of the search. Several researchers found content based video retrieval (CBVR) system as solution to the drawbacks of a concept based video retrieval system. The main objective this chapter is to provide comprehensive outlook on content based video retrieval (CBVR) system and its recent developments and a new content based video retrieval system that is going to be developed by feature fusion. The generalized algorithm of CBVR and its individual stages such as keyframe extraction and feature extraction also will be described elaborately. This chapter focuses on a brief overview of CBVR, keyframe extraction, feature extraction and feature fusion.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 149.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 199.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 199.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Mashtalir, S., Mashtalir, V.: Spatio-temporal video segmentation. In: Advances in Spatio-Temporal Segmentation of Visual, pp. 161–210. Springer, Cham (2020)

    Google Scholar 

  2. Xu, K., Wen, L., Li, G., Bo, L., Huang, Q.: Spatiotemporal cnn for video object segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1379–1388 (2019)

    Google Scholar 

  3. Tian, Y., Cheng, G., Gelernter, J., Yu, S., Song, C., Yang, B.: Joint temporal context exploitation and active learning for video segmentation. Pattern Recogn. 1(100), (2020)

    Article  Google Scholar 

  4. Jin, Y., Cheng, K., Dou, Q., Heng, P.A.: Incorporating temporal prior from motion flow for instrument segmentation in minimally invasive surgery video. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 440–448. Springer, Cham (2019)

    Google Scholar 

  5. Zhong, S.H., Wu, J., Jiang, J.: Video summarization via spatio-temporal deep architecture. Neurocomputing 7(332), 224–235 (2019)

    Article  Google Scholar 

  6. Zhang, K., Wang, L., Liu, D., Liu, B., Liu, Q., Li, Z.: Dual Temporal Memory Network for Efficient Video Object Segmentation (2003). arXiv preprint arXiv:2003.06125

  7. Ahmadi, N., Akbarizadeh, G.: Iris tissue recognition based on GLDM feature extraction and hybrid MLPNN-ICA classifier. Neural Comput. Appl. 32(7), 2267–2281 (2020)

    Article  Google Scholar 

  8. Wu, Y.: Research on feature point extraction and matching machine learning method based on light field imaging. Neural Comput. Appl. 31(12), 8157–8169 (2019)

    Article  Google Scholar 

  9. Qin, Y., Zou, J., Tang, B., Wang, Y., Chen, H.: Transient feature extraction by the improved orthogonal matching pursuit and K-SVD algorithm with adaptive transient dictionary. IEEE Trans. Industr. Inf. 16(1), 215–227 (2019)

    Article  Google Scholar 

  10. Wang, R., Shi, Y., Cao, W.: GA-SURF: a new speeded-up robust feature extraction algorithm for multispectral images based on geometric algebra. Pattern Recogn. Lett. 1(127), 11–17 (2019)

    Article  Google Scholar 

  11. Janwe, N., Bhoyar, K.: Semantic concept based video retrieval using convolutional neural network. SN Appl. Sci. 2(1), 80 (2020)

    Article  Google Scholar 

  12. Francis, D., Anh Nguyen P, Huet B, Ngo CW. Fusion of multimodal embeddings for ad-hoc video search. In: Proceedings of the IEEE International Conference on Computer Vision Workshops, pp. 0–0 (2019)

    Google Scholar 

  13. Zhang, J., Mei, K., Zheng, Y., Fan, J.: Exploiting mid-level semantics for large-scale complex video classification. IEEE Trans. Multimed. 21(10), 2518–2530 (2019)

    Article  Google Scholar 

  14. Prathiba, T., Kumari, R.S.: Content based video retrieval system based on multimodal feature grouping by KFCM clustering algorithm to promote human–computer interaction. J. Ambient Intell. Humaniz. Comput. 13, 1–5 (2020)

    Google Scholar 

  15. Zeng, J., Liu, M., Fu, X., Gu, R., Leng, L.: Curvature bag of words model for shape recognition. IEEE Access. 29(7), 57163–57171 (2019)

    Article  Google Scholar 

  16. Agosti, M., Marchesin, S., Silvello, G., Vezzani, F., Di Nunzio, G.M., Tellez, D., Hoppener, D., Verhoef, C., Grunhagen, D., Nierop, P., Drozdzal, M.: Learning unsupervised knowledge-enhanced representations to reduce the semantic gap in information retrieval. ACM Trans. Inf. Syst. 1(1) (2020)

    Google Scholar 

  17. Song, G., Tan, X.: Deep code operation network for multi-label image retrieval. Comput. Vis. Image Underst. 1(193), (2020)

    Article  Google Scholar 

  18. Bommisetty, R.M., Prakash, O., Khare, A.: Keyframe extraction using Pearson correlation coefficient and color moments. Multimed. Syst. 18, 1–33 (2019)

    Google Scholar 

  19. Mounika, B.R., Prakash, O., Khare, A.: Key frame extraction using uniform local binary pattern. In: 2018 Second International Conference on Advances in Computing, Control and Communication Technology (IAC3T), pp. 87–91. IEEE (2018)

    Google Scholar 

  20. Khare, A., Mounika, B.R., Khare, M.: Keyframe extraction using binary robust invariant scalable keypoint features. In: Twelfth International Conference on Machine Vision (ICMV 2019), vol. 11433, p. 1143308. International Society for Optics and Photonics (2020)

    Google Scholar 

  21. Mounika, B.R., Khare, A.: Shot boundary detection using second order statistics of gray level co-occurrence matrix. Res. J. Comput. Inf. Technol. Sci. 5, 1–7 (2017)

    Google Scholar 

  22. Abed, R., Bahroun, S., Zagrouba, E.: KeyFrame extraction based on face quality measurement and convolutional neural network for efficient face recognition in videos. Multimedia Tools Appl. 6, 1–22 (2020)

    Google Scholar 

  23. Li, Y., Kanemura, A., Asoh, H., Miyanishi, T., Kawanabe, M.: Multi-Sensor integration for key-frame extraction from first-person videos. IEEE Access. 9(8), 122281–122291 (2020)

    Article  Google Scholar 

  24. Lokoč, J., Bailer, W., Schoeffmann, K., Münzer, B., Awad, G.: On influential trends in interactive video retrieval: video browser showdown 2015–2017. IEEE Trans. Multimedia 20(12), 3361–3376 (2018)

    Article  Google Scholar 

  25. Dong, J., Li, X., Xu, C., Ji, S., He, Y., Yang, G., Wang, X.: Dual encoding for zero-example video retrieval. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 9346–9355 (2019)

    Google Scholar 

  26. Wu, G., Han, J., Guo, Y., Liu, L., Ding, G., Ni, Q., Shao, L.: Unsupervised deep video hashing via balanced code for large-scale video retrieval. IEEE Trans. Image Process. 28(4), 1993–2007 (2018)

    Article  MathSciNet  Google Scholar 

  27. Lokoč, J., Kovalčík, G., Souček, T., Moravec, J., Čech, P.: VIRET: A video retrieval tool for interactive known-item search. In: Proceedings of the 2019 on International Conference on Multimedia Retrieval, pp. 177–181 (2019)

    Google Scholar 

  28. Zhang, C., Lin, Y., Zhu, L., Liu, A., Zhang, Z., Huang, F.: CNN-VWII: an efficient approach for large-scale video retrieval by image queries. Pattern Recogn. Lett. 15(123), 82–88 (2019)

    Google Scholar 

  29. Kordopatis-Zilos, G., Papadopoulos, S., Patras, I., Kompatsiaris, I.: FIVR: Fine-grained incident video retrieval. IEEE Trans. Multimedia 21(10), 2638–2652 (2019)

    Article  Google Scholar 

  30. Rossetto, L., Gasser R., Lokoc, J., Bailer, W., Schoeffmann, K., Muenzer, B., Soucek, T., Nguyen, P.A., Bolettieri, P., Leibetseder, A., Vrochidis, S.: Interactive video retrieval in the age of deep learning-detailed evaluation of vbs 2019. IEEE Trans. Multimedia. (2020)

    Google Scholar 

  31. Shen, L., Hong, R., Zhang, H., Tian, X., Wang, M.: Video retrieval with similarity-preserving deep temporal hashing. ACM Trans. Multimed. Comput. Commun. Appl. (TOMM) 15(4), 1–6

    Google Scholar 

  32. Sauter, L., Parian, M.A., Gasser, R., Heller, S., Rossetto, L., Schuldt, H.: Combining boolean and multimedia retrieval in vitrivr for large-scale video search. In: International Conference on Multimedia Modeling, pp. 760–765. Springer, Cham (2020)

    Google Scholar 

  33. Zhang, C., Hu, B., Suo, Y., Zou, Z., Ji, Y.: Large-scale video retrieval via deep local convolutional features. Adv. Multimedia. 9, 2020 (2020)

    Google Scholar 

  34. Sandeep, R., Sharma, S., Thakur, M., Bora, P.K.: Perceptual video hashing based on Tucker decomposition with application to indexing and retrieval of near-identical videos. Multimedia Appl. 75(13), 7779–7797 (2016)

    Article  Google Scholar 

  35. Thomas, S.S., Gupta, S., Venkatesh, K.S.: Perceptual synoptic view-based video retrieval using metadata. SIViP 11(3), 549–555 (2017)

    Article  Google Scholar 

  36. Araujo, A., Girod, B.: Large-scale video retrieval using image queries. IEEE Trans. Circuits Syst. Video Technol. 28(6), 1406–1420 (2018)

    Article  Google Scholar 

  37. Shekar, B.H., Uma, K.P., Holla, K.R.: Video clip retrieval based on LBP variance. Procedia Comput. Sci. 1(89), 828–835 (2016)

    Article  Google Scholar 

  38. Mounika, B.R., Khare, A.: Content based video retrieval using histogram of gradients and frame fusion. In: Twelfth International Conference on Machine Vision (ICMV 2019), vol. 11433, p. 114332J. International Society for Optics and Photonics (2020)

    Google Scholar 

  39. Shi, Y., Yang, H., Gong, M., Liu, X., Xia, Y.: A fast and robust key frame extraction method for video copyright protection. J. Electr. Comput. Eng. (2017)

    Google Scholar 

  40. Kannappan, S., Liu, Y., Tiddeman, B.: DFP-ALC: automatic video summarization using distinct frame patch index and appearance based linear clustering. Pattern Recogn. Lett. 120, 8–16 (2019)

    Article  Google Scholar 

  41. Liu, X.M., Hao, A.M., Zhao, D.: Optimization-based key frame extraction for motion capture animation. Vis. Comput. 29(1), 85–95 (2013)

    Article  Google Scholar 

  42. Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)

    Article  Google Scholar 

  43. Bay, H., Tuytelaars, T., Van Gool, L.: Surf: speeded up robust features. In: European Conference on Computer Vision, pp. 404–417. Springer, Berlin, Heidelberg (2006)

    Google Scholar 

  44. Porebski, A., Vandenbroucke, N., Macaire, L.: Haralick feature extraction from LBP images for color texture classification. In: 2008 First Workshops on Image Processing Theory, Tools and Applications, pp. 1–8. IEEE (2008)

    Google Scholar 

  45. Liu, C.L.: Normalization-cooperated gradient feature extraction for handwritten character recognition. IEEE Trans. Pattern Anal. Mach. Intell. 29(8), 1465–1469 (2007)

    Article  Google Scholar 

  46. Yaji, G.S., Sarkar, S., Manikantan, K., Ramachandran, S.: DWT feature extraction based face recognition using intensity mapped unsharp masking and laplacian of gaussian filtering with scalar multiplier. Procedia Technol. 1(6), 475–484 (2012)

    Article  Google Scholar 

  47. Rosten, E., Drummond, T.: Machine learning for high-speed corner detection. In: European Conference on Computer Vision, pp. 430–443. Springer, Berlin, Heidelberg (2016)

    Google Scholar 

  48. Calonder, M., Lepetit, V., Strecha, C., Brief, F.P.: Binary robust independent elementary features. In: European Conference on Computer Vision, pp. 778–792 (2010)

    Google Scholar 

  49. Liu, F., Tang, Z., Tang, J.: WLBP: Weber local binary pattern for local image description. Neurocomputing 23(120), 325–335 (2013)

    Article  Google Scholar 

  50. Wang, W., Li, J., Huang, F., Feng, H.: Design and implementation of Log-Gabor filter in fingerprint image enhancement. Pattern Recogn. Lett. 29(3), 301–308 (2008)

    Article  Google Scholar 

  51. Poongothai, E., Suruliandi, A.: Global and local oriented gabor texture histogram for person re-identification. Braz. Arch. Biol. Technol. 62 (2019)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Electronics and Communication Engineering, National Institute of Technology Tiruchirappalli, Tiruchirappalli, India

    Reddy Mounika Bommisetty & P. Palanisamy

  2. Department of Electronics and Communication Engineering, University of Allahabad, Prayagraj, India

    Ashish Khare

Authors
  1. Reddy Mounika Bommisetty
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. P. Palanisamy
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ashish Khare
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Reddy Mounika Bommisetty .

Editor information

Editors and Affiliations

  1. Department of Computer Science and Application, North Orissa University, Bhubaneswar, India

    Sujata Dash

  2. School of Computer Science and Engineering, Orissa Engineering College, Bhubaneswar, Odisha, India

    Subhendu Kumar Pani

  3. Faculty of Electrical Engineering and Computer science, Machine Intelligence Research Labs (MIR Labs), Auburn, WA, USA

    Ajith Abraham

  4. Department of Family and Community Health, University of Maryland, Baltimore, MD, USA

    Yulan Liang

Rights and permissions

Reprints and permissions

Copyright information

© 2021 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Bommisetty, R.M., Palanisamy, P., Khare, A. (2021). Content Based Video Retrieval—Methods, Techniques and Applications. In: Dash, S., Pani, S.K., Abraham, A., Liang, Y. (eds) Advanced Soft Computing Techniques in Data Science, IoT and Cloud Computing. Studies in Big Data, vol 89. Springer, Cham. https://doi.org/10.1007/978-3-030-75657-4_4

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-75657-4_4

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-75656-7

  • Online ISBN: 978-3-030-75657-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation