Skip to main content
SpringerLink
Log in

Bayesian fuzzy clustering and deep CNN-based automatic video summarization

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

The expansion of growth in the generation of video data in various organizations causes an urgent requirement for effectual video summarization methods. This paper devises a novel optimization-driven deep learning technique for video summarization. The aim is to give an automated video summarization. Initially, the video data is extracted from the database. Then, the representative frame selection is done using Bayesian fuzzy clustering (BFC). After that, the frames are then temporally segmented, wherein each segment is modelled as a representative frame, which is generated by clustering the temporal segment into clusters. These segments are selected from each cluster closest to the cluster center. The next step is fine refining that is performed using Deep convolution neural network (Deep CNN), which helps to refine the final frame set. The Deep CNN is trained using the proposed Lion deer hunting (LDH) algorithm. The LDH algorithm is the integration of the Deer hunting optimization algorithm (DHOA) and Lion optimization algorithm (LOA). Thus, the final frames obtained by the proposed LDH-based Deep CNN are employed for video summarization. Here, the final frames are adapted to play as a continuous output video. The developed LDH-based Deep CNN offered enhanced performance than other techniques with the highest precision of 0.841, highest recall of 0.810, and highest F1-Score of 0.888.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24

Similar content being viewed by others

Data availability

The datasets analyzed during the current study are available in the VIOLENT-FLOWS repository, https://www.openu.ac.il/home/hassner/data/violentflows/, SumMe repository, https://paperswithcode.com/dataset/summe, and TvSum repository, https://paperswithcode.com/dataset/tvsum-1.

References

  1. Abonyi J, Feil B, Nemeth S, Arva P (2003) Fuzzy clustering based segmentation of time-series. In: International Symposium on Intelligent Data Analysis, Springer, pp. 275–285

  2. Acha AR, Pritch Y, Peleg S (2006) Making a long video short: Dynamic video synopsis. In: Proceedings CVPR, pp. 435–441

  3. Aote SS, Potnurwar A (2018) An automatic video annotation framework based on two level keyframe extraction mechanism. Multimed Tools Appl:1–20

  4. Boothalingam R (2018) Optimization using lion algorithm: a biological inspiration from lion’s social behavior. Evol Intel 11(1):31–52

    Article  Google Scholar 

  5. Brammya G, Praveena S, NinuPreetha NS, Ramya R, Rajakumar BR, Binu D (2019) Deer hunting optimization algorithm: A new nature-inspired meta-heuristic paradigm. Comput J

  6. Chen K, Franko K, Sang R (2021) Structured model pruning of convolutional networks on tensor processing units. arXiv preprint arXiv:2107.04191

  7. Choi T-M, Chan HK, Yue X (2016) Recent development in big data analytics for business operations and risk management. IEEE Trans Cybern 47:81–92

    Article  Google Scholar 

  8. Cong Y, Yuan JS, Luo JB (2012) Towards scalable summarization of consumer videos via sparsedictionary selection. IEEE Trans Multimed 14(1):66–75

    Article  Google Scholar 

  9. Ejaz N, Mehmood I, Baik SW (2014) Feature aggregation based visual attention model for videosummarization. ComputElectrEng 40(3):993–1005

    Google Scholar 

  10. Fei M, Jiang W, Mao W (2017) A novel compact yet rich key frame creation method for compressed video summarization. Multimed Tools Appl 77(10):11957–11977

    Article  Google Scholar 

  11. Glenn TC, Zare A, Gader PD (2014) Bayesian fuzzy clustering. IEEE Trans Fuzzy Syst 23(5):1545–1561

    Article  Google Scholar 

  12. Hannane R, Elboushaki A, Afdel K (2018) MSKVS: Adaptive mean shift-based keyframe extraction for video summarization and a new objective verification approach. J Vis Commun Image Represent 55:179–200

    Article  Google Scholar 

  13. He Y, Gao C, Sang N, Qu Z, Han J (2017) Graph coloring based surveillance video synopsis. Neurocomputing 225:64–79

    Article  Google Scholar 

  14. Huang C, Wang H (2019) Novel key-frames selection framework for comprehensive video summarization. IEEE Trans Circuit Syst Video Technol:1–1

  15. Hussain T, Muhammad K, Ullah A, Cao Z, Baik SW, de Albuquerque VHC (2019) Cloud-assisted multiview video summarization using CNN and bidirectional LSTM. IEEE Trans Indus Inform 16(1):77–86

    Article  Google Scholar 

  16. Jadhav JN, Arunkumar B (2018) Web page recommendation system using laplace correction dependent probability and Chronological dragonfly-based clustering. Int J Eng Technol (UAE) 7(3.27):290–302

    Article  Google Scholar 

  17. Jog VV, Pande V (2014) Security of outsourced data in cloud using Dynamic Auditing. Int J Sci, Eng Comput Technol 4(12):392

    Google Scholar 

  18. Li L, Zhou K, Xue GR, Video summarization via transferrable structured learning. In: Proceedings of International conference on world wide web, WWW 2011, hyderabad, India, pp 287–296, March 28 – April 2011.

  19. Muhammad K, Tanveer H, Del Ser J, Palade V, De Albuquerque VHC (2019) DeepReS: a deep learning-based video summarization strategy for resource-constrained industrial surveillance scenarios. IEEE Trans Indus Inform:1–1

  20. Mundur P, Rao Y, Yesha Y (2006) Keyframe-based video summarization using Delaunay clustering. Int J Digit Libr 6(2):219–232

    Article  Google Scholar 

  21. Ngo CW, Pong TC, Zhang HJ (2002) Motion-based video representation for scene change detection. IntJ Comput Vis 50(2):127–142

    Article  Google Scholar 

  22. Pande D, Jog VV (2014) Enhancing Security of outsourced data in cloud using Dynamic Auditing.

  23. Puttaswamy MR (2020) Improved deer hunting optimization algorithm for video based salient object detection. Multimed Res 3(3)

  24. Senthil Murugan T, Jog VV (2019) Systematic investigation and performance study of authentication and authorization techniques of Internet of Things. Int J Knowledge-based Intel Eng Syst 23(2):61–76

    Google Scholar 

  25. Song J, Gao L, Liu L, Zhu X, Sebe N (2018) Quantization-based hashing: a general framework for scalable image and video retrieval. Pattern Recogn 75:175–187

    Article  Google Scholar 

  26. SumMe database taken from, (n.d.) “https://paperswithcode.com/dataset/summe”.

  27. Taha M, Ali A, Lloret J, Gondim PRL, Canovas A (2021) An automated model for the assessment of QoE of adaptive video streaming over wireless networks. Multimed Tools Appl 80:26833–26854

    Article  Google Scholar 

  28. Taha M, Canovas A, Lloret J, Ali J (2021) A QoE adaptive management system for high definition video streaming over wireless networks. Telecommun Syst 77(1):63–81

    Article  Google Scholar 

  29. Thomas SS, Gupta S, Subramanian VK (2017) Perceptual video summarization—a new framework for video summarization. IEEE Trans Circuit Syst Video Technol 27(8):1790–1802

    Article  Google Scholar 

  30. Tu F, Yin S, Ouyang P, Tang S, Liu L, Wei S (2017) Deep convolutional neural network architecture with reconfigurable computation patterns. IEEE Trans Very Large Scale Integ (VLSI) Syst 25(8):2220–2233

    Article  Google Scholar 

  31. TvSum database taken from, (n.d.) “https://paperswithcode.com/dataset/tvsum-1”.

  32. Ullah A, Muhammad K, Del Ser J, Baik SW, Albuquerque V (2018) Activity recognition using temporal optical flow convolutional features and multi-layer LSTM. IEEE Trans Ind Electron

  33. VIOLENT-FLOWS DATABASE taken from, “https://www.openu.ac.il/home/hassner/data/violentflows/” Accessed on February 2021.

  34. Wang M, Hong R, Li G (2012) Event driven web video summarization by tag localization and key-shot identification. IEEE Trans Multimed 14(4):975–985

    Article  Google Scholar 

  35. Wang X, Nie X, Liu X, Wang B, Yin Y (2020) Modality correlation-based video summarization. Multimed Tools Appl:1–16

  36. Wu J, Zhong S, Ma Z, Heinen SJ, Jiang J, (2019) Foveated convolutional neural networks for video summarization. Multimed Tools Appl

  37. Zhang L, Jing P, Su Y, Zhang C, Shaoz L (2016) SnapVideo: personalized video generation for a sightseeing trip. IEEE Trans Cybern 47:3866–3878

    Article  Google Scholar 

  38. Zhu X, Guo K, Fang H, Chen L, Ren S, Bin H (2021) Cross view capture for stereo image super-resolution. IEEE Trans Multimed 24:3074–3086

    Article  Google Scholar 

  39. Zhu X, Guo K, Ren S, Bin H, Min H, Fang H (2021) Lightweight image super-resolution with expectation-maximization attention mechanism. IEEE Trans Circuit Syst Video Technol 32(3):1273–1284

    Article  Google Scholar 

Download references

Funding

None.

Author information

Authors and Affiliations

  1. GLA University, Mathura, India

    Anshy Singh & Manoj Kumar

Authors
  1. Anshy Singh
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Manoj Kumar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Anshy Singh.

Ethics declarations

Conflict of interest

None.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Singh, A., Kumar, M. Bayesian fuzzy clustering and deep CNN-based automatic video summarization. Multimed Tools Appl 83, 963–1000 (2024). https://doi.org/10.1007/s11042-023-15431-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-023-15431-9

Keywords

Search

Navigation