Skip to main content
SpringerLink
Log in

Hybrid multi scale hard switch YOLOv4 network for cricket video summarization

  • Original Paper
  • Published:
Wireless Networks Aims and scope Submit manuscript

Abstract

Cricket is a popular sport with a lengthy duration that makes it challenging to watch in its entirety. Therefore, video summarization techniques are essential to providing viewers with a condensed version of the match's exciting moments. Automated cricket video summarization is difficult due to the sport's regulations and extended sessions. Existing methods often include repetitive shots, making the summary less concise and informative. Therefore, this paper proposes a hybrid video summarization framework that uses audio and text features to extract exciting clips from the raw cricket video. The framework employs the Multi-Scale Hard Switch YOLOv4 (MSHS-YOLOv4) network to accurately detect and label exciting events, including small details such as a ball hitting the stumps. A significance score is computed for each event to generate a summary that includes the most exciting and significant moments. The proposed method eliminates replay shots, reducing redundancy and making the summary more concise. The proposed method combines audio and video features to identify the most exciting moments, uses the MSHS-YOLOv4 network to detect and label exciting events, computes a significance score for each event, and eliminates replay shots to generate a concise summary. The proposed method outperforms existing summarization techniques in terms of accuracy, precision, recall, F1-score, and error. The analysis shows a significant increase in performance compared to the existing methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  1. Hussain, T., Muhammad, K., Ding, W., Lloret, J., Baik, S. W., & de Albuquerque, V. H. (2021). A comprehensive survey of multi-view video summarization. Pattern Recognition., 109, 107567.

    Article  Google Scholar 

  2. Javed, A., Irtaza, A., Khaliq, Y., Malik, H., & Mahmood, M. T. (2019). Replay and key-events detection for sports video summarization using confined elliptical local ternary patterns and extreme learning machine. Applied Intelligence., 49(8), 2899–2917.

    Article  Google Scholar 

  3. Vasudevan, V., Sellappa Gounder, M. (2021). Advances in sports video summarization–a review based on cricket videos. In International conference on industrial, engineering and other applications of applied intelligent systems (pp. 347–359). Springer, Cham.

  4. Bhalla, A., Ahuja, A., Pant, P., & Mittal, A. (2019). A multimodal approach for automatic cricket video summarization. In 2019 6th International conference on signal processing and integrated networks (SPIN) (pp. 146–150). IEEE.

  5. Javed, A., & Ali Khan, A. (2022). Shot classification and replay detection for sports video summarization. Frontiers of Information Technology & Electronic Engineering., 23(5), 790–800.

    Article  Google Scholar 

  6. Agyeman, R., Muhammad, R., & Choi, G. S. (2019). Soccer video summarization using deep learning. In 2019 IEEE Conference on multimedia information processing and retrieval (MIPR) (pp. 270–273). IEEE.

  7. Wu, L., & Li, H. (2021). RETRACTED ARTICLE: Risk assessment of extreme rainfall climate change and sports stadium sports based on video summarization algorithm. Arabian Journal of Geosciences., 14(16), 1–3.

    Article  Google Scholar 

  8. Bora, A., Sharma, S. (2018). A review on video summarization approcahes: recent advances and directions. In 2018 International conference on advances in computing, communication control and networking (ICACCCN) (pp. 601–606). IEEE.

  9. Yan, C., Li, X., Li, G. (2021). A new action recognition framework for video highlights summarization in sporting events. In 2021 16th International conference on computer science & education (ICCSE) (pp. 653–666). IEEE.

  10. Tejero-de-Pablos, A., Nakashima, Y., Sato, T., Yokoya, N., Linna, M., & Rahtu, E. (2018). Summarization of user-generated sports video by using deep action recognition features. IEEE Transactions on Multimedia., 20(8), 2000–2011.

    Article  Google Scholar 

  11. Emon, S. H., Annur, A. H., Xian, A.H., Sultana, K.M., & Shahriar, S. M. (2020). Automatic video summarization from cricket videos using deep learning. In 2020 23rd International conference on computer and information technology (ICCIT) (pp. 1–6). IEEE.

  12. Nandyal, S., Kattimani, S. L. (2021). An efficient umpire key frame segmentation in cricket video using HOG and SVM. In 2021 6th International conference for convergence in technology (I2CT) (pp. 1–7). IEEE.

  13. Zanganeh, A., Jampour, M., Layeghi, K. (2022). IAUFD: A 100k images dataset for automatic football image/video analysis. IET Image processing.

  14. Kolekar, M. H., & Sengupta, S. (2015). Bayesian network-based customized highlight generation for broadcast soccer videos. IEEE Transactions on Broadcasting., 61(2), 195–209.

    Article  Google Scholar 

  15. Muhammad, K., Hussain, T., & Baik, S. W. (2020). Efficient CNN based summarization of surveillance videos for resource-constrained devices. Pattern Recognition Letters., 130, 370–375.

    Article  Google Scholar 

  16. Lin, C., Chen, Y. (2019). Sports video summarization with limited labeling datasets based on 3D neural networks. In 2019 16th IEEE International conference on advanced video and signal based surveillance (AVSS) (pp. 1–6). IEEE.

  17. Zhao, B., Li, H., Lu, X., & Li, X. (2021). Reconstructive sequence-graph network for video summarization. IEEE Transactions on Pattern Analysis and Machine Intelligence., 44(5), 2793–2801.

    Google Scholar 

  18. Zhu, W., Lu, J., Li, J., & Zhou, J. (2020). Dsnet: A flexible detect-to-summarize network for video summarization. IEEE Transactions on Image Processing., 30, 948–962.

    Article  Google Scholar 

  19. Dange, B. J., Kshirsagar, D. B., Khodke, H. E., & Gunjal, S. N. (2022). Automatic video summarization for cricket match highlights using convolutional neural network. In 2022 International conference on smart technologies and systems for next generation computing (ICSTSN) (pp. 1–7). IEEE.

  20. Guntuboina, C., Porwal, A., Jain, P., Shingrakhia, H. (2022). Video summarization for multiple sports using deep learning. In Proceedings of the International e-conference on intelligent systems and signal processing: e-ISSP 2020 (pp. 643–656). Springer Singapore.

  21. Mujtaba, G., Malik, A., & Ryu, E. S. (2022). LTC-SUM: Lightweight client-driven personalized video summarization framework using 2D CNN. IEEE Access., 10, 103041–103055.

    Article  Google Scholar 

  22. Sanabria, M., Precioso, F., Menguy, T. (2021). Hierarchical multimodal attention for deep video summarization. In 2020 25th International conference on pattern recognition (ICPR) (pp. 7977–7984). IEEE.

  23. Basak, H., Kundu, R., Singh, P. K., Ijaz, M. F., Woźniak, M., & Sarkar, R. (2022). A union of deep learning and swarm-based optimization for 3D human action recognition. Scientific Reports, 12(1), 5494.

    Article  Google Scholar 

  24. Li, P., Ye, Q., Zhang, L., Yuan, L., Xu, X., & Shao, L. (2021). Exploring global diverse attention via pairwise temporal relation for video summarization. Pattern Recognition., 111, 107677.

    Article  Google Scholar 

  25. Yaliniz, G., & Ikizler-Cinbis, N. (2021). Using independently recurrent networks for reinforcement learning based unsupervised video summarization. Multimedia Tools and Applications., 80(12), 17827–17847.

    Article  Google Scholar 

  26. Fu, H., & Wang, H. (2021). Self-attention binary neural tree for video summarization. Pattern Recognition Letters., 143, 19–26.

    Article  Google Scholar 

  27. Liu, T., Meng, Q., Huang, J. J., Vlontzos, A., Rueckert, D., & Kainz, B. (2022). Video summarization through reinforcement learning with a 3D spatio-temporal u-net. IEEE Transactions on Image Processing., 31, 1573–1586.

    Article  Google Scholar 

  28. Basavarajaiah, M., & Sharma, P. (2021). GVSUM: Generic video summarization using deep visual features. Multimedia Tools and Applications., 80(9), 14459–14476.

    Article  Google Scholar 

  29. Yasmin, G., Chowdhury, S., Nayak, J., Das, P., & Das, A. K. (2021). Key moment extraction for designing an agglomerative clustering algorithm-based video summarization framework. Neural Computing and Applications. 1–22.

  30. Sahu, A., & Chowdhury, A. S. (2021). First person video summarization using different graph representations. Pattern Recognition Letters., 146, 185–192.

    Article  Google Scholar 

  31. Mahum, R., Irtaza, A., Nawaz, M., Nazir, T., Masood, M., Shaikh, S., & Nasr, E. A. (2023). A robust framework to generate surveillance video summaries using combination of zernike moments and r-transform and deep neural network. Multimedia Tools and Applications., 82(9), 13811–13835.

    Article  Google Scholar 

  32. Ray, A., Kolekar, M. H., Balasubramanian, R., & Hafiane, A. (2023). Transfer learning enhanced vision-based human activity recognition: A decade-long analysis. International Journal of Information Management Data Insights., 3(1), 100142.

    Article  Google Scholar 

  33. Zhao, B., Gong, M., & Li, X. (2022). Hierarchical multimodal transformer to summarize videos. Neurocomputing, 468, 360–369.

    Article  Google Scholar 

  34. Xu, P., Jia, Y., & Jiang, M. (2021). Blind audio source separation based on a new system model and the savitzky-golay filter. Journal of Electrical Engineering., 72(3), 208–212.

    Article  Google Scholar 

  35. Gupta, V., & Mittal, M. (2020). A novel method of cardiac arrhythmia detection in electrocardiogram signal. International Journal of Medical Engineering and Informatics., 12(5), 489–499.

    Article  Google Scholar 

  36. Gupta, V., Mittal, M., & Mittal, V. (2021). Chaos theory and ARTFA: Emerging tools for interpreting ECG signals to diagnose cardiac arrhythmias. Wireless Personal Communications., 118, 3615–3646.

    Article  Google Scholar 

  37. Gao, Z., Feng, A., Song, X., & Wu, X. (2019). Target-dependent sentiment classification with BERT. Ieee Access., 7, 154290–154299.

    Article  Google Scholar 

  38. Yeom, S. K., Seegerer, P., Lapuschkin, S., Binder, A., Wiedemann, S., Müller, K. R., & Samek, W. (2021). Pruning by explaining: A novel criterion for deep neural network pruning. Pattern Recognition., 115, 107899.

    Article  Google Scholar 

Download references

Funding

There is no funding for this study.

Author information

Authors and Affiliations

  1. Department of Electronics and Communication Engineering, C.S.I Institute of Technology, Thovalai, Kanyakumari, Tamil Nadu, India

    D. Minola Davids

  2. Department of Electronics and Communication Engineering, Saveetha Engineering College, Chennai, Tamil Nadu, India

    A. Arul Edwin Raj

  3. Department of Computer Science and Engineering, St. Xavier’s Catholic College of Engineering, Nagercoil, Tamil Nadu, India

    C. Seldev Christopher

Authors
  1. D. Minola Davids
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. A. Arul Edwin Raj
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. C. Seldev Christopher
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors contributed to the study conception and design. Material preparation, data collection and analysis were performed by DMD, A. AERC. SC. The first draft of the manuscript was written by DMD and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript. Conceptualization: DMD; Methodology: DMD; Formal analysis and investigation: DMD, AAER; Writing—original draft preparation: DMD; Writing—review and editing: AAER; Supervision: CSC. All the authors have participated in writing the manuscript and have revised the final version. All authors read and approved the final manuscript.

Corresponding author

Correspondence to D. Minola Davids.

Ethics declarations

Conflict of interest

Authors declares that they have no conflict of interest.

Ethical approval

This article does not contain any studies with human participants and/or animals performed by any of the authors.

Informed consent

There is no informed consent for this study.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Davids, D.M., Raj, A.A.E. & Christopher, C.S. Hybrid multi scale hard switch YOLOv4 network for cricket video summarization. Wireless Netw 30, 17–35 (2024). https://doi.org/10.1007/s11276-023-03449-8

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11276-023-03449-8

Keywords

Search

Navigation