Skip to main content
SpringerLink
Log in

A survey of recent work on video summarization: approaches and techniques

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

The volume of video data generated has seen an exponential growth over the years and video summarization has emerged as a process that can facilitate efficient storage, quick browsing, indexing, fast retrieval and quick sharing of the content. In view of the vast literature available on different aspects of video summarization approaches and techniques, a need has arisen to summarize and organize various recent research findings, future research focus and trends, challenges, performance measures and evaluation and datasets for testing and validations. This paper investigates into the existing video summarization frameworks and presents a comprehensive view of the existing approaches and techniques. It highlights the recent advances in the techniques and discusses the paradigm shift that has occurred over the last two decades in the area, leading to considerable improvement. Attempts are made to consolidate the most significant findings right from the basic summarization structure to the classification of summarization techniques and noteworthy contributions in the area. Additionally, the existing datasets categorized domain-wise for the purpose of video summarization and evaluation are enumerated. The present study would be helpful in: assimilating important research findings and data for ready reference, identifying groundwork and exploring potential directions for further research.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Ajmal M, Ashraf MH, Shakir M, Abbas Y and Shah FA (2012) Video summarization: techniques and classification. In: International Conference on Computer Vision and Graphics pp. 1–13. https://doi.org/10.1007/978-3-642-33564-8_1

  2. Angadi S, Naik V (2014), “Entropy based fuzzy C means clustering and key frame extraction for sports video summarization”, in fifth international conference on signal and image processing, pp. 271-279.

  3. Aparício M, Figueiredo P, Raposo F, Martins de Matos D, Ribeiro R, Marujo L (2016) Summarization of films and documentaries based on subtitles and scripts. Pattern Recogn Lett 73:7–12

    Article  Google Scholar 

  4. Atencio P, German ST, Branch JW, Delrieux C (2019) Video summarization by deep visual and categorical diversity. IET Comput Vis 13(6):569–577

    Article  Google Scholar 

  5. Barbeiri TTDS, Goularte R (2020) Content selection criteria for news multi-video summarization based on human strategies. International Journal on Digital Libraries, 1–14

  6. Basavarajaiah M, Sharma P (2019) Survey of de domain video summarization techniques. ACM Comput Surv 52(6):1–29

    Article  Google Scholar 

  7. Baskurt KB, Samet R (2019) Video synopsis: a survey. Comput Vis Image Underst 181:26–38

    Article  Google Scholar 

  8. Cao Y et al. (2013) Recognise human activities from partially observed videos. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 2658–2665. doi: https://doi.org/10.1109/CVPR.2013.343.

  9. Chang SF (2003) Content- based video summarization and adaptation for ubiquitous media access. In: 12th international conference on image analysis and processing, pp. 494-496, doi: https://doi.org/10.1109/ICIAP.2003.1234098.

  10. Chen Y, Zhang B (2014) Surveillance video summarization by jointly applying moving object detection and tracking. International Journal of Computational Vision and Robotics 4(3):212–234

    Article  Google Scholar 

  11. Chen T, Lu A, Hu SM (2012) Visual storylines: semantic visualization of movie sequence. Comput Graph 36(4):241–249

    Article  Google Scholar 

  12. Chen B, Chen Y, Chen F (2017) Video to text summary: joint video summarization and captioning with recurrent neural networks. Proceedings of the British Machine Vision Conference (BMVC) 118:1–118.14. https://doi.org/10.5244/C.31.118

    Article  Google Scholar 

  13. Choudary C, Liu T (2007) Summarization of visual content in instructional videos. IEEE Transactions on Multimedia 9(7):1443–1455

    Article  Google Scholar 

  14. Chu WS, Song Y and Jaimes A (2015) Video co-summarization: Video summarization by visual co-occurrence. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3584–3592. doi:https://doi.org/10.1109/CVPR.2015.7298981

  15. Cong Y, Yuan J, Luo J (2012) Towards scalable summarization of consumer videos via sparse dictionary selection. IEEE Transactions on Multimedia 14(1):66–75

    Article  Google Scholar 

  16. Coppola C, Cosar S, Faria DR, Belloto N (2020) Social activity recognition on continuous RGB-D video sequences. Int J Soc Robot 12:201–215

    Article  Google Scholar 

  17. Cosar S, Donatiello G, Bogorny V, Garate C, Alvares LO, Bremond F (2017) Towards abnormal trajectory and event detection in video surveillance. IEEE Transactions on Circuits and Systems for Video Technology 27(3):683–695

    Article  Google Scholar 

  18. Dang CT, Radha H (2014) Heterogeneity image patch index and its application to consumer video summarization. IEEE Trans Image Process 23(6):2704–2718

    Article  MathSciNet  MATH  Google Scholar 

  19. De Aliva SEF et al (2011) VSUMM: a mechanism designed to produce static video summaries and a novel evaluation method. Pattern Recogn Lett 32(1):56–68

    Article  Google Scholar 

  20. De Silva GC, Yamasaki T and Aizawa K (2005) Evaluation of video summarization for a large number of cameras in ubiquitous home. In: proceedings of the 13th annual ACM international conference on multimedia, pp. 820-828. doi: https://doi.org/10.1145/1101149.1101329.

  21. Duque D, Santos H and Cortez P (2007) Prediction of abnormal behaviors for intelligent video surveillance systems. In: IEEE Symposium on Computational Intelligence and Data Mining, pp. 362–367. doi: https://doi.org/10.1109/CIDM.2007.368897.

  22. Evangelopoulos G, et al. (2008) Movie summarization based on audiovisual saliency detection. In: 15th IEEE international conference on image processing, pp. 2528-2531, doi: https://doi.org/10.1109/ICIP.2008.4712308.

  23. Evangelopoulos G et al (n.d.) Multimodal saliency and Fusion for Movie Summarization Based on Aural, Visual and Textual Attention. IEEE Transactions on Multimedia 15(7):1553–1568

  24. Fakhar B, Kanan HR, Behrad A (2019) Event detection in soccer videos using unsupervised learning of Spatio-temporal features based on pooled spatial pyramid model. Multimed Tools Appl 78(12):16995–17025

    Article  Google Scholar 

  25. Fei M, Jian W, Mao W (2017) Memorable and rich video summarization. J Vis Commun Image Represent 42:207–217

    Article  Google Scholar 

  26. Fu Y, Guo Y, Zhu Y, Liu F, Song C, Zhou Z (2010) Multi-View Video Summarization. IEEE Transactions on Multimedia 12(7):717–729

    Article  Google Scholar 

  27. Garcia AM, Tan C, Lim JH, Tan AH (2017) Summarization of egocentric videos: a comprehensive survey. IEEE Transactions on Human-Machine Systems 47(1):65–76

    Google Scholar 

  28. Goldman DB, Curless B, Salesin D, Seitz SM (2006) Schematic storyboarding for video visualization and editing. ACM Transactions on Graphics (TOG) 25(3):862–871

    Article  Google Scholar 

  29. Gong B, Chao WL, Grauman K and Sha F (2014) Diverse sequential subset selection for supervised video summarization. In advances in neural information processing systems, pp. 2069-2077.

  30. Gowsikhaa D, Abirami S, Baskaran R (2014) Automated human behavior analysis from surveillance videos: a survey. Artif Intell Rev 42(4):747–765

    Article  Google Scholar 

  31. Guo Z, Gao L, Zhen X, Zou F, Shen F, Zheng K (2016) Spatial and temporal scoring for egocentric video summarization. Neurocomputing 208:299–308

    Article  Google Scholar 

  32. Gygli M, Grabner H, Riemenschneider H and Van Gool L (2014) Creating summaries from user videos. In: European Conference on Computer Vision, pp. 505–520. https://doi.org/10.1007/978-3-319-10584-0_33

  33. Gygli M, Grabner H and Van Gool L (2015) Video summarization by learning submodular mixture of objectives. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 3090–3098. doi: https://doi.org/10.1109/CVPR.2015.7298928.

  34. Haq IU, Muhammad K, Hussain T, Kwon S, Sodanil M, Baik SW, Lee MY (2019) Movie scene segmentation using object detection and set theory. International Journal of Distributed Sensor Networks 15(6):155014771984527

    Article  Google Scholar 

  35. Herranz L, Martinez JM (2010) A framework for scalable summarization of video. IEEE Transactions on Circuits and Systems for Video Technology 20(9):1265–1270

    Article  Google Scholar 

  36. Hesham M, Hani B, Fouad N and Amer E (2018) Smart trailer: automatic generation of movie trailer using only subtitles. In: 2018 first international workshop on deep and representation learning (IWRDL), pp. 26-30. doi:https://doi.org/10.1109/IWDRL.2018.8358211.

  37. Hussein N, Gavves E and Smeulders AW (2019) VideoGraph: Recognising minutes- long human activities in videos”. arXiv preprint arXiv:1905.05143.

  38. Ide I et al. (2017) Summarization of news videos considering the consistency of auditory and visual contents. In: IEEE International Symposium on Multimedia, pp. 193–199, doi: https://doi.org/10.1109/ISM.2017.33.

  39. Javed A, Irtaza A, Malik H, Mahmood MT, Adnan S (2019) Multimodal framework based on audio-visual features for summarization of cricket videos. IET Image Process 13(4):615–622

    Article  Google Scholar 

  40. Ji Z, Zhang Y, Pang Y, Li X (2018) Hypergraph dominant set based multi-video summarization. Signal Process 148:114–123

    Article  Google Scholar 

  41. Ji H, Hooshyar D, Kim K, Lim H (2019) A semantic – based video scene segmentation using a deep neural network. J Inf Sci 45(6):833–844

    Article  Google Scholar 

  42. Ji Z, Xiong K, Pang Y, Li X (2020) Video summarization with attention-based encoder- decoder networks. IEEE Transactions on Circuits and Systems for Video Technology 30(6):1709–1717

    Article  Google Scholar 

  43. Ji Z, Zhao Y, Pang Y, Li X (2020) Cross-modal guidance based auto-encoder for multi-video summarization. Pattern Recogn Lett 135:131–137. https://doi.org/10.1016/j.patrec.2020.04.011

    Article  Google Scholar 

  44. Jiang Y, Cui K, Peng B and Xu C (2019) Comprehensive video understanding: video summarization with content-based video recommender design. In: 2019 IEEE/CVF international conference on computer vision workshop (ICCVW), pp. 1562-1569. doi: https://doi.org/10.1109/ICCVW.2019.00195.

  45. Joho H, Jose JM, Valenti R, Sebe N (2009) Exploiting facial expressions for affective video summarization. Proceedings of the ACM International Conference on Image and Video Retrieval, Article 31:1–8. https://doi.org/10.1145/1646396.1646435

    Article  Google Scholar 

  46. Kanehira A, Van Gool L, Ushiku Y and Harada T (2018) Viewpoint – aware video summarization. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7435–7444. doi: https://doi.org/10.1109/CVPR.2018.00776.

  47. Kato K, Ide I, Deguchi D and Murase H (2014) Estimation of the representative story transition in a chronological semantic structure of news topics. In: Proceedings of International Conference on Multimedia Retrieval, pp. 487–490. doi:https://doi.org/10.1145/2578726.2578800.

  48. Kavitha J, Rani PAJ (2015) Static and multi resolution feature extraction for video summarization. Procedia Computer Science 47:292–300

    Article  Google Scholar 

  49. Khan AA, Shao J, Ali W, Tumrani S (2020) Content- aware summarization of broadcast sports videos: an audio-visual feature extraction approach. Neural Process Lett 52:1–24. https://doi.org/10.1007/s11063-020-10200-3

    Article  Google Scholar 

  50. Khan G, Jabeen S, Khan MZ, Khan MUG, Iqbal R (2020) Blockchain-enabled deep semantic video-to-video summarization for IoT devices. Computers & Electrical Engineering 81:81. https://doi.org/10.1016/j.compeleceng.2019.106524

    Article  Google Scholar 

  51. Khosla A, Hamid R, Lin CJ and Sundaresan N (2013) Large-scale video summarization using web-image priors. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2698–2705. doi: https://doi.org/10.1109/CVPR.2013.348.

  52. Khosla A, Raju AS, Torallba A and Olivia A (2015) Understanding and predicting image memorability at a large scale. In: IEEE International Conference on Computer Vision, pp. 2390–2398, doi: https://doi.org/10.1109/ICCV.2015.275.

  53. Kim C, Hwang JN (2002) Object-based video abstraction for video surveillance systems. IEEE Transactions on Circuits and Systems for Video Technology 12(12):1128–1138

    Article  Google Scholar 

  54. Kim G, Sigal L and Xing EP (2014) Joint summarization of large-scale collections of web images and videos for storyline reconstruction. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 4225–4232. doi: https://doi.org/10.1109/CVPR.2014.538.

  55. Klaser A, Marszalek M, Schmid C (2008) A Spatio–temporal descriptor based on 3D gradients. Proceedings of British Machine Vision Conference 99:1–99.10. https://doi.org/10.5244/C.22.99

    Article  Google Scholar 

  56. Kota BU, Ahmed S, Stone A, Davila K, Stelur S, Govindaraju V (2019) Summarizing Lecture Videos by Key Handwritten Content Regions. In: 2019 International conference on document analysis and recognition workshops (ICDARW) 4: 13–18. IEEE.

  57. Kwon J, Lee KM (2015) A unified framework for event summarization and rare event detection from multiple views. IEEE Trans Pattern Anal Mach Intell 37(9):1737–1750

    Article  Google Scholar 

  58. Lai PK, Decombas M, Moutet K and Laganiere R (2016) Video summarization of surveillance cameras. In: IEEE International Conference on Advanced Video and Signal based Surveillance, pp. 286–294, doi:https://doi.org/10.1109/AVSS.2016.7738018.

  59. Lee YJ, Grauman K (2015) Predicting important objects for egocentric video summarization. Int J Comput Vis 114(1):38–55

    Article  MathSciNet  Google Scholar 

  60. Lee YJ, Ghosh J and Grauman K (2012) Discovering important people and objects for egocentric video summarization. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1346–1353. doi:https://doi.org/10.1109/CVPR.2012.6247820.

  61. Lee S, Sung J, Yu Y and Kim G (2018) A memory network approach for story-based temporal summarization of 360 videos. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1410–1419.

  62. Lew MS, Sebe N, Djeraba C, Jain R (2006) Content-based multimedia information retrieval: state of the art and challenges. ACM Trans Multimed Comput Commun Appl 2(1):1–19

    Article  Google Scholar 

  63. Li Y, Merialdo B (2016) Multimedia maximal marginal relevance for multi-video summarization. Multimed Tools Appl 75(1):199–220

    Article  Google Scholar 

  64. Li B, Pan H and Sezan I (2003) A general framework for sports video summarization with its application to soccer. In: IEEE international conference on acoustics, speech, and signal processing, pp. III-169. doi: https://doi.org/10.1109/ICASSP.2003.1199134.

  65. Lie WN and Lai CM (2004) News video summarization based on spatial and motion feature analysis. In: Pacific-Rim Conference on Multimedia, pp. 246–255. https://doi.org/10.1007/978-3-540-30542-2_31

  66. Liu T, Kender JR (2002) Rule-based semantic summarization of instructional videos. In: Proceedings of International Conference on Image Processing pp. I-I. IEEE. doi: https://doi.org/10.1109/ICIP.2002.1038095.

  67. Lu Z and Grauman K (2013) Story-driven summarization for egocentric video. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2714–2721, doi: https://doi.org/10.1109/CVPR.2013.350

  68. Lu G, Zhou Y, Li X, Yan P (2017) Unsupervised, efficient and scalable key-frame selection for automatic summarization of surveillance videos. Multimed Tools Appl 76(5):6309–6331

    Article  Google Scholar 

  69. Ma M, Mei S, Wan S, Hou J, Wang Z, Feng DD (2020) Video summarization via block sparse dictionary selection. Neurocomputing 378:197–209

    Article  Google Scholar 

  70. Mademlis I, Tefas A, Nikolaidis N, Pitas I (2016) Multimodal stereoscopic movie summarization conforming to narrative characteristics. IEEE Trans Image Process 25(12):5828–5840

    Article  MathSciNet  MATH  Google Scholar 

  71. Mademlis I, Tefas A, Nikolaidis N and Pitas I (2017) summarization of human activity videos via low – rank approximation. In: IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 1627-1631. doi: https://doi.org/10.1109/ICASSP.2017.7952432.

  72. Mahasseni B, Lam M and Todorovic S (2017) Unsupervised video summarization with adversarial LSTM networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2982–2991.doi: https://doi.org/10.1109/CVPR.2017.318.

  73. Matthews CE, Kuncheva LI, Yousefi P (2019) Classification and comparison of on-line video summarization methods. Mach Vis Appl 30:507–518

    Article  Google Scholar 

  74. Mendi E, Clemente HB, Bayrak C (2013) Sports video summarization based on motion analysis. Computers & Electrical Engineering 39(3):790–796

    Article  Google Scholar 

  75. Meng J, Wang S, Wang H, Yuan J, Tan YP (2018) Video summarization via multiview representative selection. IEEE Trans Image Process 27(5):2134–2145

    Article  MathSciNet  MATH  Google Scholar 

  76. Money AG, Agius H (2008) Video summarization: a conceptual framework and survey of the state of art. J Vis Commun Image Represent 19(2):121–143

    Article  Google Scholar 

  77. Moses TM and Balachandran K (2017) A classified study on semantic analysis of video summarization. In: 2017 international conference on algorithms, methodology, models and applications in emerging technologies (ICAMMAET), pp 1-6. doi: https://doi.org/10.1109/ICAMMAET.2017.8186684

  78. Niebles JC, Chen CW and Fei-Fei L (2010) Modeling temporal structure of decomposable motion segments for activity classification. In: European Conference on Computer Vision, pp. 392–405, https://doi.org/10.1007/978-3-642-15552-9_29

  79. Oh S et al (2011) A large-scale benchmark dataset for event recognition in surveillance video. CVPR 2011, pp. 3153–3160, doi: https://doi.org/10.1109/CVPR.2011.5995586.

  80. Otani M, Nakashima Y, Rahtu E and Heikkila J (2019) Rethinking the evaluation of video summaries. In: IEEE/CVF conference on computer vision and pattern recognition (CVPR), pp. 7588-7596. doi: https://doi.org/10.1109/CVPR.2019.00778.

  81. Ouyang JQ, Liu R (2013) Ontology reasoning scheme for constructing meaningful sports video summarization. IET Image Process 7(4):324–334

    Article  Google Scholar 

  82. Panda R, Roy-Chowdhury AK (2017) Multi-view surveillance video summarization via joint embedding and sparse optimization. IEEE Transactions on Multimedia 19(9):2010–2021

    Article  Google Scholar 

  83. Panda R, Mithun NC, Roy-Chowdhury AK (2017) Diversity-aware multi-video summarization. IEEE Trans Image Process 26(10):4712–4724

    Article  MathSciNet  Google Scholar 

  84. Panda R, Kuanar SK, Chowdhury AS (2018) Nyström approximated temporally constrained multisimilarity spectral clustering approach for movie scene detection. IEEE Transactions on Cybernetics 48(3):836–847

    Article  Google Scholar 

  85. Paul M, Haque SM, Chakraborty S (2013) Human detection in surveillance videos and its applications- a review. EURASIP Journal on Advances in Signal processing 2013(1):176

    Article  Google Scholar 

  86. Peng WT, Chu WT, Chang CT, Chou CN, Huang WJ, Chang WY, Hung YP (2011) Editing by viewing: automatic home video summarization by viewing behaviour analysis. IEEE Transactions on Multimedia 13(3):539–550

    Article  Google Scholar 

  87. Pereira MHR, Padua FLC, Dalip DH et al (2019) Multimodal approach for tension levels estimation in news videos. Multimed Tools Appl 78:23783–23808

    Article  Google Scholar 

  88. Pirsiavash H and Ramanan D (2012) Detecting activities of daily living in first-person camera views. In: IEEE conference on computer vision and pattern recognition (CVPR), pp. 2847-2854, doi: https://doi.org/10.1109/CVPR.2012.6248010.

  89. Potapov D, Douze M, Harchaoui Z and Schmid C (2014) Category-specific video summarization. In: European Conference on Computer Vision, pp. 540–555. https://doi.org/10.1007/978-3-319-10599-4_35

  90. Rahman MR, Subhlok J and Shah S (2020) Visual summarization of lecture video segments for enhanced navigation. arXiv preprint arXiv:2006.02434.

  91. Rani S, Kumar M (2020) Social media video summarization using multi-visual features and Kohnen's self-organizing map. Inf Process Manag 57(3):102190

    Article  Google Scholar 

  92. Rav-Acha A, Pritch Y and Peleg S (2006) Making a long video short: dynamic video synopsis. In: 2006 IEEE computer society conference on computer vision and pattern recognition (CVPR’06), pp. 435-441. doi: https://doi.org/10.1109/CVPR.2006.179

  93. Safdarnejad SM, Liu X, Udpa L, Andrus B, Wood J and Craven D (2015) Sports videos in the wild (SVW): a video dataset for sports analysis. In: 11th IEEE international conference and workshops on automatic face and gesture recognition, pp. 1-7, doi: https://doi.org/10.1109/FG.2015.7163105.

  94. Sah S et al. (2017): Semantic text summarization of long videos. In: IEEE Winter Conference on Applications of Computer Vision, pp. 989–997. doi: https://doi.org/10.1109/WACV.2017.115.

  95. Sasithradevi A, Roomi SMM (2020) A new pyramidal opponent color-shape model-based video shot boundary detection. J Vis Commun Image Represent 67:102754

    Article  Google Scholar 

  96. Scovanner P, Ali S and Shah M (2007) A 3-dimensional sift descriptor and its application to action recognition. In: Proceedings of the ACM International Conference on Multimedia, pp. 357–360. doi:https://doi.org/10.1145/1291233.1291311.

  97. Sharghi A, Gong B and Shah M (2016) Query-focused extractive video summarization. In: European conference on computer vision, pp. 3-19. Springer. doi:https://doi.org/10.1007/978-3-319-46484-8_1.

  98. Sharghi A, Laurel JS and Gong B (2017) Query-focused video summarization: dataset, evaluation and a memory network based approach. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2127–2136. doi:https://doi.org/10.1109/CVPR.2017.229.

  99. Song Y, Vallmitjana J, Stent A and Jaimes A (2015) TVSum: summarizing web videos using titles. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 5179–5187. doi: https://doi.org/10.1109/CVPR.2015.7299154

  100. Sreeja MU, Kovoor BC (2019) Towards genre-specific frameworks for video summarization: a survey. J Vis Commun Image Represent 62:340–358

    Article  Google Scholar 

  101. Tejero-de Pablos A, Nakashima Y, Sato T, Yokoya N, Linna M, Rahtu E (2018) Summarization of user-generated sports video by using deep action recognition features. IEEE Transactions on Multimedia 20(8):2000–2011

    Article  Google Scholar 

  102. Thomas SS, Gupta S, Subramanian VK (2017) Perceptual video summarization-a new framework for video summarization. IEEE Transactions on Circuits and Systems for Video Technology 27(8):1790–1802

    Article  Google Scholar 

  103. Thomas SS, Gupta S, Subramanian VK (2018) Event detection on roads using perceptual video summarization. IEEE Trans Intell Transp Syst 19(9):2944–2954

    Article  Google Scholar 

  104. Thomas SS, Gupta S, Subramanian VK (2019) Context driven optimized perceptual video summarization and retrieval. IEEE Transactions on Circuits and Systems for Video Technology 29(10):3132–3145

    Article  Google Scholar 

  105. Truong BT, Venkatesh S (2007) Video abstraction: a systematic review and classification. ACM Trans Multimed Comput Commun Appl 3(1):3

    Article  Google Scholar 

  106. Tsai C, Kang LW, Lin CW, Lin W (2013) Scene based movie summarization via role-community networks. IEEE Transactions on Circuits and Systems for Video Technology 23(11):1927–1940

    Article  Google Scholar 

  107. Ul Haq I, Ullah A, Muhammad K, Lee MY, Baik SW (2019) Personalised movie summarization using deep CNN- assisted facial expression recognition. Complexity 2019:1–10. https://doi.org/10.1155/2019/3581419

    Article  Google Scholar 

  108. Vaca-Castano G, Das S, Sousa JP, Lobo ND, Shah M (2017) Improved scene identification and object detection on egocentric vision of daily activities. Comput Vis Image Underst 156:92–103

    Article  Google Scholar 

  109. Varini P, Serra G, Cucchiara R (2017) Personalised egocentric video summarization of cultural tour on user preferences input. IEEE Transactions on Multimedia 19(12):2832–2845

    Article  Google Scholar 

  110. Vasudevan AB, Gygli M, Volokitin A and Van Gool L (2017) Query-adaptive video summarization via quality aware relevance estimation. In: proceedings of the 25th ACM international conference on multimedia, pp. 582-590. https://doi.org/10.1145/3123266.3123297.

  111. Wu J, Zhong SH, Liu Y (2020) Dynamic graph convolutional network for multi-video summarization. Pattern Recogn 107:107382. https://doi.org/10.1016/j.patcog.2020.107382

    Article  Google Scholar 

  112. Xiong B, Kim G and Sigal L (2015) Storyline representation of egocentric videos with an applications to story-based search. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 4525–4533. doi: https://doi.org/10.1109/ICCV.2015.514

  113. Xu J, Mukherjee L, Li Y, Warner J, Rehg JM and Singh V (2015) Gaze-enabled egocentric video summarization via constrained submodular maximization. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2235–2244. doi: https://doi.org/10.1109/CVPR.2015.7298836

  114. Yu Y, Lee S, Na J, Kang J, and Kim G (2018) A deep ranking model for spatio-temporal highlight detection from a 360 video. arXiv preprint arXiv:1801.10312.

  115. Zhang K, Chao WL, Sha F and Grauman K (2016) Video summarization with long short-term memory. In: European Conference on Computer Vision, pp. 766–782. doi: https://doi.org/10.1007/978-3-319-46478-7_47

  116. Zhang K, Chao W, Sha F and Grauman K (2016) Summary transfer: exemplar-based subset selection for video summarization. In: IEEE conference on computer vision and pattern recognition (CVPR), pp. 1059-1067, doi: https://doi.org/10.1109/CVPR.2016.120.

  117. Zhang Y, Lu H, Zhang L, Ruan X, Sakai S (2016) Video anomaly detection based on locality sensitive hashing filters. Pattern Recogn 59:302–311

    Article  Google Scholar 

  118. Zhang S, Zhu Y, Roy Chowdhury AK (2016) Context – aware surveillance video summarization. IEEE Trans Image Process 25(11):5469–5478

    Article  MathSciNet  MATH  Google Scholar 

  119. Zhang Y, Tao R, Wang Y (2017) Motion-state-adaptive video summarization via spatiotemporal analysis. IEEE Transactions on Circuits and Systems for Video Technology 27(6):1340–1352

    Article  Google Scholar 

  120. Zhang Y, Kampffmeyer M, Zhao X, Tan M (2019) Deep reinforcement learning for query-conditioned video summarization. Appl Sci 9(4):750

    Article  Google Scholar 

  121. Zhao B and Xing EP (2014) Quasi real-time summarization for consumer videos. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2513–2520. doi: https://doi.org/10.1109/CVPR.2014.322.

  122. Zhao B, Li X and Lu X (2018) HSA-RNN: hierarchical structure-adaptive RNN for video summarization. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7405–7414. doi:https://doi.org/10.1109/CVPR.2018.00773.

  123. Zhong H, Shi J and Visontai M (2004) Detecting unusual activity in video. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2004), pp. II-II, doi: https://doi.org/10.1109/CVPR.2004.1315249.

  124. Zhou B, Lapedriza A, Xiao J, Torralba A and Oliva A (2014) Learning deep features for scene recognition using places database. In Advances in neural information processing systems, pp 487–495.

  125. Zhou K, Qiao Y and Xiang T (2017) Deep reinforcement learning for unsupervised video summarization with diversity-representativeness reward. arXiv preprint arXiv:1801.00054.

  126. Zhu X, Wu X, Fan J, Elmagarmid AK, Aref WG (2004) Exploring video content structure for hierarchical summarization. Multimedia Systems 10:98–115

    Article  Google Scholar 

  127. Zhu X, Elmagarmid AK, Xue X, Wu L, Catlin AC (2005) InsightVideo: toward hierarchical video content Organization for Efficient Browsing, summarization and retrieval. IEEE Transactions on Multimedia 7(4):648–666

    Article  Google Scholar 

  128. Zhu X, Loy CC, Gong S (2016) Learning from multiple sources for video summarization. Int J Comput Vis 117:247–268

    Article  MathSciNet  Google Scholar 

  129. Zhukov D et al. (2019) Cross-task weakly supervised learning from instructional videos. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3537–3545

Download references

Author information

Authors and Affiliations

  1. Department of Computer Engineering and Applications, GLA University, Mathura, India

    Vasudha Tiwari & Charul Bhatnagar

Authors
  1. Vasudha Tiwari
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Charul Bhatnagar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Vasudha Tiwari.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tiwari, V., Bhatnagar, C. A survey of recent work on video summarization: approaches and techniques. Multimed Tools Appl 80, 27187–27221 (2021). https://doi.org/10.1007/s11042-021-10977-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-021-10977-y

Keywords

Search

Navigation