Circuits, Systems, and Signal Processing

, Volume 38, Issue 4, pp 1695–1716 | Cite as

Compressed Domain Video Abstraction Based on I-Frame of HEVC Coded Videos

  • Ali Reza Yamghani
  • Farzad ZargariEmail author


Video abstraction allows indexing, searching, browsing and evaluating a video only by accessing its useful contents. Several studies have been done in this field, but most of them are in pixel domain and require decoding process. It makes these methods more time and process consuming than compressed domain video abstraction. In this paper, we present a new video abstraction method in H.265/HEVC compressed domain, HVAIF. The method is based on the normalized histogram of extracted I-frame prediction modes from an H.265/HEVC coded video. The frames’ similarity is calculated by intersecting their I-frame prediction modes’ histogram. The similarity measure detects and removes redundant key-frames to increase the quality of final video abstraction. Moreover, we employ fuzzy c-means clustering to categorize similar frames and extract key-frames as representatives of the entire video frames. The interpretation of the results shows that using the proposed method achieves on average 86% accuracy and 19% error rate in compressed domain video abstraction which is higher than the other tested methods in the pixel domain. Also, it has an acceptable robustness to coding parameters, and on average it generates video key-frames that are closer to human summaries.


Video abstraction Clustering Prediction modes’ histogram Compressed video Key-frame extraction Compressed domain feature vector 


  1. 1.
    J. Almeida, N.J. Leite, R.D.S. Torres, Online video summarization on compressed domain. J. Vis. Commun. Image Represent. 24(6), 729–738 (2013). CrossRefGoogle Scholar
  2. 2.
    S.E.D. Avila, A.P.B. Lopes, A. Da Luz, A.D.A. Araújo, VSUMM: a mechanism designed to produce static video summaries and a novel evaluation method. Pattern Recognit. Lett. 32(1), 56–68 (2011). CrossRefGoogle Scholar
  3. 3.
    S. Cha, Comprehensive survey on distance/similarity measures between probability density functions. Int. J. Math. Models Methods Appl. Sci. 1(4), 300–307 (2007). Google Scholar
  4. 4.
    T. Chheng, Video Summarization Using Clustering (Department of Computer Science University of California, Irvine, 2007)Google Scholar
  5. 5.
    S. De Bruyne, D. Van Deursen, J. De Cock, W. De Neve, P. Lambert, R. Van de Walle, A compressed-domain approach for shot boundary detection on H.264/AVC bit streams. Signal Process. Image Commun. 23(7), 473–489 (2008). CrossRefGoogle Scholar
  6. 6.
    D. DeMenthon, V. Kobla, D. Doermann, Video summarization by curve simplification. In: Proceedings of the ACM International Conference on Multimedia, New York, USA (1998), pp. 211–218.
  7. 7.
    R.O. Duda, P.E. Hart, D.G. Stork, Pattern Classification, 2nd edn. (Wiley, New York, 2001)zbMATHGoogle Scholar
  8. 8.
    N. Ejaz, T.B. Tariq, S.W. Baik, Adaptive key frame extraction for video summarization using an aggregation mechanism. J. Vis. Commun. Image Represent. 23(7), 1031–1040 (2012). CrossRefGoogle Scholar
  9. 9.
    M. Furini, F. Geraci, M. Montangero, M. Pellegrini, STIMO: STIll and MOving video storyboard for the web scenario. Multimed. Tools Appl. 46(1), 47–69 (2010). CrossRefGoogle Scholar
  10. 10.
    H.265/HEVC Reference Software, Last seen on Feb 2017
  11. 11.
    A. Hanjalic, H. Zhang, An Integrated scheme for automated video abstraction based on unsupervised cluster-validity analysis. IEEE Trans. Circuits Syst. 9(8), 1280–1289 (1999). Google Scholar
  12. 12.
    J. He, F. Yang, Y. Zhou, High-speed implementation of rate-distortion optimised quantisation for H.265/HEVC. IET Image Process. 9(8), 652–661 (2015). CrossRefGoogle Scholar
  13. 13.
    L. Herranz, J.M. Martínez, An efficient summarization algorithm based on clustering and bitstream extraction. In: Proceedings of International Conference on Multimedia and Expo (2009), pp. 654–657.
  14. 14.
    W. Hu, S. Member, N. Xie, L. Li, X. Zeng, S. Maybank, A survey on visual content-based video indexing and retrieval. IEEE Trans. Syst. Man Cybern. Part C Appl. Rev. 41(6), 797–819 (2011). CrossRefGoogle Scholar
  15. 15.
    J. Kavitha, P.A.J. Rani, Static and multiresolution feature extraction for video summarization. Procedia Comput. Sci. 47(C), 292–300 (2015). CrossRefGoogle Scholar
  16. 16.
    J. Li, T. Yao, Q. Ling, T. Mei, Detecting shot boundary with sparse coding for video summarization. Neurocomputing 266, 66–78 (2017). CrossRefGoogle Scholar
  17. 17.
    Y. Li, T. Zhang, D. Tretter, An overview of video abstraction techniques an overview of video abstraction techniques. Imaging, 1–23 (2001). Retrieved from
  18. 18.
    T. Liu, X. Zhang, J. Feng, K.T. Lo, Shot reconstruction degree: a novel criterion for key frame selection. Pattern Recognit. Lett. 25(12), 1451–1457 (2004). CrossRefGoogle Scholar
  19. 19.
    A.G. Money, H. Agius, Video summarization: a conceptual framework and survey of the state of the art. J. Vis. Commun. Image Represent. 19(2), 121–143 (2008). CrossRefGoogle Scholar
  20. 20.
    P. Mundur, Y. Rao, Y. Yesha, Key-frame based video summarization using Delaunay clustering. Int. J. Digit. Libr. 6(2), 219–232 (2006). CrossRefGoogle Scholar
  21. 21.
    J.H. Oh, Q. Wen, S. Hwang, J. Lee, Video abstraction. In: Video Data Management and Information Retrieval (2005), pp. 321–346Google Scholar
  22. 22.
    F. Rahmani, F. Zargari, Compressed domain visual information retrieval based on I-frames in HEVC. Multimed. Tools Appl. (2016). Google Scholar
  23. 23.
    G.-H. Song, Q.G. Ji, Z.-M. Lu, Z.D. Fang, Z.H. Xie, A novel video abstraction method based on fast clustering of the regions of interest in keyframes. Int. J. Electron. Commun. (AEÜ) 68, 237–243 (2014). CrossRefGoogle Scholar
  24. 24.
    X. Song, G. Fan, Joint key-frame extraction and object segmentation for content-based video analysis. IEEE Trans. Circuits Syst. Video Technol. 16(7), 904–914 (2006). CrossRefGoogle Scholar
  25. 25.
    M. Srinivas, M.M.M. Pai, R.M. Pai, An improved algorithm for video summarization—a rank based approach. Procedia Comput. Sci. 89, 812–819 (2016). CrossRefGoogle Scholar
  26. 26.
    G.J. Sullivan, J. Ohm, W. Han, T. Wiegand, Overview of the high efficiency video coding. IEEE Trans. Circuits Syst. Video Technol. 22(12), 1649–1668 (2012). CrossRefGoogle Scholar
  27. 27.
    Z. Sun, K. Jia, H. Chen, Video keyframe extraction based on spatial-temporal color distribution. In: Proceedings—4th International Conference on Intelligent Information Hiding and Multimedia Signal Processing, IIH-MSP (2008), pp. 196–199.
  28. 28.
    B.T. Truong, S. Venkatesh, Video abstraction: a systematic review and classification. ACM Trans. Multimed. Comput. Commun. Appl. 3(1), 1–37 (2007). CrossRefGoogle Scholar
  29. 29.
    J. Wu, S. Zhong, J. Jiang, Y. Yang, A novel clustering method for static video summarization. Multimed. Tools Appl. 76(7), 9625–9641 (2017). CrossRefGoogle Scholar
  30. 30.
    L. Xiang-wei, Z. Li-dong, Z. Kai, Hierarchical video summarization extraction algorithm in compressed domain. Phys. Procedia 24, 2360–2366 (2012). CrossRefGoogle Scholar
  31. 31.
    W. Yao, Z. Li, S. Rahardja, Dynamic threshold-based keyframe detection and its application in rate control. IET Image Process. 6(7), 986 (2012). CrossRefGoogle Scholar
  32. 32.
    X.D. Zhang, T.Y. Liu, K.T. Lo, J. Feng, Dynamic selection and effective compression of keyframes for video abstraction. Pattern Recognit. Lett. 24(9–10), 1523–1532 (2003). CrossRefzbMATHGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Department of Computer Engineering, Science and Research BranchIslamic Azad UniversityTehranIran
  2. 2.Department of Information TechnologyResearch Institute for ICTTehranIran

Personalised recommendations