Advertisement

Circuits, Systems, and Signal Processing

, Volume 38, Issue 4, pp 1695–1716 | Cite as

Compressed Domain Video Abstraction Based on I-Frame of HEVC Coded Videos

  • Ali Reza Yamghani
  • Farzad ZargariEmail author
Article
  • 59 Downloads

Abstract

Video abstraction allows indexing, searching, browsing and evaluating a video only by accessing its useful contents. Several studies have been done in this field, but most of them are in pixel domain and require decoding process. It makes these methods more time and process consuming than compressed domain video abstraction. In this paper, we present a new video abstraction method in H.265/HEVC compressed domain, HVAIF. The method is based on the normalized histogram of extracted I-frame prediction modes from an H.265/HEVC coded video. The frames’ similarity is calculated by intersecting their I-frame prediction modes’ histogram. The similarity measure detects and removes redundant key-frames to increase the quality of final video abstraction. Moreover, we employ fuzzy c-means clustering to categorize similar frames and extract key-frames as representatives of the entire video frames. The interpretation of the results shows that using the proposed method achieves on average 86% accuracy and 19% error rate in compressed domain video abstraction which is higher than the other tested methods in the pixel domain. Also, it has an acceptable robustness to coding parameters, and on average it generates video key-frames that are closer to human summaries.

Keywords

Video abstraction Clustering Prediction modes’ histogram Compressed video Key-frame extraction Compressed domain feature vector 

References

  1. 1.
    J. Almeida, N.J. Leite, R.D.S. Torres, Online video summarization on compressed domain. J. Vis. Commun. Image Represent. 24(6), 729–738 (2013).  https://doi.org/10.1016/j.jvcir.2012.01.009 CrossRefGoogle Scholar
  2. 2.
    S.E.D. Avila, A.P.B. Lopes, A. Da Luz, A.D.A. Araújo, VSUMM: a mechanism designed to produce static video summaries and a novel evaluation method. Pattern Recognit. Lett. 32(1), 56–68 (2011).  https://doi.org/10.1016/j.patrec.2010.08.004 CrossRefGoogle Scholar
  3. 3.
    S. Cha, Comprehensive survey on distance/similarity measures between probability density functions. Int. J. Math. Models Methods Appl. Sci. 1(4), 300–307 (2007).  https://doi.org/10.1007/s00167-009-0884-z Google Scholar
  4. 4.
    T. Chheng, Video Summarization Using Clustering (Department of Computer Science University of California, Irvine, 2007)Google Scholar
  5. 5.
    S. De Bruyne, D. Van Deursen, J. De Cock, W. De Neve, P. Lambert, R. Van de Walle, A compressed-domain approach for shot boundary detection on H.264/AVC bit streams. Signal Process. Image Commun. 23(7), 473–489 (2008).  https://doi.org/10.1016/j.image.2008.04.012 CrossRefGoogle Scholar
  6. 6.
    D. DeMenthon, V. Kobla, D. Doermann, Video summarization by curve simplification. In: Proceedings of the ACM International Conference on Multimedia, New York, USA (1998), pp. 211–218.  https://doi.org/10.1145/290747.290773
  7. 7.
    R.O. Duda, P.E. Hart, D.G. Stork, Pattern Classification, 2nd edn. (Wiley, New York, 2001)zbMATHGoogle Scholar
  8. 8.
    N. Ejaz, T.B. Tariq, S.W. Baik, Adaptive key frame extraction for video summarization using an aggregation mechanism. J. Vis. Commun. Image Represent. 23(7), 1031–1040 (2012).  https://doi.org/10.1016/j.jvcir.2012.06.013 CrossRefGoogle Scholar
  9. 9.
    M. Furini, F. Geraci, M. Montangero, M. Pellegrini, STIMO: STIll and MOving video storyboard for the web scenario. Multimed. Tools Appl. 46(1), 47–69 (2010).  https://doi.org/10.1007/s11042-009-0307-7 CrossRefGoogle Scholar
  10. 10.
    H.265/HEVC Reference Software, https://hevc.hhi.fraunhofer.de/svn/svn_HEVCSoftware/branches/. Last seen on Feb 2017
  11. 11.
    A. Hanjalic, H. Zhang, An Integrated scheme for automated video abstraction based on unsupervised cluster-validity analysis. IEEE Trans. Circuits Syst. 9(8), 1280–1289 (1999).  https://doi.org/10.1109/76.809162 Google Scholar
  12. 12.
    J. He, F. Yang, Y. Zhou, High-speed implementation of rate-distortion optimised quantisation for H.265/HEVC. IET Image Process. 9(8), 652–661 (2015).  https://doi.org/10.1049/iet-ipr.2014.0849 CrossRefGoogle Scholar
  13. 13.
    L. Herranz, J.M. Martínez, An efficient summarization algorithm based on clustering and bitstream extraction. In: Proceedings of International Conference on Multimedia and Expo (2009), pp. 654–657.  https://doi.org/10.1109/icme.2009.5202581
  14. 14.
    W. Hu, S. Member, N. Xie, L. Li, X. Zeng, S. Maybank, A survey on visual content-based video indexing and retrieval. IEEE Trans. Syst. Man Cybern. Part C Appl. Rev. 41(6), 797–819 (2011).  https://doi.org/10.1109/TSMCC.2011.2109710 CrossRefGoogle Scholar
  15. 15.
    J. Kavitha, P.A.J. Rani, Static and multiresolution feature extraction for video summarization. Procedia Comput. Sci. 47(C), 292–300 (2015).  https://doi.org/10.1016/j.procs.2015.03.209 CrossRefGoogle Scholar
  16. 16.
    J. Li, T. Yao, Q. Ling, T. Mei, Detecting shot boundary with sparse coding for video summarization. Neurocomputing 266, 66–78 (2017).  https://doi.org/10.1016/j.neucom.2017.04.065 CrossRefGoogle Scholar
  17. 17.
    Y. Li, T. Zhang, D. Tretter, An overview of video abstraction techniques an overview of video abstraction techniques. Imaging, 1–23 (2001). Retrieved from http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.84.6173&rep=rep1&type=pdf
  18. 18.
    T. Liu, X. Zhang, J. Feng, K.T. Lo, Shot reconstruction degree: a novel criterion for key frame selection. Pattern Recognit. Lett. 25(12), 1451–1457 (2004).  https://doi.org/10.1016/j.patrec.2004.05.020 CrossRefGoogle Scholar
  19. 19.
    A.G. Money, H. Agius, Video summarization: a conceptual framework and survey of the state of the art. J. Vis. Commun. Image Represent. 19(2), 121–143 (2008).  https://doi.org/10.1016/j.jvcir.2007.04.002 CrossRefGoogle Scholar
  20. 20.
    P. Mundur, Y. Rao, Y. Yesha, Key-frame based video summarization using Delaunay clustering. Int. J. Digit. Libr. 6(2), 219–232 (2006).  https://doi.org/10.1007/s00799-005-0129-9 CrossRefGoogle Scholar
  21. 21.
    J.H. Oh, Q. Wen, S. Hwang, J. Lee, Video abstraction. In: Video Data Management and Information Retrieval (2005), pp. 321–346Google Scholar
  22. 22.
    F. Rahmani, F. Zargari, Compressed domain visual information retrieval based on I-frames in HEVC. Multimed. Tools Appl. (2016).  https://doi.org/10.1007/s11042-016-3391-5 Google Scholar
  23. 23.
    G.-H. Song, Q.G. Ji, Z.-M. Lu, Z.D. Fang, Z.H. Xie, A novel video abstraction method based on fast clustering of the regions of interest in keyframes. Int. J. Electron. Commun. (AEÜ) 68, 237–243 (2014).  https://doi.org/10.1016/j.aeue.2014.03.004 CrossRefGoogle Scholar
  24. 24.
    X. Song, G. Fan, Joint key-frame extraction and object segmentation for content-based video analysis. IEEE Trans. Circuits Syst. Video Technol. 16(7), 904–914 (2006).  https://doi.org/10.1109/TCSVT.2006.877419 CrossRefGoogle Scholar
  25. 25.
    M. Srinivas, M.M.M. Pai, R.M. Pai, An improved algorithm for video summarization—a rank based approach. Procedia Comput. Sci. 89, 812–819 (2016).  https://doi.org/10.1016/j.procs.2016.06.065 CrossRefGoogle Scholar
  26. 26.
    G.J. Sullivan, J. Ohm, W. Han, T. Wiegand, Overview of the high efficiency video coding. IEEE Trans. Circuits Syst. Video Technol. 22(12), 1649–1668 (2012).  https://doi.org/10.1109/TCSVT.2012.2221191 CrossRefGoogle Scholar
  27. 27.
    Z. Sun, K. Jia, H. Chen, Video keyframe extraction based on spatial-temporal color distribution. In: Proceedings—4th International Conference on Intelligent Information Hiding and Multimedia Signal Processing, IIH-MSP (2008), pp. 196–199.  https://doi.org/10.1109/iih-msp.2008.245
  28. 28.
    B.T. Truong, S. Venkatesh, Video abstraction: a systematic review and classification. ACM Trans. Multimed. Comput. Commun. Appl. 3(1), 1–37 (2007).  https://doi.org/10.1145/1198302.1198305 CrossRefGoogle Scholar
  29. 29.
    J. Wu, S. Zhong, J. Jiang, Y. Yang, A novel clustering method for static video summarization. Multimed. Tools Appl. 76(7), 9625–9641 (2017).  https://doi.org/10.1007/s11042-016-3569-x CrossRefGoogle Scholar
  30. 30.
    L. Xiang-wei, Z. Li-dong, Z. Kai, Hierarchical video summarization extraction algorithm in compressed domain. Phys. Procedia 24, 2360–2366 (2012).  https://doi.org/10.1016/j.phpro.2012.02.350 CrossRefGoogle Scholar
  31. 31.
    W. Yao, Z. Li, S. Rahardja, Dynamic threshold-based keyframe detection and its application in rate control. IET Image Process. 6(7), 986 (2012).  https://doi.org/10.1049/iet-ipr.2011.0189 CrossRefGoogle Scholar
  32. 32.
    X.D. Zhang, T.Y. Liu, K.T. Lo, J. Feng, Dynamic selection and effective compression of keyframes for video abstraction. Pattern Recognit. Lett. 24(9–10), 1523–1532 (2003).  https://doi.org/10.1016/S0167-8655(02)00391-4 CrossRefzbMATHGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Department of Computer Engineering, Science and Research BranchIslamic Azad UniversityTehranIran
  2. 2.Department of Information TechnologyResearch Institute for ICTTehranIran

Personalised recommendations