Advertisement

Multimedia Tools and Applications

, Volume 78, Issue 5, pp 6233–6252 | Cite as

A video hard cut detection using multifractal features

  • Goran ZajicEmail author
  • Ana Gavrovska
  • Irini Reljin
  • Branimir Reljin
Article
  • 31 Downloads

Abstract

Efficient management of video sequences is based on adequate video content description. This description can be used for various purposes in different applications, telecommunication services, video and multimedia systems. Video hard cut detection represents the foundation of temporal video segmentation. In this paper, a new video hard cut detection methodology is proposed using multifractal features. Transition between two shots can be described as color and texture differences within a decoded video sequence. In the proposed methodology we formed specific structures by measuring color differences between frames. The formed structures are used for hard cut candidate detection. This is followed by multifractal representation of texture changes by Hölder exponents. The proposed methodology achieves high performance using more than 750,000 frames, extracted from forty different video sequences, classified by four well known genre groups. Moreover, the proposed hard cut detection achieves high performance regardless of high level video production or complex non-linear editing for different genre groups. This is confirmed by comparison between the proposed methodology and other recent work on hard cut detection.

Keywords

Video Hard cut detection Shot Multifractals Color Texture 

Notes

References

  1. 1.
    Abd-Almageed W (2008) Online, simultaneous shot boundary detection and key frame extraction for sports videos using rank tracing. In proceedings of the IEEE International Conference on Image Processing (ICIP 2008), San Diego, USA, October 12–15, pp. 3200–3203.doi: https://doi.org/10.1109/ICIP.2008.4712476
  2. 2.
    Adcock J, Girgensohn A, Cooper M, Liu T, Wilcox L, Rieffel E (2004) FXPAL experiments for TRECVID 2004. In Proceedings of the TRECVID 2004 Workshop, Gaithersburg, MD, USA NIST, November 15–16Google Scholar
  3. 3.
    Calic J, Izquierdo E (2002) Temporal segmentation of MPEG video streams. EURASIP Journal on Applied Signal Processing 6:561–565.  https://doi.org/10.1155/S1110865702000938 CrossRefGoogle Scholar
  4. 4.
    Damnjanovic U, Izquierdo E (2007) Shot boundary detection using spectral clustering. In Proceedings of the 15th European Signal Processing Conference (EUSIPCO 2007), Poznan, Poland, September 3–9, pp. 1779–1783Google Scholar
  5. 5.
    Donate A, Liu X (2010) Shot Boundary Detection in Videos Using Robust Three-Dimensional Tracking. First International Workshop on Three Dimensional Information Extraction for Video Analysis and Mining (in conjunction with CVPR). San Francisco, California. doi: https://doi.org/10.1109/CVPRW.2010.5543811
  6. 6.
    Dutta D, Saha SK, Chanda B (2016) A shot detection technique using linear regression of shot transition pattern. Multimedia Tools and Applications 75(1):93–113.  https://doi.org/10.1007/s11042-014-2273-y CrossRefGoogle Scholar
  7. 7.
    El khattabi Z, Tabii Y, Benkaddour A (2017) Video shot boundary detection using the scale invariant feature transform and RGB color channels. International Journal of Electrical and Computer Engineering (IJECE) 7(5):2565–2673CrossRefGoogle Scholar
  8. 8.
    Falconer KJ (2003) Fractal geometry: mathematical foundations and applications, 2nd edn. John Wiley & Sons, New YorkCrossRefGoogle Scholar
  9. 9.
    Grecos C, Yang M (2009) An improved rate control algorithm based on a novel shot detection scheme for the H.264/AVC standard. Journal Real-Time Image Processing 4(1):91–106.  https://doi.org/10.1007/s11554-008-0093-x CrossRefGoogle Scholar
  10. 10.
    Gunal E, Canbek S, Adar N (2011) Fractal dimension based shot transition detection in sport videos. J Softw Eng Appl 4(4):235–243.  https://doi.org/10.4236/jsea.2011.44026 CrossRefGoogle Scholar
  11. 11.
    Hanjalic A (2002) Shot-boundary detection: unraveled and resolved? IEEE Trans Circuits and Systems for Video Technology 12(2):90–105.  https://doi.org/10.1109/76.988656 CrossRefGoogle Scholar
  12. 12.
    Jacobs A, Mine A, Ioannidis GT, Herzog O (2004) Automatic shot boundary detection combining color, edge, and motion features of adjacent frames. In proceedings of the TRECVID 2004 Workshop, Gaithersburg, MD. USA. NIST, November 15–16Google Scholar
  13. 13.
    Kawai Y, Sumiyoshi H, Yagi N (2007) Shot Boundary Detection at TRECVID 2007. In Proceedings of the TRECVID 2007 Workshop, Gaithersburg, MD, USA NISTGoogle Scholar
  14. 14.
    Kim T, Park DC, Woo DM, Jeong T, Min SY (2012) Multiclass classifier-based Adaboost algorithm. Intelligent Science and Intelligent Data Engineering 7202:122–127.  https://doi.org/10.1007/978-3-642-31919-8_16 CrossRefGoogle Scholar
  15. 15.
    Krulikovská L, Polec J, Hirner T (2012) Fast algorithm of shot cut detection. World Academy of Science Engineering and Technology International Science Index 67 6(7):317–320Google Scholar
  16. 16.
    Lawrence S, Ziou D, Auclair-Fortier MF (2004) Motion-insensitive detection of cuts and gradual transitions in digital videos. Pattern Recognition and Image Analysis 14(1):109–119Google Scholar
  17. 17.
    Le DD, Satoh S, Ngo TD, Duong DA (2008) A text segmentation based approach to video shot boundary detection. In proceedings of the IEEE international Workshop on Multimedia Signal Processing (MMSP 2008), Cairns, Australia, October 8–10, pp. 702–706. doi:  https://doi.org/10.1109/MMSP.2008.4665166
  18. 18.
    Levy Vehel J (1996) Introduction to the multufractal analysis of images. Technical Report INRIAGoogle Scholar
  19. 19.
    Li J, Ding Y, Shi Y, Zeng Q (2009) DWT-Based Shot Boundary Detection Using Support Vector Machine. In the proceedings of the Fifth International Conference on Information Assurance and Security (IAS09), Xi’an, China, August 18–20, pp. 435–438.doi: https://doi.org/10.1109/IAS.2009.16
  20. 20.
    Liang R, Zhu Q, Wei H, Liao S (2017) A Video Shot Boundary Detection Approach Based on CNN Feature, In Proc. of the IEEE International Symposium on Multimedia (ISM), IEEE, pp. 489–494Google Scholar
  21. 21.
    Liu X, Dai J (2016) A method of video shot-boundary detection based on grey modeling for histogram sequence. Int J Signal Process Image Process Pattern Recognit 9(4):265–280Google Scholar
  22. 22.
    Liu Z, Zavesky E, Gibbon D, Shahraray B, Haffner P (2007) AT & T research at TRECVID 2007. In proceedings of the TRECVID 2007 Workshop, Gaithersburg, MD, USA NIST, November 5–6Google Scholar
  23. 23.
    Lopes R, Betrouni N (2009) Fractal and multifractal analysis: a review. Med Image Anal 13(4):634–649.  https://doi.org/10.1016/j.media.2009.05.003 CrossRefGoogle Scholar
  24. 24.
    Lu ZM, Shi Y (2013) Fast video shot boundary detection based on SVD and pattern matching. IEEE Trans Image Process 22(12):5136–5514.  https://doi.org/10.1109/TIP.2013.2282081 MathSciNetCrossRefGoogle Scholar
  25. 25.
    Mandelbrot BB (1967) How long is the coast of Britain? Statistical self-similarity and fractional dimension. Science 156:636–638.  https://doi.org/10.1126/science.156.3775.636 CrossRefGoogle Scholar
  26. 26.
    Mandelbrot BB (1983) The Fractal Geometry of Nature. WH Freeman Oxford 1983Google Scholar
  27. 27.
    Mishra R, Singhai SK, Sharma M (2013) Video shot boundary detection using dual tree complex wavelet transform. IACC 3rdIEEE International Conference:1201–1206.  https://doi.org/10.1109/IAdCC.2013.6514398
  28. 28.
    Mohanta PP, Saha SK, Chanda B (2012) A model-based shot boundary detection technique using frame transition parameters. IEEE Transactions on Multimedia 14(1):223–233.  https://doi.org/10.1109/TMM.2011.2170963 CrossRefGoogle Scholar
  29. 29.
    Mondal J, Kundu MK, Das S, Chowdhury M (2017) Video shot boundary detection using multiscale geometric analysis of nsct and least squares support vector machine. Multimedia Tools and Applications:1–23.  https://doi.org/10.1007/s11042-017-4707-9 CrossRefGoogle Scholar
  30. 30.
    Petersohn C (2004) Fraunhofer HHI at TRECVID 2004: Shot boundary detection system. In the proceedings of the TRECVID 2004 Workshop, Gaithersburg, MD, USA NIST, November 15–16Google Scholar
  31. 31.
    Petersohn C (2010) Temporal Video Segmentation. Jörg Vogt Verlag BerlinGoogle Scholar
  32. 32.
    Primaux L, Benois-Pineau J, Kramer P, Domenger JP (2004) Shot boundary detection in the framework of rough indexing paradigm. In Proceedings of the TRECVID 2004 Workshop, Gaithersburg, MD, USA NIST, November 15–16Google Scholar
  33. 33.
    Quenot GM, Moraru D, Ayache S, Charhad M (2004) CLIPS-LIS-LSR-LABRI experiments at TRECVID 2004. In Proceedings of the TRECVID 2004 Workshop, Gaithersburg, MD, USA NIST, November 15–16Google Scholar
  34. 34.
    Reljin I, Reljin B, Pavlović I, Rakočević I (2000) Multifractal analysis of gray-scale images. In proc. 10th Conference, MELECON-2000 (II), Lemesos, Cyprus, May 29–31, pp. 490–493. doi: https://doi.org/10.1109/MELCON.2000.879977
  35. 35.
    Ren J, Jiang J, Chen J (2009) Shot boundary detection in MPEG videos using local and global indicators. IEEE Trans Circuits Syst Video Techn 19(8), pp. 1234–1238. doi: https://doi.org/10.1109/TCSVT.2009.2022707 CrossRefGoogle Scholar
  36. 36.
    Richardson LF (1961) The problem of contiguity. General Systems Yearbook 6:139–187Google Scholar
  37. 37.
    Song BC, Ra JB (2001) Automatic shot change detection algorithm using multi-stage clustering for MPEG-compressed videos. J Vis Commun Image Represent 12(3):364–385.  https://doi.org/10.1006/jvci.2001.0469 CrossRefGoogle Scholar
  38. 38.
    Stojic T, Reljin I, Reljin B (2006) Adaptation of multifractal analysis to segmentation of microcalcifications in digital mammograms. Physica A: Statistical Mechanics and its Applications 367:494–508.  https://doi.org/10.1016/j.physa.2005.11.030 CrossRefGoogle Scholar
  39. 39.
    Sun X, Xiaoyu L, Mingwei Z (2010) Novel shot boundary detection method based on support vector machine. In International Conference on Computer and Information Application (ICCIA), pp. 56–59.doi: https://doi.org/10.1109/ICCIA.2010.6141536
  40. 40.
    Tabii Y, Sadiq A (2014) Shot boundary detection in videos sequences using motion activities. Advances in Multimedia-An International Journal (AMIJ) 5(1):1–7Google Scholar
  41. 41.
    Turner MJ, Blackledge JM, Andrews PR (1998) Fractal geometry in digital imaging. Academic Press, NYGoogle Scholar
  42. 42.
    VirtualDub http://virtualdub.sourceforge.net/ Accessed 17 April 2017
  43. 43.
    Wang Y (2012) Cognitive Informatics for Revealing Human Cognition: Knowledge Manipulations in Natural Intelligence: Knowledge Manipulations in Natural Intelligence. IGI GlobalGoogle Scholar
  44. 44.
    Watkinson J (2004) The MPEG Handbook: MPEG-1, MPEG-2, MPEG-4. 2nd edn. Elsevier/Focal Press, Oxford, Burlington, MACrossRefGoogle Scholar
  45. 45.
    Yazici A, Koyuncu M, Yilmaz T, Sattari S, Sert M, Gulen E (2018) An intelligent multimedia information system for multimodal content extraction and querying. Multimedia Tools and Applications 77(2):2225–2260.  https://doi.org/10.1007/s11042-017-4378-6 CrossRefGoogle Scholar
  46. 46.
    Yuan J, Guo Z, Lv L, Wan W, Zhang T, Wang D, Liu X, Liu C, Zhu S, Wang D, Pang Y, Ding N, Liu Y, Wang J, Zhang X, Tie X, Wang Z, Wang H, Xiao T, Liang Y, Li J, Lin F, Zhang B (2007) THU and ICRC at TRECVID 2007.In Proc. TRECVID 2007 Workshop, Gaithersburg, MD, USA. NISTGoogle Scholar
  47. 47.
    Zabih R, Miller J, Mai K (1995) A feature-based algorithm for detecting and classifying scene breaks. In proceedings of the Third ACM International Conference on Multimedia '95, San Francisco, CA, USA, November 5–9, pp. 189–200Google Scholar
  48. 48.
    Zajić GJ (2015) Shot-change detection based on multifractal analysis. In Proceedings of the 23rd Telecommunications Forum Telfor (TELFOR), IEEE, pp. 724–731Google Scholar
  49. 49.
    Zajić G, Kojić N, Radosavljević V, Rudinac M, Rudinac S, Reljin N, Reljin I, Reljin B (2007) Accelerating of Image Retrieval in CBIR System with Relevance Feedback. EURASIP Journal on Advances in Signal Processing Spec. Issue on Knowledge Assisted Media Analysis for Interactive Multimedia Applications 2007, pp. 1–13. doi: https://doi.org/10.1155/2007/62678
  50. 50.
    Zajić GJ, Reljin IS, Reljin BD (2011) Video shot boundary detection based on multifractal Analisys. Telfor Journal 3(2):105–110Google Scholar
  51. 51.
    Zeinalpour-Tabrizi Z, Aminian-Modarres A, Fathy M, Jahed-Motlagh M (2010) Fractal based video shot cut/fade detection and classification. Active Media Technology Lecture Notes in Computer Science 6335:128–137.  https://doi.org/10.1007/978-3-642-15470-6_14 CrossRefGoogle Scholar
  52. 52.
    Zhao ZC, Zeng X, Liu T, Cai AN (2007) BUPT at TRECVID 2007: Shot Boundary Detection. In Proceedings of the TRECVID 2007Workshop, Gaithersburg, MD, USA NISTGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.ICT College of Vocational StudiesBelgradeSerbia
  2. 2.School of Electrical EngineeringUniversity of BelgradeBelgradeSerbia

Personalised recommendations