Novel Mutual Information Analysis of Attentive Motion Entropy Algorithm for Sports Video Summarization

  • Bo-Wei Chen
  • Karunanithi Bharanitharan
  • Jia-Ching Wang
  • Zhounghua Fu
  • Jhing-Fa Wang
Conference paper
Part of the Lecture Notes in Electrical Engineering book series (LNEE, volume 260)

Abstract

This study presents a novel summarization method, which utilizes attentive motion analysis, mutual information, and segmental spectro-temporal subtraction, for generating sports video abstracts. The proposed attentive motion entropy and mutual information are both based on an attentive model. To capture and detect significant segments among a video, this work uses color contrast, intensity contrast, and orientation contrast of frames to calculate saliency maps. Regional histograms of oriented gradients based on human shapes are also adopted at the preliminary stage. In the next step, a new algorithm based on mutual information is proposed to improve the smoothness problem when the system selects the boundaries of motion segments. Meanwhile, differential salient motions and oriented gradients are merged to mutual information analysis, subsequently generating an attentive curve. Furthermore, to remove non-motion boundaries, a smoothing technique based on segmental spectro-temporal subtraction is also used for selecting favorable event boundaries. The experiment results show that our proposed algorithm can detect highlights effectively and generate smooth playable clips. Compared with existing systems, the precision and recall rates of our system outperform their results by 8.6 and 11.1 %, respectively. Besides, smoothness is enhanced by 0.7 on average, which also verified feasibility of our system.

Keywords

Video summarization Attentive motion entropy Mutual information analysis Segmental spectro-temporal subtraction 

References

  1. 1.
    Bagga A, Hu J, Zhong J, Ramesh G (2002) Multi-source combined-media video tracking for summarization. In Proceedings of the 16th IEEE international conference pattern recognition, Quebec, Canada, Aug 11–15. IEEE computer society, Washington, pp 818–821Google Scholar
  2. 2.
    Liu T, Zhang H-J, Qi F (2003) A novel video key-frame-extraction algorithm based on perceived motion energy model. IEEE trans. circuits and systems for video technology 13(10):1006–1013Google Scholar
  3. 3.
    Duan L-Y, Xu M, Tian Q, Xu C-S, Jin JS (2005) A unified framework for semantic shot classification in sports video. IEEE Trans Multimedia 7(6):1066–1083CrossRefGoogle Scholar
  4. 4.
    Li Z, Schuster GM, Katsaggelos AK (2005) MINMAX optimal video summarization. IEEE trans. circuits and systems for video technology, 15(10):1245–1256Google Scholar
  5. 5.
    Liu T-Y, Ma W-Y, Zhang H-J (2005) Effective feature extraction for play detection in American football video. In: Proceedings of the 11th international multimedia modeling conference (Melbourne, Australia, Jan. 12–14). IEEE computer society, Washington, pp 164–171Google Scholar
  6. 6.
    Ma Y-F, Hua X-S, Lu L, Zhang H-J (2005) A generic framework of user attention model and its application in video summarization. IEEE Trans Multimedia 7(5):907–919CrossRefGoogle Scholar
  7. 7.
    Yeo B-L, Liu B (2005) Rapid scene analysis on compressed video. IEEE trans circuits and systems for video technology, 5(6):533–544Google Scholar
  8. 8.
    Cernekova Z, Pitas I, Nikou C (2006) Information theory-based shot cut/fade detection and video summarization. IEEE transactions circuits and systems for video technology, 16(1):82–91Google Scholar
  9. 9.
    Li Y, Lee S-H, Yeh C-H, Kuo C-CJ (2006) Techniques for movie content analysis and skimming: tutorial and overview on video abstraction techniques. IEEE Signal Process Mag 23(2):79–89CrossRefMATHGoogle Scholar
  10. 10.
    Taskiran CM, Pizlo Z, Amir A, Ponceleon D, Delp EJ (2006) Automated video program summarization using speech transcripts. IEEE Trans Multimedia 8(4):775–791CrossRefGoogle Scholar
  11. 11.
    Chen C-Y, Wang J-C, Wang J-F, Hu Y-H (2007) Event-based segmentation of sports video using motion entropy. In: Proceedings of the 9th IEEE international symposium multimedia (Taichung, Taiwan, 10–12). IEEE computer society, Washington, pp 107–111Google Scholar
  12. 12.
    You J, Liu G, Sun L, Li H (2007) A multiple visual models based perceptive analysis framework for multilevel video summarization. IEEE trans. circuits and systems for video technology, 17(3):273–285Google Scholar
  13. 13.
    Chen B-W, Wang J-C, Wang J-F (2009) A novel video summarization based on mining the story-structure and semantic relations among concept entities. IEEE Trans Multimedia 11(2):295–312CrossRefGoogle Scholar
  14. 14.
    Black MJ (1996) The robust estimation of multiple motions: parametric and piecewise-smooth flow fields. Comput Vis Image Underst 63(1):75–104MathSciNetCrossRefGoogle Scholar
  15. 15.
    Walther D, Rutishauser U, Koch C, Perona P (2005) Selective visual attention enables learning and recognition of multiple objects in cluttered scenes. Comput Vis Image Underst 100(1–2):41–63CrossRefGoogle Scholar
  16. 16.
    Walther D, Koch C (2006) Modeling attention to salient proto-objects. Neural Networks 19(9):1395–1407CrossRefMATHGoogle Scholar
  17. 17.
    Ma Y-F, Lu L, Zhang H-J, Li M (2002) A user attention model for video summarization. In: Proceedings of the 10th ACM international conference multimedia (Juan-les-Pins, France, Dec. 01–06). ACM Press, New York, pp 533–542Google Scholar
  18. 18.
    Lu S, King I, Lyu MR (2005) A novel video summarization framework for document preparation and archival applications. In: Proceedings of the 2005 IEEE aerospace conference (Big Sky, Montana, United States, Mar. 05–12). IEEE computer society, Washington, 1–10Google Scholar
  19. 19.
    Ngo C-W, Ma Y-F, Zhang H-J (2005) Video summarization and scene detection by graph modeling. IEEE transactions circuits and systems for video technology, 15(2):296–305Google Scholar
  20. 20.
    Chen Y-T, Chen C-S (2008) Fast human detection using a novel boosted cascading structure with meta stages. IEEE Trans Image Proc 17(8):1452–1464CrossRefGoogle Scholar
  21. 21.
    Kamath SD, Loizou PC (2002) A multi-band spectral subtraction method for enhancing speech corrupted by colored noise. In: Proceedings of the IEEE international conference acoustics, speech, and signal processing (Orlando, Florida, United States, May 13–17). IEEE computer society, Washington, pp 4164–4167Google Scholar
  22. 22.
    Zhang T, Kuo C-CJ (2001) Audio content analysis for online audiovisual data segmentation and classification. IEEE Trans Speech Audio Proc 9(4):441–457CrossRefGoogle Scholar
  23. 23.
    Misra H, Vepa J, Bourlard H (2006) Multi-stream ASR: an oracle perspective. In: Proceedings of the ISCA international conference spoken language processing (Pittsburgh, Pennsylvania, United States, Sep. 17–21)Google Scholar
  24. 24.
    Gray AH, Markel JD (1974) A spectral-flatness measure for studying the autocorrelation method of linear prediction of speech analysis. IEEE Trans Acoustics, Speech and Signal Processing 22(3):207–217CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2014

Authors and Affiliations

  • Bo-Wei Chen
    • 1
  • Karunanithi Bharanitharan
    • 2
  • Jia-Ching Wang
    • 3
  • Zhounghua Fu
    • 4
  • Jhing-Fa Wang
    • 1
  1. 1.Department of Electrical EngineeringNational Cheng Kung UniversityTainanTaiwan, Republic of China
  2. 2.Department of Electrical EngineeringFeng Chia UniversityTaichungTaiwan, Republic of China
  3. 3.Department of Computer Science and Information EngineeringNational Central UniversityJhongliTaiwan, Republic of China
  4. 4.School of Computer ScienceNorthwestern Polytechnical UniversityXi’anChina

Personalised recommendations