Signal, Image and Video Processing

, Volume 7, Issue 3, pp 507–520 | Cite as

Tsallis entropy-based information measures for shot boundary detection and keyframe selection

  • Màrius Vila
  • Anton Bardera
  • Qing Xu
  • Miquel Feixas
  • Mateu Sbert
Original Paper

Abstract

Automatic shot boundary detection and keyframe selection constitute major goals in video processing. We propose two different information-theoretic approaches to detect the abrupt shot boundaries of a video sequence. These approaches are, respectively, based on two information measures, Tsallis mutual information and Jensen–Tsallis divergence, that are used to quantify the similarity between two frames. Both measures are also used to find out the most representative keyframe of each shot. The representativeness of a frame is basically given by its average similarity with respect to the other frames of the shot. Several experiments analyze the behavior of the proposed measures for different color spaces (RGB, HSV, and Lab), regular binnings, and entropic indices. In particular, the Tsallis mutual information for the HSV and Lab color spaces with only 8 regular bins for each color component and an entropic index between 1.5 and 1.8 substantially improve the performance of previously proposed methods based on mutual information and Jensen–Shannon divergence.

Keywords

Information theory Tsallis entropy Video processing Shot boundary detection Keyframe selection 

References

  1. 1.
    The Open Video Project: a shared digital video collection. http://www.open-video.org/index.php
  2. 2.
    TREC video retrieval evaluation: TRECVID. http://trecvid.nist.gov/
  3. 3.
  4. 4.
    Browne, P., Smeaton, A.F., Murphy, N., O’Connor, N., Marlow, S., Berrut, C.: Evaluating and combining digital video shot boundary detection algorithms. In: Irish Machine Vision and Image Processing Conference (1999)Google Scholar
  5. 5.
    Burbea, J., Rao, C.R.: On the convexity of some divergence measures based on entropy functions. IEEE Trans. Inf. Theory 28(3), 489–495 (1982)MathSciNetMATHCrossRefGoogle Scholar
  6. 6.
    Butz, T., Thiran, J.P.: Shot boundary detection with mutual information. In: International Conference on Image Processing, ICIP ’01, pp. 422–425 (2001)Google Scholar
  7. 7.
    Cernekova, Z., Pitas, I., Nikou, C.: Information theory-based shot cut/fade detection and video summarization. IEEE Trans. Circuits Syst. Video Technol. 16(1), 82–91 (2006)CrossRefGoogle Scholar
  8. 8.
    Ciocca, G., Schettini, R.: An innovative algorithm for key frame extraction in video summarization. J. Real-Time Image Process. 1(1), 69–88 (2006)CrossRefGoogle Scholar
  9. 9.
    Cotsaces, C., Nikolaidis, N., Pitas, I.: Video shot detection and condensed representation. a review. IEEE Signal Process. Mag. 23(2), 28–37Google Scholar
  10. 10.
    Cover, T.M., Thomas, J.A.: Elements of Information Theory. Wiley Series in Telecommunications, New York (1991)MATHCrossRefGoogle Scholar
  11. 11.
    Dhawale, C.A., Jain, S.: Motion compensated video shot detection using multiple feature experts. ICGST Int. J. Graph. Vis. Image Process. GVIP 08, 1–11 (2008)Google Scholar
  12. 12.
    Gargi, U., Kasturi, R., Strayer, S.: Performance characterization of video-shot-change detection methods. IEEE Trans. Circuits Syst. Video Technol. 10(1), 1–13 (2000)CrossRefGoogle Scholar
  13. 13.
    Gonzalez, R.C., Woods, R.E.: Digital Image Processing. Prentice Hall, Upper Saddle River (NJ), USA (2002)Google Scholar
  14. 14.
    Grana, C., Cucchiara, R.: Linear transition detection as a unified shot detection approach. IEEE Trans. Circuits Syst. Video Technol. 17(4), 483–489 (2007)CrossRefGoogle Scholar
  15. 15.
    Gunsel, B., Tekalp, A.: Content-based video abstraction. In: International Conference on Image Processing, ICIP ’98, vol. 3, pp. 128–132 (1998)Google Scholar
  16. 16.
    Hanjalic, A.: Shot-boundary detection: unraveled and resolved? IEEE Trans. Circuits Syst. Video Technol. 12(2), 90–105 (2002)CrossRefGoogle Scholar
  17. 17.
    Harvda, J., Charvát, F.: Quantification method of classification processes. Concept of structural \(\alpha \)-entropy. Kybernetika 3, 30–35 (1967)MathSciNetGoogle Scholar
  18. 18.
    Hesseler, W., Eickeler, S.: MPEG-2 compressed-domain algorithms for video analysis. EURASIP J. Appl. Signal Process. 2006, 186–186 (2006)Google Scholar
  19. 19.
    Huan, Z., Xiuhuan, L., Lilei, Y.: Shot boundary detection based on mutual information and canny edge detector. In: Proceedings of the 2008 International Conference on Computer Science and Software Engineering, vol. 02, pp. 1124–1128 (2008)Google Scholar
  20. 20.
    Lee, M.H., Yoo, H.W., Jang, D.S.: Video scene change detection using neural network: improved ART2. Expert Syst. Appl. 31(1), 13–25 (2006)CrossRefGoogle Scholar
  21. 21.
    Lelescu, D., Schonfeld, D.: Statistical sequential analysis for real-time video scene change detection on compressed multimedia bitstream. IEEE Trans. Multimed. 5(1), 106–117 (2003)CrossRefGoogle Scholar
  22. 22.
    Lienhart, R.: Comparison of automatic shot boundary detection algorithms. In: Proceedings of SPIE, pp. 290–301 (1999)Google Scholar
  23. 23.
    Lienhart, R.: Reliable transition detection in videos: a survey and practitioner’s guide. Int. J. Image Graph. 1, 469–486 (2001)CrossRefGoogle Scholar
  24. 24.
    Martins, A.F.T., Figueiredo, M.A.T., Aguiar, P.M.Q., Smith, N.A., Xing, E.P.: Nonextensive entropic kernels. In: Proceedings of the 25th International Conference on Machine Learning, ICML ’08, pp. 640–647 (2008)Google Scholar
  25. 25.
    Meng, J., Juan, Y., Chang, S.F.: Scene change detection in an MPEG-compressed video sequence. In: IS &T/SPIE Symposium Proceedings, vol. 2419, pp. 14–25. SPIE (1995)Google Scholar
  26. 26.
    Mentzelopoulos, M., Psarrou, A.: Key-frame extraction algorithm using entropy difference. In: Proceedings of the 6th ACM SIGMM International Workshop on Multimedia Information Retrieval, MIR ’04, pp. 39–45 (2004)Google Scholar
  27. 27.
    Money, A.G., Agius, H.W.: Video summarisation: a conceptual framework and survey of the state of the art. J. Vis. Commun. Image Represent. 19(2), 121–143 (2008)CrossRefGoogle Scholar
  28. 28.
    Nagasaka, A., Tanaka, Y.: Automatic video indexing and full-video search for object appearances. In: Proceedings of the IFIP TC2/WG 2.6 Second Working Conference on Visual Database Systems II, pp. 113–127 (1992)Google Scholar
  29. 29.
    Peng, J., Xiao-Lin, Q.: Keyframe-based video summary using visual attention clues. IEEE MultiMed. 17(2), 64–73 (2010)Google Scholar
  30. 30.
    Portes de Albuquerque, M., Esquef, I.: Image thresholding using Tsallis entropy. Pattern Recognit. Lett. 25, 1059–1065 (2004)CrossRefGoogle Scholar
  31. 31.
    Slonim, N., Tishby, N.: Document clustering using word clusters via the information bottleneck method. In: Proceedings of the 23rd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 208–215. ACM Press, New York (2000)Google Scholar
  32. 32.
    Taneja, I.J.: Bivariate measures of type \(\alpha \) and their applications. Tamkang J. Math. 19(3), 63–74 (1988)MathSciNetMATHGoogle Scholar
  33. 33.
    Tardini, G., Grana, C., Marchi, R., Cucchiara, R.: Shot detection and motion analysis for automatic MPEG-7 annotation of sports videos. In: 13th International Conference on Image Analysis and Processing, pp. 653–660 (2005)Google Scholar
  34. 34.
    Thompson, W.B., Fleming, R.W., Creem-Regehr, S.H., Stefanucci, J.K.: Visual Perception from a Computer Graphics Perspective. CRC Press, Boca Raton, FL (2011)Google Scholar
  35. 35.
    Tsallis, C.: Possible generalization of Boltzmann-Gibbs statistics. J. Stat. Phys. 52(1/2), 479–487 (1988) Google Scholar
  36. 36.
    Tsallis, C.: Generalized entropy-based criterion for consistent testing. Phys. Rev. E 58, 1442–1445 (1998)CrossRefGoogle Scholar
  37. 37.
    Urhan, O., Güllü, K.M., Ertürk, S.: Modified phase-correlation based robust hard-cut detection with application to archive film. IEEE Trans. Circuits Syst. Video Technol. 16(6), 753–770 (2006)CrossRefGoogle Scholar
  38. 38.
    Vila, M., Bardera, A., Feixas, M., Sbert, M.: Tsallis mutual information for document classification. Entropy 13(9), 1694–1707 (2011)CrossRefGoogle Scholar
  39. 39.
    Wolf, W.: Key frame selection by motion analysis. In: IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP ’96, vol. 2, pp. 1228–1231 (1996)Google Scholar
  40. 40.
    Xu, Q., Wang, P., Long, B., Sbert, M., Feixas, M., Scopigno, R.: Selection and 3D visualization of video key frames. In: IEEE International Conference on Systems Man and Cybernetics (SMC), pp. 52–59 (2010)Google Scholar
  41. 41.
    Yoo, H.W., Ryoo, H.J., Jang, D.S.: Gradual shot boundary detection using localized edge blocks. Multimed. Tools Appl. 28, 283–300 (2006)CrossRefGoogle Scholar
  42. 42.
    Yuan, J., Wang, H., Xiao, L., Zheng, W., Li, J., Lin, F., Zhang, B.: A formal study of shot boundary detection. IEEE Trans. Circuits Syst. Video Technol. 17(2), 168–186 (2007)CrossRefGoogle Scholar
  43. 43.
    Zabih, R., Miller, J., Mai, K.: A feature-based algorithm for detecting and classifying scene breaks. In: Proc. ACM Multimedia, vol. 95, pp. 189–200 (1995)Google Scholar
  44. 44.
    Zhang, W., Lin, J., Chen, X., Huang, Q., Liu, Y.: Video shot detection using hidden markov models with complementary features. In: First International Conference on Innovative Computing, Information and Control, ICICIC ’06, vol. 3, pp. 593–596 (2006)Google Scholar
  45. 45.
    Zhuang, Y., Rui, Y., Huang, T., Mehrotra, S.: Adaptive key frame extraction using unsupervised clustering. In: International Conference on Image Processing, ICIP ’98, vol. 1, pp. 866–870 (1998)Google Scholar

Copyright information

© Springer-Verlag London 2013

Authors and Affiliations

  • Màrius Vila
    • 1
  • Anton Bardera
    • 1
  • Qing Xu
    • 2
  • Miquel Feixas
    • 1
  • Mateu Sbert
    • 1
  1. 1.Institut d’Informàtica i AplicacionsUniversity of GironaGironaSpain
  2. 2.School of Computer Science and TechnologyTianjin UniversityTianjinChina

Personalised recommendations