Skip to main content
SpringerLink
Log in

Tsallis entropy-based information measures for shot boundary detection and keyframe selection

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

Automatic shot boundary detection and keyframe selection constitute major goals in video processing. We propose two different information-theoretic approaches to detect the abrupt shot boundaries of a video sequence. These approaches are, respectively, based on two information measures, Tsallis mutual information and Jensen–Tsallis divergence, that are used to quantify the similarity between two frames. Both measures are also used to find out the most representative keyframe of each shot. The representativeness of a frame is basically given by its average similarity with respect to the other frames of the shot. Several experiments analyze the behavior of the proposed measures for different color spaces (RGB, HSV, and Lab), regular binnings, and entropic indices. In particular, the Tsallis mutual information for the HSV and Lab color spaces with only 8 regular bins for each color component and an entropic index between 1.5 and 1.8 substantially improve the performance of previously proposed methods based on mutual information and Jensen–Shannon divergence.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. The Open Video Project: a shared digital video collection. http://www.open-video.org/index.php

  2. TREC video retrieval evaluation: TRECVID. http://trecvid.nist.gov/

  3. TRECVID 2007 shot boundary determination results. http://www-nlpir.nist.gov/projects/tv2007/active/results/shot.boundaries/runTable.web

  4. Browne, P., Smeaton, A.F., Murphy, N., O’Connor, N., Marlow, S., Berrut, C.: Evaluating and combining digital video shot boundary detection algorithms. In: Irish Machine Vision and Image Processing Conference (1999)

  5. Burbea, J., Rao, C.R.: On the convexity of some divergence measures based on entropy functions. IEEE Trans. Inf. Theory 28(3), 489–495 (1982)

    Article  MathSciNet  MATH  Google Scholar 

  6. Butz, T., Thiran, J.P.: Shot boundary detection with mutual information. In: International Conference on Image Processing, ICIP ’01, pp. 422–425 (2001)

  7. Cernekova, Z., Pitas, I., Nikou, C.: Information theory-based shot cut/fade detection and video summarization. IEEE Trans. Circuits Syst. Video Technol. 16(1), 82–91 (2006)

    Article  Google Scholar 

  8. Ciocca, G., Schettini, R.: An innovative algorithm for key frame extraction in video summarization. J. Real-Time Image Process. 1(1), 69–88 (2006)

    Article  Google Scholar 

  9. Cotsaces, C., Nikolaidis, N., Pitas, I.: Video shot detection and condensed representation. a review. IEEE Signal Process. Mag. 23(2), 28–37

  10. Cover, T.M., Thomas, J.A.: Elements of Information Theory. Wiley Series in Telecommunications, New York (1991)

    Book  MATH  Google Scholar 

  11. Dhawale, C.A., Jain, S.: Motion compensated video shot detection using multiple feature experts. ICGST Int. J. Graph. Vis. Image Process. GVIP 08, 1–11 (2008)

    Google Scholar 

  12. Gargi, U., Kasturi, R., Strayer, S.: Performance characterization of video-shot-change detection methods. IEEE Trans. Circuits Syst. Video Technol. 10(1), 1–13 (2000)

    Article  Google Scholar 

  13. Gonzalez, R.C., Woods, R.E.: Digital Image Processing. Prentice Hall, Upper Saddle River (NJ), USA (2002)

    Google Scholar 

  14. Grana, C., Cucchiara, R.: Linear transition detection as a unified shot detection approach. IEEE Trans. Circuits Syst. Video Technol. 17(4), 483–489 (2007)

    Article  Google Scholar 

  15. Gunsel, B., Tekalp, A.: Content-based video abstraction. In: International Conference on Image Processing, ICIP ’98, vol. 3, pp. 128–132 (1998)

  16. Hanjalic, A.: Shot-boundary detection: unraveled and resolved? IEEE Trans. Circuits Syst. Video Technol. 12(2), 90–105 (2002)

    Article  Google Scholar 

  17. Harvda, J., Charvát, F.: Quantification method of classification processes. Concept of structural \(\alpha \)-entropy. Kybernetika 3, 30–35 (1967)

    MathSciNet  Google Scholar 

  18. Hesseler, W., Eickeler, S.: MPEG-2 compressed-domain algorithms for video analysis. EURASIP J. Appl. Signal Process. 2006, 186–186 (2006)

    Google Scholar 

  19. Huan, Z., Xiuhuan, L., Lilei, Y.: Shot boundary detection based on mutual information and canny edge detector. In: Proceedings of the 2008 International Conference on Computer Science and Software Engineering, vol. 02, pp. 1124–1128 (2008)

  20. Lee, M.H., Yoo, H.W., Jang, D.S.: Video scene change detection using neural network: improved ART2. Expert Syst. Appl. 31(1), 13–25 (2006)

    Article  Google Scholar 

  21. Lelescu, D., Schonfeld, D.: Statistical sequential analysis for real-time video scene change detection on compressed multimedia bitstream. IEEE Trans. Multimed. 5(1), 106–117 (2003)

    Article  Google Scholar 

  22. Lienhart, R.: Comparison of automatic shot boundary detection algorithms. In: Proceedings of SPIE, pp. 290–301 (1999)

  23. Lienhart, R.: Reliable transition detection in videos: a survey and practitioner’s guide. Int. J. Image Graph. 1, 469–486 (2001)

    Article  Google Scholar 

  24. Martins, A.F.T., Figueiredo, M.A.T., Aguiar, P.M.Q., Smith, N.A., Xing, E.P.: Nonextensive entropic kernels. In: Proceedings of the 25th International Conference on Machine Learning, ICML ’08, pp. 640–647 (2008)

  25. Meng, J., Juan, Y., Chang, S.F.: Scene change detection in an MPEG-compressed video sequence. In: IS &T/SPIE Symposium Proceedings, vol. 2419, pp. 14–25. SPIE (1995)

  26. Mentzelopoulos, M., Psarrou, A.: Key-frame extraction algorithm using entropy difference. In: Proceedings of the 6th ACM SIGMM International Workshop on Multimedia Information Retrieval, MIR ’04, pp. 39–45 (2004)

  27. Money, A.G., Agius, H.W.: Video summarisation: a conceptual framework and survey of the state of the art. J. Vis. Commun. Image Represent. 19(2), 121–143 (2008)

    Article  Google Scholar 

  28. Nagasaka, A., Tanaka, Y.: Automatic video indexing and full-video search for object appearances. In: Proceedings of the IFIP TC2/WG 2.6 Second Working Conference on Visual Database Systems II, pp. 113–127 (1992)

  29. Peng, J., Xiao-Lin, Q.: Keyframe-based video summary using visual attention clues. IEEE MultiMed. 17(2), 64–73 (2010)

    Google Scholar 

  30. Portes de Albuquerque, M., Esquef, I.: Image thresholding using Tsallis entropy. Pattern Recognit. Lett. 25, 1059–1065 (2004)

    Article  Google Scholar 

  31. Slonim, N., Tishby, N.: Document clustering using word clusters via the information bottleneck method. In: Proceedings of the 23rd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 208–215. ACM Press, New York (2000)

  32. Taneja, I.J.: Bivariate measures of type \(\alpha \) and their applications. Tamkang J. Math. 19(3), 63–74 (1988)

    MathSciNet  MATH  Google Scholar 

  33. Tardini, G., Grana, C., Marchi, R., Cucchiara, R.: Shot detection and motion analysis for automatic MPEG-7 annotation of sports videos. In: 13th International Conference on Image Analysis and Processing, pp. 653–660 (2005)

  34. Thompson, W.B., Fleming, R.W., Creem-Regehr, S.H., Stefanucci, J.K.: Visual Perception from a Computer Graphics Perspective. CRC Press, Boca Raton, FL (2011)

    Google Scholar 

  35. Tsallis, C.: Possible generalization of Boltzmann-Gibbs statistics. J. Stat. Phys. 52(1/2), 479–487 (1988)

    Google Scholar 

  36. Tsallis, C.: Generalized entropy-based criterion for consistent testing. Phys. Rev. E 58, 1442–1445 (1998)

    Article  Google Scholar 

  37. Urhan, O., Güllü, K.M., Ertürk, S.: Modified phase-correlation based robust hard-cut detection with application to archive film. IEEE Trans. Circuits Syst. Video Technol. 16(6), 753–770 (2006)

    Article  Google Scholar 

  38. Vila, M., Bardera, A., Feixas, M., Sbert, M.: Tsallis mutual information for document classification. Entropy 13(9), 1694–1707 (2011)

    Article  Google Scholar 

  39. Wolf, W.: Key frame selection by motion analysis. In: IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP ’96, vol. 2, pp. 1228–1231 (1996)

  40. Xu, Q., Wang, P., Long, B., Sbert, M., Feixas, M., Scopigno, R.: Selection and 3D visualization of video key frames. In: IEEE International Conference on Systems Man and Cybernetics (SMC), pp. 52–59 (2010)

  41. Yoo, H.W., Ryoo, H.J., Jang, D.S.: Gradual shot boundary detection using localized edge blocks. Multimed. Tools Appl. 28, 283–300 (2006)

    Article  Google Scholar 

  42. Yuan, J., Wang, H., Xiao, L., Zheng, W., Li, J., Lin, F., Zhang, B.: A formal study of shot boundary detection. IEEE Trans. Circuits Syst. Video Technol. 17(2), 168–186 (2007)

    Article  Google Scholar 

  43. Zabih, R., Miller, J., Mai, K.: A feature-based algorithm for detecting and classifying scene breaks. In: Proc. ACM Multimedia, vol. 95, pp. 189–200 (1995)

  44. Zhang, W., Lin, J., Chen, X., Huang, Q., Liu, Y.: Video shot detection using hidden markov models with complementary features. In: First International Conference on Innovative Computing, Information and Control, ICICIC ’06, vol. 3, pp. 593–596 (2006)

  45. Zhuang, Y., Rui, Y., Huang, T., Mehrotra, S.: Adaptive key frame extraction using unsupervised clustering. In: International Conference on Image Processing, ICIP ’98, vol. 1, pp. 866–870 (1998)

Download references

Acknowledgments

This work has been funded in part by grants from the Spanish Government (Nr. TIN2010-21089-C03-01), from the Catalan Government (Nr. 2009-SGR-643 and Nr. 2010-CONE2-00053), and from the Natural Science Foundation of China (61179067, 61103005, 60879003).

Author information

Authors and Affiliations

  1. Institut d’Informàtica i Aplicacions, University of Girona, Girona, Spain

    Màrius Vila, Anton Bardera, Miquel Feixas & Mateu Sbert

  2. School of Computer Science and Technology, Tianjin University, Tianjin, China

    Qing Xu

Authors
  1. Màrius Vila
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Anton Bardera
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Qing Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Miquel Feixas
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Mateu Sbert
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Anton Bardera.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Vila, M., Bardera, A., Xu, Q. et al. Tsallis entropy-based information measures for shot boundary detection and keyframe selection. SIViP 7, 507–520 (2013). https://doi.org/10.1007/s11760-013-0452-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-013-0452-3

Keywords

Search

Navigation