Video Repeat Recognition and Mining by Visual Features

  • Xianfeng Yang
  • Qi Tian
Part of the Studies in Computational Intelligence book series (SCI, volume 287)


Repeat video clips such as program logos and commercials are widely used in video productions, and mining them is important for video content analysis and retrieval. In this chapter we present methods to identify known and unknown video repeats respectively. For known video repeat recognition, we focus on robust feature extraction and classifier learning problems. A clustering model of visual features (e.g. color, texture) is proposed to represent video clip and subspace discriminative analysis is adopted to improve classification accuracy, which results in good results for short video clip recognition. We also propose a novel method to explore statistics of video database to estimate nearest neighbor classification error rate and learn the optimal classification threshold. For unknown video repeat mining, we address robust detection, searching efficiency and learning issues. Two detectors in a cascade structure are employed to efficiently detect unknown video repeats of arbitrary length, and this approach combines video segmentation, color fingerprinting, self-similarity analysis and Locality-Sensitive Hashing (LSH) indexing. A reinforcement learning approach is also adopted to efficiently learn optimal parameters. Experiment results show that very short video repeats and long ones can be detected with high accuracy. Video structure analysis by short video repeats mining is also presented in results.


Video Clip Video Segment Video Database Locality Sensitive Hash Video Feature 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Yang, X., Xue, P., Tian, Q.: A repeated video clip identification system. In: Proc. ACM Multimedia, Singapore (2005)Google Scholar
  2. 2.
    Yang, X., Tian, Q., Xue, P.: Efficient Short Video Repeat Identification With Application to News Video Structure Analysis. IEEE Trans. Multimedia 9, 600–609 (2007)CrossRefGoogle Scholar
  3. 3.
    Cheung, S.-C., Ngueyen, T.P.: Mining Arbitrary-length Repeated Patterns in Television Broadcast. In: Proc. IEEE Int. Conf. on Image Processing (2005)Google Scholar
  4. 4.
    Lienhart, R., Kuhmunch, C., Effelsberg, W.: On the detection and Recognition of Television Commercials. In: Proc. IEEE Int. Conf. Multimedia Computing and Systems (1997)Google Scholar
  5. 5.
    Snchez, J.M., Binefa, X., Vitri, J.: Shot partitioning based recognition of TV commercials. Multimedia Tools and Applications 18, 233–247 (2002)CrossRefGoogle Scholar
  6. 6.
    Kashino, K., Kurozumi, T., Murase, H.: A quick search method for audio and video signals based on histogram pruning. IEEE Trans. Multimedia 5(3), 348–357 (2003)CrossRefGoogle Scholar
  7. 7.
    Yuan, J., Duan, L.-Y., Tian, Q., Xu, C.: Fast and robust short video clip search using an index structure. In: Proc. ACM Multimedia’s Multimedia Information Retrieval Workshop (2004)Google Scholar
  8. 8.
    Cheung, S.-C., Zakhor, A.: Estimation of web video multiplicity. In: Proc. SPIE, vol. 3964, pp. 34–46 (2000)Google Scholar
  9. 9.
    Pua, K.M., Gauch, J.M.: Real time repeated video sequence identification. Computer Vision and Image Understanding 93(3), 310–327 (2004)CrossRefGoogle Scholar
  10. 10.
    Lowe, D.: Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision 60(2), 91–120 (2004)CrossRefGoogle Scholar
  11. 11.
    Oostveen, J.C., Kalker, A.A.C., Haitsma, J.A.: Visual hashing of digital Video: applications and techniques. In: SPIE applications of digital image processing XXIV, San Diego, pp. 121–131 (2001)Google Scholar
  12. 12.
    Yang, X., Tian, Q., Chang, E.C.: A Color Fingerprint of Video Shot for Content Identification. In: Proc. ACM Multimedia, NY, USA (2004)Google Scholar
  13. 13.
    Cevikalp, H., Neamtu, M., Wilkes, M., Barkana, A.: Discriminant Common Vectors for Face Recognition. IEEE Trans. Pattern Analysis and Machine Intelligence 27(1), 4–13 (2005)CrossRefGoogle Scholar
  14. 14.
    Chakrabarti, S., Roy, S., Soundalgekar, M.: Fast and Accurate Text Classification via Multiple Linear Discriminant Projections. In: Proc. Int’l. Conf. Very Large Data Bases, pp. 658–669 (2002)Google Scholar
  15. 15.
    Belhumeur, P., Hespanha, J., Kriegman, D.: Eigenfaces versus Fisherfaces: Recognition Using Class Specific Linear Projection. IEEE Trans. Pattern Analysis and Machine Intelligence 19(7), 711–720 (1997)CrossRefGoogle Scholar
  16. 16.
    Yang, X., Yuan, M.: Video Copy Recognition By Oriented PCA and Statistical Analysis. In: Proc. IEEE Int. Conf. on Image Processing, Cario, Egypt (2009)Google Scholar
  17. 17.
    Burges, C.J.C., Platt, J.C., Jana, S.: Distortion Discriminant Analysis for Audio Fingerprinting. IEEE Trans. Speech and Audio Processing 11(3), 165–174 (2003)CrossRefGoogle Scholar
  18. 18.
    Herley, C.: ARGOS: Automatically Extracting Repeating Objects From Multimedia Streams. IEEE Trans. Multimedia 8(1), 113–129 (2006)CrossRefGoogle Scholar
  19. 19.
    Haralick, R.M., Shanmugam, K., Dinstein, I.: Textural features for Image Classification. IEEE Transactions on Systems, Man, and Cybernetics 3(6), 610–621 (1973)CrossRefGoogle Scholar
  20. 20.
    Diamantaras, K., Kung, S.: Principal Component Neural Networks. John Wiley, Chichester (1996)MATHGoogle Scholar
  21. 21.
    Gionis, A., Indyky, P., Motwaniz, R.: Similarity Search in High Dimensions via Hashing. In: Proc. Int. Conf. Very Large Data Bases, pp. 518–529 (1999)Google Scholar
  22. 22.
    Williams, R.J.: Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning. Machine Learning 8, 229–256 (1992)MATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Xianfeng Yang
    • 1
  • Qi Tian
    • 2
  1. 1.Faculty of EngineeringNational University of SingaporeSingapore
  2. 2.Institute for Infocomm ResearchSingapore

Personalised recommendations