Video Genre Inference Based on Camera Capturing Models

  • Ping-Hao Wu
  • Sanjay Purushotham
  • C. -C. Jay Kuo
Part of the Studies in Computational Intelligence book series (SCI, volume 287)

Abstract

On-line video collection is getting larger nowadays. It becomes difficult for users to go through the whole collection to find the video of their interest. To allow efficient browsing, search and retrieval, one intuitive solution is to cluster video clips according to their genres automatically. Then, users’ choices can be narrowed down. Besides on-line video repositories, other applications include managing television broadcasting archives, video conferencing records, etc. The goal of video classification is to automatically place each video title in different categories, such as news, sports, etc. The classification process involves extracting the information from the video clips and classifying them into different classes. In this chapter, we first review related work in this field. Then, two novel features based on the camera shooting process is proposed for video genre classification. These new camera based features exploit the fact that a different genre tends to have different camera effects and user perception. Although a lot of work has been proposed with the consideration of cinematic principles, most extracted features are low-level features without much semantic information.We propose a feature that estimates the number of cameras used in a short time interval. Then, we propose another feature by calculating the distribution of the camera distance, which is approximated by the normalized foreground area of each frame. The block-based motion vector field is adopted to reduce the complexity involved in foreground/background modeling. Preliminary experiment results show that the proposed features capture additional genre-related information so that the video genre can be inferred from the proposed features well.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Araki, S., Matsuoka, T., Takemura, H., Yokoya, N.: Real-time tracking of multiple moving objects in moving camera image sequences using robust statistics. In: International Conference on Pattern Recognition, vol. 2, p. 1433 (1998), http://doi.ieeecomputersociety.org/10.1109/ICPR.1998.711972
  2. 2.
    Brezeale, D., Cook, D.: Automatic video classification: A survey of the literature. IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews 38(3), 416–430 (2008), doi:10.1109/TSMCC.2008.919173CrossRefGoogle Scholar
  3. 3.
    Brezeale, D., Cook, D.J.: Using closed captions and visual features to classify movies by genre. In: 7th Int. Workshop Multimedia Data Min, MDM/KDD (2006)Google Scholar
  4. 4.
    Chien, S.Y., Ma, S.Y., Chen, L.G.: Efficient moving object segmentation algorithm using background registration technique. IEEE Transactions on Circuits and Systems for Video Technology 12(7), 577–586 (2002), doi:10.1109/TCSVT.2002.800516CrossRefGoogle Scholar
  5. 5.
    Drew, M.S., Au, J.: Video keyframe production by efficient clustering of compressed chromaticity signatures (poster session). In: MULTIMEDIA 2000: Proceedings of the eighth ACM international conference on Multimedia, pp. 365–367. ACM, New York (2000), http://doi.acm.org/10.1145/354384.354534
  6. 6.
    Fan, J., Luo, H., Xiao, J., Wu, L.: Semantic video classification and feature subset selection under context and concept uncertainty, pp. 192–201 (2004), doi:10.1109/JCDL.2004.1336120Google Scholar
  7. 7.
    Fischer, S., Lienhart, R., Effelsberg, W.: Automatic recognition of film genres. In: MULTIMEDIA 1995: Proceedings of the third ACM international conference on Multimedia, pp. 295–304. ACM, New York (1995), http://doi.acm.org/10.1145/217279.215283
  8. 8.
    Haritaoglu, I., Harwood, D., Davis, L.: W4: real-time surveillance of people and their activities. IEEE Transactions on Pattern Analysis and Machine Intelligence 22(8), 809–830 (2000), doi:10.1109/34.868683CrossRefGoogle Scholar
  9. 9.
    Hauptmann, A., Yan, R., Qi, Y., Jin, R., Christel, M., Derthick, M., Chen, M.Y., Baron, R., Lin, W.H., Ng, T.D.: Video classification and retrieval with the informedia digital video library system. In: Text Retrieval Conf., TREC 2002 (2002)Google Scholar
  10. 10.
    Horn, B.K., Schunck, B.G.: Determining optical flow. Tech. rep., Cambridge, MA, USA (1980)Google Scholar
  11. 11.
    Huang, J., Liu, Z., Wang, Y., Chen, Y., Wong, E.: Integration of multimodal features for video scene classification based on hmm, pp. 53–58 (1999), doi:10.1109/MMSP.1999.793797Google Scholar
  12. 12.
    Iyengar, G., Lippman, A.: Models for automatic classification of video sequences. In: Storage and Retrieval for Image and Video Databases (SPIE), pp. 216–227 (1998)Google Scholar
  13. 13.
    Jadon, R.S., Chaudhury, S., Biswas, K.K.: Generic video classification: An evolutionary learning based fuzzy theoretic approach. In: Indian Conf. Comput. Vis. Graph. Image Process, ICVGIP (2002)Google Scholar
  14. 14.
    Jasinschi, R., Louie, J.: Automatic tv program genre classification based on audio patterns, pp. 370–375 (2001), doi:10.1109/EURMIC.2001.952477Google Scholar
  15. 15.
    Kim, C., Hwang, J.N.: Fast and automatic video object segmentation and tracking for content-based applications. IEEE Transactions on Circuits and Systems for Video Technology 12(2), 122–129 (2002), doi:10.1109/76.988659CrossRefGoogle Scholar
  16. 16.
    Kobla, V., DeMenthon, D., Doermann, D.: Identifying sports videos using replay, text, and camera motion features. In: Proc. SPIE Conf. Storage Retrieval Media Databases, pp. 332–343 (2000)Google Scholar
  17. 17.
    Lee, M., Nepal, S., Srinivasan, U.: Edge-based semantic classification of sports video sequences, vol. 1, pp. I-157– I-160 (2003), doi:10.1109/ICME.2003.1220878Google Scholar
  18. 18.
    Li, Y., Lee, S.H., Yeh, C.H., Kuo, C.C.J.: Techniques for movie content analysis and skimming: tutorial and overview on video abstraction techniques. IEEE Signal Processing Magazine 23(2), 79–89 (2006), doi:10.1109/MSP.2006.1621451CrossRefMATHGoogle Scholar
  19. 19.
    Lin, W.H., Hauptmann, A.: News video classification using svm-based multimodal classifiers and combination strategies. In: MULTIMEDIA 2002: Proceedings of the tenth ACM international conference on Multimedia, pp. 323–326. ACM, New York (2002), doi:10.1145/641007.641075CrossRefGoogle Scholar
  20. 20.
    Liu, Z., Wang, Y., Chen, T.: Audio feature extraction and analysis for scene segmentation and classification. J. VLSI Signal Process. Syst. 20(1-2), 61–79 (1998)CrossRefGoogle Scholar
  21. 21.
    Morellas, V., Pavlidis, I., Tsiamyrtzis, P.: Deter: detection of events for threat evaluation and recognition. Mach. Vision Appl. 15(1), 29–45 (2003), http://dx.doi.org/10.1007/s00138-003-0121-6 CrossRefGoogle Scholar
  22. 22.
    Murray, D., Basu, A.: Motion tracking with an active camera. IEEE Transactions on Pattern Analysis and Machine Intelligence 16(5), 449–459 (1994), doi:10.1109/34.291452CrossRefGoogle Scholar
  23. 23.
    Nagasaka, A., Tanaka, Y.: Automatic video indexing and full-video search for object appearances. In: Proceedings of the IFIP TC2/WG 2.6 Second Working Conference on Visual Database Systems II, pp. 113–127. North-Holland Publishing Co, Amsterdam (1992)Google Scholar
  24. 24.
    Nam, J., Alghoniemy, M., Tewfik, A.: Audio-visual content-based violent scene characterization, vol. 1, pp. 353–357 (1998), doi:10.1109/ICIP.1998.723496Google Scholar
  25. 25.
    Rasheed, Z., Shah, M.: Movie genre classification by exploiting audio-visual features of previews, vol. 2, pp. 1086–1089 (2002)Google Scholar
  26. 26.
    Ren, Y., Chua, C.S., Ho, Y.K.: Motion detection with nonstationary background. Mach. Vision Appl. 13(5-6), 332–343 (2003), doi: http://dx.doi.org/10.1007/s00138-002-0091-0 Google Scholar
  27. 27.
    Roach, M., Mason, J., Pawlewski, M.: Motion-based classification of cartoons, pp. 146–149 (2001), doi:10.1109/ISIMP, 925353Google Scholar
  28. 28.
    Roach, M., Mason, J., Pawlewski, M.: Video genre classification using dynamics. In: IEEE International Conference on Acoustics, Speech, and Signal Processing, 2001. Proceedings (ICASSP 2001), vol. 3, pp. 1557–1560 (2001), doi:10.1109/ICASSP.2001.941230Google Scholar
  29. 29.
    Stauffer, C., Grimson, W.E.L.: Learning patterns of activity using real-time tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence 22(8), 747–757 (2000), doi: http://doi.ieeecomputersociety.org/10.1109/34.868677
  30. 30.
    Truong, B.T., Dorai, C.: Automatic genre identification for content-based video categorization. In: Proceedings of 15th International Conference on Pattern Recognition, vol. 4, pp. 230–233 (2000), doi:10.1109/ICPR.2000.902901Google Scholar
  31. 31.
    Wan, X., Kuo, C.C.J.: A new approach to image retrieval with hierarchical color clustering. IEEE Transactions on Circuits and Systems for Video Technology 8(5), 628–643 (1998), doi:10.1109/76.718509CrossRefGoogle Scholar
  32. 32.
    Wang, P., Cai, R., Yang, S.Q.: A hybrid approach to news video classification multimodal features, vol. 2, pp. 787–791 (2003), doi:10.1109/ICICS.2003.1292564Google Scholar
  33. 33.
    Wei, G., Agnihotri, L., Dimitrova, N.: Tv program classification based on face and text processing, vol. 3, pp. 1345–1348 (2000), doi:10.1109/ICME.2000.871015Google Scholar
  34. 34.
    Xu, L.Q., Li, Y.: Video classification using spatial-temporal features and pca, vol. 3, pp. III-485–III-488 (2003)Google Scholar
  35. 35.
    Zhang, H., Wu, J., Zhong, D., Smoliar, S.W.: An integrated system for content-based video retrieval and browsing. Pattern Recognition 30(4), 643–658 (1997)CrossRefGoogle Scholar
  36. 36.
    Zhang, T., Kuo, C.C.J.: Audio content analysis for online audiovisual data segmentation and classification. IEEE Transactions on Speech and Audio Processing, 9(4), 441–457 (2001), doi:10.1109/89.917689CrossRefGoogle Scholar
  37. 37.
    Zhou, W., Vellaikal, A., Kuo, C.C.J.: Rule-based video classification system for basketball video indexing. . In: MULTIMEDIA 2000: Proceedings of the 2000 ACM workshops on Multimedia, pp. 213–216. ACM, New York (2000), http://doi.acm.org/10.1145/357744.357941
  38. 38.
    Zhu, W., Toklu, C., Liou, S.P.: Automatic news video segmentation and categorization based on closed-captioned text. In: IEEE International Conference on Multimedia and Expo., ICME 2001, pp. 829–832 (2001)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Ping-Hao Wu
    • 1
  • Sanjay Purushotham
    • 1
  • C. -C. Jay Kuo
    • 1
  1. 1.Ming Hsieh Department of Electrical EngineeringUniversity of Southern CaliforniaLos Angeles

Personalised recommendations