Using High-Level Semantic Features in Video Retrieval

  • Wujie Zheng
  • Jianmin Li
  • Zhangzhang Si
  • Fuzong Lin
  • Bo Zhang
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4071)


Extraction and utilization of high-level semantic features are critical for more effective video retrieval. However, the performance of video retrieval hasn’t benefited much despite of the advances in high-level feature extraction. To make good use of high-level semantic features in video retrieval, we present a method called pointwise mutual information weighted scheme(PMIWS). The method makes a good judgment of the relevance of all the semantic features to the queries, taking the characteristics of semantic features into account. The method can also be extended for the fusion of multi-modalities. Experiment results based on TRECVID2005 corpus demonstrate the effectiveness of the method.


Semantic Feature Mean Average Precision Video Retrieval Text Retrieval Multimedia Document 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Smeulders, A.W.M., Worring, M., Santini, S., Gupta, A., Jain, R.: Content-based image retrieval at the end of the early years. IEEE Transactions on Pattern Analysis and Machine Intelligence 22(12), 1349–1380 (2000)CrossRefGoogle Scholar
  2. 2.
    Natsev, A., Naphade, M.R., Smith, J.R.: Semantic representation: search and mining of multimedia content. In: KDD 2004, pp. 641–646 (2004)Google Scholar
  3. 3.
    Christel, M.G., Hauptmann, A.G.: The Use and Utility of High-Level Semantic Features in Video Retrieval. In: Leow, W.-K., Lew, M., Chua, T.-S., Ma, W.-Y., Chaisorn, L., Bakker, E.M. (eds.) CIVR 2005. LNCS, vol. 3568, pp. 134–144. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  4. 4.
    Manning, C.D., Schutze, H.: Foundations of Statistical Natural Language Processing. The MIT Press, Cambridge (1999)zbMATHGoogle Scholar
  5. 5.
    Over, P., Ianeva, T., Kraaij, W., Smeaton, A.: TRECVID 2005 - An Introduction. In: Proceedings of TRECVID 2005 (2005)Google Scholar
  6. 6.
    Amir, A., Argillandery, J., Campbellz, M., Hauboldz, A., Iyengar, G., Ebadollahiz, S., Kangz, F., Naphadez, M.R. (Paul) Natsevz, A., Smithz, J.R., Tesicz, J., Volkmer, T.: BM Research TRECVID-2005 Video Retrieval System. In: Proceedings of TRECVID 2005 (2005)Google Scholar
  7. 7.
    Jinhui, Y., Huiyi, W., Lan, X., Dong, W., Dayong, D., Yuanyuan, Z., Zijian T., Xiaobing, L., Shuping, X., Wujie, Z., Xirong, L., Zhangzhang, S., Jianmin, L., Fuzong, L., Bo, Z.: Tsinghua University at TRECVID 2005. In: Proceedings of TRECVID 2005 (2005)Google Scholar
  8. 8.
    Iyengar, G., Duygulu, P., Feng, S., Ircing, P., Khudanpur, S.P., Klakow, D., Krause, M.R., Manmatha, R., Nock, H.J., Petkova, D., Pytlik, B., Virga Pages, P.: Joint Visual-Text Modeling for Automatic Retrieval of Multimedia Documents. In: Proceedings of the 13th ACM international conference on Multimedia, pp. 21–30 (2005)Google Scholar
  9. 9.
    TREC-10 Proceedings Appendix on Common Evaluation Measures,
  10. 10.
    Yan, R., Yang, J., Hauptmann, A.G.: Learning query-class dependent weights in automatic video retrieval. In: Proceedings of ACM Multimedia 2004, October 2004, pp. 548–555 (2004)Google Scholar
  11. 11.
    Eidenberger, H., Breiteneder, C.: Semantic Feature Layers in Content-Based Image Retrieval. In: Proceedings IEEE International Conference on Control, Automation, Robotic and Vision, Singapore (2002)Google Scholar
  12. 12.
    Naphade, M.R., Kennedy, L., Kender, J.R., Chang, S-F., C., Smith, J.R., Over, P., Hauptmann, A.: A Light Scale Concept Ontology for Multimedia Understanding for TRECVID 2005. IBM Research Report RC23612 (W0505-104) (May 2005)Google Scholar
  13. 13.
    Platt, J.: Probabilities for SV machines. In: Advances in Large Margin Classiers, pp. 61–74. MIT Press, Cambridge (2000)Google Scholar
  14. 14.
    Chang, S.-F., Hsu, W., Kennedy, L., Xie, L., Yanagawa, A., Zavesky, E., Zhang, D.-Q.: Columbia University TRECVID-2005 Video Search and High-Level Feature Extraction. In: Proceedings of TRECVID 2005 (2005)Google Scholar
  15. 15.
    Chua, T.-S., Neo, S.-Y., Goh, H.-K., Ming, Z.,Yang, X., Gang, W.: TRECVID 2005 by NUS PRIS. In: Proceedings of TRECVID 2005 (2005)Google Scholar
  16. 16.
    Snoek, C.G.M., van Gemert, J.C., Geusebroek, J.M., Huurnink, B., Koelma, D.C., Nguyen, G.P., de Rooij, O., Seinstra, F.J., Smeulders, A.W.M., Veenman, C.J., Worring, M.: The MediaMill TRECVID 2005 Semantic Video Search Engine. In: Proceedings of TRECVID 2005 (2005)Google Scholar
  17. 17.
    Koskela, M., Laaksonen, J., Sjoberg, M., Muurinen, H.: PicSOM Experiments in TRECVID 2005. In: Proceedings of TRECVID 2005 (2005)Google Scholar
  18. 18.
    Hauptmann, A.G., Christel, M., Concescu, R., Gao, J., Jin, Q., Lin, W.-H., Pan, J.-Y., Stevens, S.M., Yan, R., Yang, J., Zhang, Y.: CMU Informedia’s TRECVID 2005 Skirmishes. In: Proceedings of TRECVID 2005 (2005)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Wujie Zheng
    • 1
  • Jianmin Li
    • 1
  • Zhangzhang Si
    • 1
  • Fuzong Lin
    • 1
  • Bo Zhang
    • 1
  1. 1.State Key Laboratory of Intelligent Technology and System, Department of Computer Science and TechnologyTsinghua UniversityBeijingChina

Personalised recommendations