Video Retrieval Using High Level Features: Exploiting Query Matching and Confidence-Based Weighting

  • Shi-Yong Neo
  • Jin Zhao
  • Min-Yen Kan
  • Tat-Seng Chua
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4071)


Recent research in video retrieval has focused on automated, high-level feature indexing on shots or frames. One important application of such indexing is to support precise video retrieval. We report on extensions of this semantic indexing on news video retrieval. First, we utilize extensive query analysis to relate various high-level features and query terms by matching the textual description and context in a time-dependent manner. Second, we introduce a framework to effectively fuse the relation weights with the detectors’ confidence scores. This results in individual high level features that are weighted on a per-query basis. Tests on the TRECVID 2005 dataset show that the above two enhancements yield significant improvement in performance over a corresponding state-of-the-art video retrieval baseline.


Automatic Speech Recognition News Article Query Term Expanded Query Mean Average Precision 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Hauptmann, A., Chen, M.Y., Christel, M., Huang, C., Lin, W.H., Ng, T., Papernick, N., Velivelli, A., Yang, J., Yan, R., Yang, H., Wactlar, H.D.: Confounded expectations: Informedia at TRECVID 2004. In: TRECVID 2004 (2004)Google Scholar
  2. 2.
    Miller, G.: Wordnet: An on-line lexical database. International Journal of Lexicography (1995)Google Scholar
  3. 3.
    Neo, S., Goh, H., Chua, T.: Multimodal event-based model for retrieval of multi-lingual news video. In: IWAIT (2006)Google Scholar
  4. 4.
    Over, P., Ianeva, T.: TRECVID 2005: An introduction. In: TRECVID 2005 (2005)Google Scholar
  5. 5.
    Smeaton, A.F., Kraaij, W., Over, P.: TRECVID - an overview. In: TRECVID 2003 (2003)Google Scholar
  6. 6.
    Amir, A., Iyengar, G., Argillander, J., Campbell, M., Haubold, A., Ebadollahi, S., Kang, F., Naphade, M.R., Natsev, A.P., Smith, J.R., Tesic, J., Volkmer, T.: IBM research TRECVID 2005 video retrieval system. In: TRECVID 2005 (2005)Google Scholar
  7. 7.
    Chang, S.F., Hsu, W., Kennedy, L., Xie, L., Yanagawa, A., Zavesky, E., Zhang, D.Q.: Columbia university TRECVID-2005 video search and high-level feature extraction. In: TRECVID 2005 (2005)Google Scholar
  8. 8.
    Snoek, C.G.M., van Gemert, J., Geusebroek, J.M., Huurnink, B., Koelma, D.C., Nguyen, G.P., de Rooij, O., Seinstra, F.J., Smeulders, A.W.M., Veenman, C.J., Worring, M.: The MediaMill TRECVID 2005 semantic video search engine. In: Proceedings of the 3rd TRECVID Workshop, NIST (2005)Google Scholar
  9. 9.
    Hauptmann, A.G., Christel, M., Concescu, R., Gao, J., Jin, Q., Lin, W.H., Pan, J.Y., Stevens, S.M., Yan, R., Yang, J., Zhang, Y.: CMU Informedia’s TRECVID 2005 skirmishes. In: TRECVID 2005 (2005)Google Scholar
  10. 10.
    Foley, C., Gurrin, C., Jones, G., Lee, H., McGivney, S., O’Connor, N.E., Sav, S., Smeaton, A.F., Wilkins, P.: TRECVid 2005 experiments at dublin city university. In: TRECVID 2005 (2005)Google Scholar
  11. 11.
    Chua, T.S., Neo, S.Y., Goh, H.K., Zhao, M., Xiao, Y., Wang, G.: TRECVID 2005 by NUS PRIS. In: TRECVID 2005 (2005)Google Scholar
  12. 12.
    Chua, T.S., Neo, S.Y., Li, K., Wang, G., Shi, R., Zhao, M., Xu, H.: TRECVID 2004 search and feature extraction task by NUS PRIS. In: TRECVID 2004 (2004)Google Scholar
  13. 13.
    Yang, H., Chua, T.S., Wang, S., Koh, C.K.: Structured use of external knowledge for event-based open-domain question-answering. In: SIGIR 2003, Canada (July 2003)Google Scholar
  14. 14.
    Neo, S., Chua, T.: Query-dependent retrieval on news video. In: MMIR 2005 workshop in SIGIR 2005 (2005)Google Scholar
  15. 15.
    Resnik, P.: Semantic similarity in a taxonomy: An information- based measure and its applications to problems of ambiguity in natural language. Journal of Artificial Intelligence Research 11, 95–130 (1999)MATHGoogle Scholar
  16. 16.
    Kennedy, L.S., Natsev, A.P., Chang, S.F.: Automatic discover of query-class-dependent models for multimodal search. In: ACM Multimedia (MM 2005), pp. 882–891 (2005)Google Scholar
  17. 17.
    Christel, M.G., Hauptmann, A.G.: The use and utility of high-level semantic features in video retrieval. In: Leow, W.-K., Lew, M., Chua, T.-S., Ma, W.-Y., Chaisorn, L., Bakker, E.M. (eds.) CIVR 2005. LNCS, vol. 3568, pp. 134–144. Springer, Heidelberg (2005)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Shi-Yong Neo
    • 1
  • Jin Zhao
    • 1
  • Min-Yen Kan
    • 1
  • Tat-Seng Chua
    • 1
  1. 1.Department of Computer Science, School of ComputingNational University of SingaporeSingapore

Personalised recommendations