Multimedia Evidence Fusion for Video Concept Detection via OWA Operator

  • Ming Li
  • Yan-Tao Zheng
  • Shou-Xun Lin
  • Yong-Dong Zhang
  • Tat-Seng Chua
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5371)


We present a novel multi-modal evidence fusion method for highlevel feature (HLF) detection in videos. The uni-modal features, such as color histogram, transcript texts, etc, tend to capture different aspects of HLFs and hence share complementariness and redundancy in modeling the contents of such HLFs. We argue that such inter-relation are key to effective multi-modal fusion. Here, we formulate the fusion as a multi-criteria group decision making task, in which the uni-modal detectors are coordinated for a consensus final detection decision, based on their inter-relations. Specifically, we mine the complementariness and redundancy inter-relation of uni-modal detectors using the Ordered Weighted Average (OWA) operator. The ‘or-ness’ measure in OWA models the inter-relation of uni-modal detectors as combination of pure complementariness and pure redundancy. The resulting weights of OWA can then yield a consensus fusion, by optimally leveraging the decisions of uni-modal detectors. The experiments on TRECVID 07 dataset show that the proposed OWA aggregation operator can significantly outperform other fusion methods, by achieving a state-of-art MAP of 0.132.


OWA Fusion Video Concept Detection 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Chang, S.-F., Hsu, W., Kennedy, L., Xie, L., Yanagawa, A., Zavesky, E., Zhang., D.-Q.: Columbia university trecvid 2005 video search and high-level feature extraction. In: TREC Video Retrieval Evaluation Proceedings (March 2006)Google Scholar
  2. 2.
    Dorai, C., Venkatesh., S.: Bridging the semantic gap with computational media aesthetics. IEEE MultiMedia 10(2), 15–17 (2003)CrossRefGoogle Scholar
  3. 3.
    Hauptmann, A.G., Chen, M.-Y., Christel, M., Lin, W.-H., Yan, R., Yang, J.: 2006. Multi-lingual broadcast news retrieval. In: Proceedings of TREC Video Retrieval Evaluation Proceedings (March 2006)Google Scholar
  4. 4.
    Mei, T., Hua, X., Lai, W., Yang, L., Zha, Z., Liu, Y., Gu, Z., Qi, G., Wang, M., Tang, J., Yuan, X., Lu, Z., Liu, J.: MSRA-USTC-SJTU at TRECVID 2007: High-level feature extraction and search (2007),
  5. 5.
    Le, H.D., Satoh, S., Matsui, T.: NII-ISM, Japan at TRECVID 2007: High Level Feature Extraction (2007),
  6. 6.
    Snoek, C., Worring, M., Gemert, J., Geusebroek, J.-M., Smeulders, A.: 2006. The challenge problem for automated detection of 101 semantic concepts in multimedia. In: Proceedings of ACM MM, pp. 421–430 (2006)Google Scholar
  7. 7.
    Kacprzyk, J., Fedrizzi, M., Nurmi, H.: OWA operators in group decision making and consensus reaching under fuzzy preferences and fuzzy majority. In: Yager, R.R., Kacprzyk, J. (eds.) The Ordered Weighted Averaging Operators: Theory and Applications, pp. 193–206. Kluwer Academic Publishers, Dordrecht (1997)CrossRefGoogle Scholar
  8. 8.
    Yager, R.R.: Ordered weighted averaging aggregation operators in multi-criteria decision making. IEEE Tran. On Systems, Man and Cybernetics 18, 183–190 (1988)CrossRefzbMATHGoogle Scholar
  9. 9.
    Marchant, T.: Maximal orness weights with a fixed variability for OWA operators. International Journal of Uncertainty Fuzziness and Knowledge Based Systems 14, 271–276 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  10. 10.
    Fuller, R., Majlender, P.: An analytic approach for obtaining maximal entropy OWA operator weights. Fuzzy Sets and System 124, 53–57 (2001)MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    Smeaton, A.F., Over, P., Kraaij, W.: Evaluation campaigns and TRECVid. In: Proceedings of the 8th ACM International Workshop on Multimedia Information Retrieval MIR 2006, pp. 321–330. ACM Press, New York (2006)Google Scholar
  12. 12.
    Ngo, C., Jiang, Y., Wei, X., Wang, F., Zhao, W., Tan, H., Wu, X.: Experimenting vireo-374: Bag-of-visual-words and visual-based ontology for semantic video indexing and search. In: TREC Video Retrieval Evaluation Proceedings (November 2007)Google Scholar
  13. 13.
    Magalhães, J., Rüger, S.: Information-theoretic semantic multimedia indexing. In: Proceedings of the 6th ACM international conference on Image and video retrieval (CIVR 2007) (July 2007)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Ming Li
    • 1
    • 2
    • 3
  • Yan-Tao Zheng
    • 2
  • Shou-Xun Lin
    • 1
  • Yong-Dong Zhang
    • 1
  • Tat-Seng Chua
    • 2
  1. 1.Key Laboratory of Intelligent Information Processing, ICT, CASBeijingChina
  2. 2.Department of Computer ScienceNational University of SingaporeSingapore
  3. 3.Graduate School of the Chinese Academy of SciencesBeijingChina

Personalised recommendations