Knowledge-Based Concept Score Fusion for Multimedia Retrieval

Abstract

Automated detection of semantic concepts in multimedia documents has been attracting intensive research efforts over the last years. These efforts can be generally classified in two categories of methodologies: the ones that attempt to solve the problem using discriminative methods (classifiers) and those that build knowledge-based models, as driven by the W3C consortium. This paper proposes a methodology that tries to combine both approaches for multimedia retrieval. Our main contribution is the adoption of a formal model for defining concepts using logic and the incorporation of the output of concept classifiers to the computation of annotation scores. Our method does not require the computationally intensive training of new classifiers for the concepts defined. Instead, it employs a knowledge-based mechanism to combine the output score of existing classifiers and can be used for either detecting new concepts or enhancing the accuracy of existing detectors. Optimization procedures are employed to adapt the concept definitions to the multimedia corpus in hand, further improving the attained accuracy. Experiments using the TRECVID2005 video collection demonstrate promising results.