Chapter

Advances in Multimedia Information Processing - PCM 2009

Volume 5879 of the series Lecture Notes in Computer Science pp 637-646

Multi-modal Correlation Modeling and Ranking for Retrieval

  • Hong ZhangAffiliated withCollege of Computer Science & Technology, Wuhan University of Science & Technology
  • , Fanlian MengAffiliated withCollege of Computer Science & Technology, Wuhan University of Science & Technology

* Final gross prices may vary according to local VAT.

Get Access

Abstract

Correlation measure is a new hot topic in multimedia retrieval compared to distance metric like Euclidean and Mahalanobis distances. However, most correlation learning algorithms focused on multimedia data of single modality. For heterogeneous multi-modal data of different modalities correlation learning is more complicated. In this paper, we analyze multi-modal correlation among text, image and audio to understand underlying semantics for multi-modal retrieval. First, Kernel Canonical Correlation is used to build a kernel space where global inter-media correlation is analyzed; based on local geometrical topology in the kernel space a weighted graph and corresponding affinity matrix are formed for data and correlation representation; then correlation ranking is used to generate retrieval results; we also provide active learning strategies in relevance feedback to improve retrieval results. Experiment and comparison results are encouraging and show that the performance of our approach is effective.

Keywords

Multi-modal Kernel CCA Correlation Ranking Active Learning