Multi-modal Correlation Modeling and Ranking for Retrieval
- Cite this paper as:
- Zhang H., Meng F. (2009) Multi-modal Correlation Modeling and Ranking for Retrieval. In: Muneesawang P., Wu F., Kumazawa I., Roeksabutr A., Liao M., Tang X. (eds) Advances in Multimedia Information Processing - PCM 2009. PCM 2009. Lecture Notes in Computer Science, vol 5879. Springer, Berlin, Heidelberg
Correlation measure is a new hot topic in multimedia retrieval compared to distance metric like Euclidean and Mahalanobis distances. However, most correlation learning algorithms focused on multimedia data of single modality. For heterogeneous multi-modal data of different modalities correlation learning is more complicated. In this paper, we analyze multi-modal correlation among text, image and audio to understand underlying semantics for multi-modal retrieval. First, Kernel Canonical Correlation is used to build a kernel space where global inter-media correlation is analyzed; based on local geometrical topology in the kernel space a weighted graph and corresponding affinity matrix are formed for data and correlation representation; then correlation ranking is used to generate retrieval results; we also provide active learning strategies in relevance feedback to improve retrieval results. Experiment and comparison results are encouraging and show that the performance of our approach is effective.
KeywordsMulti-modal Kernel CCA Correlation Ranking Active Learning
Unable to display preview. Download preview PDF.