Multi-modal Correlation Modeling and Ranking for Retrieval

  • Hong Zhang
  • Fanlian Meng
Conference paper

DOI: 10.1007/978-3-642-10467-1_56

Part of the Lecture Notes in Computer Science book series (LNCS, volume 5879)
Cite this paper as:
Zhang H., Meng F. (2009) Multi-modal Correlation Modeling and Ranking for Retrieval. In: Muneesawang P., Wu F., Kumazawa I., Roeksabutr A., Liao M., Tang X. (eds) Advances in Multimedia Information Processing - PCM 2009. PCM 2009. Lecture Notes in Computer Science, vol 5879. Springer, Berlin, Heidelberg

Abstract

Correlation measure is a new hot topic in multimedia retrieval compared to distance metric like Euclidean and Mahalanobis distances. However, most correlation learning algorithms focused on multimedia data of single modality. For heterogeneous multi-modal data of different modalities correlation learning is more complicated. In this paper, we analyze multi-modal correlation among text, image and audio to understand underlying semantics for multi-modal retrieval. First, Kernel Canonical Correlation is used to build a kernel space where global inter-media correlation is analyzed; based on local geometrical topology in the kernel space a weighted graph and corresponding affinity matrix are formed for data and correlation representation; then correlation ranking is used to generate retrieval results; we also provide active learning strategies in relevance feedback to improve retrieval results. Experiment and comparison results are encouraging and show that the performance of our approach is effective.

Keywords

Multi-modal Kernel CCA Correlation Ranking Active Learning 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Hong Zhang
    • 1
  • Fanlian Meng
    • 1
  1. 1.College of Computer Science & TechnologyWuhan University of Science & TechnologyChina

Personalised recommendations