Multi-feature Subspace Learning via Sparse Correlation Fusion and Embedding

Abstract

Subspace learning is most traditional and important in multimedia analysis. Numerous researches have focused on how to introduce machine learning and statistical methods to multimedia subspace learning for semantic understanding and denoising, and have gained remarkable achievement in different multimedia applications, such as content-based retrieval, data clustering, face recognition, etc. However, most of these researches are based on multimedia data of single modality. Nowadays, with the rapid development of multimedia and information technology, multimedia data of different modalities often coexist, and the presence of one has a complementary effect on the other to some extent. Because different multimedia data are usually represented with heterogeneous low-level features and there exists the well-known semantic gap, it is interesting and challenging to learn multimedia semantics by multi-feature subspace learning of different modalities. In this paper, we analyze sparse canonical correlation between feature matrices of different multimedia data, construct an isomorphic sparse multi-feature subspace; moreover, we propose subspace optimization strategy with correlation fusion, which explores both geometrical-based content correlation and graph-based semantic correlation. Our algorithm has been applied to content-based multimodal retrieval and data classification. Comprehensive experiments have demonstrated the superiority of our method over several existing algorithms.