Multimedia Tools and Applications

, Volume 25, Issue 1, pp 5–35

Multimodal Video Indexing: A Review of the State-of-the-art

Authors

  • Cees G.M. Snoek
    • Intelligent Sensory Information SystemsInformatics Institute, University of Amsterdam
  • Marcel Worring
    • Intelligent Sensory Information SystemsInformatics Institute, University of Amsterdam
Article

DOI: 10.1023/B:MTAP.0000046380.27575.a5

Cite this article as:
Snoek, C.G. & Worring, M. Multimedia Tools and Applications (2005) 25: 5. doi:10.1023/B:MTAP.0000046380.27575.a5

Abstract

Efficient and effective handling of video documents depends on the availability of indexes. Manual indexing is unfeasible for large video collections. In this paper we survey several methods aiming at automating this time and resource consuming process. Good reviews on single modality based video indexing have appeared in literature. Effective indexing, however, requires a multimodal approach in which either the most appropriate modality is selected or the different modalities are used in collaborative fashion. Therefore, instead of separately treating the different information sources involved, and their specific algorithms, we focus on the similarities and differences between the modalities. To that end we put forward a unifying and multimodal framework, which views a video document from the perspective of its author. This framework forms the guiding principle for identifying index types, for which automatic methods are found in literature. It furthermore forms the basis for categorizing these different methods.

review multimodal video indexing video segmentation multimodal integration analysis framework

Copyright information

© Kluwer Academic Publishers 2005