Retrieval of Multimedia Objects by Combining Semantic Information from Visual and Textual Descriptors
We propose a method of content-based multimedia retrieval of objects with visual, aural and textual properties. In our method, training examples of objects belonging to a specific semantic class are associated with their low-level visual descriptors (such as MPEG-7) and textual features such as frequencies of significant keywords. A fuzzy mapping of a semantic class in the training set to a class of similar objects in the test set is created by using Self-Organizing Maps (SOMs) trained from automatically extracted low-level descriptors. We have performed several experiments with different textual features to evaluate the potential of our approach in bridging the gap from visual features to semantic concepts by the use textual presentations. Our initial results show a promising increase in retrieval performance.
KeywordsTextual Feature Video Clip Machine Translation Retrieval Performance Semantic Concept
Unable to display preview. Download preview PDF.
- 1.Laaksonen, J., Koskela, M., Oja, E.: PicSOM—Self-organizing image retrieval with MPEG-7 content descriptions. IEEE Transactions on Neural Networks, Special Issue on Intelligent Multimedia Processing 13(4), 841–853 (2002)Google Scholar
- 3.Koskela, M., Laaksonen, J., Sjöberg, M., Muurinen, H.: PicSOM experiments in TRECVID 2005. In: Proceedings of the TRECVID 2005 Workshop, Gaithersburg, MD, USA, pp. 262–270 (2005)Google Scholar
- 4.MPEG: MPEG-7 visual part of the eXperimentation Model (version 9.0) (2001), ISO/IEC JTC1/SC29/WG11 N3914Google Scholar
- 5.Davis, S.B., Mermelstein, P.: Comparison of parametric representations for monosyllabic word recognition in continuously spoken sentences. In: Waibel, A., Lee, K. (eds.) Readings in speech recognition, pp. 65–74. Morgan Kaufmann Publishers Inc., San Francisco (1990)Google Scholar
- 6.Salton, G., McGill, M.J.: Introduction to Modern Information Retrieval. Computer Science Series. McGraw-Hill, New York (1983)Google Scholar
- 7.Koehn, P.: Europarl: A multilingual corpus for evaluation of machine translation, http://people.csail.mit.edu/~koehn/publications/europarl.ps (Draft, Unpublished)
- 9.Porter, M.: An algorithm for suffix stripping. Program 14(3), 130–137 (1980)Google Scholar