Abstract
In this article an XML-driven approach for multimedia information retrieval is presented and evaluated, which uses principal component analysis to derive a composite ranking for a set of XML elements that have a multimedia character. The multimedia strategies that implement the PCA module on top of the B 3-SDR system allow for the integration of image retrieval with the already present text retrieval modules.
Three different strategies are defined. The first strategy implements annotation-based image retrieval, which uses the caption of an image to find related images using a keyword-based search. The second component enables content-based multimedia retrieval by using PCA to derive a composite ranking, which reflects the combined relevance for text and images that are present within an XML element. A simple content-based image retrieval system is build for this purpose, which uses ‘query by example’. The last strategy allows for a bidirectional combination of the first two strategies, where the content-based image retrieval component benefits from the additional images retrieved by the annotation-based search, and vice versa.
The multimedia strategies are evaluated within the INEX 2005 multimedia track, where based on the Lonelyplanet Worldguide and a set of related topics the retrieval performance is measured in terms of recall and precision. The outcome of the experiment shows that the multimedia strategies have a positive influence on the retrieval performance when compared to the text-based XML retrieval system. However, the PCA component did not yet fully live up to its expectation, which is probably due to the poor performance of the ad hoc build image retrieval system that is used for the experiment.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Fuhr, N., Lalmas, M., Malik, S., Szlávik, Z. (eds.): INEX 2004. LNCS, vol. 3493. Springer, Heidelberg (2005)
Smeaton, A., Kraaij, W., Over, P.: The trec video retrieval evaluation (trecvid): A case study and status report. In: Coupling approaches, coupling media and coupling languages for information retrieval - RIAO 2004, Vaucluse, France, pp. 25–37 (2004)
Westerveld, T.: Using generative probabilistic models for multimedia retrieval. PhD thesis, University of Twente, Enschede, The Netherlands (2004)
Smith, L.I.: A Tutorial on Principal Component Analysis (2002)
Fernandez, G.: Data Mining Using SAS Applications. Taylor & francis, CRC Press (2002)
Iskandar, D., Pehcevski, J.: Combining Image and Structured Text Retrieval. In: Advances in XML Information Retrieval. LNCS. Springer, Heidelberg (2006)
Hyvonen, E., Styrman, A., Saarela, S.: Ontology-Based Image Retrieval. In: Towards the semantic web and web services, Proceedings of XML Finland 2002 Conference, pp. 15–27 (2002)
Flickner, M., Sawhney, H., Niblack, W., Ashley, J., Huang, Q., Dom, B., Gorkani, M., Hafner, J., Lee, D., Petkovic, D., Steele, D., Yanker, P.: Query by image and video content: The qbic system. IEEE Computer 28(9), 23–32 (1995)
Urban, J., Jose, J.M., van Rijsbergen, C.: An Adaptive Technique for Content-Based Image Retrieval. In: Multimedia Tools and Applications. Kluwer, Dordrecht (2005)
van Zwol, R., Kazai, G., Lalmas, M.: INEX 2005 Multimedia Track. In: Fuhr, N., Lalmas, M., Malik, S., Kazai, G. (eds.) INEX 2005. LNCS, vol. 3977, pp. 497–510. Springer, Heidelberg (2006)
Yang, J., Zhuang, Y., Li, Q.: Multi-Modal Retrieval for Multimedia Digital Libraries: Issues, Architecture, and Mechanisms. In: Multimedia Information Systems, Capri, Italy, pp. 81–88 (2001)
Feng, D., Siu, W.C., Zhang, H.J. (eds.): Multimedia Information Retrieval and Management: Technological Fundamentals and Applications. Springer, Berlin, Germany (2003)
van Zwol, R., Baas, J., van Oostendorp, H., Wiering, F.: Bricks: The Building Blocks to Tackle Query Formulation in Structured Document Retrieval. In: Lalmas, M., MacFarlane, A., Rüger, S.M., Tombros, A., Tsikrika, T., Yavlinsky, A. (eds.) ECIR 2006. LNCS, vol. 3936, pp. 314–325. Springer, Heidelberg (2006)
Baeza-Yates, R.A., Ribeiro-Neto, B.: Modern Information Retrieval. Addison-Wesley Longman Publishing Co., Inc., Boston (1999)
Jain, A., Nandakumar, K., Ross, A.: Score normalisation in multimodal biometric systems. Pattern Recognition 38(12), 2270–2285 (2005)
van Zwol, R.: B3-SDR and Effective Use of Structural Hints. In: Advances in XML Information Retrieval. LNCS. Springer, Heidelberg (2006)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
van Zwol, R. (2006). Multimedia Strategies for B 3-SDR, Based on Principal Component Analysis. In: Fuhr, N., Lalmas, M., Malik, S., Kazai, G. (eds) Advances in XML Information Retrieval and Evaluation. INEX 2005. Lecture Notes in Computer Science, vol 3977. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-34963-1_41
Download citation
DOI: https://doi.org/10.1007/978-3-540-34963-1_41
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-34962-4
Online ISBN: 978-3-540-34963-1
eBook Packages: Computer ScienceComputer Science (R0)