Abstract
In the ImageCLEF image retrieval competition multimodal image retrieval has been evaluated over the past seven years. For ICPR 2010 a contest was organized for the fusion of visual and textual retrieval as this was one task where most participants had problems. In this paper, classical approaches such as the maximum combinations (combMAX), the sum combinations (combSUM) and the multiplication of the sum and the number of non–zero scores (combMNZ) were employed and the trade–off between two fusion effects (chorus and dark horse effects) was studied based on the sum of n maxima. Various normalization strategies were tried out. The fusion algorithms are evaluated using the best four visual and textual runs of the ImageCLEF medical image retrieval task 2008 and 2009. The results show that fused runs outperform the best original runs and multi–modality fusion statistically outperforms single modality fusion. The logarithmic rank penalization shows to be the most stable normalization. The dark horse effect is in competition with the chorus effect and each of them can produce best fusion performance depending on the nature of the input data.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Dietterich, T.G.: Ensemble methods in machine learning. In: Kittler, J., Roli, F. (eds.) MCS 2000. LNCS, vol. 1857, pp. 1–15. Springer, Heidelberg (2000)
Müller, H., Michoux, N., Bandon, D., Geissbuhler, A.: A review of content-based image retrieval systems in medicine–clinical benefits and future directions. International Journal of Medical Informatics 73(1), 1–23 (2004)
Hersh, W., Müller, H., Kalpathy-Cramer, J., Kim, E., Zhou, X.: The consolidated ImageCLEFmed medical image retrieval task test collection. Journal of Digital Imaging 22(6), 648–655 (2009)
Müller, H., Kalpathy–Cramer, J., Eggel, I., Bedrick, S., Radhouani, S., Bakke, B., Kahn Jr., C.E., Hersh, W.: Overview of the CLEF 2009 medical image retrieval track. In: Peters, C., Caputo, B., Gonzalo, J., Jones, G.J.F., Kalpathy-Cramer, J., Müller, H., Tsikrika, T. (eds.) CLEF 2009. LNCS, vol. 6242, pp. 72–84. Springer, Heidelberg (2010)
Müller, H., Kalpathy-Cramer, J.: The ImageCLEF medical retrieval task at icpr 2010 — information fusion to combine viusal and textual information. In: Ünay, D., Çataltepe, Z., Aksoy, S. (eds.) ICPR 2010. LNCS, vol. 6388, pp. 101–110. Springer, Heidelberg (2010)
Wu, S., Mcclean, S.: Performance prediction of data fusion for information retrieval. Information Processing & Management 42(4), 899–915 (2006)
Valet, L., Mauris, G., Bolon, P.: A statistical overview of recent literature in information fusion. IEEE Aerospace and Electronic Systems Magazine 16(3), 7–14 (2001)
Croft, W.B.: Combining approaches to information retrieval. In: Advances in Information Retrieval, pp. 1–36. Springer US, Heidelberg (2000)
Kludas, J., Bruno, E., Marchand-Maillet, S.: Information fusion in multimedia information retrieval. In: Proceedings of 5th International Workshop on Adaptive Multimedia Retrieval (AMR), vol. 4918, pp. 147–159. ACM, New York (June 2008)
Dwork, C., Kumar, R., Naor, M., Sivakumar, D.: Rank aggregation methods for the web. In: WWW 2001: Proceedings of the 10th International Conference on World Wide Web, New York, NY, USA, pp. 613–622 (2001)
Renda, E.M., Straccia, U.: Web metasearch: rank vs. score based rank aggregation methods. In: SAC 2003: Proceedings of the 2003 ACM Symposium on Applied Computing, pp. 841–846. ACM Press, New York (2003)
Fox, E.A., Shaw, J.A.: Combination of multiple searches. In: Text REtrieval Conference, pp. 243–252 (1993)
Aslam, J.A., Montague, M.: Models for metasearch. In: SIGIR 2001: Proceedings of the 24th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 276–284. ACM, New York (2001)
Montague, M., Aslam, J.A.: Condorcet fusion for improved retrieval. In: CIKM 2002: Proceedings of the Eleventh International Conference on Information and Knowledge Management, pp. 538–548. ACM, New York (2002)
Lillis, D., Toolan, F., Collier, R., Dunnion, J.: Probfuse: a probabilistic approach to data fusion. In: SIGIR 2006: Proceedings of the 29th ACM SIGIR Conference on Research and Development in Information Retrieval, New York, USA, pp. 139–146 (2006)
Vogt, C.C., Cottrell, G.W.: Fusion via a linear combination of scores. Information Retrieval 1(3), 151–173 (1999)
Lee, J.H.: Analyses of multiple evidence combination. In: SIGIR 1997: Proceedings of the 20th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 267–276. ACM, New York (1997)
Wu, S., Crestani, F., Bi, Y.: Evaluating score normalization methods in data fusion. In: Information Retrieval Technology, AIRS 2006, pp. 642–648 (2006)
Wu, S., Bi, Y., Zeng, X., Han, L.: Assigning appropriate weights for the linear combination data fusion method in information retrieval. Information Processing & Management 45(4), 413–426 (2009)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Zhou, X., Depeursinge, A., Müller, H. (2010). Information Fusion for Combining Visual and Textual Image Retrieval in ImageCLEF@ICPR. In: Ünay, D., Çataltepe, Z., Aksoy, S. (eds) Recognizing Patterns in Signals, Speech, Images and Videos. ICPR 2010. Lecture Notes in Computer Science, vol 6388. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-17711-8_14
Download citation
DOI: https://doi.org/10.1007/978-3-642-17711-8_14
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-17710-1
Online ISBN: 978-3-642-17711-8
eBook Packages: Computer ScienceComputer Science (R0)