Abstract
Medical image retrieval is one of the crucial tasks in everyday medical practices. This paper investigates three forms of medical image retrieval: text, visual and multimodal retrieval. We investigate by evaluating different weighting models for text retrieval. In the case of the visual retrieval, we focused on extracting low-level features and examining their performance. For, the multimodal retrieval we used late fusion to combine the best text and visual results. We found that the choice of weighting model for text retrieval dramatically influences the outcome of the multimodal retrieval. The results from the text and visual retrieval are fused using linear combination, which is among the simplest and most frequently used methods. Our results clearly show that the fusion of text and visual retrieval with an appropriate fusion technique improves the retrieval performance.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Kalpathy–Cramer, J., Muller, H., Bedrick, S., Eggel, I., de Herrera, A.G.S., Tsikrika, T.: Overview of the CLEF 2011 medical image classification and retrieval tasks (2011)
Sonka, M., Hlavac, V., Boyle, R., et al.: Image processing, analysis, and machine vision, vol. 2. PWS publishing Pacific Grove, CA (1999)
Deb, S., Zhang, Y.: An overview of content-based image retrieval techniques. In: 18th International Conference on Advanced Information Networking and Applications, vol. 1, pp. 59–64 (2004)
Müller, H., Kalpathy–Cramer, J., Eggel, I., Bedrick, S., Radhouani, S., Bakke, B., Kahn Jr., C.E., Hersh, W.: Overview of the CLEF 2009 medical image retrieval track. In: Peters, C., Caputo, B., Gonzalo, J., Jones, G.J.F., Kalpathy-Cramer, J., Müller, H., Tsikrika, T., et al. (eds.) CLEF 2009. LNCS, vol. 6242, pp. 72–84. Springer, Heidelberg (2010)
Alpkocak, A., Ozturkmenoglu, O., Berber, T., Vahid, A.H., Hamed, R.G.: DEMIR at ImageCLEFMed 2011: Evaluation of Fusion Techniques for Multimodal Content-based Medical Image Retrieval. In: 12th Workshop of the Cross-Language Evaluation Forum (CLEF), Amsterdam, Netherlands (2011)
Csurka, G., Clinchant, S., Jacquet, G.: XRCE’s Participation at Medical Image Modality Classification and Ad-hoc Retrieval Tasks of ImageCLEF (2011)
Gkoufas, Y., Morou, A., Kalamboukis, T.: IPL at ImageCLEF 2011 Medical Retrieval Task. Working Notes of CLEF (2011)
Castellanos, A., Benavent, X., Benavent, J., Garcia-Serrano, A.: UNED-UV at Medical Retrieval Task of ImageCLEF (2011)
Amati, G., Van Rijsbergen, C.J.: Probabilistic models of information retrieval based on measuring the divergence from randomness. ACM Transactions on Information Systems (TOIS) 20, 357–389 (2002)
Hiemstra, D.: A probabilistic justification for using tf-idf term weighting in information retrieval. International Journal on Digital Libraries 3, 131–139 (2000)
Chatzichristofis, S.A., Boutalis, Y.S.: CEDD: Color and edge directivity descriptor: A compact descriptor for image indexing and retrieval. In: Gasteratos, A., Vincze, M., Tsotsos, J.K. (eds.) ICVS 2008. LNCS, vol. 5008, pp. 312–322. Springer, Heidelberg (2008)
Chatzichristofis, S.A., Boutalis, Y.S.: Fcth: Fuzzy color and texture histogram-a low level feature for accurate image retrieval. In: Ninth International Workshop on Image Analysis for Multimedia Interactive Services, pp. 191–196 (2008)
Chatzichristofis, S.A., Boutalis, Y.S.: Content based radiology image retrieval using a fuzzy rule based scalable composite descriptor. Multimedia Tools and Applications 46, 493–519 (2010)
Atrey, P.K., Hossain, M.A., El Saddik, A., Kankanhalli, M.S.: Multimodal fusion for multimedia analysis: a survey. Multimedia Systems 16, 345–379 (2010)
Ounis, I., Amati, G., Plachouras, V., He, B., Macdonald, C., Johnson, D.: Terrier information retrieval platform. In: Losada, D.E., Fernández-Luna, J.M. (eds.) ECIR 2005. LNCS, vol. 3408, pp. 517–519. Springer, Heidelberg (2005)
Porter, M.F.: An algorithm for suffix stripping (1980)
Chatzichristofis, S.A., Boutalis, Y.S., Lux, M.: Img (rummager): An interactive content based image retrieval system. In: Second International Workshop on Similarity Search and Applications, pp. 151–153 (2009)
Croft, W.B.: Combining approaches to information retrieval. In: Advances in Information Retrieval, pp. 1–36 (2002)
Jain, A., Nandakumar, K., Ross, A.: Score normalization in multimodal biometric systems. Pattern Recognition 38, 2270–2285 (2005)
Manning, C.D., Raghavan, P., Schutze, H.: Introduction to information retrieval. Cambridge University Press, Cambridge (2008)
He, B., Ounis, I.: Term frequency normalisation tuning for BM25 and DFR models. In: Losada, D.E., Fernández-Luna, J.M. (eds.) ECIR 2005. LNCS, vol. 3408, pp. 200–214. Springer, Heidelberg (2005)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Kitanovski, I., Trojacanec, K., Dimitrovski, I., Loskovska, S. (2013). Multimodal Medical Image Retrieval. In: Markovski, S., Gusev, M. (eds) ICT Innovations 2012. ICT Innovations 2012. Advances in Intelligent Systems and Computing, vol 207. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-37169-1_8
Download citation
DOI: https://doi.org/10.1007/978-3-642-37169-1_8
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-37168-4
Online ISBN: 978-3-642-37169-1
eBook Packages: EngineeringEngineering (R0)