Combining Textual and Visual Features for Cross-Language Medical Image Retrieval

  • Pei-Cheng Cheng
  • Been-Chian Chien
  • Hao-Ren Ke
  • Wei-Pang Yang
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4022)


In this paper we describe the technologies and experimental results for the medical retrieval task and automatic annotation task. We combine textual and content-based approaches to retrieve relevant medical images. The content-based approach containing four image features and the text-based approach using word expansion are developed to accomplish these tasks. Experimental results show that combining both the content-based and text-based approaches is better than using only one approach. In the automatic annotation task we use Support Vector Machines (SVM) to learn image feature characteristics for assisting the task of image classification. Based on the SVM model, we analyze which image feature is more promising in medical image retrieval. The results show that the spatial relationship between pixels is an important feature in medical image data because medical image data always has similar anatomic regions. Therefore, image features emphasizing spatial relationship have better results than others.


Support Vector Machine Image Feature Visual Feature Image Retrieval Support Vector Machine Model 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Miller, G.: WordNet: A Lexical Database for English. Communications of the ACM, 39–45 (1995)Google Scholar
  2. 2.
    Cheng, P.C., Chien, B.C., Ke, H.R., Yang, W.P.: KIDS’s evaluation in medical image retrieval task at ImageCLEF 2004. In: Working Notes for the CLEF 2004 Workshop, Bath, UK, September 2004, pp. 585–593 (2004)Google Scholar
  3. 3.
    Han, J., Kamber, M.: Data Mining: Concepts and Techniques. Academic Press, San Diego (2001)Google Scholar
  4. 4.
    Keysers, D., Macherey, W., Ney, H., Dahmen, J.: Adaptation in Statistical Pattern Recogni-tion using Tangent Vectors. IEEE transactions on Pattern Analysis and Machine Intelligence 26(2), 269–274 (2004)CrossRefGoogle Scholar
  5. 5.
    Swain, M.J., Ballard, D.H.: Color Indexing. International Journal of Computer Vision 7, 11–32 (1991)CrossRefGoogle Scholar
  6. 6.
    Salton, G., McGill, M.J.: Introduction to Modern Information Retrieval. McGraw-Hill, New York (1983)MATHGoogle Scholar
  7. 7.
    Manjunath, B.S., Ma, W.Y.: Texture features for browsing and retrieval of large image data. IEEE Transactions on Pattern Analysis and Machine Intelligence 18(8), 837–842 (1996)CrossRefGoogle Scholar
  8. 8.
    Flickner, M., Sawhney, H., Niblack, W., Ashley, J., Huang, Q., Dom, B., Gorkani, M., Hafner, J., Lee, D., Petkovic, D., Steele, D., Yanker, P.: Query by Image and Video Content: The QBIC system. IEEE Computer 28(9), 23–32 (1995)Google Scholar
  9. 9.
    Carson, C., Thomas, M., Belongie, S., Hellerstein, J.M., Malik, J.: Blobworld: A System for Region-Based Image Indexing and Retrieval. In: Huijsmans, D.P., Smeulders, A.W.M. (eds.) VISUAL 1999. LNCS, vol. 1614, pp. 509–517. Springer, Heidelberg (1999)CrossRefGoogle Scholar
  10. 10.
    Belongie, S., Carson, C., Greenspan, H., Malik, J.: Color and texture based image segmentation using EM and its application to content-based image retrieval. In: Proceedings of the International Conference on Computer Vision (ICCV 1998), Bombay, India, pp. 675–682 (1998)Google Scholar
  11. 11.
    Squire, D.M., Muller, W., Muller, H., Raki, J.: Content-Based Query of Image Databases, Inspirations from Text Retrieval: Inverted Files, Frequency-Based Weights and Relevance Feedback. In: Scandinavian Conference on Image Analysis, Kangerlussuaq, Greenland, June 1999, pp. 143–149 (1999)Google Scholar
  12. 12.
    Wang, J.Z., Li, J., Wiederhold, G.: SIMPLIcity: Semantics-Sensitive Integrated Matching for Picture Libraries. IEEE Transaction on Pattern Analysis and Machine Intelligence 23(9), 947–963 (2001)CrossRefGoogle Scholar
  13. 13.
    Boser, B., Guyon, I., Vapnik, V.: A training algorithm for optimal margin classifiers. In: Proceedings of the Fifth Annual Workshop on Computational Learning Theory (1992)Google Scholar
  14. 14.
    Manjunath, B.S., Ma, W.Y.: Texture features for browsing and retrieval of large image data. IEEE Transactions on Pattern Analysis and Machine Intelligence 18(8), 837–842 (1996)CrossRefGoogle Scholar
  15. 15.
    Chang, C.-C., Lin, C.-J.: LIBSVM: a library for support vector machines (2001),
  16. 16.
    Clough, P., Müller, H., Deselaers, T., Grubinger, M., Lehmann, T., Jensen, J., Hersh, W.: The CLEF 2005 Cross–Language Image Retrieval Track. In: Peters, C., Gey, F.C., Gonzalo, J., Müller, H., Jones, G.J.F., Kluck, M., Magnini, B., de Rijke, M., Giampiccolo, D. (eds.) CLEF 2005. LNCS, vol. 4022, pp. 535–557. Springer, Heidelberg (2006)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Pei-Cheng Cheng
    • 1
  • Been-Chian Chien
    • 2
  • Hao-Ren Ke
    • 3
  • Wei-Pang Yang
    • 1
    • 4
  1. 1.Department of Computer & Information ScienceNational Chiao Tung UniversityHsinchuTaiwan, R.O.C.
  2. 2.Department of Computer Science and Information EngineeringNational University of TainanTainanTaiwan, R.O.C.
  3. 3.Library and Institute of Information ManagementNational Chiao Tung UniversityHsinchuTaiwan, R.O.C.
  4. 4.Department of Information ManagementNational Dong Hwa UniversityHualienTaiwan, R.O.C.

Personalised recommendations