Automated Object Extraction for Medical Image Retrieval Using the Insight Toolkit (ITK)

  • Henning Müller
  • Joris Heuberger
  • Adrien Depeursinge
  • Antoine Geissbühler
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4182)


Visual information retrieval is an emerging domain in the medical field as it has been in computer vision for more than ten years. It has the potential to help better managing the rising amount of visual medical data. One of the most frequent application fields for content–based medical image retrieval (CBIR) is diagnostic aid. By submitting an image showing a certain pathology to a CBIR system, the medical expert can easily find similar cases. A major problem is the background surrounding the object in many medical images. System parameters of the imaging modalities are stored around the images in text as well as patient name or a logo of the institution. With such noisy input data, image retrieval often rather finds images where the object appears in the same area and is surrounded by similar structures. Whereas in specialised application domains, segmentation can focus the research on a particular area, PACS–like (Picture Archiving and Communication System) databases containing a large variety of images need a more general approach. This article describes an algorithm to extract the important object of the image to reduce the amount of data to be analysed for CBIR and focuses analysis to the important object. Most current solutions index the entire image without making a difference between object and background when using varied PACS–like databases or radiology teaching files. Our requirement is to have a fully automatic algorithm for object extraction. Medical images have the advantage to normally have one particular object more or less in the centre of the image. The database used for evaluating this task is taken from a radiology teaching file called casimage and the retrieval component is an open source retrieval engine called medGIFT.


Image Retrieval Medical Informatics Mean Average Precision Image Retrieval System Object Extraction 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Smeulders, A.W.M., Worring, M., Santini, S., Gupta, A., Jain, R.: Content–based image retrieval at the end of the early years. IEEE Transactions on Pattern Analysis and Machine Intelligence 22(12), 1349–1380 (2000)CrossRefGoogle Scholar
  2. 2.
    Rui, Y., Huang, T.S., Chang, S.F.: Image retrieval: Past, present and future. In: Liao, M. (ed.) Proceedings of the International Symposium on Multimedia Information Processing, Taipei, Taiwan (1997)Google Scholar
  3. 3.
    Carson, C., Thomas, M., Belongie, S., Hellerstein, J.M., Malik, J.: Blobworld: A system for region–based image indexing and retrieval. In: Huijsmans, D.P., Smeulders, A.W.M. (eds.) VISUAL 1999. LNCS, vol. 1614, pp. 509–517. Springer, Heidelberg (1999)CrossRefGoogle Scholar
  4. 4.
    Flickner, M., Sawhney, H., Niblack, W., Ashley, J., Huang, Q., Dom, B., Gorkani, M., Hafner, J., Lee, D., Petkovic, D., Steele, D., Yanker, P.: Query by Image and Video Content: The QBIC system. IEEE Computer 28, 23–32 (1995)Google Scholar
  5. 5.
    Bach, J.R., Fuller, C., Gupta, A., Hampapur, A., Horowitz, B., Humphrey, R., Jain, R., Shu, C.F.: The Virage image search engine: An open framework for image management. In: Sethi, I.K., Jain, R.C. (eds.) Storage & Retrieval for Image and Video Databases IV, IS&T/SPIE Proceedings, San Jose, CA, USA, vol. 2670, pp. 76–87 (1996)Google Scholar
  6. 6.
    Graham, M.E., Eakins, J.P.: Artisan: A prototype retrieval system for trademark images. Vine 107, 73–80 (1998)Google Scholar
  7. 7.
    Müller, H., Rosset, A., Vallée, J.P., Geissbuhler, A.: Integrating content–based visual access methods into a medical case database. In: Proceedings of the Medical Informatics Europe Conference (MIE 2003), St. Malo, France (2003)Google Scholar
  8. 8.
    Tagare, H.D., Jaffe, C., Duncan, J.: Medical image databases: A content–based retrieval approach. Journal of the American Medical Informatics Association 4, 184–198 (1997)Google Scholar
  9. 9.
    Lowe, H.J., Antipov, I., Hersh, W., Arnott Smith, C.: Towards knowledge–based retrieval of medical images. The role of semantic indexing, image content representation and knowledge–based retrieval. In: Proceedings of the Annual Symposium of the American Society for Medical Informatics (AMIA), Nashville, TN, USA, pp. 882–886 (1998)Google Scholar
  10. 10.
    Tang, L.H.Y., Hanka, R., Ip, H.H.S.: A review of intelligent content–based indexing and browsing of medical images. Health Informatics Journal 5, 40–49 (1999)CrossRefGoogle Scholar
  11. 11.
    Müller, H., Michoux, N., Bandon, D., Geissbuhler, A.: A review of content–based image retrieval systems in medicine – clinical benefits and future directions. International Journal of Medical Informatics 73, 1–23 (2004)CrossRefGoogle Scholar
  12. 12.
    Le Bozec, C., Zapletal, E., Jaulent, M.C., Heudes, D., Degoulet, P.: Towards content–based image retrieval in HIS–integrated PACS. In: Proceedings of the Annual Symposium of the American Society for Medical Informatics (AMIA), Los Angeles, CA, USA, pp. 477–481 (2000)Google Scholar
  13. 13.
    Westerveld, T.: Image retrieval: Content versus context. In: Recherche d’Informations Assistée par Ordinateur (RIAO 2000) Computer–Assisted Information Retrieval, Paris, France, vol. 1, pp. 276–284 (2000)Google Scholar
  14. 14.
    Orphanoudakis, S.C., Chronaki, C.E., Vamvaka, D.: I2Cnet: Content–based similarity search in geographically distributed repositories of medical images. Computerized Medical Imaging and Graphics 20, 193–207 (1996)CrossRefGoogle Scholar
  15. 15.
    Liu, C.T., Tai, P.L., Chen, A.Y.J., Peng, C.H., Lee, T., Wang, J.S.: A content– based CT lung retrieval system for assisting differential diagnosis images collection. In: Proceedings of the second International Conference on Multimedia and Exposition (ICME 2001), Tokyo, Japan, pp. 241–244. IEEE Computer Society, Los Alamitos (2001)Google Scholar
  16. 16.
    Aisen, A.M., Broderick, L.S., Winer-Muram, H., Brodley, C.E., Kak, A.C., Pavlopoulou, C., Dy, J., Shyu, C.R., Marchiori, A.: Automated storage and retrieval of thin–section CT images to assist diagnosis: System description and preliminary assessment. Radiology 228, 265–270 (2003)CrossRefGoogle Scholar
  17. 17.
    Shyu, C.R., Brodley, C.E., Kak, A.C., Kosaka, A., Aisen, A.M., Broderick, L.S.: ASSERT: A physician–in–the–loop content–based retrieval system for HRCT image databases. Computer Vision and Image Understanding (special issue on content–based access for image and video libraries) 75, 111–132 (1999)Google Scholar
  18. 18.
    Lehmann, T., Güld, M.O., Thies, C., Spitzer, K., Keysers, D., Ney, H., Kohnen, M., Schubert, H., Wein, B.B.: Content–based image retrieval in medical applications. Methods of Information in Medicine 43, 354–361 (2004)Google Scholar
  19. 19.
    Güld, M.O., Kohnen, M., Keysers, D., Schubert, H., Wein, B.B., Bredno, J., Lehmann, T.M.: Quality of DICOM header information for image categorization. In: SPIE Proceedings of International Symposium on Medical Imaging, San Diego, CA, USA, vol. 4685, pp. 280–287 (2002)Google Scholar
  20. 20.
    Lehmann, T.M., Schubert, H., Keysers, D., Kohnen, M., Wein, B.B.: The IRMA code for unique classification of medical images. In: SPIE Proceedings of Medical Imaging, San Diego, California, USA, vol. 5033 (2003)Google Scholar
  21. 21.
    Squire, D.M., Müller, W., Müller, H., Pun, T.: Content–based query of image databases: inspirations from text retrieval. In: Ersboll, B.K., Johansen, P. (eds.) Pattern Recognition Letters (Selected Papers from The 11th Scandinavian Conference on Image Analysis SCIA 1999), vol. 21, pp. 1193–1198 (2000)Google Scholar
  22. 22.
    Chen, D., Odobez, J.M., Thiran, J.P.: A localization/verification scheme for finding text in images and video frames based on contrats independent features and machine learning methods. Signal Processing: Image Communication 19, 205–217 (2004)CrossRefGoogle Scholar
  23. 23.
    Agnithotri, L., Dimitrova, N.: Text detection for video analysis. In: IEEE Workshop on Content–based Access of Image and Video Libraries (CBAIVL 1999), Fort Collins, Colorado, USA, pp. 109–113 (1999)Google Scholar
  24. 24.
    Cates, J., Lefohn, A., Whitaker, R.: GIST: An interactive GPU-based level-set segmentation tool for 3d medical images. Journal on Medical Image Analysis 8 (2004) (to appear)Google Scholar
  25. 25.
    Revet, B.: DICOM Cook Book for Implementations in Modalities. In: Philips Medical Systems, Eindhoven, Netherlands (1997)Google Scholar
  26. 26.
    Rosset, A., Müller, H., Martins, M., Dfouni, N., Vallée, J.P., Ratib, O.: Casimage project – a digital teaching files authoring environment. Journal of Thoracic Imaging 19, 1–6 (2004)CrossRefGoogle Scholar
  27. 27.
    Clough, P., Sanderson, M., Müller, H.: A proposal for the CLEF cross language image retrieval track (imageCLEF) 2004. In: CIVR 2004. LNCS, Springer, Heidelberg (2004)Google Scholar
  28. 28.
    Müller, H., Rosset, A., Geissbuhler, A., Terrier, F.: A reference data set for the evaluation of medical image retrieval systems. Computerized Medical Imaging and Graphics (2004) (to appear)Google Scholar
  29. 29.
    Zobel, J.: How reliable are the results of large–scale information retrieval experiments? In: Croft, W.B., Moffat, A., van Rijsbergen, C.J., Wilkinson, R., Zobel, J. (eds.) Proceedings of the 21st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 307–314. ACM Press, New York (1998)CrossRefGoogle Scholar
  30. 30.
    Sparck Jones, K., van Rijsbergen, C.: Report on the need for and provision of an ideal information retrieval test collection. In: British Library Research and Development Report 5266, Computer Laboratory, University of Cambridge (1975)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Henning Müller
    • 1
  • Joris Heuberger
    • 1
  • Adrien Depeursinge
    • 1
  • Antoine Geissbühler
    • 1
  1. 1.Medical InformaticsUniversity and Hospitals of GenevaGenevaSwitzerland

Personalised recommendations