Evaluation of Fast 2D and 3D Medical Image Retrieval Approaches Based on Image Miniatures

  • René Donner
  • Sebastian Haas
  • Andreas Burner
  • Markus Holzer
  • Horst Bischof
  • Georg Langs
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7075)


The present work evaluates four medical image retrieval approaches based on features derived from image miniatures. We argue that due to the restricted domain of medical image data, the standardized acquisition protocols and the absence of a potentially cluttered background a holistic image description is sufficient to capture high-level image similarities. We compare four different miniature 2D and 3D descriptors and corresponding metrics, in terms of their retrieval performance: (A) plain miniatures together with euclidean distances in a k Nearest Neighbor based retrieval backed by kD-trees; (B) correlations of rigidly aligned miniatures, initialized using the kD-tree; (C) distribution fields together with the l 1-norm; (D) SIFT-like histogram of gradients using the χ 2-distance. We evaluate the approaches on two data sets: the ImageClef 2009 benchmark of 2D radiographs with the aim to categorize the images and a large set of 3D-CTs representing a realistic sample in a hospital PACS with the objective to estimate the location of the query volume.


medical image retrieval 3D retrieval image descriptors anatomical region localization 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    André, B., Vercauteren, T., Perchant, A., Wallace, M.B., Buchner, A.M., Ayache, N.: Endomicroscopic image retrieval and classification using invariant visual features. In: Proceedings of the Sixth IEEE International Symposium on Biomedical Imaging (ISBI 2009), pp. 346–349. IEEE, Boston (2009)Google Scholar
  2. 2.
    Avni, U., Goldberger, J., Greenspan, H.: Addressing the ImageCLEF 2009 challenge using a patch-based visual words representation. In: Working Notes for the CLEF 2009 Workshop. The Cross-Language Evaluation Forum (CLEF), Corfu, Greece (2009)Google Scholar
  3. 3.
    Dimitrovski, I., Kocev, D., Loskovska, S., Džeroski, S.: ImageCLEF 2009 Medical Image Annotation Task: PCTs for Hierarchical Multi-Label Classification. In: Peters, C., Caputo, B., Gonzalo, J., Jones, G.J.F., Kalpathy-Cramer, J., Müller, H., Tsikrika, T. (eds.) CLEF 2009. LNCS, vol. 6242, pp. 231–238. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  4. 4.
    Donner, R., Langs, G., Micusik, B., Bischof, H.: Generalized Sparse MRF Appearance Models. Image and Vision Computing 28(6), 1031–1038 (2010)CrossRefGoogle Scholar
  5. 5.
    Feulner, J., Zhou, S.K., Seifert, S., Cavallaro, A., Hornegger, J., Comaniciu, D.: Estimating the body portion of CT volumes by matching histograms of visual words. In: Medical Imaging 2009: Image Processing (Proceedings Volume). vol. 7259, p. 72591V. SPIE (2009)Google Scholar
  6. 6.
    Haas, S., Donner, R., Burner, A., Holzer, M., Langs, G.: SuperPixel-Based Interest Points for Effective Bags of Visual Words Medical Image Retrieval. In: Müller, H., et al. (eds.) MCBR-CDS 2011. LNCS, vol. 7075, pp. 58–68. Springer, Heidelberg (2011)Google Scholar
  7. 7.
    Keysers, D., Dahmen, J., Ney, H., Wein, B., Lehmann, T.: Statistical Framework for Model-Based Image Retrieval in Medical Applications. Journal of Electron Imaging 12(1), 59–68 (2003)CrossRefGoogle Scholar
  8. 8.
    Lehmann, T.M., Schubert, H., Keysers, D., Kohnen, M., Wein, B.B.: The IRMA code for unique classification of medical images. In: Medical Imaging 2003: PACS and Integrated Medical Information Systems: Design and Evaluation (Proceedings Volume), vol. 5033, pp. 440–451. SPIE (2003)Google Scholar
  9. 9.
    Lowe, D.G.: Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision 60, 91–110 (2004)CrossRefGoogle Scholar
  10. 10.
    Sevilla, L., Learned-Miller, E.: Distribution Fields. Technical Report UM-CS-2011-027, Dept. of Computer Science, University of Massachusetts Amherst (2011)Google Scholar
  11. 11.
    Tommasi, T., Caputo, B., Welter, P., Güld, M.O., Deserno, T.M.: Overview of the CLEF 2009 Medical Image Annotation Track. In: Peters, C., Caputo, B., Gonzalo, J., Jones, G.J.F., Kalpathy-Cramer, J., Müller, H., Tsikrika, T. (eds.) CLEF 2009. LNCS, vol. 6242, pp. 85–93. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  12. 12.
    Torralba, A., Fergus, R., Freeman, W.: 80 Million Tiny Images: A large Data Set for Nonparametric Object and Scene Recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence (2008)Google Scholar
  13. 13.
    Ünay, D., Soldea, O., Akyüz, S., Çetin, M., Erçil, A.: Medical image retrieval and automatic annotation: VPA-SABANCI at ImageCLEF 2009. In: Working Notes for the CLEF 2009 Workshop, Corfu, Greece (2009)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • René Donner
    • 1
  • Sebastian Haas
    • 1
  • Andreas Burner
    • 1
  • Markus Holzer
    • 1
  • Horst Bischof
    • 2
  • Georg Langs
    • 1
    • 3
  1. 1.Computational Image Analysis and Radiology Lab, Department of RadiologyMedical University of ViennaAustria
  2. 2.Institute for Computer Graphics and VisionGraz University of TechnologyAustria
  3. 3.Computer Science and Artificial Intelligence LaboratoryMassachusetts Institute of TechnologyCambridgeUSA

Personalised recommendations