Journal of Digital Imaging

, Volume 24, Issue 4, pp 739–748 | Cite as

Managing Biomedical Image Metadata for Search and Retrieval of Similar Images

  • Daniel Korenblum
  • Daniel Rubin
  • Sandy Napel
  • Cesar Rodriguez
  • Chris Beaulieu


Radiology images are generally disconnected from the metadata describing their contents, such as imaging observations (“semantic” metadata), which are usually described in text reports that are not directly linked to the images. We developed a system, the Biomedical Image Metadata Manager (BIMM) to (1) address the problem of managing biomedical image metadata and (2) facilitate the retrieval of similar images using semantic feature metadata. Our approach allows radiologists, researchers, and students to take advantage of the vast and growing repositories of medical image data by explicitly linking images to their associated metadata in a relational database that is globally accessible through a Web application. BIMM receives input in the form of standard-based metadata files using Web service and parses and stores the metadata in a relational database allowing efficient data query and maintenance capabilities. Upon querying BIMM for images, 2D regions of interest (ROIs) stored as metadata are automatically rendered onto preview images included in search results. The system’s “match observations” function retrieves images with similar ROIs based on specific semantic features describing imaging observation characteristics (IOCs). We demonstrate that the system, using IOCs alone, can accurately retrieve images with diagnoses matching the query images, and we evaluate its performance on a set of annotated liver lesion images. BIMM has several potential applications, e.g., computer-aided detection and diagnosis, content-based image retrieval, automating medical analysis protocols, and gathering population statistics like disease prevalences. The system provides a framework for decision support systems, potentially improving their diagnostic accuracy and selection of appropriate therapies.

Key words

Imaging informatics data mining databases decision support body imaging cancer detection computed tomography computer-aided diagnosis (CAD) image retrieval PACS ROC curve ROC-based analysis web technology digital imaging and communications in medicine (DICOM) algorithms 



This study is supported in part by NIH CA72023.


  1. 1.
    Rubin GD: Data explosion: the challenge of multidetector-row CT. Eur J Radiol 36:74–80, 2000PubMedCrossRefGoogle Scholar
  2. 2.
    Huang HK: Some historical remarks on picture archiving and communication systems. Comput Med Imaging Graph 27:93–99, 2003PubMedCrossRefGoogle Scholar
  3. 3.
    Lowe HJ, Antipov I, Hersh W, Smith CA: Towards knowledge-based retrieval of medical images. The role of semantic indexing, image content representation and knowledge-based retrieval. Proc AMIA Symp:882–886, 1998Google Scholar
  4. 4.
    Dina D-F, Sameer A, Mohammad-Reza S, Hamid S-Z, Farshad F, Kost E: Automatically Finding Images for Clinical Decision Support: IEEE Computer Society, 2007Google Scholar
  5. 5.
    Hai J, et al: Content and semantic context based image retrieval for medical image grid: IEEE Computer Society, 2007Google Scholar
  6. 6.
    Oria V, et al: Modeling Images for Content-Based Queries: The DISIMA Approach, 1997Google Scholar
  7. 7.
    Solomon A, Richard C, Lionel B: Content-Based and Metadata Retrieval in Medical Image Database: IEEE Computer Society, 2002Google Scholar
  8. 8.
    Warren R, et al: MammoGrid—a prototype distributed mammographic database for Europe. Clin Radiol 62:1044–1051, 2007PubMedCrossRefGoogle Scholar
  9. 9.
    Wesley WC, Chih-Cheng H, Alfonso FC, Cardenas AF, Ricky KT: Knowledge-Based Image Retrieval with Spatial and Temporal Constructs. IEEE Trans on Knowl and Data Eng 10:872–888, 1998CrossRefGoogle Scholar
  10. 10.
    Channin DS, Mongkolwat P, Kleper V, Sepukar K, Rubin DL: The caBIG Annotation and Image Markup Project. J Digit Imaging 23:217–25, 2010PubMedCrossRefGoogle Scholar
  11. 11.
    Rubin DL, Mongkolwat P, Kleper V, Supekar K, Channin DS: Medical Imaging on the Semantic Web: Annotation and Image Markup, Stanford University, 2008Google Scholar
  12. 12.
    Rubin DL, Supekar K, Mongkolwat P, Kleper V, Channin DS: Annotation and Image Markup: Accessing and Interoperating with the Semantic Content in Medical Imaging. Ieee Intelligent Systems 24:57–65, 2009CrossRefGoogle Scholar
  13. 13.
    Rubin DL, Rodriguez C, Shah P, Beaulieu C: iPad: Semantic annotation and markup of radiological images. AMIA Annu Symp Proc:626–630, 2008Google Scholar
  14. 14.
    Rosset A, Spadola L, Ratib O: OsiriX: an open-source software for navigating in multidimensional DICOM images. J Digit Imaging 17:205–216, 2004PubMedCrossRefGoogle Scholar
  15. 15.
    Kundu S, et al: The IR Radlex Project: an interventional radiology lexicon—a collaborative project of the Radiological Society of North America and the Society of Interventional Radiology. J Vasc Interv Radiol 20:S275–277, 2009PubMedCrossRefGoogle Scholar
  16. 16.
    Koutelakis GV, Lymperopoulos DK: PACS through web compatible with DICOM standard and WADO service: advantages and implementation. Conf Proc IEEE Eng Med Biol Soc 1:2601–2605, 2006PubMedCrossRefGoogle Scholar
  17. 17.
    Kahn CE Jr, Thao C: GoldMiner: a radiology image search engine. AJR Am J Roentgenol 188:1475–1478, 2007PubMedCrossRefGoogle Scholar
  18. 18.
    Xu S, McCusker J, Krauthammer M: Yale Image Finder (YIF): a new search engine for retrieving biomedical images. Bioinformatics 24:1968–1970, 2008PubMedCrossRefGoogle Scholar
  19. 19.
    Zhou XS, et al: Semantics and CBIR: a medical imaging perspective. ACM, New York, NY, USA, 2008Google Scholar
  20. 20.
    Lober WB, Trigg LJ, Bliss D, Brinkley JM: IML: An image markup language. Journal of the American Medical Informatics Association:403–407, 2001Google Scholar
  21. 21.
    Oster S, et al: caGrid 1.0: an enterprise Grid infrastructure for biomedical research. J Am Med Inform Assoc 15:138–149, 2008PubMedCrossRefGoogle Scholar
  22. 22.
    Napel S, et al: Automated retrieval of CT images of liver lesions on the basis of image similarity: method and preliminary results. Radiology 256(1): 243–52, 2010Google Scholar

Copyright information

© Society for Imaging Informatics in Medicine 2010

Authors and Affiliations

  • Daniel Korenblum
    • 1
  • Daniel Rubin
    • 1
    • 2
  • Sandy Napel
    • 1
  • Cesar Rodriguez
    • 3
  • Chris Beaulieu
    • 1
  1. 1.Department of RadiologyStanford UniversityStanfordUSA
  2. 2.Department of Stanford Medical InformaticsStanford UniversityStanfordUSA
  3. 3.BIOFABUniversity of California at BerkeleyBerkeleyUSA

Personalised recommendations