Semantic Retrieval of Radiological Images with Relevance Feedback

  • Camille Kurtz
  • Paul-André Idoux
  • Avinash Thangali
  • Florence Cloppet
  • Christopher F. Beaulieu
  • Daniel L. Rubin
Conference paper

DOI: 10.1007/978-3-319-24471-6_2

Part of the Lecture Notes in Computer Science book series (LNCS, volume 9059)
Cite this paper as:
Kurtz C., Idoux PA., Thangali A., Cloppet F., Beaulieu C.F., Rubin D.L. (2015) Semantic Retrieval of Radiological Images with Relevance Feedback. In: Müller H., Jimenez del Toro O., Hanbury A., Langs G., Foncubierta Rodriguez A. (eds) Multimodal Retrieval in the Medical Domain. Lecture Notes in Computer Science, vol 9059. Springer, Cham

Abstract

Content-based image retrieval can assist radiologists by finding similar images in databases as a means to providing decision support. In general, images are indexed using low-level features, and given a new query image, a distance function is used to find the best matches in the feature space. However, using low-level features to capture the appearance of diseases in images is challenging and the semantic gap between these features and the high-level visual concepts in radiology may impair the system performance. In addition, the results of these systems are fixed and cannot be updated based on user’s intention. We present a new framework that enables retrieving similar images based on high-level semantic image annotations and user feedback. In this framework, database images are automatically annotated with semantic terms. Image retrieval is then performed by computing the similarity between image annotations using a new similarity measure, which takes into account both image-based and ontological inter-term similarities. Finally, a relevance feedback mechanism allows the user to iteratively mark the returned answers, informing which images are relevant according to the query. This information is used to infer user-defined inter-term similarities that are then injected in the image similarity measure to produce a new set of retrieved images. We validated this approach for the retrieval of liver lesions from CT images and annotated with terms of the RadLex ontology.

Keywords

Image retrieval Riesz wavelets Image annotation RadLex Semantic gap Relevance feedback Computed tomographic (CT) images 

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Camille Kurtz
    • 1
  • Paul-André Idoux
    • 1
  • Avinash Thangali
    • 2
  • Florence Cloppet
    • 1
  • Christopher F. Beaulieu
    • 2
  • Daniel L. Rubin
    • 2
  1. 1.LIPADEUniversity Paris DescartesParisFrance
  2. 2.Department of Radiology, School of MedicineStanford UniversityStanfordUSA

Personalised recommendations