International Journal of Computer Assisted Radiology and Surgery

, Volume 8, Issue 6, pp 1003–1014

Designing user interfaces to enhance human interpretation of medical content-based image retrieval: application to PET-CT images

  • Ashnil Kumar
  • Jinman Kim
  • Lei Bi
  • Michael Fulham
  • Dagan Feng
Original Article

DOI: 10.1007/s11548-013-0896-5

Cite this article as:
Kumar, A., Kim, J., Bi, L. et al. Int J CARS (2013) 8: 1003. doi:10.1007/s11548-013-0896-5


Purpose Content-based image retrieval (CBIR) in medicine has been demonstrated to improve evidence-based diagnosis, education, and teaching. However, the low clinical adoption of CBIR is partially because the focus of most studies has been the development of feature extraction and similarity measurement algorithms with limited work on facilitating better understanding of the similarity between complex volumetric and multi-modality medical images. In this paper, we present a method for defining user interfaces (UIs) that enable effective human user interpretation of retrieved images.

Methods We derived a set of visualisation and interaction requirements based on the characteristics of modern volumetric medical images. We implemented a UI that visualised multiple views of a single image, displayed abstractions of image data, and provided access to supplementary non-image data. We also defined interactions for refining the search and visually indicating the similarities between images. We applied the UI for the retrieval of multi-modality positron emission tomography and computed tomography (PET-CT) images. We conducted a user survey to evaluate the capabilities of our UI.

Results Our proposed method obtained a high rating (\(\ge \)4 out of 5) in the majority of survey questions. In particular, the survey responses indicated the UI presented all the information necessary to understand the retrieved images, and did so in an intuitive manner.

Conclusion Our proposed UI design improved the ability of users to interpret and understand the similarity between retrieved PET-CT images. The implementation of CBIR UIs designed to assist human interpretation could facilitate wider adoption of medical CBIR systems.


Content-based image retrievalRetrieval interpretationUser interfacePET-CT



Content-based image retrieval


Computed tomography


Picture archiving and communications systems


Positron emission tomography


Combined positron emission tomography and computed tomography

Copyright information

© CARS 2013

Authors and Affiliations

  • Ashnil Kumar
    • 1
  • Jinman Kim
    • 1
  • Lei Bi
    • 1
  • Michael Fulham
    • 1
    • 2
    • 3
  • Dagan Feng
    • 1
    • 4
  1. 1.Biomedical and Multimedia Information Technology (BMIT) Research GroupSchool of Information Technologies, University of SydneySydneyAustralia
  2. 2.Department of Molecular ImagingRoyal Prince Alfred HospitalSydneyAustralia
  3. 3.Sydney Medical SchoolUniversity of SydneySydneyAustralia
  4. 4.Med-X Research InstituteShanghai Jiao Tong University ShanghaiChina