Building and Querying RDF/OWL Database of Semantically Annotated Nuclear Medicine Images
- 338 Downloads
As the use of positron emission tomography-computed tomography (PET-CT) has increased rapidly, there is a need to retrieve relevant medical images that can assist image interpretation. However, the images themselves lack the explicit information needed for query. We constructed a semantically structured database of nuclear medicine images using the Annotation and Image Markup (AIM) format and evaluated the ability the AIM annotations to improve image search. We created AIM annotation templates specific to the nuclear medicine domain and used them to annotate 100 nuclear medicine PET-CT studies in AIM format using controlled vocabulary. We evaluated image retrieval from 20 specific clinical queries. As the gold standard, two nuclear medicine physicians manually retrieved the relevant images from the image database using free text search of radiology reports for the same queries. We compared query results with the manually retrieved results obtained by the physicians. The query performance indicated a 98 % recall for simple queries and a 89 % recall for complex queries. In total, the queries provided 95 % (75 of 79 images) recall, 100 % precision, and an F1 score of 0.97 for the 20 clinical queries. Three of the four images missed by the queries required reasoning for successful retrieval. Nuclear medicine images augmented using semantic annotations in AIM enabled high recall and precision for simple queries, helping physicians to retrieve the relevant images. Further study using a larger data set and the implementation of an inference engine may improve query results for more complex queries.
KeywordsImage retrieval Nuclear medicine PET Controlled vocabulary Protégé AIM ePAD
This work was supported in part by grants from the National Cancer Institute, National Institutes of Health, U01CA142555 and 1U01CA190214.
Compliance with Ethical Standards
This study was approved by the Institutional Review Board and written consent was waived.
- 5.Smith JR, Chang SF: VisualSeek: a fully automatic content-based query system. Proceedings of the Fourth ACM International Conference on Multimedia, ACM Multimedia’96, Boston, MA, Nov 1996Google Scholar
- 6.Ma WY, Manjunath B: Netra: a toolbox for navigating large image databases. Proceedings of the IEEE International Conference on Image Processing 1:1997, pp 568–571Google Scholar
- 8.Brodley C, Kak A, Shyu C, Dy J, Broderick L, Aisen AM: Content-based retrieval from medical image database: a synergy of human interaction, machine learning and computer vision. Proceedings of the Sixteenth National Conference on Artificial Intelligence (AAAI-99), Orlando, FL, Jul 1999Google Scholar
- 9.Bray T, Paoli J, Sperberg-McQueen CM: Extensible Markup Language (XML) 1.0, W3C recommendation. Available at http://www.w3.org/TR/REC-xml. Accessed 15 September 2016
- 10.Lassila O, Swick RR: Resource Description Framework (RDF) Model and Syntax Specification, W3C recommendation. Available at http://www.w3.org/TR/PR-rdf-syntax. Accessed 15 September 2016
- 11.Bao J, Kendall EF, McGuinness DL, Patel-Schneider PF: OWL 2 Web Ontology Language Quick Reference Guide (Second Edition), W3C recommendation. Available at http://www.w3.org/TR/2012/REC-owl2-quick-reference-20121211/. Accessed 15 September 2016
- 16.Sherter AL: Building a vocabulary. A new, improved version of SNOMED has the potential to ease the collection and analysis of clinical data. Health Data Manag 6:76–77, 1998Google Scholar
- 19.Rubin DL, Mongkolwat P, Kleper V, Supekar K, Channin DS: Medical Imaging on the Semantic Web: Annotation and Image Markup, Association for the Advancement of Artificial Intelligence, 2008. Spring Symposium Series, Stanford, 2008Google Scholar
- 21.Rubin DL, Rodriguez C, Shah P, Beaulieu C: iPad: Semantic annotation and markup of radiological images. AMIA Annu Symp Proc:626–630, 2008Google Scholar
- 24.Moreira DA, Hage C, Luque EF, Willrett D, Rubin DL: 3D markup of radiological images in ePAD, a web-based image annotation tool. IEEE 28th International Symposium on Computer-Based Medical Systems, Proc, 2015, pp 97–102Google Scholar
- 25.Prud'hommeaux E, Seaborne A: SPARQL Query Language for RDF. W3C Recommendation. Available at http://www.w3.org/TR/rdf-sparql-query. Accessed 25 September 2015
- 27.Mate S, Kopcke F, Toddenroth D, Martin M, Prokosch H-U, Burkle T, Ganslandt T: Ontology-based data integration between clinical and research systems. PLoS ONE, 2015. doi: 10.1371/journal.pone.0116656