Journal of Digital Imaging

, Volume 26, Issue 6, pp 1025–1039 | Cite as

Content-Based Medical Image Retrieval: A Survey of Applications to Multidimensional and Multimodality Data

  • Ashnil KumarEmail author
  • Jinman Kim
  • Weidong Cai
  • Michael Fulham
  • Dagan Feng


Medical imaging is fundamental to modern healthcare, and its widespread use has resulted in the creation of image databases, as well as picture archiving and communication systems. These repositories now contain images from a diverse range of modalities, multidimensional (three-dimensional or time-varying) images, as well as co-aligned multimodality images. These image collections offer the opportunity for evidence-based diagnosis, teaching, and research; for these applications, there is a requirement for appropriate methods to search the collections for images that have characteristics similar to the case(s) of interest. Content-based image retrieval (CBIR) is an image search technique that complements the conventional text-based retrieval of images by using visual features, such as color, texture, and shape, as search criteria. Medical CBIR is an established field of study that is beginning to realize promise when applied to multidimensional and multimodality medical data. In this paper, we present a review of state-of-the-art medical CBIR approaches in five main categories: two-dimensional image retrieval, retrieval of images with three or more dimensions, the use of nonimage data to enhance the retrieval, multimodality image retrieval, and retrieval from diverse datasets. We use these categories as a framework for discussing the state of the art, focusing on the characteristics and modalities of the information used during medical image retrieval.


Content-based image retrieval Medical images Multimodality data Multidimensional data Review 



We are grateful to our collaborators at the Royal Prince Alfred Hospital, Sydney, Australia for their direct and indirect contributions to this work. This work was supported in part by ARC grants.


  1. 1.
    Doi K: Computer-aided diagnosis in medical imaging: Historical review, current status and future potential. Comput Med Imaging Graph 31(4–5):198–211, 2007PubMedCrossRefGoogle Scholar
  2. 2.
    Zaidi H, Vees H, Wissmeyer M: Molecular PET/CT imaging-guided radiation therapy treatment planning. Acad Radiol 16(9):1108–33, 2009PubMedCrossRefGoogle Scholar
  3. 3.
    Marcus C, Ladam-Marcus V, Cucu C, Bouché O, Lucas L, Hoeffel C: Imaging techniques to evaluate the response to treatment in oncology: Current standards and perspectives. Crit Rev Oncol/Hematol 72(3):217–38, 2009CrossRefGoogle Scholar
  4. 4.
    Holt A, Bichindaritz I, Schmidt R, Perner P: Medical applications in case-based reasoning. Knowl Eng Rev 20(03):289–92, 2005CrossRefGoogle Scholar
  5. 5.
    Sedghi S, Sanderson M, Clough P: How do health care professionals select medical images they need? ASLIB Proc 64(4):437–56, 2012CrossRefGoogle Scholar
  6. 6.
    Haux R: Health information systems—Past, present, future. Int J Med Inform 75(3–4):268–81, 2006PubMedCrossRefGoogle Scholar
  7. 7.
    Huang HK. PACS and Imaging Informatics: Basic Principles and Applications. New York: Wiley, 2004Google Scholar
  8. 8.
    Müller H, Michoux N, Bandon D, Geissbuhler A: A review of content-based image retrieval systems in medical applications—Clinical benefits and future directions. Int J Med Inform 73(1):1–23, 2004PubMedCrossRefGoogle Scholar
  9. 9.
    Müller H, Zhou X, Depeursinge A, Pitkanen M, Iavindrasana J, Geissbuhler A: Medical visual information retrieval: State of the art and challenges ahead. In: Proceedings of the IEEE International Conference on Multimedia and Expo, Beijing, 2007, pp 683–686Google Scholar
  10. 10.
    Müller H, Kalpathy-Cramer J, Caputo B, Syeda-Mahmood T, Wang F: Overview of the first workshop on medical content-based retrieval for clinical decision support at MICCAI 2009. In: Caputo B, Müller H, Syeda-Mahmood T, Duncan J, Wang F, Kalpathy-Cramer J Eds. Medical Content-Based Retrieval for Clinical Decision Support, Vol. 5853 of Lecture Notes in Computer Science. Berlin: Springer, 2010, pp 1–17Google Scholar
  11. 11.
    Huang HK: Utilization of medical imaging informatics and biometrics technologies in healthcare delivery. Int J Comput Assist Radiol Surg 3:27–39, 2008CrossRefGoogle Scholar
  12. 12.
    Tagare HD, Jaffe CC, Duncan J: Medical image databases: A content-based retrieval approach. J Am Med Inform Assoc 4(3):184–98, 1997PubMedCrossRefGoogle Scholar
  13. 13.
    Lehmann TM, Guld MO, Thies C, Fischer B, Keysers D, Kohnen M, et al: Content-based image retrieval in medical applications for picture archiving and communication systems. In: Huang HK, Ratib OM Eds. Proceedings of SPIE 5033, 2003, pp 109–117Google Scholar
  14. 14.
    Brown KR, Silver I, Musgrave J, Roberts A: The use of μCT technology to identify skull fracture in a case involving blunt force trauma. Forensic Sci Int 206(1–3):8–11, 2011CrossRefGoogle Scholar
  15. 15.
    Blodgett TM, Meltzer CC, Townsend DW: PET/CT: Form and function. Radiology 242(2):360–85, 2007PubMedCrossRefGoogle Scholar
  16. 16.
    Smeulders A, Worring M, Santini S, Gupta A, Jain R: Content-based image retrieval at the end of the early years. IEEE Trans Pattern Anal Mach Intell 22(12):1349–80, 2000CrossRefGoogle Scholar
  17. 17.
    Cai TW, Kim J, Feng DD: Content-based medical image retrieval. In: Feng DD Ed. Biomedical Information Technology. Burlington: Academic Press, 2008, pp 83–113Google Scholar
  18. 18.
    Long LR, Antani S, Deserno TM, Thoma GR: Content-based image retrieval in medicine: Retrospective assessment, state of the art, and future directions. Int J Healthcare Inf Syst Inform 4(1):1–16, 2009CrossRefGoogle Scholar
  19. 19.
    Akgül C, Rubin D, Napel S, Beaulieu C, Greenspan H, Acar B: Content-based image retrieval in radiology: Current status and future directions. J Digit Imaging 24:208–22, 2011PubMedCrossRefGoogle Scholar
  20. 20.
    Lew MS, Sebe N, Djeraba C, Jain R: Content-based multimedia information retrieval: State of the art and challenges. ACM Trans Multimed Comput Commun Appl 2(1):1–19, 2006CrossRefGoogle Scholar
  21. 21.
    Rui Y, Huang TS, Chang SF: Image retrieval: Current techniques, promising directions, and open issues. J Vis Commun Image Represent 10(1):39–62, 1999CrossRefGoogle Scholar
  22. 22.
    Datta R, Joshi D, Li J, Wang JZ: Image retrieval: Ideas, influences, and trends of the new age. ACM Comput Surv 40(2):5:1–5:60, 2008Google Scholar
  23. 23.
    Flickner M, Sawhney H, Niblack W, Ashley J, Huang Q, Dom B, et al: Query by image and video content: The QBIC system. Computer 28(9):23–32, 1995CrossRefGoogle Scholar
  24. 24.
    Bach JR, Fuller C, Gupta A, Hampapur A, Horowitz B, Humphrey R, et al: Virage image search engine: an open framework for image management. In: Sethi IK, Jain RC Eds. Proceedings of SPIE 2670, 1, 1996, pp 76–87Google Scholar
  25. 25.
    Pentland A, Picard RW, Sclaroff S: Photobook: Content-based manipulation of image databases. Int J Comput Vis 18:233–54, 1996CrossRefGoogle Scholar
  26. 26.
    Chechik G, Sharma V, Shalit U, Bengio S: Large scale online learning of image similarity through ranking. J Mach Learn Res 11:1109–35, 2010Google Scholar
  27. 27.
    Duncan JS, Ayache N: Medical image analysis: Progress over two decades and the challenges ahead. IEEE Trans Pattern Anal Mach Intell 22(1):85–106, 2000CrossRefGoogle Scholar
  28. 28.
    Townsend DW, Beyer T: A combined PET/CT scanner: The path to true image fusion. Br J Radiol 75(Supplement 9):S24–30, 2002PubMedGoogle Scholar
  29. 29.
    Townsend DW, Beyer T, Blodgett TM: PET/CT scanners: A hardware approach to image fusion. Semin Nucl Med 33(3):193–204, 2003PubMedCrossRefGoogle Scholar
  30. 30.
    Judenhofer MS, Catana C, Swann BK, Siegel SB, Jung WI, Nutt RE, et al: PET/MR images acquired with a compact MR-compatible PET detector in a 7-T magnet. Radiology 244(3):807–14, 2007PubMedCrossRefGoogle Scholar
  31. 31.
    Shyu CR, Brodley CE, Kak AC, Kosaka A, Aisen AM, Broderick LS: ASSERT: A physician-in-the-loop content-based retrieval system for HRCT image databases. Comp Vision Image Underst 75(1–2):111–32, 1999CrossRefGoogle Scholar
  32. 32.
    Aisen AM, Broderick LS, Winer-Muram H, Brodley CE, Kak AC, Pavlopoulou C, et al: Automated storage and retrieval of thin-section CT images to assist diagnosis: System description and preliminary assessment. Radiology 228(1):265–70, 2003PubMedCrossRefGoogle Scholar
  33. 33.
    Napel SA, Beaulieu CF, Rodriguez C, Cui J, Xu J, Gupta A, et al: Automated retrieval of CT images of liver lesions on the basis of image similarity: Method and preliminary results. Radiology 256(1):243–52, 2010PubMedCrossRefGoogle Scholar
  34. 34.
    Müller H, Rosset A, Garcia A, Vallée JP, Geissbuhler A: Benefits of content-based visual data access in radiology. Radiographics 25(3):849–58, 2005PubMedCrossRefGoogle Scholar
  35. 35.
    Keysers D, Dahmen J, Ney H, Wein BB, Lehmann TM: Statistical framework for model-based image retrieval in medical applications. J Electron Imaging 12(1):59–68, 2003CrossRefGoogle Scholar
  36. 36.
    Güld MO, Thies C, Fischer B, Lehmann TM: A generic concept for the implementation of medical image retrieval systems. Int J Med Inform 76(2–3):252–9, 2007PubMedCrossRefGoogle Scholar
  37. 37.
    Iakovidis D, Pelekis N, Kotsifakos E, Kopanakis I, Karanikas H, Theodoridis Y: A pattern similarity scheme for medical image retrieval. IEEE Trans Inf Technol Biomed 13(4):442–50, 2009PubMedCrossRefGoogle Scholar
  38. 38.
    Antani S, Lee D, Long LR, Thoma GR: Evaluation of shape similarity measurement methods for spine X-ray images. J Vis Commun Image Represent 15(3):285–302, 2004CrossRefGoogle Scholar
  39. 39.
    Antani S, Long LR, Thoma GR, Lee DJ: Evaluation of shape indexing methods for content-based retrieval of X-ray images. In: Yeung MM, Lienhart RW, Li CS Eds. Proceedings of SPIE 5021, 2003, pp 405–416Google Scholar
  40. 40.
    Lee DJ, Antani S, Long LR: Similarity measurement using polygon curve representation and Fourier descriptors for shape-based vertebral image retrieval. In: Sonka M, Fitzpatrick JM Eds. Proceedings of SPIE 5032, 2003, pp 1283–1291Google Scholar
  41. 41.
    Xu X, Lee DJ, Antani S, Long L: A spine X-ray image retrieval system using partial shape matching. IEEE Trans Inf Technol Biomed 12(1):100–8, 2008PubMedCrossRefGoogle Scholar
  42. 42.
    Hsu W, Antani S, Long LR, Neve L, Thoma GR: SPIRS: A web-based image retrieval system for large biomedical databases. Int J Med Inform 78(Supplement 1):S13–24, 2009PubMedCrossRefGoogle Scholar
  43. 43.
    Lee DJ, Antani S, Chang Y, Gledhill K, Long LR, Christensen P: CBIR of spine X-ray images on inter-vertebral disc space and shape profiles using feature ranking and voting consensus. Data Knowl Eng 68(12):1359–69, 2009CrossRefGoogle Scholar
  44. 44.
    Qian X, Tagare HD, Fulbright RK, Long R, Antani S: Optimal embedding for shape indexing in medical image databases. Med Image Anal 14(3):243–54, 2010PubMedCrossRefGoogle Scholar
  45. 45.
    Xue Z, Antani S, Long LR, Jeronimo J, Thoma GR: Investigating CBIR techniques for cervicographic images. In: Proceedings of the Annual Symposium of American Medical Information Association, 2007, pp 826–830Google Scholar
  46. 46.
    Xue Z, Antani S, Long L, Thoma G: A system for searching uterine cervix images by visual attributes. In: IEEE International Symposium on Computer-Based Medical Systems, 2009, pp 1–5Google Scholar
  47. 47.
    Korn P, Sidiropoulos N, Faloutsos C, Siegel E, Protopapas Z: Fast and effective retrieval of medical tumor shapes. IEEE Trans Knowl Data Eng 10(6):889–904, 1998CrossRefGoogle Scholar
  48. 48.
    Yang L, Jin R, Mummert L, Sukthankar R, Goode A, Zheng B, et al: A boosting framework for visuality-preserving distance metric learning and its application to medical image retrieval. IEEE Trans Pattern Anal Mach Intell 32(1):30–44, 2010PubMedCrossRefGoogle Scholar
  49. 49.
    Quellec G, Lamard M, Cazuguel G, Cochener B, Roux C: Wavelet optimization for content-based image retrieval in medical databases. Med Image Anal 14(2):227–41, 2010PubMedCrossRefGoogle Scholar
  50. 50.
    Quellec G, Lamard M, Bekri L, Cazuguel G, Roux C, Cochener B: Medical case retrieval from a committee of decision trees. IEEE Trans Inf Technol Biomed 14(5):1227–35, 2010PubMedCrossRefGoogle Scholar
  51. 51.
    Quellec G, Lamard M, Cazuguel G, Roux C, Cochener B: Case retrieval in medical databases by fusing heterogeneous information. IEEE Trans Med Imaging 30(1):108–18, 2011PubMedCrossRefGoogle Scholar
  52. 52.
    Dy JG, Brodley CE, Kak A, Broderick LS, Aisen AM: Unsupervised feature selection applied to content-based retrieval of lung images. IEEE Trans Pattern Anal Mach Intell 25(3):373–8, 2003CrossRefGoogle Scholar
  53. 53.
    Unay D, Ekin A, Jasinschi R: Local structure-based region-of-interest retrieval in brain MR images. IEEE Trans Inf Technol Biomed 14(4):897–903, 2010PubMedCrossRefGoogle Scholar
  54. 54.
    Petrakis EG: Design and evaluation of spatial similarity approaches for image retrieval. Image Vis Comput 20(1):59–76, 2002CrossRefGoogle Scholar
  55. 55.
    Alajlan N, Kamel M, Freeman G: Geometry-based image retrieval in binary image databases. IEEE Trans Pattern Anal Mach Intell 30(6):1003–13, 2008PubMedCrossRefGoogle Scholar
  56. 56.
    Cai W, Feng D, Fulton R: Content-based retrieval of dynamic pet functional images. IEEE Trans Inf Technol Biomed 4(2):152–8, 2000PubMedCrossRefGoogle Scholar
  57. 57.
    Kim J, Cai W, Feng D, Wu H: A new way for multidimensional medical data management: Volume of interest (VOI)-based retrieval of medical images with visual and functional features. IEEE Trans Inf Technol Biomed 10(3):598–607, 2006PubMedCrossRefGoogle Scholar
  58. 58.
    Kim J, Constantinescu L, Cai W, Feng DD: Content-based dual-modality biomedical data retrieval using co-aligned functional and anatomical features. In: Proceedings of the MICCAI Workshop on Content-Based Image Retrieval for Biomedical Image Archives: Achievements, Problems and Prospects, 2007, pp 45–52Google Scholar
  59. 59.
    Song Y, Cai W, Eberl S, Fulham M, Feng D: A content-based image retrieval framework for multi-modality lung images. In: IEEE International Symposium on Computer-Based Medical Systems, 2010, pp 285–290Google Scholar
  60. 60.
    Song Y, Cai W, Eberl S, Fulham M, Feng D: Structure-adaptive feature extraction and representation for multi-modality lung images retrieval. In: International Conference on Digital Image Computing: Techniques and Applications, 2010, pp 152–157Google Scholar
  61. 61.
    Song Y, Cai W, Eberl S, Fulham M, Feng D: Thoracic image case retrieval with spatial and contextual information. In: 2011 I.E. International Symposium on Biomedical Imaging: From Nano to Macro, 2011, pp 1885–1888Google Scholar
  62. 62.
    Song Y, Cai W, Eberl S, Fulham M, Feng D: Thoracic image matching with appearance and spatial distribution. In: International Conference of the IEEE Engineering in Medicine and Biology Society, 2011, pp 4469–4472Google Scholar
  63. 63.
    Song Y, Cai W, Feng D: Hierarchical spatial matching for medical image retrieval. In: Proceedings of the International ACM Multimedia Workshop on Medical Multimedia Analysis and Retrieval, 2011, pp 1–6Google Scholar
  64. 64.
    Cai W, Song Y, Feng DD: Regression and classification based distance metric learning for medical image retrieval. In: IEEE International Symposium on Biomedical Imaging, 2012, pp 1775–1778Google Scholar
  65. 65.
    Kumar A, Kim J, Cai W, Eberl S, Feng D: A graph-based approach to the retrieval of dual-modality biomedical images using spatial relationships. In: International Conference of the IEEE Engineering in Medicine and Biology Society, 2008, pp 390–393Google Scholar
  66. 66.
    Kumar A, Kim J, Wen L, Feng D: A graph-based approach to the retrieval of volumetric PET-CT lung images. In: Proceedings of the 34th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, 2012, pp 5408–5411Google Scholar
  67. 67.
    Kumar A, Kim J, Fulham M, Feng D: Graph-based retrieval of multi-modality medical images: A comparison of representations using simulated images. In: IEEE International Symposium on Computer-Based Medical Systems, 2012, pp 1–6Google Scholar
  68. 68.
    Kumar A, Haraguchi D, Kim J, Wen L, Eberl S, Fulham M, et al: A query and visualisation interface for a PET-CT image retrieval system. Int J Comput Assist Radiol Surg 6(Supplement 1):69, 2011Google Scholar
  69. 69.
    Kumar A, Kim J, Bi L, Feng D: An image retrieval interface for volumetric multi-modal medical data: Application to PET-CT content-based image retrieval. Int J Comput Assist Radiol Surg 7(Supplement 1):475–7, 2012Google Scholar
  70. 70.
    Radhouani S, Lim J, Chevallet JP, Falquet G: Combining textual and visual ontologies to solve medical multimodal queries. In: IEEE International Conference on Multimedia and Expo, 2006, pp 1853–1856Google Scholar
  71. 71.
    Lacoste C, Lim JH, Chevallet JP, Le D: Medical-image retrieval based on knowledge-assisted text and image indexing. IEEE Trans Circ Syst Video Technol 17(7):889–900, 2007CrossRefGoogle Scholar
  72. 72.
    Gobeill J, Müller H, Ruch P: Translation by text categorisation: Medical image retrieval in ImageCLEFmed 2006. In: Peters C, Clough P, Gey F, Karlgren J, Magnini B, Oard D, et al. Eds. Evaluation of Multilingual and Multi-modal Information Retrieval, Vol. 4730 of Lecture Notes in Computer Science, 2007, pp 706–710Google Scholar
  73. 73.
    Villena-Román J, Lana-Serrano S, González-Cristóbal J: MIRACLE at ImageCLEFmed 2007: Merging textual and visual strategies to improve medical image retrieval. In: Peters C, Jijkoun V, Mandl T, Müller H, Oard D, Peñas A, et al. Eds. Advances in Multilingual and Multimodal Information Retrieval, Vol. 5152 of Lecture Notes in Computer Science, 2008, pp 593–596Google Scholar
  74. 74.
    Caicedo JC, Moreno JG, Niño EA, González FA: Combining visual features and text data for medical image retrieval using latent semantic kernels. In: Proceedings of the International Conference on Multimedia Information Retrieval, ACM, 2010, pp 359–366Google Scholar
  75. 75.
    Rahman M, Antani S, Long R, Demner-Fushman D, Thoma G: Multi-modal query expansion based on local analysis for medical image retrieval. In: Caputo B, Müller H, Syeda-Mahmood T, Duncan J, Wang F, Kalpathy-Cramer J Eds. Medical Content-Based Retrieval for Clinical Decision Support, Vol. 5853 of Lecture Notes in Computer Science, 2010, pp 110–119Google Scholar
  76. 76.
    Müller H, Kalpathy-Cramer J, Charles E. Kahn J, Hersh W: Comparing the quality of accessing medical literature using content-based visual and textual information retrieval. In: Siddiqui KM, Liu BJ Eds. Proceedings of SPIE 7264, 2009, pp 726405:1–726405:11Google Scholar
  77. 77.
    Chu WW, Ieong IT, Taira RK: A semantic modeling approach for image retrieval by content. VLDB J—Int J Very Large Data Bases 3(4):445–77, 1994CrossRefGoogle Scholar
  78. 78.
    Chu W, Hsu CC, Cardenas A, Taira R: Knowledge-based image retrieval with spatial and temporal constructs. IEEE Trans Knowl Data Eng 10(6):872–88, 1998CrossRefGoogle Scholar
  79. 79.
    Névéol A, Deserno TM, Darmoni SJ, Güld MO, Aronson AR: Natural language processing versus content-based image analysis for medical document retrieval. J Am Soc Inf Sci Technol 60(1):123–34, 2009CrossRefGoogle Scholar
  80. 80.
    Langlotz CP: RadLex: A new method for indexing online educational materials. Radiographics 26(6):1595–7, 2006PubMedCrossRefGoogle Scholar
  81. 81.
    Müller H, Deselaers T, Deserno T, Kalpathy-Cramer J, Kim E, Hersh W: Overview of the ImageCLEFmed 2007 medical retrieval and medical annotation tasks. In: Peters C, Jijkoun V, Mandl T, Müller H, Oard D, Peñas A, et al. Eds. Advances in Multilingual and Multimodal Information Retrieval, Vol. 5152 of Lecture Notes in Computer Science, 2008, pp 472–491Google Scholar
  82. 82.
    Müller H, Kalpathy-Cramer J, Kahn C, Hatt W, Bedrick S, Hersh W: Overview of the ImageCLEFmed 2008 medical image retrieval task. In: Peters C, Deselaers T, Ferro N, Gonzalo J, Jones G, Kurimo M, et al. Eds. Evaluating Systems for Multilingual and Multimodal Information Access, Vol. 5706 of Lecture Notes in Computer Science, 2009, pp 512–522Google Scholar
  83. 83.
    Müller H, Kalpathy-Cramer J, Eggel I, Bedrick S, Radhouani S, Bakke B, et al: Overview of the CLEF 2009 medical image retrieval track. In: Peters C, Caputo B, Gonzalo J, Jones G, Kalpathy-Cramer J, Müller H, et al. Eds. Multilingual Information Access Evaluation II. Multimedia Experiments, Vol. 6242 of Lecture Notes in Computer Science, 2010, pp 72–84Google Scholar
  84. 84.
    Liu J, Hu Y, Li M, Ma S, ying Ma W: Medical image annotation and retrieval using visual features. In: Evaluation of Multilingual and Multi-modal Information Retrieval, Vol. 4730 of Lecture Notes in Computer Science, 2007, pp 678–685Google Scholar
  85. 85.
    Rahman MM, Desai BC, Bhattacharya P: Medical image retrieval with probabilistic multi-class support vector machine classifiers and adaptive similarity fusion. Comput Med Imaging Graph 32(2):95–108, 2008PubMedCrossRefGoogle Scholar
  86. 86.
    Akakin H, Gurcan M: Content-based microscopic image retrieval system for multi-image queries. IEEE Trans Inf Technol Biomed 16(4):758–69, 2012PubMedCrossRefGoogle Scholar
  87. 87.
    Allampalli-Nagaraj G, Bichindaritz I: Automatic semantic indexing of medical images using a web ontology language for case-based image retrieval. Eng Appl Artif Intell 22(1):18–25, 2009CrossRefGoogle Scholar
  88. 88.
    Zhou X, Stern R, Müller H: Case-based fracture image retrieval. Int J Comput Assist Radiol Surg 7:401–11, 2012PubMedCrossRefGoogle Scholar
  89. 89.
    Huang SC, Phelps ME, Hoffman EJ, Sideris K, Selin CJ, Kuhl DE: Noninvasive determination of local cerebral metabolic rate of glucose in man. Am J Physiol—Endocrinol Metab 238(1):E69–82, 1980Google Scholar
  90. 90.
    Chang E, Goh K, Sychay G, Wu G: CBSA: Content-based soft annotation for multimodal image retrieval using Bayes point machines. IEEE Trans Circ Syst Video Technol 13(1):26–38, 2003CrossRefGoogle Scholar
  91. 91.
    Hersh W, Müller H, Kalpathy-Cramer J: The ImageCLEFmed medical image retrieval task test collection. J Digit Imaging 22:648–55, 2009PubMedCrossRefGoogle Scholar
  92. 92.
    Tian G, Fu H, Feng D: Automatic medical image categorization and annotation using LBP and MPEG-7 edge histograms. In: International Conference on Information Technology and Applications in Biomedicine, 2008, pp 51–53Google Scholar
  93. 93.
    Spitzer V, Ackerman MJ, Scherzinger AL, Whitlock D: The visible human male: A technical report. J Am Med Inform Assoc 3(2):118–30, 1996PubMedCrossRefGoogle Scholar
  94. 94.
    Lowe DG: Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60:91–110, 2004CrossRefGoogle Scholar
  95. 95.
    Czernin J, Dahlbom M, Ratib O, Schiepers C: Atlas of PET/CT Imaging in Oncology. Springer, Berlin, 2004CrossRefGoogle Scholar
  96. 96.
    Goerres GW, von Schulthess GK, Steinert HC: Why most PET of lung and head-and-neck cancer will be PET/CT. J Nucl Med 45(Supplement 1):66S–71S, 2004PubMedGoogle Scholar
  97. 97.
    Fu KS: A step towards unification of syntactic and statistical pattern recognition. IEEE Trans Pattern Anal Mach Intell 8(3):398–404, 1986PubMedCrossRefGoogle Scholar
  98. 98.
    Jing Y, Rowley H, Rosenberg C, Wang J, Zhao M, Covell M: Google image swirl, a large-scale content-based image browsing system. In: IEEE International Conference on Multimedia and Expo, 2010, p 267Google Scholar
  99. 99.
    Tory M, Moller T: Human factors in visualization research. IEEE Trans Vis Comput Graph 10(1):72–84, 2004PubMedCrossRefGoogle Scholar
  100. 100.
    Wilson ML: Search user interface design. Synth Lect Inf Concepts Retr Serv 3(3):1–143, 2011Google Scholar
  101. 101.
    Etzold J, Brousseau A, Grimm P, Steiner T: Context-aware querying for multimodal search engines. In: Schoeffmann K, Merialdo B, Hauptmann A, Ngo CW, Andreopoulos Y, Breiteneder C Eds. Advances in Multimedia Modeling, Vol. 7131 of Lecture Notes in Computer Science. Berlin: Springer, 2012, pp 728–739Google Scholar
  102. 102.
    Ekin A, Jasinschi R, van der Grond J, Van Buchem M: Improving information quality of MR brain images by fully automatic and robust image analysis methods. J Soc Inf Disp 15(6):367–76, 2007CrossRefGoogle Scholar
  103. 103.
    van Rikxoort EM, Isgum I, Arzhaeva Y, Staring M, Klein S, Viergever MA, et al: Adaptive local multi-atlas segmentation: Application to the heart and the caudate nucleus. Medical Image Analysis 14(1):39–49, 2010PubMedCrossRefGoogle Scholar
  104. 104.
    Jones KN, Woode DE, Panizzi K, Anderson PG: PEIR digital library: Online resources and authoring system. In: Proceedings of the American Medical Informatics Association Symposium, 2001, p 1075Google Scholar
  105. 105.
    Long LR, Antani SK, Thoma GR: Image informatics at a national research center. Comput Med Imaging Graph 29(2–3):171–93, 2005PubMedCrossRefGoogle Scholar
  106. 106.
    The Cancer Imaging Archive. 2011.
  107. 107.
    Armato III, SG, McLennan G, McNitt-Gray MF, Meyer CR, Yankelevitz D, Aberle DR, et al: Lung image database consortium: Developing a resource for the medical imaging research community. Radiology 232(3):739–48, 2004PubMedCrossRefGoogle Scholar
  108. 108.
    Langs G, Müller H, Menze BH, Hanbury A: VISCERAL: Towards large data in medical imaging—Challenges and directions. In: MICCAI Workshop on Medical Content-Based Retrieval for Clinical Decision Support 2012, Vol. 7723 of Springer LNCS, 2013, pp 92–98Google Scholar
  109. 109.
    Detterbeck FC, Boffa DJ, Tanoue LT: The new lung cancer staging system. Chest 136(1):260–71, 2009PubMedCrossRefGoogle Scholar
  110. 110.
    Edge SB, Byrd DR, Compton CC, Frtiz AG, Greene FL, Trotti A Eds. AJCC Cancer Staging Manual. New York: Springer, 2010Google Scholar
  111. 111.
    Bodenreider O: The unified medical language system (UMLS): Integrating biomedical terminology. Nucleic Acids Res 32(Supplement 1):D267–70, 2004PubMedCrossRefGoogle Scholar
  112. 112.
    Depeursinge A, Vargas A, Gaillard F, Platon A, Geissbuhler A, Poletti PA, et al: Case-based lung image categorization and retrieval for interstitial lung diseases: Clinical workflows. Int J Comput Assist Radiol Surg 7(1):97–110, 2012PubMedCrossRefGoogle Scholar
  113. 113.
    Antani S, Xue Z, Long LR, Bennett D, Ward S, Thoma GR: Is there a need for biomedical CBIR systems in clinical practice? Outcomes from a usability study. In: Proceedings of SPIE 7967, 2011, pp 796708-1–796708-7Google Scholar

Copyright information

© Society for Imaging Informatics in Medicine 2013

Authors and Affiliations

  • Ashnil Kumar
    • 1
    Email author
  • Jinman Kim
    • 1
  • Weidong Cai
    • 1
  • Michael Fulham
    • 1
    • 2
    • 3
  • Dagan Feng
    • 1
    • 4
  1. 1.Biomedical and Multimedia Information Technology (BMIT) Research Group, School of Information TechnologiesUniversity of SydneySydneyAustralia
  2. 2.Department of Molecular ImagingRoyal Prince Alfred HospitalSydneyAustralia
  3. 3.Sydney Medical SchoolUniversity of SydneySydneyAustralia
  4. 4.Med-X Research InstituteShanghai Jiao Tong UniversityShanghaiChina

Personalised recommendations