Skip to main content

Advertisement

SpringerLink
Log in

Automatic medical image annotation and keyword-based image retrieval using relevance feedback

  • Published:
Journal of Digital Imaging Aims and scope Submit manuscript

Abstract

This paper presents novel multiple keywords annotation for medical images, keyword-based medical image retrieval, and relevance feedback method for image retrieval for enhancing image retrieval performance. For semantic keyword annotation, this study proposes a novel medical image classification method combining local wavelet-based center symmetric–local binary patterns with random forests. For keyword-based image retrieval, our retrieval system use the confidence score that is assigned to each annotated keyword by combining probabilities of random forests with predefined body relation graph. To overcome the limitation of keyword-based image retrieval, we combine our image retrieval system with relevance feedback mechanism based on visual feature and pattern classifier. Compared with other annotation and relevance feedback algorithms, the proposed method shows both improved annotation performance and accurate retrieval results.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Müller H, Ruch P, Geissbuhler A: Enriching content-based image retrieval with multi-lingual search terms. Swiss Med Inform 54:6–11, 2005

    Google Scholar 

  2. Rahman M, Desai BC, Bhattacharya P: Medical image retrieval with probabilistic multi-class support vector machine classifiers and adaptive similarity fusion. Comput Med Imaging Graph 32:95–108, 2008

    Article  PubMed  Google Scholar 

  3. Julio VR, José GC, José GM, José MF: MIRACLE’s naïve approach to medical images annotation. Proceeding of the Workshop on Cross Language Evaluation Forum: 1–9, 2005.

  4. Setia L, Teynor A, Halawani A, Burkhardt H: Grayscale medical image annotation using local relational features. Pattern Recognit Lett 29:2039–2045, 2008

    Article  Google Scholar 

  5. Mueen A, Zainuddin, Baba MS: Automatic multilevel medical image annotation and retrieval. J Digit Imaging 21:290–295, 2008

    Article  PubMed  CAS  Google Scholar 

  6. Amaral IF, Coelho F, Costa JF, Cardoso JS: Hierarchical medical image annotation using SVM-based approaches. Proceedings of the 10th IEEE International Conference on Information Technology and Applications in Biomedicine: 1–5, 2010

  7. Xu X, Lee DJ, Antani SK, Long LR, Archibald JK: Using relevance feedback with short-term memory for content-based spine X-ray image retrieval. Neurocomputing 72:2259–2269, 2009

    Article  Google Scholar 

  8. Tong H, He J, Li M, Ma WY, Zhang HJ, Zhang C: Manifold-ranking based keyword propagation for image retrieval. Comput Med Imaging Graph 32:95–108, 2008

    Article  Google Scholar 

  9. Bao Y, Zhang Y, Wang D, Shi J: Soft SVM and novel sampling rule based relevance feedback for medical image retrieval. Proceeding of Fourth International Conference on Computer Sciences and Convergence Information Technology: 483–488, 2009.

  10. Liu H, Zhang CM, Han H: Medical image retrieval based on semi-supervised learning. J Adv Mater Res 108:201–206, 2010

    Article  Google Scholar 

  11. Oh JH, Naqa IE: Adaptive learning for relevance feedback: application to digital mammography. Med Phys 37:4432–4445, 2010

    Article  PubMed  Google Scholar 

  12. Wei CH, Li CT: Learning pathological characteristics from user’s relevance feedback for content-based mammogram retrieval. Proceedings of eighth IEEE International Symposium on Multimedia pp: 738–741, 2006.

  13. Ko BC, Kim SH, Nam JY: X-ray image classification using random forests with local wavelet-based CS-local binary patterns. J Digit Imaging, 2011. doi:10.1007/s10278-011-9380-3O

  14. MacArthur SD, Brodley CE, Shyu CR: Relevance feedback decision trees in content-based image retrieval. Proceedings of the IEEE Workshop on Content-Based Access of Image and Video Libraries: 68–73, 2000.

  15. Lakdashti A, Ajorloo H: Content-based image retrieval based on relevance feedback and reinforcement learning for medical images. ETRI J 33:240–250, 2011

    Article  Google Scholar 

  16. ImageCLEF. Available at http://www.imageclef.org/2007/photo. Accessed 25 April 2011.

  17. Tommasi T, Orabona F, Caputo B: An SVM confidence-based approach to medical image annotation. Lect Notes Comp Sci 5706:696–703, 2009

    Article  Google Scholar 

  18. Ojala T, Pietikainen M, Maenpaa T: Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans Pattern Anal Mach Intell 24:971–987, 2002

    Article  Google Scholar 

  19. Heikkilä M, Pietikäinen M, Schmid C: Description of interest regions with local binary patterns. Pattern Recognit 42:425–436, 2009

    Article  Google Scholar 

  20. Breiman L: Random forests. Mach Leaning 45:5–32, 2001

    Article  Google Scholar 

  21. Heidemann G: Unsupervised image categorization. Image Vision Comput 23:861–876, 2005

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Shindang-Dong Dalseo-Gu, Dept. Of Computer Engineering, Keimyung University, Daegu, 704-701, South Korea

    Byoung Chul Ko, JiHyeon Lee & Jae-Yeal Nam

Authors
  1. Byoung Chul Ko
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. JiHyeon Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jae-Yeal Nam
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Byoung Chul Ko.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Ko, B.C., Lee, J. & Nam, JY. Automatic medical image annotation and keyword-based image retrieval using relevance feedback. J Digit Imaging 25, 454–465 (2012). https://doi.org/10.1007/s10278-011-9443-5

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10278-011-9443-5

Keywords

Search

Navigation