Skip to main content
SpringerLink
Log in

Hybrid query refinement based approach for enhanced biomedical image retrieval

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

With the advent of technology, abundant electronic biomedical data is available and extracting relevant information from huge data is a fundamental need. In medical image retrieval, majority of the information is extracted either through content-based or context-based retrieval. The semantic information in images is not considered in previous research. In this work, the proposed hybrid query refinement framework for medical image retrieval shows promising results in improving the accuracy of search results compared to existing approaches. By incorporating both low-level features and high-level semantic terms based on the MeSH hierarchy, the framework addresses some existing limitations of content-based image retrieval (CBIR) approach based on relevance feedback and semantic knowledge. The framework uses the OpenI search engine to retrieve a pool of images based on user-generated queries related to a medical disease. Then, the images are processed using Scikit Image library to extract features like size, orientation, shape, and color. Edge feature extraction and region-based segmentation are performed to identify important features and analyze them more closely. The results show that the proposed approach achieves improved precision@10 and MAP for OpenI search engine compared to baseline CBIR. The number of retrieved images is also significantly reduced, indicating the relevance of the retrieved images. The proposed approach provides a more accurate and efficient medical image retrieval system that can help medical practitioners conclude diagnoses faster. Future research can focus on expanding the framework to incorporate more advanced machine learning algorithms for feature extraction and classification.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Pseudocode 1
Pseudocode 2
Pseudocode 3
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Data availability

The data that support the findings of this study are available from the corresponding author [Dr. Lokendra Singh Umrao], upon reasonable request.

References

  1. Singhal A (2001) Modern information retrieval: a brief overview. IEEE Data Eng 24(4):35–43

    Google Scholar 

  2. Manning C, Raghavan P, Schütze H (2010) Introduction to information retrieval. Nat Lang Eng 16(1):100–103

    Google Scholar 

  3. Lalmas M (1998) Logical models in information retrieval: introduction and overview. Inf Process Manag 34(1):19–33

    Article  Google Scholar 

  4. Mitra M, Chaudhuri BB (2000) Information retrieval from documents: a survey. Inf Retr Boston 2(2–3):141–163

    Article  Google Scholar 

  5. Hliaoutakis A, Varelas G, Voutsakis E, Petrakis EGM, Milios E (2006) Information retrieval by semantic similarity. Int J Semant Web Inf Syst 2(3):55–73

    Article  Google Scholar 

  6. Broder A (2002) A taxonomy of web search. ACM SIGIR Forum 36(2):3

    Article  Google Scholar 

  7. Rui Y, Huang TS, Chang S (1999) Image retrieval: past, present, and future. J Vis Commun Image Represent 10(1):1–23

    Google Scholar 

  8. Rui Y, Huang TS, Chang S (1999) Image retrieval: current techniques, promising directions, and open issues. J Vis Commun Image Represent 10(1):39–62

    Article  Google Scholar 

  9. Deselaers T, Keysers D, Ney H (2008) Features for image retrieval: an experimental comparison. Inform Retr vol 11(2):77–107

    Article  Google Scholar 

  10. Choe J, Hwang HJ, Seo JB, Lee SM, Yun J, Kim MJ, Jeong J, Lee Y, Jin K, Park R, Kim J (2022) Content-based image retrieval by using deep learning for interstitial lung disease diagnosis with chest CT. Radiology 302(1):187–197

    Article  Google Scholar 

  11. Reddy KS, Sreedhar K (2016) Image retrieval techniques: a survey. Int J Electron Commun Eng 9(1):19–27

    Google Scholar 

  12. Alsmadi MK (2020) Content-based image retrieval using color, shape and texture descriptors and features. Arab J Sci Eng 45(4):3317–3330

    Article  Google Scholar 

  13. Irtaza A, Adnan SM, Ahmed KT, Jaffar A, Khan A, Javed A, Mahmood MT (2018) An ensemble based evolutionary approach to the class imbalance problem with applications in CBIR. Appl Sci 8(4):495

    Article  Google Scholar 

  14. Alsmadi MK (2017) An efficient similarity measure for content-based image retrieval using memetic algorithm. Egypt J Basic Appl Sci 4:112–122

    Google Scholar 

  15. Alsmadi MK (2018) Query-sensitive similarity measure for content-based image retrieval using meta-heuristic algorithm. J King Saud Univ Comput Inform Sci 30:373–381

    Google Scholar 

  16. Zin NAM, Yusof R, Lashari SA, Mustapha A, Senan N, Ibrahim R (2018) Content-based image retrieval in medical domain: a review. In J Phys: Conf Ser 1019(1):012044

    Google Scholar 

  17. Choras RS (2007) Image feature extraction techniques and their applications for CBIR and biometrics systems. Int J Biol Biomed Eng 1(1):6–16

    Google Scholar 

  18. Yang CC (2004) Content-based image retrieval: a comparison between query by example and image browsing map approaches. J Inform Sci 30(3):254–267

    Article  Google Scholar 

  19. Ballerini L, Li X, Fisher RB, Rees J (2010) A query-by-example content-based image retrieval system of non-melanoma skin lesions. In: Caputo B, Muller H, SyedaMahmood T, Duncan JS, Wang F, Kalpathy Cramer  J (eds) Medical content-based retrieval for clinical decision support. Springer-Verlag GmbH, pp 31–38. https://doi.org/10.1007/978-3-642-11769-5_3

  20. Jaworska T (2016) Query techniques for CBIR. In: Andreasen T et al (eds) Flexible query answering systems 2015. Advances in intelligent systems and computing, vol 400. Springer, Cham. https://doi.org/10.1007/978-3-319-26154-6_31

  21. Robyn Speer and Catherine Havasi (2012) Representing general relational knowledge in conceptnet 5. In: Proceedings of the eighth international conference on language resources and evaluation (LREC'12). European Language Resources Association (ELRA), Istanbul, Turkey, pp 3679–3686

  22. Mihalcea R, Corley C, Strapparava C (2006) Corpus-based and knowledge-based measures of text semantic similarity. Proc 21st Natl Conf Artif Intell 1:775–780

    Google Scholar 

  23. Akinribido CT, Afolabi BS, Akhigbe BI, Udo IJ (2011) A fuzzy-ontology based information retrieval system for relevant feedback. Int J Comput Sci 8(1):382–389

    Google Scholar 

  24. Aversano L, Bruno M, Canfora G, Di Penta M, Distante D (2006) Using Concept lattices to support service selection. Int J Web Serv Res 3(4):32–51

    Article  Google Scholar 

  25. Abulaish M, Dey L (2006) Information extraction and imprecise query answering from web documents. Gene 4:407–429

    Google Scholar 

  26. Sahni L, Sehgal A, Kochar S, Ahmad F, Ahmad T (2014) A novel approach to find semantic similarity measure between words. Comput Bus Intell (ISCBI), 2nd Int. Symp., pp. 89–92, 2014

  27. Rajasurya S, Muralidharan T, Devi S, Swamynathan S (2012) Semantic information retrieval using ontology in university domain. arXiv preprint arXiv:1207.5745

  28. Saritha RR, Paul V, Kumar PG (2019) Content based image retrieval using deep learning process. Cluster Comput 22(2):4187–4200

    Article  Google Scholar 

  29. Latif A, Rasheed A, Sajid U, Ahmed J, Ali N, Ratyal NI., ... Khalil T (2019) Content-based image retrieval and feature extraction: a comprehensive review. Mathematical problems in engineering 2019

  30. Goyenka R, Kshirsagar DB (2017) Working of CBIR system using relevance feedback and navigation patterns. Feedback 6(7)

  31. Niblack CW, Barber R, Equitz W, Flickner MD, Glasman EH, Petkovic D, ...  Taubin G (1993) QBIC project: querying images by content, using color, texture, and shape. In: Storage and retrieval for image and video databases, vol. 1908. Spie, pp 173–187

  32. Banerjee I, Kurtz C, Devorah AE, Bao D, Rubin DL, Beaulieu CF (2018) Relevance feedback for enhancing content-based image retrieval and automatic prediction of semantic image features: application to bone Tumor radiographs. J Biomed Inform 84:123–135

    Article  Google Scholar 

  33. Fushman DD, Antani S, Simpson M, Thoma GR (2012) Design and development of a multimodal biomedical information retrieval system. J Comput Sci Eng Vol 6(2):168–177

    Article  Google Scholar 

  34. Kalpathy CJ, Herrera AGSD, Fushman DD, Antani S, Bedrick S, Müller H (2015) Evaluating performance of biomedical image retrieval systems—An overview of the medical image retrieval task at ImageCLEF 2004–2013. Comput Med Imaging Graph 39:55–61

    Article  Google Scholar 

  35. Ahmed A (2020) Implementing relevance feedback for content-based medical image retrieval. IEEE Access 8:79969–79976

    Article  Google Scholar 

  36. Ghosh P, Antani S, Long LR, Thoma GR (2011) Review of medical image retrieval systems and future directions. In: International Symposium on Computer-Based Medical Systems (CBMS). IEEE, pp 1–6

  37. Zhai J, Li M, Li J (2012) Semantic information retrieval based on RDF and fuzzy ontology for university scientific research management. Adv Intell Soft Comput 137(AISC):661–668

    Article  Google Scholar 

  38. Bhogal J, Macfarlane A, Smith P (2007) A review of ontology-based query expansion. Inf Process Manag 43(4):866–886

    Article  Google Scholar 

  39. Deserno TM, Güld MO, Plodowski B, Spitzer K, Wein BB, Schubert H, Ney H, Seidl T (2008) Extended query refinement for medical image retrieval. J Digit Imaging 21(3):280–289

    Article  Google Scholar 

  40. Malik S, Shoaib U, El-Sayed H, Khan MA (2020) Query expansion framework leveraging clinical diagnosis information ontology. In: International Conference on Innovations in Information Technology. IEEE, pp 18–23

  41. Nastar C, Mitschke M, Meilhac C (1998) Efficient query refinement for image retrieval. In: Proceedings Computer Society Conference on Computer Vision and Pattern Recognition. IEEE, pp 547–552

  42. Kammoun H, Gabsi I, Amous I (2022) Mesh-based semantic indexing approach to enhance biomedical information retrieval. Comput J 65(3):516–536

    Article  Google Scholar 

  43. Ayadi H, Khemakhem MT, Daoud M, Huang JX, Jemaa MB (2018) MF-Re-Rank: a modality feature-based re-ranking model for medical image retrieval. J Assoc Inform Sci Technol 69:1095–1108

    Article  Google Scholar 

  44. Mandikal V, Anantharaman A, Suhas BS, Kamath S (2018) Multimodal medical image retrieval based on latenttopic modelling. Conference on Neural Information Processing Systems

  45. Souissi N, Ayadi H, Torjmen M (2017) Towards medical image re-ranking based on convolutional neural networks. In: The International Conference of Engineering Sciences for Biology and Medicine (ESBM). ESBM, Tunisia

  46. Torjmen-Khemakhem M, Gasmi K (2019) Document/query expansion based on selecting significant concepts for context based retrieval of medical images. Journal of biomedical informatics 95:103210

  47. Malik S, Shoaib U, El-Sayed H, Khan MA (2020) Query expansion framework leveraging clinical diagnosis information ontology. In: International Conference on Innovations in Information Technology, pp 18–23

Download references

Author information

Authors and Affiliations

  1. Dr. APJ Abdul Kalam Technical University, Lucknow, UP, India

    Yatin Kumar Agarwal

  2. Krishna Institute of Engineering and Technology, Ghaziabad, UP, India

    Dilkeshwar Pandey

  3. Department of Computer Science & Engineering, Institute of Engineering and Technology, Dr. Rammanohar Lohia Avadh University, Ayodhya, UP, India

    Lokendra Singh Umrao

Authors
  1. Yatin Kumar Agarwal
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dilkeshwar Pandey
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Lokendra Singh Umrao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Lokendra Singh Umrao.

Ethics declarations

Conflict of interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Agarwal, Y.K., Pandey, D. & Umrao, L.S. Hybrid query refinement based approach for enhanced biomedical image retrieval. Multimed Tools Appl 83, 49515–49536 (2024). https://doi.org/10.1007/s11042-023-17469-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-023-17469-1

Keywords

Search

Navigation