Skip to main content

Advertisement

SpringerLink
Log in

Content-Based Medical Image Retrieval System for Skin Melanoma Diagnosis Based on Optimized Pair-Wise Comparison Approach

  • Original Paper
  • Published:
Journal of Digital Imaging Aims and scope Submit manuscript

Abstract

Medical image analysis for perfect diagnosis of disease has become a very challenging task. Due to improper diagnosis, required medical treatment may be skipped. Proper diagnosis is needed as suspected lesions could be missed by the physician’s eye. Hence, this problem can be settled up by better means with the investigation of similar case studies present in the healthcare database. In this context, this paper substantiates an assistive system that would help dermatologists for accurate identification of 23 different kinds of melanoma. For this, 2300 dermoscopic images were used to train the skin-melanoma similar image search system. The proposed system uses feature extraction by assigning dynamic weights to the low-level features based on the individual characteristics of the searched images. Optimal weights are obtained by the newly proposed optimized pair-wise comparison (OPWC) approach. The uniqueness of the proposed approach is that it provides the dynamic weights to the features of the searched image instead of applying static weights. The proposed approach is supported by analytic hierarchy process (AHP) and meta-heuristic optimization algorithms such as particle swarm optimization (PSO), JAYA, genetic algorithm (GA), and gray wolf optimization (GWO). The proposed approach has been tested with images of 23 classes of melanoma and achieved significant precision and recall. Thus, this approach of skin melanoma image search can be used as an expert assistive system to help dermatologists/physicians for accurate identification of different types of melanomas.

Graphical abstract

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Data Availability

The datasets generated during and/or analyzed during the current study are available in the ISIC Challenge 2018 repository [Webpage: http://www.dermnet.com/, Direct link: https://www.kaggle.com/shubhamgoel27/dermnet.

References

  1. Muller H, Michoux N, Bandon D, Geissbuhler A (2004) A review of content-based image retrieval systems in medical applications - Clinical benefits and future directions. Int. J. Med. Inform. 73(1): 1–23. https://doi.org/10.1016/j.ijmedinf.2003.11.024.

    Article  PubMed  Google Scholar 

  2. Owais M, Arsalan M, Choi J, Park KR (2019) Effective Diagnosis and Treatment through Content-Based Medical Image Retrieval (CBMIR) by Using Artificial Intelligence. Journal of clinical medicine 8(4): 462. https://doi.org/10.3390/jcm8040462

    Article  PubMed  PubMed Central  Google Scholar 

  3. Miranda E, Aryuni M (2016) A Survey of Medical Image Classification Techniques. In 2016 International Conference on Information Management and Technology (ICIMTech). pp 56–61. https://doi.org/10.1109/ICIMTech.2016.7930302

  4. Ma L, Liu X, Gao Y, Zhao Y, Zhao X, Zhou C (2017) A newmethod of content based medical image retrieval and its applications to CT imaging sign retrieval. J. of Biomed. Inform. 66: 148–158. https://doi.org/10.1016/j.jbi.2017.01.002

    Article  Google Scholar 

  5. Tang J, Agaian S, Thompson I(2014) Guest editorial: computer-aided detection or diagnosis (CAD) systems. IEEE Systems Journal 8(3): 907-909. https://doi.org/10.1109/JSYST.2014.2317378.

    Article  Google Scholar 

  6. Lei Z, Fuzong L, Bo Z (1999) A CBIR method based on color-spatial feature, IEEE Reg. 10 Annu. Int. Conf. Proceedings/TENCON 1: 166–169. https://doi.org/10.1109/TENCON.1999.818376.

    Article  Google Scholar 

  7. Su Z, Zhang H, Li S, Ma S (2003) Relevance Feedback in Content-Based Image Retrieval: Bayesian Framework, Feature Subspaces,and Progressive Learning. IEEE Trans. Image Process 12(8): 924–937. https://doi.org/10.1109/TIP.2003.815254

    Article  PubMed  Google Scholar 

  8. Swain MJ, Ballard DH (1991) Color Indexing. International journal of computer vision 32: 11–32. https://doi.org/10.1007/BF00130487.

    Article  Google Scholar 

  9. Huang J, Kumar SR, Mitra M, Zhu W-J, Zabih R (1997) Image indexing using color correlograms. Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit. pp 762–768. https://doi.org/10.1109/CVPR.1997.609412

  10. Dudani SA, Breeding KJ, McGhee RB (1977) Aircraft Identification by Moment Invariants. IEEE Trans. Comput. 100(1): 39–46. https://doi.org/10.1109/TC.1977.5009272

    Article  Google Scholar 

  11. Mingqiang Y, Kidiyo K, Joseph R (2008) A survey of shape feature extraction techniques. Pattern recognition 15(7): 43–90. https://hal.archives-ouvertes.fr/hal-00446037

  12. Tsai HH, Chang B, Liou S (2014) Rotation-invariant texture image retrieval using particle swarm optimization and support vector regression. Appl. Soft Comput. 17: 127-139. https://doi.org/10.1016/j.asoc.2013.12.003.

    Article  Google Scholar 

  13. Haralick RM (1979) Statistical and structural approach to texture. Proceeding IEEE 67(5): 786–804. https://doi.org/10.1109/PROC.1979.11328

    Article  Google Scholar 

  14. Wang X-Y, Zhang B-B, Yang H-Y (2014) Content-based image retrieval by integrating color and texture features. Multimed. Tools Appl. 68(3): 545–569. https://doi.org/10.1007/s11042-012-1055-7

    Article  Google Scholar 

  15. Varish N, Pal AK (2018) A novel image retrieval scheme using gray level co-occurrence matrix descriptors of discrete cosine transform based residual image. Appl. Intell. 48(9) : 2930-2953. https://doi.org/10.1007/s10489-017-1125-7

    Article  Google Scholar 

  16. Ahirwal MK, Kumar A, Singh GK (2017) An Approach to Design Self Assisted CBIR System. Proc. Int. Conf. Graph. Signal Process-ICGSP ’17. pp 21–25. https://doi.org/10.1145/3121360.3121378

  17. Rout NK, Ahirwal MK (2018) A Content Based Image Retrieval System: Analysis of Individual and Mixed Image Features, 2018 Inter. Conf. on Recent Innov. in Elec., Electronics & Comm. Engg. (ICRIEECE). pp 2561–2566. https://doi.org/10.1109/ICRIEECE44171.2018.9009128

  18. Rout NK, Atulkar M, Ahirwal MK (2021) A Review on Content Based Image Retrieval System: Present Trends and Future Challenges. International Journal of Computational Vision and Robotics 11(5): 461–485. https://doi.org/10.1504/IJCVR.2021.117578

  19. Cheng CH, Liu W-X (2018) Identifying Degenerative Brain Disease Using Rough Set Classifier Based on Wavelet Packet Method. Journal of Clinical Medicine 7(6): 124. https://doi.org/10.3390/jcm7060124

    Article  PubMed  PubMed Central  Google Scholar 

  20. Murala S, Wu QMJ (2014) Local Mesh Patterns Versus Local Binary Patterns: Biomedical Image Indexing and Retrieval. IEEE J. Biomed. Heal. Informatics 18(3): 929–938. https://doi.org/10.1109/JBHI.2013.2288522

    Article  Google Scholar 

  21. Zhang X, Liu W, Dundar M, Badve S, Zhang S (2014) Towards Large-Scale Histopathological Image Analysis : Hashing-Based Image Retrieval. IEEE Transactions on Medical Imaging 34(2): 496–506. https://doi.org/10.1109/TMI.2014.2361481

    Article  PubMed  Google Scholar 

  22. Liu X, Ma L, Song L, Zhao Y, Zhao X, Zhou C (2015) Recognizing Common CT Imaging Signs of Lung Diseases through a New Feature Selection Method based on Fisher Criterion and Genetic optimization. IEEE J. Biomed. Health Informatics 19(2) : 635-647. https://doi.org/10.1109/JBHI.2014.2327811

    Article  Google Scholar 

  23. Shinde A, Rahulkar A, Patil C (2019) Content based medical image retrieval based on new efficient local neighborhood wavelet feature descriptor. Biomed. Eng. Lett. 9(3): 387-394. https://doi.org/10.1007/s13534-019-00112-0

    Article  PubMed  PubMed Central  Google Scholar 

  24. El-Naqa I, Yang Y, Galatsanos NP, Nishikawa RM, Wernick MN (2004) A Similarity Learning Approach to Content-Based Image Retrieval: Application to Digital Mammography. IEEE Trans. Med. Imaging 23(10): 1233–1244. https://doi.org/10.1109/TMI.2004.834601

    Article  PubMed  Google Scholar 

  25. Shyu C, Brodley CE, Kak AC, Kosaka A, Aisen AM, Broderick LS (1999) ASSERT : A Physician-in-the-Loop Content-Based Retrieval System for HRCT Image Databases. Computer Vision and Image Understanding 75(1-2): 111–132. https://doi.org/10.1006/cviu.1999.0768

    Article  Google Scholar 

  26. Baldi A, Murace R, Dragonetti E, Manganaro M, Guerra O, Bizzi S, Galli L (2009) Definition of an automated Content-Based Image Retrieval (CBIR) skin lesions. Biomedical engineering online 10(1): 1–10. https://doi.org/10.1186/1475-925X-8-18

    Article  Google Scholar 

  27. Renita DB, Christopher CS (2020) Novel real time content based medical image retrieval scheme with GWO-SVM, Multimedia Tools and Applications 1-17. https://doi.org/10.1007/s11042-019-07777-w

    Article  Google Scholar 

  28. Shamna P, Govindan VK, Nazeer KAA (2019) Content based medical image retrieval using topic and location model. J. Biomed. Inform. 91: 103112. https://doi.org/10.1016/j.jbi.2019.103112

    Article  CAS  PubMed  Google Scholar 

  29. Anand V, Gupta S, Koundal D, Nayak SR, Nayak J, Vimal S (2022) Multi-class Skin Disease Classification Using Transfer Learning Model. International Journal on Artificial Intelligence Tools 31(02):2250029. https://doi.org/10.1142/S0218213022500294

    Article  Google Scholar 

  30. Anand V, Gupta S, Koundal D, Nayak SR, Barsocchi P, Bhoi AK (2022) Modified U-NET Architecture for Segmentation of Skin Lesion. Sensors 22(3): 867. https://doi.org/10.3390/s22030867

    Article  PubMed  PubMed Central  Google Scholar 

  31. Hossen MN, Panneerselvam V, Koundal D, Ahmed K, Bui FM, Ibrahim SM (2022) Federated machine learning for detection of skin diseases and enhancement of internet of medical things (IoMT) security. IEEE journal of biomedical and health informatics. https://doi.org/10.1109/JBHI.2022.3149288

    Article  PubMed  Google Scholar 

  32. Xiaoling W, Kanglin XIE (2011) Content-Based Image Retrieval Incorporating the AHP Method. International Journal of Information Technology 11(1) : 25–37. http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.106.4726.

  33. Moussaoui F, Cherrared M, Akli M (2018) A genetic algorithm to optimize consistency ratio in AHP method for energy performance assessment of residential buildings-Application of top-down and bottom-up approaches in Algerian case study. Sustainable Cities and Society 42: 622–636.https://doi.org/10.1016/j.scs.2017.08.008

    Article  Google Scholar 

  34. Awad GA, Sultan EI, Ahmad N, Ithnan N, Beg AH (2011) Multi-Objectives Model to Process Security Risk Assessment Based on AHP- Multi-Objectives Model to Process Security Risk Assessment Based on AHP-PSO. Modern Applied Science 5(3): 246. https://doi.org/10.5539/mas.v5n3p246

    Article  Google Scholar 

  35. Nayak AK, Mishra BSP, Das H (2019) Computational intelligence in sensor networks, Edited by B. B. Mishra, Satchidanand Dehuri, and Bijaya Ketan Panigrahi. Springer, 2019

  36. Younus ZS, Mohamad D, Saba T, Alkawaz MH, Rehman A, Al-Rodhaan M, Al-Dhelaan A (2015) Content-based image retrieval using PSO and k-means clustering algorithm. Arab. J. Geosci. 8(8) :6211–6224. https://doi.org/10.1007/s12517-014-1584-7

    Article  Google Scholar 

  37. Jiji GW, Durairaj PJ (2015) Content-based image retrieval techniques for the analysis of dermatological lesions using particle swarm optimization technique. Appl. Soft Comput. J. 30: 650–662. https://doi.org/10.1016/j.asoc.2015.01.058

    Article  Google Scholar 

  38. Huang PC, Tong LI, Chang WW, Yeh WC (2011) A two-phase algorithm for product part change utilizing AHP and PSO. Expert Syst. Appl. 38(7): 8458–8465. https://doi.org/10.1016/j.eswa.2011.01.043

    Article  Google Scholar 

  39. Kumar A, Suvendu S (2019) Automated face retrieval using bag ‑ of ‑ features and sigmoidal grey wolf optimization. Evol. Intell. pp 1–12. https://doi.org/10.1007/s12065-019-00213-w

  40. Wang S, Phillips P, Dong Z, Zhang Y (2018) Intelligent facial emotion recognition based on stationary wavelet entropy and Jaya algorithm. Neurocomputing 272: 668–676. https://doi.org/10.1016/j.neucom.2017.08.015

    Article  Google Scholar 

  41. Satapathy SC, Rajinikanth V (2018) Jaya Algorithm Guided Procedure to Segment Tumor from brain MRI. Journal of Optimization. https://doi.org/10.1155/2018/3738049

    Article  Google Scholar 

  42. Srikanth MV, Yadaiah N (2020) Optimal parameter tuning of Modified Active Disturbance Rejection Control for unstable time-delay systemsusing an AHP combined Multi-Objective Quasi-Oppositional Jaya Algorithm. Applied Soft Computing 86: 1058881. https://doi.org/10.1016/j.asoc.2019.105881

    Article  Google Scholar 

  43. Codella NCF, Gutman D, Emre Celebi M, Helba B, Marchetti MA, Dusza SW, Kalloo A, Liopyris K, Mishra N, Kittler H, Halpern A (2018) Skin Lesion Analysis Toward Melanoma Detection: A Challenge at the 2017 International Symposium on Biomedical Imaging (ISBI), Hosted by the International Skin Imaging Collaboration (ISIC), pp 168–172. https://doi.org/10.1109/ISBI.2018.8363547

  44. Combalia M, Codella NCF, Rotemberg V, Helba B, Vilaplana V, Reiter O, Halpern AC, Puig S, Malvehy J (2019) BCN20000: Dermoscopic Lesions in the Wild. arXiv:1908.02288. https://arxiv.org/abs/1908.02288

  45. Tschandl P, Rosendahl C, Kittler H (2018) The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions Scientific Data, 5(1): 1–9. https://doi.org/10.1038/sdata.2018.161

  46. Cheng S, Chou T, Yang C, Chang H (2005) A semantic learning for content-based image retrieval using analytical hierarchy process. Expert Systems with Applications 28(3): 495-505. https://doi.org/10.1016/j.eswa.2004.12.011.

    Article  Google Scholar 

  47. Saaty, Thomas L. "What is the analytic hierarchy process?." In Mathematical models for decision support, pp. 109–121. Springer, Berlin, Heidelberg, 1988.

  48. Wang X, Nianzu L, Kanglin M (2008) A novel AHP-based image retrieval interface. Chinese Control and Decision Conference, pp 2334–2337. https://doi.org/10.1109/CCDC.2008.4597741

  49. Yalcinoz T, Altun H, Uzam M (2001) Economic Dispatch Solution Using A Genetic Algorithm Based on Arithmetic Crossover. In 2001 IEEE Porto Power Tech Proceedings (Cat. No. 01EX502), 2:4. https://doi.org/10.1109/PTC.2001.964734

  50. Mirjalili S, Mohammad S, Lewis A (2014) Advances in Engineering Software Grey Wolf Optimizer. Adv. Eng. Software, 69: 46–61. https://doi.org/10.1016/j.advengsoft.2013.12.007

    Article  Google Scholar 

  51. Rao R (2016) Jaya: A simple and new optimization algorithm for solving constrained and unconstrained optimization problems. International Journal of Industrial Engineering Computations 7(1) : 19-34. https://doi.org/10.5267/j.ijiec.2015.8.004

    Article  Google Scholar 

  52. Migallon H, Morenilla AJ, Sanchez-Romero JL (2018) Parallel improvements of the Jaya optimization algorithm. Applied Sciences 8(5) :819. https://doi.org/10.3390/app8050819

    Article  Google Scholar 

  53. Mane SU, Narsingrao MR, Patil VC (2018) A many-objective Jaya algorithm for many-objective optimization problems. Decision science letters 7(4) : 567–582. https://doi.org/10.5267/j.dsl.2017.11.001

    Article  Google Scholar 

  54. Rout NK, Atulkar M, Ahirwal MK (2021) Analytic hierarchy process- based automatic feature weight assignment method for content-based satellite image retrieval system. Soft Computing, 1–11.

  55. Rout NK, Atulkar M, Ahirwal MK (2021) Identification of Similar Gastrointestinal Images through Content Based Image Retrieval System based on Analytical Hierarchical Process. In 2021 2nd International Conference on Range Technology (ICORT), pp. 1–6. IEEE, 2021.

  56. Rout NK, Atulkar M, Ahirwal MK (2022) Assimilation of pair-wise comparison method to decide weights of features based on the content of image. International Journal of Information Technology and Decision Making (1–26). https://doi.org/10.1142/S0219622022500407

  57. Anand V, Gupta S, Koundal D (2022) Skin disease diagnosis: challenges and opportunities. In Proceedings of Second Doctoral Symposium on Computational Intelligence (pp. 449–459). Springer, Singapore. https://doi.org/10.1007/978-981-16-3346-1_63

  58. Anand V, Gupta S, Koundal D (2022) Detection and Classification of Skin Disease Using Modified Mobilenet Architecture. ECS Transactions 107(1):5059. https://doi.org/10.1149/10701.5059ecst

    Article  Google Scholar 

Download references

Acknowledgements

The authors would like to thank the Department of Dermatology of the Hospital Clínic de Barcelona for the preparation of the ISIC Challenge 2018.

Author information

Authors and Affiliations

  1. Department of Computer Application, NIT, Raipur, C.G., 492010, India

    Narendra Kumar Rout & Mithilesh Atulkar

  2. Department of Computer Science and Engineering, MANIT, Bhopal, M.P., 462003, India

    Mitul Kumar Ahirwal

Authors
  1. Narendra Kumar Rout
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mitul Kumar Ahirwal
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mithilesh Atulkar
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All the authors contributed to the study conception and design. We confirm that the manuscript has been read and approved by all the named authors and that there are no other persons who satisfied the criteria for authorship but are not listed. We further confirm that the order of authors listed in the manuscript has been approved by all of us.

Corresponding author

Correspondence to Mitul Kumar Ahirwal.

Ethics declarations

Ethics Approval

Ethical approval not required for this study. Human or animal subjects’ data is not used in this study.

Informed Consent

Not applicable.

Conflict of Interest

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (DOCX 44 KB)

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rout, N.K., Ahirwal, M.K. & Atulkar, M. Content-Based Medical Image Retrieval System for Skin Melanoma Diagnosis Based on Optimized Pair-Wise Comparison Approach. J Digit Imaging 36, 45–58 (2023). https://doi.org/10.1007/s10278-022-00710-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10278-022-00710-y

Keywords

Search

Navigation