Skip to main content

Advertisement

SpringerLink
Log in

Intelligent fusion-assisted skin lesion localization and classification for smart healthcare

  • S.I.: Improving Healthcare outcomes using Multimedia Big Data Analytics
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

With the rapid development of information technology, the conception of smart healthcare has progressively come to the fore. Smart healthcare utilizes next-generation technologies, such as artificial intelligence, the Internet of Things (IoT), big data and cloud computing to transform intelligently the existing medical system-making it more efficient, more reliable, and personalized. In this work, skin data are collected using dedicated hardware from mobile health units-working as nodes. The collected samples are uploaded to the cloud for further processing using a novel multi-modal information fusion framework, which performs skin lesion segmentation, followed by classification. The proposed framework has two main functional blocks: Segmentation and classification. In each block, we have a performance booster, which works on the principle of information fusion. For lesion segmentation, a hybrid framework is proposed, which utilizes the complementary strengths of two convolutional neural network (CNN) architectures to generate the segmented images. The resultant binary images are later fused using joint probability distribution and marginal distribution function. For lesion classification, a 30-layered CNN architecture is designed, which is trained on the HAM10000 dataset. A novel summation discriminant correlation analysis technique is used to fuse the extracted features from two fully connected layers. To avoid feature redundancy, a feature selection method “Regular Falsi” is developed, which down samples the extracted features into the lower dimensions. The selected features are finally classified using an extreme learning machine classifier. Five skin benchmark datasets (ISBI2016, ISIC2017, ISBI2018, ISIC2019, and HAM10000) are used to evaluate both segmentation and classification frameworks using average accuracy, false-negative rate, sensitivity, and computational time, whose results are impressive compared to state-of-the-art methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Availability of data and material

All datasets used in this work are publically available such as Ph2, ISIC, and HAM10000.

Notes

  1. *represent iterations, e represents number of epochs, i-2016 represents the ISBI2016 dataset, i-2017 represents the ISBI2017 dataset, and i-2018 represents the ISBI2018 dataset

References

  1. Ratanjee-Vanmali H, Swanepoel DW, Laplante-Lévesque A (2020) Patient uptake, experience, and satisfaction using web-based and face-to-face hearing health services: process evaluation study. J. med. Intern. res. 22:e15875

    Google Scholar 

  2. Mewada HK, Patel AV, Hassaballah M, Alkinani MH, Mahant K (2020) Spectral–spatial features integrated convolution neural network for breast cancer classification. Sensors 20:4747

    Article  Google Scholar 

  3. Gálvez JM, Castillo-Secilla D, Herrera LJ, Valenzuela O, Caba O, Prados JC et al (2019) Towards improving skin cancer diagnosis by integrating microarray and RNA-seq datasets. IEEE J Biomed Health Inform 24:2119–2130

    Article  Google Scholar 

  4. Rehman A, Khan MA, Mehmood Z, Saba T, Sardaraz M, Rashid M, (2020) "Microscopic melanoma detection and classification: A framework of pixel‐based fusion and multilevel features reduction." Microscopy Research and Technique

  5. Song L,LinJP, Wang ZJ, Wang H (2020) "An End-to-end Multi-task Deep Learning Framework for Skin Lesion Analysis," IEEE J. Biomed. Health Informat.

  6. Nahar VK, Wilkerson AH, Pearlman RL, Ferris TS, Zardoost P, Payson SN et al., "Skin cancer-related knowledge, attitudes, beliefs, and practices among the population in Gulf Cooperation Council countries: a systematic search and literature review", Archives of Dermatological Research. 1–12, 2020

  7. Khan MA, Sharif M, Akram T, Bukhari SAC, Nayak RS (2020) Developed newton-Raphson based deep features selection framework for skin lesion recognition. Pattern Recogn Lett 129:293–303

    Article  Google Scholar 

  8. SCF, "Skin Cancer Facts & Statistics (https://www.skincancer.org/skin-cancer-information/skin-cancer-facts/)," 2020.

  9. A. C. Society, "Key Statistics for Melanoma Skin Cancer," https://www.cancer.org/cancer/melanoma-skin-cancer/about/key-statistics.html, 2021.

  10. Celebi ME, Codella N, Halpern A (2019) Dermoscopy image analysis: overview and future directions. IEEE J Biomed Health Inform 23:474–478

    Article  Google Scholar 

  11. Hagerty JR, Stanley RJ, Almubarak HA, Lama N, Kasmi R, Guo P et al (2019) Deep learning and handcrafted method fusion: higher diagnostic accuracy for melanoma dermoscopy images. IEEE J Biomed Health Inform 23:1385–1391

    Article  Google Scholar 

  12. George YM, Antony B, Ishikawa H, Wollstein G, Schuman J,Garnavi R, "Attention-guided 3D-CNN Framework for Glaucoma Detection and Structural-Functional Association using Volumetric Images." IEEE J. Biomed. Health Informat. 2020

  13. Olender ML, Athanasiou LS, Michalis LK, Fotiadis DI, Edelman ER (2020) A Domain enriched deep learning approach to classify atherosclerosis using intravascular ultrasound imaging. IEEE J. Selected Topics in Signal Processing 14:1210–1220

    Article  Google Scholar 

  14. Boshkoska BM, Miljković D, Valmarska A, Gatsios D, Rigas G, Konitsiotis S et al (2020) Decision support for medication change of parkinson’s disease patients. Comput. Meth. Progr. Biomed. 196:105552

    Article  Google Scholar 

  15. Castiglione A, Vijayakumar P, Nappi M, Sadiq S, Umer M,(2021) "COVID-19: Automatic Detection of the Novel Coronavirus Disease from CT Images Using an Optimized Convolutional Neural Network". IEEE Transactions on Industrial Informatics

  16. Rajinikanth V, RajaNSM, Arunmozhi S (2019) "ABCD rule implementation for the skin melanoma assesment–a study," in 2019 IEEE Int. Conf. on System, Comput., Autom. and Network. (ICSCAN), 2019. 1–4

  17. Yi H, Li J, Lin Q, Wang H, Song H, Ming Z et al (2019) A rainbow-based authentical scheme for securing smart connected health systems. J Med Syst 43:276

    Article  Google Scholar 

  18. Qian X, Fu H, Shi W, Chen T, Fu Y, Shan F et al (2020) M $^ 3$ Lung-sys: a deep learning system for multi-class lung pneumonia screening from ct imaging. IEEE J Biomed Health Inform 24:3539–3550

    Article  Google Scholar 

  19. Khan MA, Akram T, Sharif M, Kadry S, Nam Y (2021) Computer decision support system for skin cancer localization and classification. CMC-Computers Materials & Continua 68:1041–1064

    Article  Google Scholar 

  20. S. Wan, Y. Xia, L. Qi, Y.-H. Yang, M. Atiquzzaman, "Automated colorization of a grayscale image with seed points propagation," IEEE Trans. Multim. 2020

  21. S. AlZu’bi, M. Shehab, M. Al-Ayyoub, Y. Jararweh, B. Gupta (2020) "Parallel implementation for 3d medical volume fuzzy segmentation", Patt. Recogn.Lett. 130 312-318

  22. Khan MA, Lali MIU, Sharif M, Javed K, Aurangzeb K, Haider SI et al (2019) An optimized method for segmentation and classification of apple diseases based on strong correlation and genetic algorithm based feature selection. IEEE Access 7:46261–46277

    Article  Google Scholar 

  23. Khan MA, Sharif M, Akram T, Damaševičius R, Maskeliūnas R (2021) Skin lesion segmentation and multiclass classification using deep learning features and improved moth flame optimization. Diagnostics 11:811

    Article  Google Scholar 

  24. Zhao Y, Li H, Wan S, Sekuboyina A, Hu X, Tetteh G et al (2019) Knowledge-aided convolutional neural network for small organ segmentation. IEEE J Biomed Health Inform 23:1363–1373

    Article  Google Scholar 

  25. Li Y, Shen L (2018) Skin lesion analysis towards melanoma detection using deep learning network. Sensors 18:556

    Article  Google Scholar 

  26. D. Gutman, N. C. Codella, E. Celebi, B. Helba, M. Marchetti, N. Mishra, et al., "Skin lesion analysis toward melanoma detection: A challenge at the international symposium on biomedical imaging (ISBI) 2016, hosted by the international skin imaging collaboration (ISIC)," arXiv preprint,"arXiv:1605.01379" 2016.

  27. V. Rajinikanth, S. C. Satapathy, N. Dey, S. L. Fernandes, and K. S. Manic, "Skin melanoma assessment using Kapur’s entropy and level set—a study with bat algorithm," in Smart intelligent computing and applications, ed: Springer, 2019, 193–202.

  28. Dey N, Rajinikanth V, Ashour AS, Tavares JMR (2018) Social group optimization supported segmentation and evaluation of skin melanoma images. Symmetry 10:51

    Article  Google Scholar 

  29. Khan MA, Akram T, Sharif M, Shahzad A, Aurangzeb K, Alhussein M et al (2018) An implementation of normal distribution based segmentation and entropy controlled features selection for skin lesion detection and classification. BMC Cancer 18:638

    Article  Google Scholar 

  30. Wang H, Li Z, Li Y, Gupta BB, Choi C (2020) Visual saliency guided complex image retrieval. Pattern Recogn Lett 130:64–72

    Article  Google Scholar 

  31. Albahar MA (2019) Skin lesion classification using convolutional neural network with novel regularizer. IEEE Access 7:38306–38313

    Article  Google Scholar 

  32. Al-Smadi M, Qawasmeh O, Al-Ayyoub M, Jararweh Y, Gupta B (2018) Deep Recurrent neural network vs. support vector machine for aspect-based sentiment analysis of Arabic hotels’ reviews. J. comput. Sci. 27:386–393

    Article  Google Scholar 

  33. M. A. Khan, K. Muhammad, M. Sharif, T. Akram, V. H. C. de Albuquerque (2021) "Multi-Class Skin Lesion Detection and Classification via Teledermatology," IEEE j. biomed. health inform, 2021.

  34. A. Gupta, M. Bhatnagar, A. Issac, M. K. Dutta, C. M. Travieso,(2019) "Imaging method for noise removal and segmentation of skin lesions from dermoscopic images," in Proceedings of the 2nd International Conference on Applications of Intelligent Systems, 1–5

  35. Garcia-Arroyo JL, Garcia-Zapirain B (2019) Segmentation of skin lesions in dermoscopy images using fuzzy classification of pixels and histogram thresholding. Comput Methods Programs Biomed 168:11–19

    Article  Google Scholar 

  36. Tschandl P, Sinz C, Kittler H (2019) Domain-specific classification-pretrained fully convolutional network encoders for skin lesion segmentation. Comput Biol Med 104:111–116

    Article  Google Scholar 

  37. Pour MP, Seker H (2020) Transform domain representation-driven convolutional neural networks for skin lesion segmentation. Expert Sys. with Appl. 144:113129

    Article  Google Scholar 

  38. Sikkandar MY, Alrasheadi BA, Prakash N, Hemalakshmi G, Mohanarathinam A, Shankar K (2021) Deep learning based an automated skin lesion segmentation and intelligent classification model. J Ambient Intell Humaniz Comput 12:3245–3255

    Article  Google Scholar 

  39. Xie F, Yang J, Liu J, Jiang Z, Zheng Y, Wang Y (2020) Skin lesion segmentation using high-resolution convolutional neural network. Computer methods and programs in biomedicine. 186:105241

    Article  Google Scholar 

  40. S. Chatterjee, D. Dey, S. Munshi, S. Gorai, (2020) "Dermatological expert system implementing the ABCD rule of dermoscopy for skin disease identification", Expert Sys. Appl. 114204

  41. Liu L, Tsui YY, Mandal M (2021) Skin lesion segmentation using deep learning with auxiliary task. Journal of Imaging 7:67

    Article  Google Scholar 

  42. H W Huang, B W Y Hsu, C H Lee, V S Tseng, "Development of a light‐weight deep learning model for cloud applications and remote diagnosis of skin cancers," The J. Dermat, 2020

  43. Kumar M, Alshehri M, AlGhamdi R, Sharma P, Deep V (2020) A de-ann inspired skin cancer detection approach using fuzzy c-means clustering. Mobile Net. Appl. 25:1319–1329

    Article  Google Scholar 

  44. Z. Gao, Y. Li, and S. Wan. 2020. "Exploring deep learning for view-based 3D model retrieval". ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM). 16 (1) 1-21

  45. Khan MA, Zhang Y-D, Sharif M, Akram T (2021) Pixels to classes: intelligent learning framework for multiclass skin lesion localization and classification. Comput. Electr. Eng. 90:106956

    Article  Google Scholar 

  46. Thomas SM, Lefevre JG, Baxter G, Hamilton NA (2021) Interpretable deep learning systems for multi-class segmentation and classification of non-melanoma skin cancer. Medical Image Analys. 68:101915

    Article  Google Scholar 

  47. M Abdel-Basset, H Hawash, R K Chakrabortty, M Ryan, M Elhoseny, and H Song, "ST-DeepHAR: Deep Learning Model for Human Activity Recognition in IoHT Applications", IEEE Internet of Things J. 2020.

  48. M Hassaballah A I Awad, Deep learning in computer vision: principles and applications: CRC Press, 2020

  49. Haghighat MBA, Aghagolzadeh A, Seyedarabi H (2011) A non-reference image fusion metric based on mutual information of image features. Comput Electr Eng 37:744–756

    Article  Google Scholar 

  50. Chen J, Li W (2006) An exponential regula falsi method for solving nonlinear equations. Numerical algorithms 4:327–338

    Article  MathSciNet  Google Scholar 

  51. Ridders C (1979) A new algorithm for computing a single root of a real continuous function. IEEE Trans. on circu. Syst. 26:979–980

    Article  Google Scholar 

  52. N C Codella, D Gutman, M E Celebi, B Helba, M A. Marchetti, S W Dusza, et al., "Skin lesion analysis toward melanoma detection: A challenge at the 2017 international symposium on biomedical imaging (isbi), hosted by the international skin imaging collaboration (isic)," in 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018), 2018. 168–172

  53. N Codella, V Rotemberg, P Tschandl, M E Celebi, S Dusza, D Gutman, et al., "Skin lesion analysis toward melanoma detection 2018: A challenge hosted by the international skin imaging collaboration (isic)," arXiv preprint arXiv:1605.01397, 2019

  54. Tschandl P, Rosendahl C, Kittler H (2018) The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions. Scientific data. 5:180161

    Article  Google Scholar 

  55. Hekler A, Utikal JS, Enk AH, Hauschild A, Weichenthal M, Maron RC et al (2019) Superior skin cancer classification by the combination of human and artificial intelligence. Eur J Cancer 120:114–121

    Article  Google Scholar 

  56. W. Sae-Lim, W. Wettayaprasit, and P. Aiyarak, "Convolutional Neural Networks Using MobileNet for Skin Lesion Classification," in 2019 16th International Joint Conference on Computer Science and Software Engineering (JCSSE), 2019. 242–247

  57. Zhang J, Xie Y, Xia Y, Shen C (2019) Attention residual learning for skin lesion classification. IEEE Trans Med Imaging 38:2092–2103

    Article  Google Scholar 

Download references

Funding

There is no specific funding reported for this study.

Author information

Authors and Affiliations

  1. Department of Computer Science, COMSATS University Islamabad, Wah Campus, Pakistan

    Muhammad Attique Khan & Muhammad Sharif

  2. Visual Analytics for Knowledge Laboratory (VIS2KNOW Lab), Department of Software, Sejong University, Seoul, 143-747, Republic of Korea

    Khan Muhammad

  3. Department of Electrical & Computer Engineering, COMSATS University Islamabad, Wah Campus, Pakistan

    Tallha Akram

  4. Faculty of Applied Computing and Technology, Noroff University College, Kristiansand, Norway

    Seifedine Kadry

Authors
  1. Muhammad Attique Khan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Khan Muhammad
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Muhammad Sharif
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Tallha Akram
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Seifedine Kadry
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Muhammad Attique Khan or Khan Muhammad.

Ethics declarations

Conflicts of interest

There is no conflict of interest in this work.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Khan, M.A., Muhammad, K., Sharif, M. et al. Intelligent fusion-assisted skin lesion localization and classification for smart healthcare. Neural Comput & Applic 36, 37–52 (2024). https://doi.org/10.1007/s00521-021-06490-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-021-06490-w

Keywords

Search

Navigation