Skip to main content
SpringerLink
Log in

A new method proposed to Melanoma-skin cancer lesion detection and segmentation based on hybrid convolutional neural network

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

The number of deaths due to melanoma skin cancer has rapidly increased in recent years. The timely diagnosis of the lesions of melanoma skin cancer can potentially increase the survival rate of such a chronic disease. However, the detection of these lesions is a challenging task, especially in the presence of occlusions, such as clinical artifacts, blood vessels, and color contrast variation, etc. The current state-of-the-art detection and segmentation methods are based on fully convolutional neural networks, which utilize an encoder-decoder method. However, these methods produce coarse segmentation masks due to the loss of location information during the encoding layers. To overcome these challenges, this study proposes a highly effective Hybrid detection and segmentation method based on the integration of RetinaNet and MaskRCNN, which utilizes a pyramid module of lateral connections and top-down paths to compensate for the loss of spatial features information. The proposed method is trained and validated on Melanoma-ISIC-2018 and PH2 datasets. Experiment results on the unseen PH2 dataset illustrate the improved generalization ability of the method. The efficacy with other methods such as Encoder-Decoder, Generative Adversarial Network(GAN), UNet Deep Convolutional Neural Network-support vector machine(DCNN-SVM), Encoder-Fully Connected Network(EFCN), Enhanced Convolutional-Deconvolutional Networks(ECDNs), UNet, and Handcrafted has also been compared. The results, show that the proposed method outperforms above methods by 7.7%, 12.9%, 11.4%, 14.4%, 14.9%, 18.6%, 25.1%, respectively, in terms of accuracy. It is envisaged that with reliable accuracy, this method can be introduced for clinical practices in the future.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Algorithm 1
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  1. Abdulla W (2017) Mask r-cnn for object detection and instance segmentation on keras and tensorflow. https://github.com/matterport/Mask_RCNN. Accessed 18 March 2018

  2. Abraham N, Khan NM (2019) A novel focal tversky loss function with improved attention u-net for lesion segmentation. In: 2019 IEEE 16th international symposium on biomedical imaging (ISBI 2019), IEEE, pp 683–687

  3. Al-Masni MA, Al-antari MA, Choi MT, Han SM, Kim TS (2018) Skin lesion segmentation in dermoscopy images via deep full resolution convolutional networks. Comput Methods Prog Biomed 162:221–231

    Article  Google Scholar 

  4. Al Nazi Z, Abir TA (2020) Automatic skin lesion segmentation and melanoma detection: transfer learning approach with u-net and dcnn-svm. In: Proceedings of international joint conference on computational intelligence, Springer, pp 371–381

  5. Arjovsky M, Chintala S, Bottou L (2017) Wasserstein gan. arXiv:1701.07875

  6. Bhandary A, Prabhu GA, Rajinikanth V, Thanaraj KP, Satapathy SC, Robbins DE, Shasky C, Zhang Y, Tavares JMR, Raja NSM (2020) Deep-learning framework to detect lung abnormality–a study with chest x-ray and lung ct scan images. Pattern Recog Lett 129:271–278

    Article  Google Scholar 

  7. Bissoto A, Perez F, Ribeiro V, Fornaciali M, Avila S, Valle E (2018) Deep-learning ensembles for skin-lesion segmentation, analysis, classification: Recod titans at isic challenge 2018. arXiv:1808.08480

  8. Brinker TJ, Hekler A, Enk AH, Berking C, Haferkamp S, Hauschild A, Weichenthal M, Klode J, Schadendorf D, Holland-Letz T, et al. (2019) Deep neural networks are superior to dermatologists in melanoma image classification. Eur J Cancer 119:11–17

    Article  Google Scholar 

  9. Chu J, Guo Z, Leng L (2018) Object detection based on multi-layer convolution feature fusion and online hard example mining. IEEE Access 6:19959–19967

    Article  Google Scholar 

  10. Ciompi F, Chung K, Van Riel SJ, Setio AAA, Gerke PK, Jacobs C, Scholten ET, Schaefer-Prokop C, Wille MM, Marchiano A, et al. (2017) Towards automatic pulmonary nodule management in lung cancer screening with deep learning. Sci Rep 7:46479

    Article  Google Scholar 

  11. Codella N, Rotemberg V, Tschandl P, Celebi ME, Dusza S, Gutman D, Helba B, Kalloo A, Liopyris K, Marchetti M, et al. (2019) Skin lesion analysis toward melanoma detection 2018: A challenge hosted by the international skin imaging collaboration (isic). arXiv:1902.03368

  12. Esteva A, Kuprel B, Novoa RA, Ko J, Swetter SM, Blau HM, Thrun S (2017) Dermatologist-level classification of skin cancer with deep neural networks. Nature 542:115–118

    Article  Google Scholar 

  13. Esteva A, Kuprel B, Novoa RA, Ko J, Swetter SM, Blau HM, Thrun S (2018) Man against machine: diagnostic performance of a deep learning convolutional neural network for dermoscopic melanoma recognition in comparison to 58 dermatologists. Ann Oncol 29:1836–1842

    Article  Google Scholar 

  14. Fan C, Peng Y, Peng S, Zhang H, Wu Y, Kwong S (2021) Detection of train driver fatigue and distraction based on forehead eeg: a time-series ensemble learning method. IEEE Transactions on Intelligent Transportation Systems :1–11

  15. Fan DP, Zhou T, Ji GP, Zhou Y, Chen G, Fu H, Shen J, Shao L (2020) Inf-net: automatic covid-19 lung infection segmentation from ct images. IEEE Trans Med Imaging 39:2626–2637

    Article  Google Scholar 

  16. Fu CY, Shvets M, Berg AC (2019) Retinamask: Learning to predict masks improves state-of-the-art single-shot detection for free. arXiv:1901.03353

  17. Ghiasi G, Fowlkes CC (2016) Laplacian pyramid reconstruction and refinement for semantic segmentation. In: European conference on computer vision, Springer, pp 519–534

  18. Girshick R (2015) Fast r-cnn. In: Proceedings of the IEEE international conference on computer vision, pp 1440–1448

  19. Gulshan V, Peng L, Coram M, Stumpe MC, Wu D, Narayanaswamy A, Venugopalan S, Widner K, Madams T, Cuadros J, et al. (2016) Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. J Am Med Assoc 316:2402–2410

    Article  Google Scholar 

  20. Hardie RC, Ali R, De Silva MS, Kebede TM (2018) Skin lesion segmentation and classification for isic 2018 using traditional classifiers with hand-crafted features. arXiv:1807.07001

  21. He K, Gkioxari G, Dollár P, Girshick R (2017) Mask r-cnn. In: Proceedings of the IEEE international conference on computer vision

  22. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

  23. Johnson JW (2018) Adapting mask-rcnn for automatic nucleus segmentation. arXiv:1805.00500

  24. Kamnitsas K, Ledig C, Newcombe VF, Simpson JP, Kane A, Menon DK, Rueckert D, Glocker B (2017) Efficient multi-scale 3d cnn with fully connected crf for accurate brain lesion segmentation. Med Image Anal 36:61–78

    Article  Google Scholar 

  25. Leng L, Yang Z, Kim C, Zhang Y (2020) A light-weight practical framework for feces detection and trait recognition. Sensors 20:2644

    Article  Google Scholar 

  26. Lin TY, Dollár P, Girshick R, He K, Hariharan B, Belongie S (2017a) Feature pyramid networks for object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2117–2125

  27. Lin TY, Goyal P, Girshick R, He K, Dollár P (2017b) Focal loss for dense object detection. In: Proceedings of the IEEE international conference on computer vision, pp 2980–2988

  28. Lin TY, Maire M, Belongie S, Hays J, Perona P, Ramanan D, Dollár P, Zitnick CL (2014) Microsoft coco: Common objects in context. In: European conference on computer vision, pp 750–755

  29. Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3431–3440

  30. Mendonça T, Ferreira PM, Marques JS, Marcal AR, Rozeira J (2013) Ph 2-a dermoscopic image database for research and benchmarking. In: 2013 35th annual international conference of the IEEE engineering in medicine and biology society (EMBC), IEEE, pp 5437–5440

  31. Miller M, Ackerman AB (1992) How accurate are dermatologists in the diagnosis of melanoma? degree of accuracy and implications. Archives of dermatology 128:559–560

    Article  Google Scholar 

  32. Naqvi S, Miller S, Garibaldi JM (2014) A general type-ii similarity based model for breast cancer grading with ftir spectral data. In: 2014 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), IEEE, pp 834–841

  33. Nida N, Irtaza A, Javed A, Yousaf MH, Mahmood MT (2019) Melanoma lesion detection and segmentation using deep region based convolutional neural network and fuzzy c-means clustering. Int J Med Inf 124:37–48

    Article  Google Scholar 

  34. Ninh QC, Tran TT, Tran TT, Tran TAX, Pham VT (2019) Skin lesion segmentation based on modification of segnet neural networks. In: 2019 6th NAFOSTED conference on information and computer science (NICS), IEEE, pp 575–578

  35. Ren S, He K, Girshick R, Sun J (2015) Faster r-cnn: Towards real-time object detection with region proposal networks. In: Advances in neural information processing systems, pp 91–99

  36. Ronneberger O, Fischer P, Brox T (2015) U-net: Convolutional networks for biomedical image segmentation. In: International Conference on Medical image computing and computer-assisted intervention

  37. Shahin AH, Amer K, Elattar MA (2019) Deep convolutional encoder-decoders with aggregated multi-resolution skip connections for skin lesion segmentation. arXiv:1901.09197

  38. Tan X, Xu K, Cao Y, Zhang Y, Ma L, Lau RWH (2021) Night-time scene parsing with a large real dataset. IEEE Trans Image Process 30:9085–9098

    Article  Google Scholar 

  39. Tschandl P, Rosendahl C, Kittler H (2018) The ham10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions. Sci Data 5:180–161

    Article  Google Scholar 

  40. Tschandl P, Sinz C, Kittler H (2019) Domain-specific classification-pretrained fully convolutional network encoders for skin lesion segmentation. Comput Biol Med 104:111–116

    Article  Google Scholar 

  41. Vesal S, Patil SM, Ravikumar N, Maier AK (2018) A multi-task framework for skin lesion detection and segmentation. In: OR 2.0 context-aware operating theaters, computer assisted robotic endoscopy, clinical image-based procedures, and skin image analysis, pp 285–293

  42. Wang Y, Chen Y, Yang N, Zheng L, Dey N, Ashour AS, Rajinikanth V, Tavares JMR, Shi F (2019) Classification of mice hepatic granuloma microscopic images based on a deep convolutional neural network. Appl Soft Comput 74:40–50

    Article  Google Scholar 

  43. Wolterink JM, Leiner T, de Vos BD, van Hamersvelt RW, Viergever MA, Išgum I (2016) Automatic coronary artery calcium scoring in cardiac ct angiography using paired convolutional neural networks. Med Image Anal 34:123–136

    Article  Google Scholar 

  44. Wyant T (2021) Melanoma survival rates: Melanoma survival statistics. https://www.cancer.org/cancer/melanoma-skin-cancer/detection-diagnosis-staging/survival-rates-for-melanoma-skin-cancer-by-stage/https://www.cancer.org/cancer/melanoma-skin-cancer/detection-diagnosis-staging/survival-rates-for-melanoma-skin-cancer-by-stage/. Accessed 20 Jan 2019

  45. Xue Y, Xu T, Huang X (2018) Adversarial learning with multi-scale loss for skin lesion segmentation. In: 2018 IEEE 15th international symposium on biomedical imaging (ISBI 2018), pp 859–863

  46. Xue Y, Xu T, Zhang H, Long R, Huang X (2017) Segan: adversarial network with multi-scale l_1 loss for medical image segmentation. arXiv:1706.01805

  47. Yang R, Yu Y (2021) Artificial convolutional neural network in object detection and semantic segmentation for medical imaging analysis. Front Oncol 11:573

    Google Scholar 

  48. Yang Z, Leng L, Kim BG (2019) Stoolnet for color classification of stool medical images. Electronics 8:1464

    Article  Google Scholar 

  49. Yuan Y, Lo YC (2017) Improving dermoscopic image segmentation with enhanced convolutional-deconvolutional networks. IEEE J Biomed Health Inform 23:519–526

    Article  Google Scholar 

  50. Zhang Y, Chu J, Leng L, Miao J (2020) Mask-refined r-cnn: A network for refining object details in instance segmentation. Sensors 20:1010

    Article  Google Scholar 

  51. Zheng Q, Yang M, Yang J, Zhang Q, Zhang X (2018) Improvement of generalization ability of deep cnn via implicit regularization in two-stage training process. IEEE Access 6:15844–15869

    Article  Google Scholar 

  52. Zheng X, Tan X, Zhou J, Ma L, Lau RWH (2021) Weakly-supervised saliency detection via salient object subitizing. IEEE Trans Circ Syst Video Technol 31:4370–4380

    Article  Google Scholar 

  53. Zuiderveld K (1994) Contrast limited adaptive histogram equalization. In: Graphics gems IV. Academic Press Professional, Inc., United States, pp 474–485

Download references

Acknowledgments

We would thank the anonymous reviewers for their constructive suggestions and insightful comments. This work is partially supported by Shanghai Jiao Tong University (ZH2018ZDA25), Shanghai Municipal Science, Technology Major Project (2021SHZDZX0102), and Shanghai Science and Technology Commission (21511101200).

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, Shanghai Jiao Tong University, Shanghai, China

    Noor Ahmed, Xin Tan & Lizhuang Ma

  2. MoE Key Lab of Artificial Intelligence, Shanghai Jiao Tong University, Shanghai, China

    Noor Ahmed, Xin Tan & Lizhuang Ma

  3. School of Computer Science Technology, East China Normal University, Shanghai, China

    Lizhuang Ma

Authors
  1. Noor Ahmed
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xin Tan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Lizhuang Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Noor Ahmed.

Ethics declarations

Conflict of Interests/Competing Interests

The authors declare that they have no conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ahmed, N., Tan, X. & Ma, L. A new method proposed to Melanoma-skin cancer lesion detection and segmentation based on hybrid convolutional neural network. Multimed Tools Appl 82, 11873–11896 (2023). https://doi.org/10.1007/s11042-022-13618-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-13618-0

Keywords

Search

Navigation