Abstract
Melanoma is one of the most dangerous skin cancers. The current melanoma segmentation is mainly based on FCNs (fully connected networks) and U-Net. Nevertheless, these two kinds of neural networks are prone to parameter redundancy, and the gradient of neural networks disappears that occurs when the neural network backpropagates as the neural network gets deeper, which will reduce the Jaccard index of the skin lesion image segmentation model. To solve the above problems and improve the survival rate of melanoma patients, an improved skin lesion segmentation model based on deformable 3D convolution and ResU-NeXt++ (D3DC- ResU-NeXt++) is proposed in this paper. The new modules in D3DC-ResU-NeXt++ can replace ordinary modules in the existing 2D convolutional neural networks (CNNs) that can be trained efficiently through standard backpropagation with high segmentation accuracy. In particular, we introduce a new data preprocessing method with dilation, crop operation, resizing, and hair removal (DCRH), which improves the Jaccard index of skin lesion image segmentation. Because rectified Adam (RAdam) does not easily fall into a local optimal solution and can converge quickly in segmentation model training, we also introduce RAdam as the training optimizer. The experiments show that our model has excellent performance on the segmentation of the ISIC2018 Task I dataset, and the Jaccard index achieves 86.84%. The proposed method improves the Jaccard index of segmentation of skin lesion images and can also assist dermatological doctors in determining and diagnosing the types of skin lesions and the boundary between lesions and normal skin, so as to improve the survival rate of skin cancer patients.
Graphical abstract
Similar content being viewed by others
References
Schadendorf D, van Akkooi ACJ, Berking C, Griewank KG, Gutzmer R, Hauschild A, Stang A, Roesch A, Ugurel S (2018) Melanoma. Lancet 392:971–984. https://doi.org/10.1016/s0140-6736(18)31559-9
Lucas RM, Yazar S, Young AR, Norval M, de Gruijl FR, Takizawa Y, Rhodes LE, Sinclair CA, Neale RE (2019) Human health in relation to exposure to solar ultraviolet radiation under changing stratospheric ozone and climate. Photochem Photobiol Sci 18:641–680. https://doi.org/10.1039/c8pp90060d
Hu L, Jin SF, Chen L, Wang YL (2018) Trends in the incidence and mortality of cutaneous melanoma in Hong Kong between 1983 and 2015. Int J Clin Exp Med 11:8259–8266
Fong ZV, Tanabe KK (2014) Comparison of melanoma guidelines in the USA, Canada, Europe, Australia and New Zealand: a critical appraisal and comprehensive review. Br J Dermatol 170:20–30. https://doi.org/10.1111/bjd.12687
Kasmi R, Mokrani K (2016) Classification of malignant melanoma and benign skin lesions: implementation of automatic ABCD rule. IET Image Process 10:448–455. https://doi.org/10.1049/iet-ipr.2015.0385
Celebi ME, Kingravi HA, Uddin B, Iyatomi H, Aslandogan YA, Stoecker WV, Moss RH (2007) A methodological approach to the classification of dermoscopy images. Comput Med Imaging Graph 31:362–373. https://doi.org/10.1016/j.compmedimag.2007.01.003
Wu J, Chen EZ, Rong R, Li X, Xu D, Jiang H (2019) Skin Lesion Segmentation with C-UNet. In: 2019 41st Annual International Conference of the IEEE. Eng Med Bio Soc:2785–2788. https://doi.org/10.1109/EMBC.2019.8857773
Yang D, Salciccioli J, Marshall D, Sheri A, Shalhoub J (2020) Trends in malignant melanoma mortality in 31 countries from 1985 to 2015. Br J Dermatol 183:1056–1064. https://doi.org/10.1111/bjd.19010
Ma L, Shuai R, Ran X, Liu W, Ye C (2020) Combining DC-GAN with ResNet for blood cell image classification. Med Biol Eng Comput 58:1–14. https://doi.org/10.1007/s11517-020-02163-3
Young K, Booth G, Simpson B, Dutton R, Shrapnel S (2019) Deep neural network or dermatologist? In: Interpretability of machine intelligence in medical image computing and multimodal learning for clinical decision support. Springer, pp 48-55. https://doi.org/10.1007/978-3-030-33850-3_6
Chan S, Reddy V, Myers B, Thibodeaux Q, Brownstone N, Liao W (2020) Machine learning in dermatology: current applications, opportunities, and limitations. Dermatol Ther 10:1–22. https://doi.org/10.1007/s13555-020-00372-0
Mohammed ZF, Abdulla AA (2021) An efficient CAD system for ALL cell identification from microscopic blood images. Multimed Tools Appl 80:6355–6368
Mohammed ZF, Abdulla AA (2020) Thresholding-based white blood cells segmentation from microscopic blood images. J Sci Technol 4:9–17
Jain S, Pise N (2015) Computer aided melanoma skin cancer detection using image processing. Proc Comput Sci 48:735–740. https://doi.org/10.1016/j.procs.2015.04.209
Ronneberger O, Fischer P, Brox T (2015) U-net: convolutional networks for biomedical image segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention:234–241. https://doi.org/10.1007/978-3-319-24574-4_28
Zhang N, Cai Y-X, Wang Y-Y, Tian Y-T, Wang X-L, Badami B (2020) Skin cancer diagnosis based on optimized convolutional neural network. Artif Intell Med 102:101756. https://doi.org/10.1016/j.artmed.2019.101756
Dai J, Qi H, Xiong Y, Li Y, Zhang G, Hu H, Wei Y (2017) Deformable convolutional networks. In: Proceedings of the IEEE international conference on computer vision. 2017:7645–773
Oh CM, Cho H, Won YJ, Kong HJ, Roh YH, Jeong KH, Jung KW (2018) Nationwide trends in the incidence of melanoma and non-melanoma skin cancers from 1999 to 2014 in South Korea. Cancer Res Treat 50:729–737. https://doi.org/10.4143/crt.2017.166
Gardner LJ, Strunck JL, Wu YP, Grossman D (2019) Current controversies in early-stage melanoma questions on incidence, screening, and histologic regression. J Am Acad Dermatol 80:1–12. https://doi.org/10.1016/j.jaad.2018.03.053
Siegel RL, Miller KD, Jemal A (2019) Cancer statistics, 2019. CA Cancer J Clin 69:7–34. https://doi.org/10.3322/caac.21551
Feigelson HS, Powers JD, Kumar M, Carroll NM, Pathy A, Ritzwoller DP (2019) Melanoma incidence, recurrence, and mortality in an integrated healthcare system: a retrospective cohort study. Cancer Med 8:4508–4516. https://doi.org/10.1002/cam4.2252
Duarte AF, Sousa-Pinto B, Freitas A, Delgado L, Costa-Pereira A, Correia O (2018) Skin cancer healthcare impact: a nation-wide assessment of an administrative database. Cancer Epidemiol 56:154–160. https://doi.org/10.1016/j.canep.2018.08.004
Duan Y, Wang J, Hu M, Zhou M, Li Q, Sun L, Qiu S, Wang Y (2019) Leukocyte classification based on spatial and spectral features of microscopic hyperspectral images. Opt Laser Technol 112:530–538. https://doi.org/10.1016/j.optlastec.2018.11.057
Liu L, Jiang H, He P, Chen W, Liu X, Gao J, Han J (2019) On the variance of the adaptive learning rate and beyond. arXiv. arXiv:190803265
Keskar NS, Socher R (2017) Improving generalization performance by switching from adam to sgd. arXiv. arXiv:171207628
Hua K-L, Hsu C-H, Hidayati SC, Cheng W-H, Chen Y-J (2015) Computer-aided classification of lung nodules on computed tomography images via deep learning technique. OncoTargets Ther:8. https://doi.org/10.2147/OTT.S80733
Mahbod A, Tschandl P, Langs G, Ecker R, Ellinger I (2020) The effects of skin lesion segmentation on the performance of dermatoscopic image classification. Comput Methods Prog Biomed 197:105725
Ballerini L, Fisher RB, Aldridge B, Rees J (2013) A color and texture based hierarchical K-NN approach to the classification of non-melanoma skin lesions. In: Color medical image analysis. Springer, pp 63-86. https://doi.org/10.1007/978-94-007-5389-1_4
Mishra NK, Celebi ME (2016) An overview of melanoma detection in dermoscopy images using image processing and machine learning. arXiv. arXiv:160107843
Cheng Y, Swamisai R, Umbaugh SE, Moss RH, Stoecker WV, Teegala S, Srinivasan SK (2008) Skin lesion classification using relative color features. Skin Res Technol 14:53–64. https://doi.org/10.1111/j.1600-0846.2007.00261.x
Jha D, Smedsrud PH, Riegler MA, Johansen D, De Lange T, Halvorsen P, Johansen HD (2019) ResUNet++: an advanced architecture for medical image segmentation. In: 2019 IEEE International Symposium on Multimedia (ISM), pp 225–2255. https://doi.org/10.1109/ISM46123.2019.00049
Huang X (2017) Belongie S Arbitrary style transfer in real-time with adaptive instance normalization. In: Proceedings of the IEEE International Conference on Computer Vision, pp 1501–1510
Jing Y, Liu X, Ding Y, Wang X, Wen S (2020) Dynamic instance normalization for arbitrary style transfer. Proceedings of the AAAI Conference on Artificial Intelligence 34:4369–4376
Alom MZ, Hasan M, Yakopcic C, Taha TM, Asari VK (2018) Recurrent residual convolutional neural network based on u-net (r2u-net) for medical image segmentation. arXiv:180206955
Zhang L, Yang G, Ye X (2019) Automatic skin lesion segmentation by coupling deep fully convolutional networks and shallow network with textons. J Med Imaging 6:024001. https://doi.org/10.1117/1.JMI.6.2.024001
Ali A-R, Li J, O’Shea SJ, Yang G, Trappenberg T, Ye X A (2019) Deep learning based approach to skin lesion border extraction with a novel edge detector in dermoscopy images. In: 2019 International Joint Conference on Neural Networks (IJCNN). IEEE, pp 1-7 https://doi.org/10.1109/IJCNN.2019.8852134
Yu L, Chen H, Dou Q, Qin J, Heng P-A (2016) Automated melanoma recognition in dermoscopy images via very deep residual networks. IEEE Trans Med Imaging 36:994–1004. https://doi.org/10.1109/TMI.2016.2642839
Zhang Z, Liu Q, Wang Y (2017) Road extraction by deep residual U-Net. IEEE Geoence and. Remote Sens Lett:1–5
He K, Zhang X, Ren S (2016) Sun J Deep residual learning for image recognition. In: IEEE Conference on Computer Vision & Pattern Recognition
Zhou Z, Siddiquee MMR, Tajbakhsh N, Liang J (2018) Unet++: A nested u-net architecture for medical image segmentation. In: Deep learning in medical image analysis and multimodal learning for clinical decision support. Springer, pp 3-11. https://doi.org/10.1007/978-3-030-00889-5_1
Xie S, Girshick R, Dollár P, Tu Z, He K (2016) Aggregated residual transformations for deep neural networks. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016
Jie H, Li S, Samuel A, Gang S, Enhua W (2019) Squeeze-and-excitation networks. IEEE Trans Pattern Anal Mach Intell
Jia X, De Brabandere B, Tuytelaars T, Van Gool L (2016) Dynamic filter networks[J]. Advances in neural information processing systems 29:667–675
Tschandl P, Rosendahl C, Kittler H (2018) The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions. Sci Data 5:180161. https://doi.org/10.1038/sdata.2018.161
Codella N, Rotemberg V, Tschandl P, Celebi ME, Dusza S, Gutman D, Helba B, Kalloo A, Liopyris K, Marchetti M (2019) Skin lesion analysis toward melanoma detection 2018: a challenge hosted by the international skin imaging collaboration. arXiv:190203368
Telea A (2004) An image inpainting technique based on the fast marching method. J Graph Tools 9:23–34. https://doi.org/10.1080/10867651.2004.10487596
Wang G, Wang Y, Li H, Chen X, Lu H, Ma Y, Peng C, Wang Y, Tang L (2014) Morphological background detection and illumination normalization of text image with poor lighting. PLoS ONE 9:e110991. https://doi.org/10.1371/journal.pone.0110991
Thapar S, Garg S (2012) Study and implementation of various morphology based image contrast enhancement techniques. CSE Department, IT Department, pp 2–5
Arora R, Raman B, Nayyar K, Awasthi R Automated skin lesion segmentation using attention-based deep convolutional neural network. Biomed Signal Process Control 65:102358
Zhu L, Feng S, Zhu W, Chen X (2020) ASNet: An adaptive scale network for skin lesion segmentation in dermoscopy images. In: Medical Imaging 2020: Biomedical Applications in Molecular, Structural, and Functional Imaging. International Society for Optics and Photonics 113170W https://doi.org/10.1117/12.2549178
Jafari M, Auer D, Francis S, Garibaldi J, Chen X (2020) DRU-Net: an efficient deep convolutional neural network for medical image segmentation. In: 2020 IEEE 17th International Symposium on Biomedical Imaging, pp 1144-1148. https://doi.org/10.1109/ISBI45749.2020.9098391
Saini S, Gupta D, Tiwari AK (2020) Detector-SegMentor network for skin lesion localization and segmentation. arXiv 200506550
Ibtehaz N, Rahman MS (2020) MultiResUNet: rethinking the U-Net architecture for multimodal biomedical image segmentation. Neural Netw 121:74–87. https://doi.org/10.1016/j.neunet.2019.08.025
Al Nazi Z, Abir TA (2020) Automatic skin lesion segmentation and melanoma detection: transfer learning approach with U-Net and DCNN-SVM. In: Proceedings of International Joint Conference on Computational Intelligence. Springer, pp 371-381. https://doi.org/10.1007/978-981-13-7564-4_32
Jha D, Riegler MA, Johansen D, Halvorsen P, Johansen HD (2020) DoubleU-Net: a deep convolutional neural network for medical image segmentation. arXiv 200604868
Funding
This work was supported in part by the National Natural Science Foundation of China no. 61701222.
Author information
Authors and Affiliations
Contributions
C.Z. contributed to the writing and editing of the paper and the operation and editing of the code. R.S. (corresponding author) contributed to technological guidance, provided experimental equipment and major financial support. L.M. contributed technical support and guidance for the paper concept. W.L. contributed to the technical guidance and as a consultant in the medical consultant field, D.H. contributed part of the paper correction, and M.W. contributed to the direction of the paper and the funding of support and related work.
Corresponding author
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Zhao, C., Shuai, R., Ma, L. et al. Segmentation of dermoscopy images based on deformable 3D convolution and ResU-NeXt + +. Med Biol Eng Comput 59, 1815–1832 (2021). https://doi.org/10.1007/s11517-021-02397-9
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11517-021-02397-9