Abstract
Nowadays, skin cancer has become a common disease and is growing worldwide at an increasing rate. Its manual examination by dermatologists demands significant time and cost in terms of instruments. Also, practical diagnosis demands experienced and skilled dermatologists. These challenges show the impracticality of manual diagnosis over the increased rate of skin cancer patients and thus demand robust end-to-end computer-aided diagnosis (CAD) methods. This paper proposes a deep learning-based skin lesion classification approach that utilizes the visual attention-based mechanism over Convolutional Neural Networks (CNNs) to improve visual context. We use the information from skin lesion images and patient demographics to enhance visual attention, which further improves classification. The proposed method accurately classifies deadly melanoma skin cancer for the PAD-UFES-20 dataset, an essential but challenging task. Our proposed approach has been evaluated over multimodel data, i.e., clinical and dermoscopic images, using two publicly available datasets named PAD-UFES-20 and ISIC-2019. During experimentation, our approach surpasses the available state-of-the-art techniques over five commonly used Convolutional Neural Networks (CNNs) architectures which validate its generalizability and applicability in different scenarios. Our approach achieved efficient performance for small datasets like PAD-UFES-20 using a lightweight model (MobileNet), making it suitable for the CAD system. The effectiveness of our method has been shown by various quantitative and qualitative measures, which demonstrate its efficacy in addressing challenging lesion diagnoses. Our source code is publicly available to reproduce the work.
A. Pundhir, A. Agarwal and S. DadhichāEqual Contribution.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Haenssle, H.A., et al.: Man against machine: diagnostic performance of a deep learning convolutional neural network for dermoscopic melanoma recognition in comparison to 58 dermatologists. Ann. Oncol. 29(8), 1836ā1842 (2018)
Liu, Y., et al.: A deep learning system for differential diagnosis of skin diseases. Nat. Med. 26(6), 900ā908 (2020)
Pacheco, A.G., et al.: PAD-UFES-20: a skin lesion dataset composed of patient data and clinical images collected from smartphones. Data Brief 32, 106221 (2020)
Arroyo, R., Alcantarilla, P.F., Bergasa, L.M., Romera, E.: Fusion and binarization of CNN features for robust topological localization across seasons. In: 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 4656ā4663. IEEE (2016)
Atrey, P.K., Hossain, M.A., El Saddik, A., Kankanhalli, M.S.: Multimodal fusion for multimedia analysis: a survey. Multimed. Syst. 16(6), 345ā379 (2010)
Brinker, T.J., et al.: Deep learning outperformed 136 of 157 dermatologists in a head-to-head dermoscopic melanoma image classification task. Eur. J. Cancer 113, 47ā54 (2019)
Brinker, T.J., et al.: Skin cancer classification using convolutional neural networks: systematic review. J. Med. Internet Res. 20(10), e11936 (2018)
Celebi, M.E., Codella, N., Halpern, A.: Dermoscopy image analysis: overview and future directions. IEEE J. Biomed. Health Inform. 23(2), 474ā478 (2019)
Codella, N.C., et al.: Skin lesion analysis toward melanoma detection: a challenge at the 2017 international symposium on biomedical imaging (ISBI), hosted by the international skin imaging collaboration (ISIC). In: 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018), pp. 168ā172. IEEE (2018)
Codella, N.C., et al.: Deep learning ensembles for melanoma recognition in dermoscopy images. IBM J. Res. Dev. 61(4/5), 5ā1 (2017)
Combalia, M., et al.: BCN20000: dermoscopic lesions in the wild. arXiv preprint arXiv:1908.02288 (2019)
Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: ImageNet: a large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248ā255. IEEE (2009)
Esteva, A., et al.: Dermatologist-level classification of skin cancer with deep neural networks. Nature 542(7639), 115ā118 (2017)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770ā778 (2016)
Huang, G., Liu, Z., Van Der Maaten, L., Weinberger, K.Q.: Densely connected convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4700ā4708 (2017)
ISIC: Skin lesion analysis towards melanoma detection. International skin imaging collaboration (2019). https://www.isic-archive.com. Accessed 26 Feb 2022
Jetley, S., Lord, N.A., Lee, N., Torr, P.H.: Learn to pay attention. In: International Conference on Learning Representations (2018)
Kharazmi, P., Kalia, S., Lui, H., Wang, Z., Lee, T.: A feature fusion system for basal cell carcinoma detection through data-driven feature learning and patient profile. Skin Res. Technol. 24(2), 256ā264 (2018)
Li, W., Zhuang, J., Wang, R., Zhang, J., Zheng, W.S.: Fusing metadata and dermoscopy images for skin disease diagnosis. In: 2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI), pp. 1996ā2000. IEEE (2020)
Lin, T.Y., RoyChowdhury, A., Maji, S.: Bilinear CNN models for fine-grained visual recognition. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1449ā1457 (2015)
Liu, Y., Chen, X., Cheng, J., Peng, H.: A medical image fusion method based on convolutional neural networks. In: 2017 20th International Conference on Information Fusion (Fusion), pp. 1ā7. IEEE (2017)
Pacheco, A.G., Krohling, R.A.: Recent advances in deep learning applied to skin cancer detection. arXiv preprint arXiv:1912.03280 (2019)
Pacheco, A.G., Krohling, R.A.: The impact of patient clinical information on automated skin cancer detection. Comput. Biol. Med. 116, 103545 (2020)
Pacheco, A.G.C., Krohling, R.: An attention-based mechanism to combine images and metadata in deep learning models applied to skin cancer classification. IEEE J. Biomed. Health Inform. 25, 3554ā3563 (2021)
Pedregosa, F., et al.: Scikit-learn: machine learning in Python. J. Mach. Learn. Res. 12, 2825ā2830 (2011)
Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., Chen, L.C.: MobileNetV 2: inverted residuals and linear bottlenecks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4510ā4520 (2018)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)
Tan, M., Le, Q.: EfficientNet: rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105ā6114. PMLR (2019)
Tschandl, P., Rosendahl, C., Kittler, H.: The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions. Sci. Data 5(1), 1ā9 (2018)
WHO: Ultraviolet (UV) radiation and skin cancer (2022). https://www.who.int/news-room/questions-and-answers/item/radiation-ultraviolet-(uv)-radiation-and-skin-cancer. Accessed on 26.02.2022
Yu, Z., Jiang, X., Zhou, F., Qin, J., Ni, D., Chen, S., Lei, B., Wang, T.: Melanoma recognition in dermoscopy images via aggregated deep convolutional features. IEEE Trans. Biomed. Eng. 66(4), 1006ā1016 (2018)
Acknowledgment
This work was supported by Indian Institute of Technology Roorkee and University Grants Commission (UGC) INDIA with grant number: 190510040512.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
Ā© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Pundhir, A., Agarwal, A., Dadhich, S., Raman, B. (2022). Visually Aware Metadata-Guided Supervision forĀ Improved Skin Lesion Classification Using Deep Learning. In: Baxter, J.S.H., et al. Ethical and Philosophical Issues in Medical Imaging, Multimodal Learning and Fusion Across Scales for Clinical Decision Support, and Topological Data Analysis for Biomedical Imaging. EPIMI ML-CDS TDA4BiomedicalImaging 2022 2022 2022. Lecture Notes in Computer Science, vol 13755. Springer, Cham. https://doi.org/10.1007/978-3-031-23223-7_6
Download citation
DOI: https://doi.org/10.1007/978-3-031-23223-7_6
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-23222-0
Online ISBN: 978-3-031-23223-7
eBook Packages: Computer ScienceComputer Science (R0)