Exploring the Correlation Between Deep Learned and Clinical Features in Melanoma Detection

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 12722)


Despite the recent success of deep learning methods in automated medical image analysis tasks, their acceptance in the medical community is still questionable due to the lack of explainability in their decision-making process. The highly opaque feature learning process of deep models makes it difficult to rationalize their behavior and exploit the potential bottlenecks. Hence it is crucial to verify whether these deep features correlate with the clinical features, and whether their decision-making process can be backed by conventional medical knowledge. In this work, we attempt to bridge this gap by closely examining how the raw pixel-based neural architectures associate with the clinical feature based learning algorithms at both the decision level as well as feature level. We have adopted skin lesion classification as the test case and present the insight obtained in this pilot study. Three broad kinds of raw pixel-based learning algorithms based on convolution, spatial self-attention and attention as activation were analyzed and compared with the ABCD skin lesion clinical features based learning algorithms, with qualitative and quantitative interpretations.


Explainable artificial intelligence Melanoma classification Digital dermatoscopy Attention mechanisms Deep machine learning 


  1. 1.
    LeCun, Y., Bengio, Y.: Convolutional networks for images, speech, and time series. In: The Handbook of Brain Theory and Neural Networks (1995)Google Scholar
  2. 2.
    Jensen, D., Elewski, B.E.: The ABCDEF rule: combining the ABCDE rule and the ugly duckling sign in an effort to improve patient self-screening examinations. J. Clin. Aesthetic Dermatol. 8(2), 15 (2015)Google Scholar
  3. 3.
    Van Molle, P., De Strooper, M., Verbelen, T., Vankeirsbilck, B., Simoens, P., Dhoedt, B.: Visualizing convolutional neural networks to improve decision support for skin lesion classification. In: Stoyanov, D., et al. (eds.) MLCN/DLF/IMIMIC -2018. LNCS, vol. 11038, pp. 115–123. Springer, Cham (2018). Scholar
  4. 4.
    Young, K., Booth, G., Simpson, B., Dutton, R., Shrapnel, S.: Deep neural network or dermatologist? In: Suzuki, K., et al. (eds.) ML-CDS/IMIMIC -2019. LNCS, vol. 11797, pp. 48–55. Springer, Cham (2019). Scholar
  5. 5.
    Selvaraju, R.R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., Batra, D.: Grad-cam: visual explanations from deep networks via gradient-based localization. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV), pp. 618–626 (2017)Google Scholar
  6. 6.
    Lundberg, S.M., Lee, S.-I.: A unified approach to interpreting model predictions. In: Advances in Neural Information Processing Systems, pp. 4765–4774 (2017)Google Scholar
  7. 7.
    Aggarwal, A., Das, N., Sreedevi, I.: Attention-guided deep convolutional neural networks for skin cancer classification. In: IEEE International Conference on Image Processing Theory, Tools and Applications, pp. 1–6 (2019)Google Scholar
  8. 8.
    Zhang, J., Xie, Y., Xia, Y., Shen, C.: Attention residual learning for skin lesion classification. IEEE Trans. Med. Imaging 38(9), 2092–2103 (2019)CrossRefGoogle Scholar
  9. 9.
    Tschandl, P., Rosendahl, C., Kittler, H.: The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions. Sci. Data 5(1), 1–9 (2018)Google Scholar
  10. 10.
    Pizer, S.M., et al.: Adaptive histogram equalization and its variations. Comput. Vis. Graphics Image Process. 39(3), 355–368 (1987)Google Scholar
  11. 11.
    Otsu, N.: A threshold selection method from gray-level histograms. IEEE Trans. Syst. Man Cybern. 9(1), 62–66 (1979)MathSciNetCrossRefGoogle Scholar
  12. 12.
    Zaqout, I.: Diagnosis of skin lesions based on dermoscopic images using image processing techniques. Pattern Recognition-Selected Methods and Applications Intech Open (2019)Google Scholar
  13. 13.
    Amaliah, B., Fatichah, C., Widyanto, M.R.: ABCD feature extraction of image dermatoscopic based on morphology analysis for melanoma skin cancer diagnosis. Jurnal Ilmu Komputer dan Informasi 3(2), 82–90 (2010)Google Scholar
  14. 14.
    Breiman, L.: Random forests. Mach. Learn. 45(1), 5–32 (2001)CrossRefGoogle Scholar
  15. 15.
    Cortes, C., Vapnik, V.: Support-vector networks. Mach. Learn. 20(3), 273–297 (1995)zbMATHGoogle Scholar
  16. 16.
    Bergstra, J., Bardenet, R., Bengio, Y., Kégl, B.: Algorithms for hyper-parameter optimization. Advances in Neural Information Processing Systems (2011)Google Scholar
  17. 17.
    Lin, M., Chen, Q., Yan, S.: Network in network. arXiv:1312.4400 (2013)
  18. 18.
    Jetley, S., Lord, N.A., Lee, N., Torr, P.H.S.: Learn to pay attention. arXiv:1804.02391 (2018)
  19. 19.
    Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems (2017)Google Scholar
  20. 20.
    Ramachandran, P., Parmar, N., Vaswani, A., Bello, I., Levskaya, A., Shlens, J.: Stand-alone self-attention in vision models. arXiv:1906.05909 (2019)
  21. 21.
    Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., Torralba, A.: Learning deep features for discriminative localization. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2016)Google Scholar
  22. 22.
    Kingma, D., Ba, J.: Adam: a method for stochastic optimization. In: International Conference on Learning Representations (2014)Google Scholar
  23. 23.
    Chen, D., Li, J., Xu, K.: AReLU: attention-based rectified linear unit. arXiv:2006.13858 (2020)
  24. 24.
    Dai, Y., Oehmcke, S., Gieseke, F., Wu, Y., Barnard, K.: Attention as activation. arXiv:2007.07729 (2020)
  25. 25.
    Eelbode, T., et al.: Optimization for medical image segmentation: theory and practice when evaluating with Dice score or Jaccard index. IEEE Trans. Med. Imaging 39(11), 3679–3690 (2020)CrossRefGoogle Scholar
  26. 26.
    Nida, N., Irtaza, A., Javed, A., Yousaf, M.H., Mahmood, M.T.: Melanoma lesion detection and segmentation using deep region based convolutional neural network and fuzzy C-means clustering. Int. J. Med. Inform. 124, 37–48 (2019)CrossRefGoogle Scholar
  27. 27.
    Bisla, D., Choromanska, A., Berman, R.S., Stein, J.A., Polsky, D.: Towards automated melanoma detection with deep learning: data purification and augmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (2019)Google Scholar
  28. 28.
    Adekanmi, A.A., Viriri, S.: Deep learning-based system for automatic melanoma detection. IEEE Access 8, 7160–7172 (2019)Google Scholar
  29. 29.
    Adekanmi, A.A., Viriri, S.: Deep learning techniques for skin lesion analysis and melanoma cancer detection: a survey of state-of-the-art. Artif. Intell. Rev. 54(2), 811–841 (2021)CrossRefGoogle Scholar
  30. 30.
    Codella, N., et al.: Skin lesion analysis toward melanoma detection 2018: a challenge hosted by the international skin imaging collaboration (ISIC). arXiv:1902.03368 (2019)

Copyright information

© Springer Nature Switzerland AG 2021

Authors and Affiliations

  1. 1.CVPR UnitIndian Statistical InstituteKolkataIndia
  2. 2.National Institute of Technology (NIT)DurgapurIndia
  3. 3.National Institute of Technology (NIT)TiruchirappalliIndia
  4. 4.IBME/BDI, Dept. of Engineering ScienceUniversity of OxfordOxfordUK

Personalised recommendations