Deep Learning, Sparse Coding, and SVM for Melanoma Recognition in Dermoscopy Images

  • Noel CodellaEmail author
  • Junjie Cai
  • Mani Abedini
  • Rahil Garnavi
  • Alan Halpern
  • John R. Smith
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9352)


This work presents an approach for melanoma recognition in dermoscopy images that combines deep learning, sparse coding, and support vector machine (SVM) learning algorithms. One of the beneficial aspects of the proposed approach is that unsupervised learning within the domain, and feature transfer from the domain of natural photographs, eliminates the need of annotated data in the target task to learn good features. The applied feature transfer also allows the system to draw analogies between observations in dermoscopic images and observations in the natural world, mimicking the process clinical experts themselves employ to describe patterns in skin lesions. To evaluate the methodology, performance is measured on a dataset obtained from the International Skin Imaging Collaboration, containing 2624 clinical cases of melanoma (334), atypical nevi (144), and benign lesions (2146). The approach is compared to the prior state-of-art method on this dataset. Two-fold cross-validation is performed 20 times for evaluation (40 total experiments), and two discrimination tasks are examined: 1) melanoma vs. all non-melanoma lesions, and 2) melanoma vs. atypical lesions only. The presented approach achieves an accuracy of 93.1% (94.9% sensitivity, and 92.8% specificity) for the first task, and 73.9% accuracy (73.8% sensitivity, and 74.3% specificity) for the second task. In comparison, prior state-of-art ensemble modeling approaches alone yield 91.2% accuracy (93.0% sensitivity, and 91.0% specificity) first the first task, and 71.5% accuracy (72.7% sensitivity, and 68.9% specificity) for the second. Differences in performance were statistically significant (p \(<\) 0.05), suggesting the proposed approach is an effective improvement over prior state-of-art.


Melanoma recognition Dermoscopy Dermatology Deep learning Sparse coding 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Cancer Facts & Figures 2014. American Cancer Society (2014)Google Scholar
  2. 2.
    Melanoma Research Gathers Momentum. The Lancet 385(9985), 2323Google Scholar
  3. 3.
    Oliveria, S.A., Selvam, N., Mehregan, D., Marchetti, M.A., Divan, H.A., Dasgeb, B., Halpern, A.C.: Biopsies of Nevi in Children and Adolescents in the United States, 2009 Through 2013. JAMA Dermatology, December 2014Google Scholar
  4. 4.
    Kittler, H., Pehamberger, H., Wolff, K., Binder, M.: Diagnostic accuracy of dermoscopy. The Lancet Oncology 3(3), 159–165 (2002)CrossRefGoogle Scholar
  5. 5.
    Abder-Rahman, A.A., Deserno, T.M.: A systematic review of automated melanoma detection in dermatoscopic images and its ground truth data. In: Proc. SPIE, Medical Imaging 2012: Image Perception, Observer Performance, and Technology Assessment, vol. 8318 (2012)Google Scholar
  6. 6.
    Braun, R.P., Rabinovitz, H.S., Oliviero, M., Kopf, A.W., Saurat, J.H.: Dermoscopy of pigmented skin lesions. J. Am. Acad. Dermatol. 52(1), 109–121 (2005)CrossRefGoogle Scholar
  7. 7.
    Carli, P., Quercioli, E., Sestini, S., Stante, M., Ricci, L., Brunasso, G., De Giorgi, V.: Pattern analysis, not simplified algorithms, is the most reliable method for teaching dermoscopy for melanoma diagnosis to residents in dermatology. Br. J. Dermatol. 148(5), 981–984 (2003)CrossRefGoogle Scholar
  8. 8.
    Rezze, G.G., Soares de Sá, B.C., Neves, R.I.: Dermoscopy: the pattern analysis. An. Bras. Dermatol. 3, 261–268 (2006)CrossRefGoogle Scholar
  9. 9.
    Aubusson, P.J., Harrison A.G., Ritchie S.M.: Metaphor and Analogy in Science and Education. Springer Science & Technology Education Library, vol. 30 (2006)Google Scholar
  10. 10.
    Gachon, J., et al.: First Prospective Study of the Recognition Process of Melanoma in Dermatological Practice. Arch. Dermatol. 141(4), 434–438 (2005)CrossRefGoogle Scholar
  11. 11.
    Garnavi, R., Aldeen, M., Bailey, J.: Computer-aided diagnosis of melanoma using border and wavelet-based texture analysis. IEEE Trans. Inf. Technol. Biomed. 16(6), 1239–1252 (2012)CrossRefGoogle Scholar
  12. 12.
    Ganster, H., Pinz, A., Röhrer, R., Wildling, E., Binder, M., Kittler, H.: Automated Melanoma Recognition. IEEE Transactions on Medical Imaging 20(3) (2001)CrossRefGoogle Scholar
  13. 13.
    Colot, O., Devinoy, R., Sombo, A., de Brucq, D.: A colour image processing method for melanoma detection. In: Wells, W.M., Colchester, A.C.F., Delp, S.L. (eds.) MICCAI 1998. LNCS, vol. 1496, p. 562. Springer, Heidelberg (1998) CrossRefGoogle Scholar
  14. 14.
    Madooei, A., Drew, M.S., Sadeghi, M., Stella Atkins, M.: Automatic Detection of Blue-White Veil by Discrete Colour Matching in Dermoscopy Images. Medical Image Computing and Computer-Assisted Intervention, 453–460 (2013)Google Scholar
  15. 15.
    Celebi, M.E., Iyatomi, H., Stoecker, W.V., Moss, R.H., Rabinovitz, H.S., Argenziano, G., Soyer, H.P.: Automatic detection of blue-white veil and related structures in dermoscopy images. Comput. Med. Imaging Graph. 32(8), 670–677 (2008)CrossRefGoogle Scholar
  16. 16.
    Mendonca, T., Ferreira, P.M., Marques, J.S., Marcal, A.R., Rozeira, J.: PH2 - a dermoscopic image database for research and benchmarking. In: Conf. Proc. IEEE Eng. Med. Biol. Soc., pp. 5437–5440 (2013)Google Scholar
  17. 17.
    Barata, C., Ruela, M., et al.: Two Systems for the Detection of Melanomas in Dermoscopy Images using Texture and Color Features. IEEE Systems Journal 99, 1–15 (2013)Google Scholar
  18. 18.
    Mairal, J., Bach, F., Ponce, J.: Sparse Modeling for Image and Vision Processing. Foundations and Trends in Computer Graphics and Vision 8(2/3), 85–283 (2014)CrossRefGoogle Scholar
  19. 19.
    International Skin Imaging Collaboration Website.
  20. 20.
    Abedini, M., Codella, N.C.F., Connell, J.H., Garnavi, R., Merler, M., Pankanti, S., Smith, J.R., Syeda-Mahmood, T.: A generalized framework for medical image classification and recognition. IBM Journal of Research and Development 59(2/3) (2015)CrossRefGoogle Scholar
  21. 21.
    Jia, Y., Shelhamer, E, Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., Darrell, T.: Caffe: Convolutional Architecture for Fast Feature Embedding (2014). arXiv preprint arXiv:1408.5093
  22. 22.
    Kender, J.R.: Separability and refinement of hierarchical semantic video labels and their ground truth. In: 2008 IEEE International Conference on Multimedia and Expo, pp. 673–676, 23 June 2008Google Scholar
  23. 23.
    Codella, N., Connell, J., Pankanti, S., Merler, M., Smith, J.R.: Automated medical image modality recognition by fusion of visual and text information. In: Golland, P., Hata, N., Barillot, C., Hornegger, J., Howe, R. (eds.) MICCAI 2014, Part II. LNCS, vol. 8674, pp. 487–495. Springer, Heidelberg (2014) Google Scholar
  24. 24.
    Zhu, C., Bichot, C., Chen, L.: Multi-scale color local binary patterns for visual object classes recognition. In: 20th IAPR International Conference on Pattern Recognition (ICPR), pp. 3065–3068. IEEE Press, New York (2010)Google Scholar
  25. 25.
    Yosinski, J., Clune, J., Bengio, Y., Lipson, H.: How transferable are features in deep neural networks? In: Advances in Neural Information Processing Systems, vol. 27, pp. 3320–3328 (2014)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Open Access This chapter is licensed under the terms of the Creative Commons Attribution-NonCommercial 2.5 International License (, which permits any noncommercial use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

Authors and Affiliations

  • Noel Codella
    • 1
    Email author
  • Junjie Cai
    • 1
  • Mani Abedini
    • 2
  • Rahil Garnavi
    • 2
  • Alan Halpern
    • 3
  • John R. Smith
    • 1
  1. 1.IBM T.J. Watson Research CenterYorktown HeightsUSA
  2. 2.IBM Australia Research LabsMelbourneAustralia
  3. 3.Memorial Sloan-Kettering Cancer CenterNew YorkUSA

Personalised recommendations