Webly Supervised Learning for Skin Lesion Classification

  • Fernando NavarroEmail author
  • Sailesh Conjeti
  • Federico Tombari
  • Nassir Navab
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11071)


Within medical imaging, manual curation of sufficient well-labeled samples is cost, time and scale-prohibitive. To improve the representativeness of the training dataset, for the first time, we present an approach to utilize large amounts of freely available web data through web-crawling. To handle noise and weak nature of web annotations, we propose a two-step transfer learning based training process with a robust loss function, termed as Webly Supervised Learning (WSL) to train deep models for the task. We also leverage search by image to improve the search specificity of our web-crawling and reduce cross-domain noise. Within WSL, we explicitly model the noise structure between classes and incorporate it to selectively distill knowledge from the web data during model training. To demonstrate improved performance due to WSL, we benchmarked on a publicly available 10-class fine-grained skin lesion classification dataset and report a significant improvement of top-1 classification accuracy from 71.25% to 80.53% due to the incorporation of web-supervision.



The authors gratefully acknowledge CONACYT for the financial support.


  1. 1.
    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: CVPR (2016)Google Scholar
  2. 2.
    Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. In: CVPR (2017)Google Scholar
  3. 3.
    Szegedy, C., Ioffe, S., Vanhoucke, V., Alemi, A.A.: Inception-v4, inception-resnet and the impact of residual connections on learning. In: AAAI (2017)Google Scholar
  4. 4.
  5. 5.
    Yu, L., Chen, H., Dou, Q., Qin, J., Heng, P.A.: Automated melanoma recognition in dermoscopy images via very deep residual networks. IEEE TMI 36, 994–1004 (2017)Google Scholar
  6. 6.
    Matsunaga, K., Hamada, A., Minagawa, A., Koga, H.: Image classification of melanoma, nevus and seborrheic keratosis by deep neural network ensemble. arXiv:1703.03108 (2017)
  7. 7.
    Codella, N.C., et al.: Skin lesion analysis toward melanoma detection. arXiv:1710.05006 (2017)
  8. 8.
    Esteva, A., et al.: Dermatologist-level classification of skin cancer with deep neural networks. Nature 542, 115 (2017)CrossRefGoogle Scholar
  9. 9.
    Krause, J., et al.: The unreasonable effectiveness of noisy data for fine-grained recognition. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9907, pp. 301–320. Springer, Cham (2016). Scholar
  10. 10.
    Massouh, N., Babiloni, F., Tommasi, T., Young, J., Hawes, N., Caputo, B.: Learning deep visual object models from noisy web data: how to make it work. arXiv:1702.08513 (2017)
  11. 11.
    Xiao, T., Xia, T., Yang, Y., Huang, C., Wang, X.: Learning from massive noisy labeled data for image classification. In: CVPR (2015)Google Scholar
  12. 12.
    Vahdat, A.: Toward robustness against label noise in training deep discriminative neural networks. In: NIPS (2017)Google Scholar
  13. 13.
    Sukhbaatar, S., Fergus, R.: Learning from noisy labels with deep neural networks. arXiv:1406.2080 (2014). 2(3), 4
  14. 14.
    Patrini, G., Rozza, A., Menon, A., Nock, R., Qu, L.: Making neural networks robust to label noise: a loss correction approach. arXiv:1609.03683 (2016)
  15. 15.
    Chen, X., Gupta, A.: Webly supervised learning of convolutional networks. In: ICCV (2015)Google Scholar
  16. 16.
    Oquab, M., Bottou, L., Laptev, I., Sivic, J.: Is object localization for free?-weakly-supervised learning with convolutional neural networks. In: CVPR (2015)Google Scholar
  17. 17.
    Ballerini, L., Fisher, R.B., Aldridge, B., Rees, J.: A color and texture based hierarchical K-NN approach to the classification of non-melanoma skin lesions. In: Celebi, M., Schaefer, G. (eds.) Color Medical Image Analysis, pp. 63–86. Springer, Dordrecht (2013). Scholar
  18. 18.
    Abadi, M., et al.: TensorFlow: large-scale machine learning on heterogeneous distributed systems. CoRR abs/1603.04467 (2016)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Fernando Navarro
    • 1
    Email author
  • Sailesh Conjeti
    • 1
    • 2
  • Federico Tombari
    • 1
  • Nassir Navab
    • 1
    • 3
  1. 1.Computer Aided Medical ProceduresTechnische Universität MünchenMunichGermany
  2. 2.German Center for Neurodegenrative Diseases (DZNE)BonnGermany
  3. 3.Computer Aided Medical ProceduresJohns Hopkins UniversityBaltimoreUSA

Personalised recommendations