Skip to main content

End-to-End Ugly Duckling Sign Detection for Melanoma Identification with Transformers

  • 4540 Accesses

Part of the Lecture Notes in Computer Science book series (LNIP,volume 12907)


The concept of ugly ducklings was introduced in dermatology to improve the likelihood of detecting melanoma by comparing a suspicious lesion against its surrounding lesions. The ugly duckling sign suggests nevi in the same individual tend to resemble one another while malignant melanoma often deviates from this nevus pattern. Differentiating the ugly duckling sign was more discriminatory between malignant melanoma and other nevi than quantitatively assessing dermoscopic patterns. In this study, we propose a framework for modeling ugly duckling context in melanoma identification (called UDTR hereafter). To this end, we construct our model in three parts: Firstly, we extract multi-scale features using a deep neural network from lesions in the same individuals; Then, we learn lesion context by modeling the dependency among features of lesions using a transformer encoder; Finally, we design a two branch architecture for performing both patient-level prediction and lesion-level prediction concurrently. Also, we propose a group contrastive learning strategy to enforce a large margin between benign and malignant lesions in feature space for better contextual feature learning. We evaluate our method on ISIC 2020 dataset which consists of \(\sim \)30,000 images from \(\sim \)2,000 patients. Extensive experiments evidence the effectiveness of our approach and highlight the importance of detecting lesions with clues from surrounding lesions than that of only evaluating lesion in question.


  • Melanoma diagnosis
  • Ugly duckling sign
  • Deep learning

This is a preview of subscription content, access via your institution.

Buying options

USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-030-87234-2_17
  • Chapter length: 9 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
USD   89.00
Price excludes VAT (USA)
  • ISBN: 978-3-030-87234-2
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   119.99
Price excludes VAT (USA)
Fig. 1.
Fig. 2.
Fig. 3.
Fig. 4.


  1. 1.

    We repeatedly sample images to the required input number of our model for those patients with insufficient lesion images.


  1. Abnar, S., Zuidema, W.: Quantifying attention flow in transformers. arXiv preprint arXiv:2005.00928 (2020)

  2. Brinker, T.J., et al.: Deep learning outperformed 136 of 157 dermatologists in a head-to-head dermoscopic melanoma image classification task. Eur. J. Cancer 113, 47–54 (2019)

    CrossRef  Google Scholar 

  3. Esteva, A., et al.: Dermatologist-level classification of skin cancer with deep neural networks. Nature 542(7639), 115–118 (2017)

    CrossRef  Google Scholar 

  4. Gachon, J., et al.: First prospective study of the recognition process of melanoma in dermatological practice. Arch. Dermatol. 141(4), 434–438 (2005)

    CrossRef  Google Scholar 

  5. Ge, Z., Demyanov, S., Chakravorty, R., Bowling, A., Garnavi, R.: Skin disease recognition using deep saliency features and multimodal learning of dermoscopy and clinical images. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., Duchesne, S. (eds.) MICCAI 2017. LNCS, vol. 10435, pp. 250–258. Springer, Cham (2017).

    CrossRef  Google Scholar 

  6. Grob, J., Bonerandi, J.: The ‘ugly duckling’ sign: identification of the common characteristics of nevi in an individual as a basis for melanoma screening. Arch. Dermatol. 134(1), 103–104 (1998)

    CrossRef  Google Scholar 

  7. Radosavovic, I., Kosaraju, R.P., Girshick, R., He, K., Dollár, P.: Designing network design spaces. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10428–10436 (2020)

    Google Scholar 

  8. Schadendorf, D., et al.: Melanoma. The Lancet 392(10151), 971–984 (2018)

    CrossRef  Google Scholar 

  9. Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, vol. 30, pp. 6000–6010 (2017)

    Google Scholar 

  10. Yu, L., Chen, H., Dou, Q., Qin, J., Heng, P.A.: Automated melanoma recognition in dermoscopy images via very deep residual networks. IEEE Trans. Med. Imaging 36(4), 994–1004 (2016)

    CrossRef  Google Scholar 

  11. Yu, Z., et al.: Melanoma recognition in dermoscopy images via aggregated deep convolutional features. IEEE Trans. Biomed. Eng. 66(4), 1006–1016 (2018)

    CrossRef  Google Scholar 

  12. Yuan, T., Deng, W., Tang, J., Tang, Y., Chen, B.: Signal-to-noise ratio: a robust distance metric for deep metric learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4815–4824 (2019)

    Google Scholar 

Download references

Author information

Authors and Affiliations


Corresponding author

Correspondence to Zongyuan Ge .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Verify currency and authenticity via CrossMark

Cite this paper

Yu, Z. et al. (2021). End-to-End Ugly Duckling Sign Detection for Melanoma Identification with Transformers. In: , et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2021. MICCAI 2021. Lecture Notes in Computer Science(), vol 12907. Springer, Cham.

Download citation

  • DOI:

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-87233-5

  • Online ISBN: 978-3-030-87234-2

  • eBook Packages: Computer ScienceComputer Science (R0)