Semantic Segmentation of Histopathological Slides for the Classification of Cutaneous Lymphoma and Eczema

Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 1248)


Mycosis fungoides (MF) is a rare, potentially life threatening skin disease, which in early stages clinically and histologically strongly resembles Eczema, a very common and benign skin condition. In order to increase the survival rate, one needs to provide the appropriate treatment early on. To this end, one crucial step for specialists is the evaluation of histopathological slides (glass slides), or Whole Slide Images (WSI), of the patients’ skin tissue. We introduce a deep learning aided diagnostics tool that brings a two-fold value to the decision process of pathologists. First, our algorithm accurately segments WSI into regions that are relevant for an accurate diagnosis, achieving a Mean-IoU of \(69\%\) and a Matthews Correlation score of \(83\%\) on a novel dataset. Additionally, we also show that our model is competitive with the state of the art on a reference dataset. Second, using the segmentation map and the original image, we are able to predict if a patient has MF or Eczema. We created two models that can be applied in different stages of the diagnostic pipeline, potentially eliminating life-threatening mistakes. The classification outcome is considerably more interpretable than using only the WSI as the input, since it is also based on the segmentation map. Our segmentation model, which we call EU-Net, extends a classical U-Net with an EfficientNet-B7 encoder which was pre-trained on the Imagenet dataset.


Semantic segmentation Histopathological slides Cutaneous lymphoma Eczema U-Net EfficientNet Transfer learning Classification 



We would like to thank everybody at the Kempf und Pfaltz Histologische Diagnostik lab that provided crucial insights about the task, all the necessary data and for their time and commitment to this collaboration. Finally, we would like to thank PAIGE ( for providing the software that was used to annotate the WSI.


  1. 1.
    Anthimopoulos, M., Christodoulidis, S., Ebner, L., Geiser, T., Christe, A., Mougiakakou, S.: Semantic segmentation of pathological lung tissue with dilated fully convolutional networks. IEEE J. Biomed. Health Inform. 23(2), 714–722 (2018)CrossRefGoogle Scholar
  2. 2.
    Aresta, G., et al.: Bach: grand challenge on breast cancer histology images. Med. Image Anal. 56, 122–139 (2019)CrossRefGoogle Scholar
  3. 3.
    Barz, B., Denzler, J.: Deep learning on small datasets without pre-training using cosine loss (2019)Google Scholar
  4. 4.
    Chan, L., Hosseini, M.S., Rowsell, C., Plataniotis, K.N., Damaskinos, S.: HistoSegNet: semantic segmentation of histological tissue type in whole slide images. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 10662–10671 (2019)Google Scholar
  5. 5.
    Chaurasia, A., Culurciello, E.: LinkNet: exploiting encoder representations for efficient semantic segmentation. CoRR abs/1707.03718 (2017).
  6. 6.
    Christodoulidis, S., Anthimopoulos, M., Ebner, L., Christe, A., Mougiakakou, S.: Multisource transfer learning with convolutional neural networks for lung pattern analysis. IEEE J. Biomed. Health Inform. 21(1), 76–84 (2016)CrossRefGoogle Scholar
  7. 7.
    Dabass, M., Vig, R., Vashisth, S.: Review of histopathological image segmentation via current deep learning approaches, May 2019.
  8. 8.
    Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: ImageNet: a large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255. IEEE (2009)Google Scholar
  9. 9.
    Fu, X., Liu, T., Xiong, Z., Smaill, B.H., Stiles, M.K., Zhao, J.: Segmentation of histological images and fibrosis identification with a convolutional neural network. Comput. Biol. Med. 98, 147–158 (2018)CrossRefGoogle Scholar
  10. 10.
    Garbe, C., Dummer, R., Kaufmann, R., Tilgen, W.: Dermatologische Onkologie. Springer, Heidelberg (1997). Scholar
  11. 11.
    Guitart, J., Kennedy, J., Ronan, S., Chmiel, J.S., Hsiegh, Y.C., Variakojis, D.: Histologic criteria for the diagnosis of mycosis fungoides: proposal for a grading system to standardize pathology reporting. J. Cutan. Pathol. 28(4), 174–183 (2001)CrossRefGoogle Scholar
  12. 12.
    Haggerty, J.M., Wang, X.N., Dickinson, A., O’Malley, C.J., Martin, E.B.: Segmentation of epidermal tissue with histopathological damage in images of haematoxylin and eosin stained human skin. BMC Med. Imaging 14(1), 7 (2014)CrossRefGoogle Scholar
  13. 13.
    Hongming, X.H., Mandal, M.: Epidermis segmentation in skin histopathological images based on thickness measurement and k-means algorithm. EURASIP J. Image Video Process. 2015, 18 (2015)CrossRefGoogle Scholar
  14. 14.
    Kempf, W., Mitteldorf, C.: Pathologic diagnosis of cutaneous lymphomas. Dermatol. Clin. 33(4), 655–681 (2015)CrossRefGoogle Scholar
  15. 15.
    Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization (2014)Google Scholar
  16. 16.
    Kłeczek, P., Dyduch, G., Jaworek-Korjakowska, J., Tadeusiewicz, R.: Automated epidermis segmentation in histopathological images of human skin stained with hematoxylin and eosin. In: Medical Imaging 2017: Digital Pathology, vol. 10140, p. 101400M. International Society for Optics and Photonics (2017)Google Scholar
  17. 17.
    Kwok, S.: Multiclass classification of breast cancer in whole-slide images. In: Campilho, A., Karray, F., ter Haar Romeny, B. (eds.) ICIAR 2018. LNCS, vol. 10882, pp. 931–940. Springer, Cham (2018). Scholar
  18. 18.
    Lin, T.Y., Dollár, P., Girshick, R., He, K., Hariharan, B., Belongie, S.: Feature pyramid networks for object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2117–2125 (2017)Google Scholar
  19. 19.
    Lu, C., Mandal, M.: Automated analysis and diagnosis of skin melanoma on whole slide histopathological images. Pattern Recogn. 48(8), 2738–2750 (2015)CrossRefGoogle Scholar
  20. 20.
    Oskal, K.R., Risdal, M., Janssen, E.A., Undersrud, E.S., Gulsrud, T.O.: A U-net based approach to epidermal tissue segmentation in whole slide histopathological images. SN Appl. Sci. 1(7), 672 (2019)CrossRefGoogle Scholar
  21. 21.
    Pimpinelli, N., Olsen, E.A., Santucci, M., et al.: Defining early mycosis fungoides. J. Am. Acad. Dermatol. 53(6), 1053–1063 (2005).
  22. 22.
    Ronneberger, O., Fischer, P., Brox, T.: U-net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). CoRR abs/1505.04597. Scholar
  23. 23.
    Rovner, R., Smith, H.L., Katz, P.J., Liu, V.: Influence of clinical and pathologic features on the pathologist’s diagnosis of mycosis fungoides: a pilot study. J. Cutan. Pathol. 42(7), 471–479 (2015)CrossRefGoogle Scholar
  24. 24.
    Santucci, M., Biggeri, A., Feller, A.C., Massi, D., Burg, G.: Efficacy of histologic criteria for diagnosing early mycosis fungoides: an eortc cutaneous lymphoma study group investigation. Am. J. Surg. Pathol. 24(1), 40 (2000)CrossRefGoogle Scholar
  25. 25.
    Tan, M., Le, Q.V.: EfficientNet: rethinking model scaling for convolutional neural networks (2019)Google Scholar
  26. 26.
    Wang, J., MacKenzie, J.D., Ramachandran, R., Chen, D.Z.: A deep learning approach for semantic segmentation in histology tissue images. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) MICCAI 2016. LNCS, vol. 9901, pp. 176–184. Springer, Cham (2016). Scholar
  27. 27.
    Xie, P., Zuo, K., Zhang, Y., Li, F., Yin, M., Lu, K.: Interpretable classification from skin cancer histology slides using deep learning: a retrospective multicenter study. arXiv preprint arXiv:1904.06156 (2019)
  28. 28.
    Xu, H., Lu, C., Berendt, R., Jha, N., Mandal, M.: Automated analysis and classification of melanocytic tumor on skin whole slide images. Comput. Med. Imaging Graph. 66, 124–134 (2018)CrossRefGoogle Scholar
  29. 29.
    Yakubovskiy, P.: Segmentation models (2019).
  30. 30.
    Zhao, H., Shi, J., Qi, X., Wang, X., Jia, J.: Pyramid scene parsing network. CoRR abs/1612.01105 (2016).

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.ETH ZürichZürichSwitzerland
  2. 2.Kempf und Pfaltz Histologische DiagnostikZürichSwitzerland

Personalised recommendations