Negative Pseudo Labeling Using Class Proportion for Semantic Segmentation in Pathology

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 12360)


In pathological diagnosis, since the proportion of the adenocarcinoma subtypes is related to the recurrence rate and the survival time after surgery, the proportion of cancer subtypes for pathological images has been recorded as diagnostic information in some hospitals. In this paper, we propose a subtype segmentation method that uses such proportional labels as weakly supervised labels. If the estimated class rate is higher than that of the annotated class rate, we generate negative pseudo labels, which indicate, “input image does not belong to this negative label,” in addition to standard pseudo labels. It can force out the low confidence samples and mitigate the problem of positive pseudo label learning which cannot label low confident unlabeled samples. Our method outperformed the state-of-the-art semi-supervised learning (SSL) methods.


Pathological image Semantic segmentation Negative learning Semi-supervised learning Learning from label proportion 



This work was supported by JSPS KAKENHI Grant Number 20H04211.


  1. 1.
    Agrawal, P., Girshick, R., Malik, J.: Analyzing the performance of multilayer neural networks for object recognition. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8695, pp. 329–344. Springer, Cham (2014). Scholar
  2. 2.
    Alsubaie, N., Shaban, M., Snead, D., Khurram, A., Rajpoot, N.: A multi-resolution deep learning framework for lung adenocarcinoma growth pattern classification. In: Nixon, M., Mahmoodi, S., Zwiggelaar, R. (eds.) MIUA 2018. CCIS, vol. 894, pp. 3–11. Springer, Cham (2018). Scholar
  3. 3.
    Altunbay, D., Cigir, C., Sokmensuer, C., GunduzDemi, C.: Color graphs for automated cancer diagnosis and grading. IEEE Trans. Biomed. Eng. 57(3), 665–674 (2010)CrossRefGoogle Scholar
  4. 4.
    Bandi, P., Geessink, O., Manson, Q., van Dijk, M., et al.: From detection of individual metastases to classification of lymph node status at the patient level: the camelyon17 challenge. IEEE Trans. Med. Imag. 38(2), 550-560 (2018)Google Scholar
  5. 5.
    Bejnordi, B.E., Veta, M., van Diest, P.J., van Ginneken, B., et al.: Diagnostic assessment of deep learning algorithms for detection of lymph node metastases in women with breast cancer. Jama 318(22), 2199–2210 (2017)Google Scholar
  6. 6.
    Berthelot, D., Carlini, N., Goodfellow, I., Papernot, N., Oliver, A., Raffel, C.A.: MixMatch: a holistic approach to semi-supervised learning. In: NeurIPS, pp. 5050–5060 (2019)Google Scholar
  7. 7.
    Chang, H., Zhou, Y., Borowsky, A., Barner, K., Spellman, P., Parvin, B.: Stacked predictive sparse decomposition for classification of histology sections. Int. J. Comput. Vis. 113(1), 3–18 (2014)MathSciNetCrossRefGoogle Scholar
  8. 8.
    Chen, B., Chen, L., Ramakrishnan, R., Musicant, D.: Learning from aggregate views. In: ICDE, p. 3 (2006)Google Scholar
  9. 9.
    Cruz-Roa, A., Basavanhally, A., Gonzalez, F., Gilmore, H., et al.: Automatic detection of invasive ductal carcinoma in whole slide images with convolutional neural networks. In: SPIE Medical Imaging (2014)Google Scholar
  10. 10.
    Csurka, G., Larlus, D., Perronnin, F.: What is a good evaluation measure for semantic segmentation? In: CVPR (2013)Google Scholar
  11. 11.
    Hendrik, K., de Nando, F.: SVM classifier estimation from group probabilities. In: CUAI, pp. 332–339 (2005)Google Scholar
  12. 12.
    Hou, L., Samaras, D., Kurc, T.M., Gao, Y., Davis, J.E., Saltz, J.H.: Patch-based convolutional neural network for whole slide tissue image classification. In: CVPR, pp. 2424–2433 (2016)Google Scholar
  13. 13.
    Hou, L., Samaras, D., Kurc, T.M., Gao, Y. et al.: Patch-based convolutional neural network for whole slide tissue image classification. In: CVPR, pp. 2424–2433 (2016)Google Scholar
  14. 14.
    Ishida, T., Niu, G., Menon, A.K., Sugiyama, M.: Complementary-label learning for arbitrary losses and models. arXiv preprint arXiv:1810.04327 (2018)
  15. 15.
    Liu, J., Wang, B., Qi, Z., Tian, Y., Shi, Y.: Learning from label proportions with generative adversarial networks. In: NeurIPS (2019)Google Scholar
  16. 16.
    Kikkawa, R., Sekiguchi, H., Tsuge, I., Saito, S., Bise, R.: Semi-supervised learning with structured knowledge for body hair detection in photoacoustic image. In: ISBI (2019)Google Scholar
  17. 17.
    Kim, Y., Yim, J., Yun, J., Kim, J.: NLNL: negative learning for noisy labels. In: ICCV, pp. 101–110 (2019)Google Scholar
  18. 18.
    Kong, B., Wang, X., Li, Z., Song, Q., Zhang, S.: Cancer metastasis detection via spatially structured deep network. In: Niethammer, M. (ed.) IPMI 2017. LNCS, vol. 10265, pp. 236–248. Springer, Cham (2017). Scholar
  19. 19.
    Lee, D.H.: Pseudo-label: the simple and efficient semi-supervised learning method for deep neural networks. In: ICML Workshops, vol. 3, p. 2 (2013)Google Scholar
  20. 20.
    Liu, L., et al.: On the variance of the adaptive learning rate and beyond. arXiv preprint arXiv:1908.03265 (2019)
  21. 21.
    Miyato, T., Maeda, S.I., Koyama, M., Ishii, S.: Virtual adversarial training: a regularization method for supervised and semi-supervised learning. IEEE Trans. Pattern Anal. Mach. Intell. 41(8), 1979–1993 (2018)CrossRefGoogle Scholar
  22. 22.
    Mousavi, H., Monga, V., Rao, G., Rao, A.U.: Automated discrimination of lower and higher grade gliomas based on histopathological image analysis. J. Pathol. Inform. 6, 15 (2015)CrossRefGoogle Scholar
  23. 23.
    Musicant, D., Christensen, J., Olson, J.: Supervised learning by training on aggregate outputs. In: ICDM, pp. 252–261 (2007)Google Scholar
  24. 24.
    Oliver, A., Odena, A., Raffel, C.A., Cubuk, E.D., Goodfellow, I.: Realistic evaluation of deep semi-supervised learning algorithms. In: NeurIPS, pp. 3235–3246 (2018)Google Scholar
  25. 25.
    Qi, Z., Wang, B., Meng, F.: Learning with label proportions via NPSVM. IEEE Trans. Cybern. 47(10), 3293–3305 (2017)CrossRefGoogle Scholar
  26. 26.
    Rueping, S.: SVM classifier estimation from group probabilities. In: ICML (2010)Google Scholar
  27. 27.
    Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., Chen, L.C.: MobileNetV2: inverted residuals and linear bottlenecks. In: CVPR, pp. 4510–4520 (2018)Google Scholar
  28. 28.
    Shusuke, T., et al.: Multi-stage pathological image classification using semantic segmentation. In: ICCV (2019)Google Scholar
  29. 29.
    Sirinukunwattana, K., Alham, N.K., Verrill, C., Rittscher, J.: Improving whole slide segmentation through visual context - a systematic study. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11071, pp. 192–200. Springer, Cham (2018). Scholar
  30. 30.
    Sohn, K., et al.: Fixmatch: simplifying semi-supervised learning with consistency and confidence. arXiv preprint arXiv:2001.07685 (2020)
  31. 31.
    Takashi, I., Gang, N., Weihua, H., Masashi, S.: Learning from complementary labels. In: NeurIPS (2017)Google Scholar
  32. 32.
    Tarvainen, A., Valpola, H.: Mean teachers are better role models: weight-averaged consistency targets improve semi-supervised deep learning results. In: NeurIPS, pp. 1195–1204 (2017)Google Scholar
  33. 33.
    Tokunaga, H., Teramoto, Y., Yoshizawa, A., Bise, R.: Adaptive weighting multi-field-of-view CNN for semantic segmentation in pathology. In: CVPR (2019)Google Scholar
  34. 34.
    Verma, V., Lamb, A., Kannala, J., Bengio, Y., Lopez-Paz, D.: Interpolation consistency training for semi-supervised learning. In: IJCAI, pp. 3635–3641 (2019)Google Scholar
  35. 35.
    Wang, D., Khosla, A., Gargeya, R., Irshad, H., Beck, A.H.: Deep learning for identifying metastatic breast cancer. arXiv preprint arXiv:1606.05718 (2016)
  36. 36.
    Xie, Q., Hovy, E., Luong, M.T., Le, Q.V.: Self-training with noisy student improves ImageNet classification. arXiv preprint arXiv:1911.04252 (2019)
  37. 37.
    Xu, Y., Jia, Z., Ai, Y., Zhang, F., Lai, M., Chang, E.I.C.: Deep convolutional activation features for large scale brain tumor histopathology image classification and segmentation. In: ICASSP (2015)Google Scholar
  38. 38.
    Yoshizawa, A., Motoi, N., Riely, G.J., Sima, C., et al.: Impact of proposed IASLC/ATS/ERS classification of lung adenocarcinoma: prognostic subgroups and implications for further revision of staging based on analysis of 514 stage I cases. Mod. Pathol. 24(5), 653 (2011)CrossRefGoogle Scholar
  39. 39.
    Yu, F., Liu, D., Kumar, S., Tony, J., Chang, S.F.: \(\propto \)SVM for learning with label proportions. In: ICML, pp. 504–512 (2013)Google Scholar
  40. 40.
    Zhang, H., Cisse, M., Dauphin, Y.N., Lopez-Paz, D.: mixup: beyond empirical risk minimization. In: ICLR (2018)Google Scholar
  41. 41.
    Zhou, Y., Chang, H., Barner, K., Spellman, P., Parvin, B.: Classification of histology sections via multispectral convolutional sparse coding. In: CVPR Workshop, pp. 3081–3088 (2014)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.Kyushu UniversityFukuokaJapan
  2. 2.Kyoto University HospitalKyotoJapan
  3. 3.Research Center for Medical BigdataNational Institute of InformaticsTokyoJapan

Personalised recommendations