Abstract
Nowadays, there is an urgent requirement of self-supervised learning (SSL) on whole slide pathological images (WSIs) to relieve the demand of finely expert annotations. However, the performance of SSL algorithms on WSIs has long lagged behind their supervised counterparts. To close this gap, in this paper, we fully explore the intrinsic characteristics of WSIs and propose SSLP: Spatial Guided Self-supervised Learning on Pathological Images. We argue the patch-wise spatial proximity is a significant characteristic of WSIs, if properly employed, shall provide abundant supervision for free. Specifically, we explore three semantic invariance from 1) self-invariance: the same patch of different augmented views, 2) intra-invariance: the patches within spatial neighbors and 3) inter-invariance: their corresponding neighbors in the feature space. As a result, our SSLP model achieves \(82.9\%\) accuracy and \(85.7\%\) AUC on CAMELYON linear classification and \(95.2\%\) accuracy fine-tuning on cross-disease classification on NCTCRC, which outperforms previous state-of-the-art algorithm and matches the performance of a supervised counterpart.
J. Li and T. Lin—These authors have contributed equally.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Abbet, C., Zlobec, I., Bozorgtabar, B., Thiran, J.P.: Divide-and-rule: self-supervised learning for survival analysis in colorectal cancer. In: Martel, A.L. et al. (eds.) Medical Image Computing and Computer Assisted Intervention–MICCAI 2020. MICCAI 2020. Lecture Notes in Computer Science, 12265, 480–489. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59722-1_46
Bachman, P., Hjelm, R.D., Buchwalter, W.: Learning representations by maximizing mutual information across views (2019). arXiv preprint: arXiv:1906.00910
Bejnordi, B.E., et al.: Diagnostic assessment of deep learning algorithms for detection of lymph node metastases in women with breast cancer. JAMA 318(22), 2199–2210 (2017)
Bulten, W., et al.: Automated deep-learning system for Gleason grading of prostate cancer using biopsies: a diagnostic study. Lancet Oncol. 21(2), 233–241 (2020)
Caron, M., Misra, I., Mairal, J., Goyal, P., Bojanowski, P., Joulin, A.: Unsupervised learning of visual features by contrasting cluster assignments (2020). arXiv preprint: arXiv:2006.09882
Chen, T., Kornblith, S., Norouzi, M., Hinton, G.: A simple framework for contrastive learning of visual representations. In: International Conference on Machine Learning. pp. 1597–1607. PMLR (2020)
Chen, X., Fan, H., Girshick, R., He, K.: Improved baselines with momentum contrastive learning (2020). arXiv preprint: arXiv:2003.04297
Chen, X., He, K.: Exploring simple Siamese representation learning (2020). arXiv preprint: arXiv:2011.10566
Coudray, N., et al.: Classification and mutation prediction from non-small cell lung cancer histopathology images using deep learning. Nat. Med. 24(10), 1559–1567 (2018)
Dagogo-Jack, I., Shaw, A.T.: Tumour heterogeneity and resistance to cancer therapies. Nat. Rev. Clin. Oncol. 15(2), 81 (2018)
Dehaene, O., et al.: Self-supervision closes the gap between weak and strong supervision in histology (2020). arXiv preprint: arXiv:2012.03583
Egeblad, M., Nakasone, E.S., Werb, Z.: Tumors as organs: complex tissues that interface with the entire organism. Dev. Cell 18(6), 884–901 (2010)
Frankle, J., et al.: Are all negatives created equal in contrastive instance discrimination? (2020). arXiv preprint: arXiv:2010.06682
Gidaris, S., Singh, P., Komodakis, N.: Unsupervised representation learning by predicting image rotations (2018). arXiv preprint: arXiv:1803.07728
Grill, J.B., et al.: Bootstrap your own latent: a new approach to self-supervised learning (2020). arXiv preprint: arXiv:2006.07733
He, K., Fan, H., Wu, Y., Xie, S., Girshick, R.: Momentum contrast for unsupervised visual representation learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9729–9738 (2020)
Huang, J., Dong, Q., Gong, S., Zhu, X.: Unsupervised deep learning by neighbourhood discovery (2019). arXiv preprint: arXiv:1904.11567
Kather, J.N., Halama, N., Marx, A.: 100,000 histological images of human colorectal cancer and healthy tissue (2018). https://doi.org/10.5281/zenodo.1214456
Kather, J.N., et al.: Multi-class texture analysis in colorectal cancer histology. Sci. Rep. 6(1), 1–11 (2016)
Laine, S., Aila, T.: Temporal ensembling for semi-supervised learning (2016). arXiv preprint: arXiv:1610.02242
Lu, M.Y., Chen, R.J., Mahmood, F.: Semi-supervised breast cancer histology classification using deep multiple instance learning and contrast predictive coding (conference presentation). In: Medical Imaging 2020: Digital Pathology, vol. 11320, p. 113200J. International Society for Optics and Photonics (2020)
Mercan, C., et al.: Virtual staining for mitosis detection in breast histopathology. In: 2020 IEEE 17th International Symposium on Biomedical Imaging, pp. 1770–1774. IEEE (2020)
Mobadersany, P., et al.: Predicting cancer outcomes from histology and genomics using convolutional networks. Proc. Natl. Acad. Sci. 115(13), E2970–E2979 (2018)
Ngiam, K.Y., Khor, W.: Big data and machine learning algorithms for health-care delivery. Lancet Oncol. 20(5), e262–e273 (2019)
Niazi, M.K.K., Parwani, A.V., Gurcan, M.N.: Digital pathology and artificial intelligence. Lancet Oncol. 20(5), e253–e261 (2019)
Oord, A.v.d., Li, Y., Vinyals, O.: Representation learning with contrastive predictive coding (2018). arXiv preprint: arXiv:1807.03748
Sahiner, B., et al.: Deep learning in medical imaging and radiation therapy. Med. Phys. 46(1), e1–e36 (2019)
Wu, Z., Xiong, Y., Yu, S.X., Lin, D.: Unsupervised feature learning via non-parametric instance discrimination. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3733–3742 (2018)
Xie, J., Zhan, X., Liu, Z., Ong, Y.S., Loy, C.C.: Delving into inter-image invariance for unsupervised visual representations (2020). arXiv preprint: arXiv:2008.11702
Xie, X., et al.: Instance-aware self-supervised learning for nuclei segmentation. In: Martel, A.L. et al. (eds.) Medical Image Computing and Computer Assisted Intervention–MICCAI 2020. MICCAI 2020. Lecture Notes in Computer Science, 12265, 341–350. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59722-1_33
Xie, Z., Lin, Y., Zhang, Z., Cao, Y., Lin, S., Hu, H.: Propagate yourself: exploring pixel-level consistency for unsupervised visual representation learning (2020). arXiv preprint: arXiv:2011.10043
Yuan, Y., et al.: Quantitative image analysis of cellular heterogeneity in breast tumors complements genomic profiling. Sci. Transl. Med. 4(157), 157ra143 (2012)
Zhuang, C., Zhai, A.L., Yamins, D.: Local aggregation for unsupervised learning of visual embeddings. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 6002–6012 (2019)
Acknowledgement
This work was supported in part by Shanghai Municipal Science and Technology Major Project (2021SHZDZX0102), 111 project (BP0719010), Shanghai Science and Technology Committee (18DZ2270700) and Shanghai Jiao Tong University Science and Technology Innovation Special Fund (ZH2018ZDA17).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Li, J., Lin, T., Xu, Y. (2021). SSLP: Spatial Guided Self-supervised Learning on Pathological Images. In: de Bruijne, M., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2021. MICCAI 2021. Lecture Notes in Computer Science(), vol 12902. Springer, Cham. https://doi.org/10.1007/978-3-030-87196-3_1
Download citation
DOI: https://doi.org/10.1007/978-3-030-87196-3_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-87195-6
Online ISBN: 978-3-030-87196-3
eBook Packages: Computer ScienceComputer Science (R0)
