Improving Whole Slide Segmentation Through Visual Context - A Systematic Study

  • Korsuk Sirinukunwattana
  • Nasullah Khalid Alham
  • Clare Verrill
  • Jens Rittscher
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11071)


While challenging, the dense segmentation of histology images is a necessary first step to assess changes in tissue architecture and cellular morphology. Although specific convolutional neural network architectures have been applied with great success to the problem, few effectively incorporate visual context information from multiple scales. With this paper, we present a systematic comparison of different architectures to assess how including multi-scale information affects segmentation performance. A publicly available breast cancer and a locally collected prostate cancer datasets are being utilised for this study. The results support our hypothesis that visual context and scale plays a crucial role in histology image classification problems.


Digital pathology Whole slide imaging Dense segmentation Deep learning 



This research was funded by the National Institute for Health Research (NIHR) Oxford Biomedical Research Centre (BRC) and the EPSRC SeeBiByte Programme Grant (EP/M013774/1). The views expressed are those of the authors and not necessarily those of the NHS, the NIHR or the Department of Health.

Supplementary material

473975_1_En_22_MOESM1_ESM.pdf (197 kb)
Supplementary material 1 (pdf 196 KB)


  1. 1.
    Shah, M., Rubadue, C., Suster, D., Wang, D.: Deep learning assessment of tumor proliferation in breast cancer histological images. arXiv preprint arXiv:1610.03467 (2016)
  2. 2.
    Schaumberg, A.J., Rubin, M.A., Fuchs, T.J.: H&E-stained whole slide deep learning predicts SPOP mutation state in prostate cancer. bioRxiv, page 064279 (2016)Google Scholar
  3. 3.
    Bychkov, D., et al.: Deep learning based tissue analysis predicts outcome in colorectal cancer. Sci. Rep. 8(1), 3395 (2018)CrossRefGoogle Scholar
  4. 4.
    Liang, X., Shen, X., Xiang, D., Feng, J., Lin, L., Yan, S.: Semantic object parsing with local-global long short-term memory. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3185–3193 (2016)Google Scholar
  5. 5.
    Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)CrossRefGoogle Scholar
  6. 6.
    Araújo, T., et al.: Classification of breast cancer histology images using convolutional neural networks. PloS one 12(6), e0177544 (2017)CrossRefGoogle Scholar
  7. 7.
    Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). Scholar
  8. 8.
    Zhang, W., et al.: Deep convolutional neural networks for multi-modality isointense infant brain image segmentation. NeuroImage 108, 214–224 (2015)CrossRefGoogle Scholar
  9. 9.
    Wang, D., Khosla, A., Gargeya, R., Irshad, H., Beck, A.H.: Deep learning for identifying metastatic breast cancer. arXiv preprint arXiv:1606.05718 (2016)
  10. 10.
    Hou, L., Samaras, D., Kurc, T.M., Gao, Y., Davis, J.E., Saltz, J.H.: Patch-based convolutional neural network for whole slide tissue image classification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2424–2433 (2016)Google Scholar
  11. 11.
    Moeskops, P., Viergever, M.A., Mendrik, A.M., de Vries, L.S., Benders, M.J.N.L., Išgum, I.: Automatic segmentation of MR brain images with a convolutional neural network. IEEE Trans. Med. Imaging 35(5), 1252–1261 (2016)CrossRefGoogle Scholar
  12. 12.
    Kong, B., Wang, X., Li, Z., Song, Q., Zhang, S.: Cancer metastasis detection via spatially structured deep network. In: Niethammer, M., et al. (eds.) IPMI 2017. LNCS, vol. 10265, pp. 236–248. Springer, Cham (2017). Scholar
  13. 13.
    Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. CoRR, abs/1412.6980 (2014)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  1. 1.Institute of Biomedical Engineering, Department of Engineering ScienceUniversity of OxfordOxfordUK
  2. 2.Nuffield Department of Surgical Sciences and Oxford NIHR Biomedical Research Centre (BRC), University of Oxford, John Radcliffe HospitalOxfordUK

Personalised recommendations