Uncertainty Driven Pooling Network for Microvessel Segmentation in Routine Histology Images

  • M. M. FrazEmail author
  • M. Shaban
  • S. Graham
  • S. A. Khurram
  • N. M. Rajpoot
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11039)


Lymphovascular invasion (LVI) and tumor angiogenesis are correlated with metastasis, cancer recurrence and poor patient survival. In most of the cases, the LVI quantification and angiogenic analysis is based on microvessel segmentation and density estimation in immunohistochemically (IHC) stained tissues. However, in routine H&E stained images, the microvessels display a high level of heterogeneity in terms of size, shape, morphology and texture which makes microvessel segmentation a non-trivial task. Manual delineation of microvessels for biomarker analysis is labor-intensive, time consuming, irreproducible and can suffer from subjectivity among pathologists. Moreover, it is often beneficial to account for the uncertainty of a prediction when making a diagnosis. To address these challenges, we proposed a framework for microvessel segmentation in H&E stained histology images. The framework extends DeepLabV3+ by using an improved dice coefficient based custom loss function and also incorporating an uncertainty prediction mechanism. The proposed method uses an aligned Xception model, followed by atrous spatial pyramid pooling for feature extraction at multiple scales. This architecture counters the challenge of segmenting blood vessels of varying morphological appearance. To incorporate uncertainty, random transformations are introduced at test time for a superior segmentation result and simultaneous uncertainty map generation, highlighting ambiguous regions. The method is evaluated using 1167 images of size \(512\times 512\) pixels, extracted from 13 WSIs of oral squamous cell carcinoma (OSCC) tissue at 20x magnification. The proposed net-work achieves state-of-the-art performance compared to current semantic segmentation deep neural networks (FCN-8, U-Net, SegNet and DeepLabV3+).


Microvessel detection Tumor angiogenesis Lymphovascular invasion Separable convolution Pyramid pooling based neural network Uncertainty quantification 


  1. 1.
    Badrinarayanan, V., Kendall, A., Cipolla, R.: SegNet: a deep convolutional encoder-decoder architecture for image segmentation. arXiv preprint arXiv:1511.00561 (2015)
  2. 2.
    Carmeliet, P.: Angiogenesis in life, disease and medicine. Nature 438(7070), 932 (2005)CrossRefGoogle Scholar
  3. 3.
    Chen, L.C., Papandreou, G., Kokkinos, I., Murphy, K., Yuille, A.L.: DeepLab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Trans. Pattern Anal. Mach. Intell. 40(4), 834–848 (2018)CrossRefGoogle Scholar
  4. 4.
    Chen, L.C., Zhu, Y., Papandreou, G., Schroff, F., Adam, H.: Encoder-decoder with atrous separable convolution for semantic image segmentation. arXiv preprint arXiv:1802.02611 (2018)
  5. 5.
    Chollet, F.: Xception: deep learning with depthwise separable convolutions. arXiv preprint arXiv:1610–02357 (2017)
  6. 6.
    Gal, Y., Ghahramani, Z.: Dropout as a Bayesian approximation: representing model uncertainty in deep learning. In: International Conference on Machine Learning, pp. 1050–1059 (2016)Google Scholar
  7. 7.
    Graham, S., Chen, H., Dou, Q., Heng, P.A., Rajpoot, N.: MILD-Net: minimal information loss dilated network for gland instance segmentation in colon histology images. arXiv preprint arXiv:1806.01963 (2018)
  8. 8.
    Kather, J.N., Marx, A., Reyes-Aldasoro, C.C., Schad, L.R., Zöllner, F.G., Weis, C.A.: Continuous representation of tumor microvessel density and detection of angiogenic hotspots in histological whole-slide images. Oncotarget 6(22), 19163 (2015)CrossRefGoogle Scholar
  9. 9.
    Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3431–3440 (2015)Google Scholar
  10. 10.
    Noma, D., et al.: Prognostic effect of Lymphovascular Invasion on TNM staging in stage i non-small-cell lung cancer. Clin. Lung Cancer 19(1), e109–e122 (2018)CrossRefGoogle Scholar
  11. 11.
    Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015 Part III. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). Scholar
  12. 12.
    Saha, M., Chakraborty, C., Racoceanu, D.: Efficient deep learning model for mitosis detection using breast histopathology images. Comput. Med. Imaging Graph. 64, 29–40 (2018)CrossRefGoogle Scholar
  13. 13.
    Saltz, J., et al.: Spatial organization and molecular correlation of tumor-infiltrating lymphocytes using deep learning on pathology images. Cell Rep. 23(1), 181 (2018)CrossRefGoogle Scholar
  14. 14.
    Sirinukunwattana, K., Raza, S.E.A., Tsang, Y.W., Snead, D.R., Cree, I.A., Rajpoot, N.M.: Locality sensitive deep learning for detection and classification of nuclei in routine colon cancer histology images. IEEE Trans. Med. Imaging 35(5), 1196–1206 (2016)CrossRefGoogle Scholar
  15. 15.
    Xu, J., Luo, X., Wang, G., Gilmore, H., Madabhushi, A.: A deep convolutional neural network for segmenting and classifying epithelial and stromal regions in histopathological images. Neurocomputing 191, 214–223 (2016)CrossRefGoogle Scholar
  16. 16.
    Yi, F., et al.: Microvessel prediction in H&E stained pathology images using fully convolutional neural networks. BMC Bioinf. 19(1), 64 (2018)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • M. M. Fraz
    • 1
    • 2
    • 4
    Email author
  • M. Shaban
    • 1
  • S. Graham
    • 1
  • S. A. Khurram
    • 5
  • N. M. Rajpoot
    • 1
    • 2
    • 3
  1. 1.Department of Computer ScienceUniversity of WarwickCoventryUK
  2. 2.The Alan Turing InstituteLondonUK
  3. 3.University Hospitals Coventry and Warwickshire, NHS TrustCoventryUK
  4. 4.National University of Sciences and TechnologyIslamabadPakistan
  5. 5.The University of SheffieldSheffieldUK

Personalised recommendations