Propagating Uncertainty Across Cascaded Medical Imaging Tasks for Improved Deep Learning Inference

  • Raghav MehtaEmail author
  • Thomas Christinck
  • Tanya Nair
  • Paul Lemaitre
  • Douglas Arnold
  • Tal Arbel
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11840)


Although deep networks have been shown to perform very well on a variety of tasks, inference in the presence of pathology in medical images presents challenges to traditional networks. Given that medical image analysis typically requires a sequence of inference tasks to be performed (e.g. registration, segmentation), this results in an accumulation of errors over the sequence of deterministic outputs. In this paper, we explore the premise that, by embedding uncertainty estimates across cascaded inference tasks, the final prediction results should improve over simply cascading the deterministic classification results or performing inference in a single stage. Specifically, we develop a deep learning framework that propagates voxel-based uncertainty measures (e.g. Monte Carlo (MC) dropout sample variance) across inference tasks in order to improve the detection and segmentation of focal pathologies (e.g. lesions, tumours) in brain MR images. We apply the framework to two different contexts. First, we demonstrate that propagating multiple sclerosis T2 lesion segmentation results along with their associated uncertainty measures improves subsequent T2 lesion detection accuracy when evaluated on a proprietary large-scale, multi-site, clinical trial dataset. Second, we show how by propagating uncertainties associated with a regressed 3D MRI volume as an additional input to a follow-on brain tumour segmentation task, one can improve segmentation results on the publicly available BraTS-2018 dataset.



This work was supported by a Canadian Natural Science and Engineering Research Council (NSERC) Collaborative Research and Development Grant (CRDPJ 505357 - 16), Synaptive Medical, the Canadian NSERC Discovery and CREATE grants, and an award from the International Progressive MS Alliance (PA-1603-08175).


  1. 1.
    Chartsias, A., Joyce, T., Giuffrida, M.V., Tsaftaris, S.A.: Multimodal MR synthesis via modality-invariant latent representation. IEEE Trans. Med. Imaging 37(3), 803–814 (2017)CrossRefGoogle Scholar
  2. 2.
    Mehta, R., Arbel, T.: RS-Net: regression-segmentation 3D CNN for synthesis of full resolution missing brain MRI in the presence of tumours. In: Gooya, A., Goksel, O., Oguz, I., Burgos, N. (eds.) SASHIMI 2018. LNCS, pp. 119–129. Springer, Cham (2018). Scholar
  3. 3.
    Dalca, A.V., Balakrishnan, G., Guttag, J., Sabuncu, M.R.: Unsupervised learning for fast probabilistic diffeomorphic registration. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11070, pp. 729–738. Springer, Cham (2018). Scholar
  4. 4.
    Isensee, F., Kickingereder, P., Wick, W., Bendszus, M., Maier-Hein, K.H.: No new-net. In: Crimi, A., Bakas, S., Kuijf, H., Keyvan, F., Reyes, M., van Walsum, T. (eds.) BrainLes 2018. LNCS, vol. 11384, pp. 234–244. Springer, Cham (2019). Scholar
  5. 5.
    Nair, T., Precup, D., Arnold, D.L., Arbel, T.: Exploring uncertainty measures in deep networks for multiple sclerosis lesion detection and segmentation. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11070, pp. 655–663. Springer, Cham (2018). Scholar
  6. 6.
    Tousignant, A., Lemaître, P., Precup, D., Arnold, D.L., Arbel, T.: Prediction of disease progression in multiple sclerosis patients using deep learning analysis of MRI data. In: International Conference on Medical Imaging with Deep Learning, pp. 483–492, May 2019Google Scholar
  7. 7.
    Gal, Y., Ghahramani, Z.: Dropout as a Bayesian approximation: representing model uncertainty in deep learning. In: International Conference on Machine Learning, pp. 1050–1059, June 2016Google Scholar
  8. 8.
    Ozdemir, O., Woodward, B., Berlin, A.A.: Propagating uncertainty in multi-stage Bayesian convolutional neural networks with application to pulmonary nodule detection. arXiv preprint arXiv:1712.00497 (2017)
  9. 9.
    Roy, A.G., Conjeti, S., Navab, N., Wachinger, C., Alzheimer’s Disease Neuroimaging Initiative: Bayesian QuickNAT: model uncertainty in deep whole-brain segmentation for structure-wise quality control. NeuroImage 195, 11–22 (2019)CrossRefGoogle Scholar
  10. 10.
    Leibig, C., Allken, V., Ayhan, M.S., Berens, P., Wahl, S.: Leveraging uncertainty information from deep neural networks for disease detection. Sci. Rep. 7(1), 17816 (2017)CrossRefGoogle Scholar
  11. 11.
    van Tulder, G., de Bruijne, M.: Why does synthesized data improve multi-sequence classification? In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9349, pp. 531–538. Springer, Cham (2015). Scholar
  12. 12.
    Çiçek, Ö., Abdulkadir, A., Lienkamp, S.S., Brox, T., Ronneberger, O.: 3D U-Net: learning dense volumetric segmentation from sparse annotation. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) MICCAI 2016. LNCS, vol. 9901, pp. 424–432. Springer, Cham (2016). Scholar
  13. 13.
    Bakas, S., et al.: Identifying the best machine learning algorithms for brain tumor segmentation, progression assessment, and overall survival prediction in the BRATS challenge. arXiv preprint arXiv:1811.02629 (2018)
  14. 14.
    Wu, Y., He, K.: Group normalization. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11217, pp. 3–19. Springer, Cham (2018). Scholar
  15. 15.
    Ulyanov, D., Vedaldi, A., Lempitsky, V.: Instance normalization: the missing ingredient for fast stylization. arXiv preprint arXiv:1607.08022 (2016)

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Raghav Mehta
    • 1
    Email author
  • Thomas Christinck
    • 1
  • Tanya Nair
    • 1
  • Paul Lemaitre
    • 1
  • Douglas Arnold
    • 2
    • 3
  • Tal Arbel
    • 1
  1. 1.Centre for Intelligent MachinesMcGill UniversityMontrealCanada
  2. 2.Montreal Neurological InstituteMcGill UniversityMontrealCanada
  3. 3.NeuroRx ResearchMontrealCanada

Personalised recommendations