Abstract
The accurate estimation of predictive uncertainty carries importance in medical scenarios such as lung node segmentation. Unfortunately, most existing works on predictive uncertainty do not return calibrated uncertainty estimates, which could be used in practice. In this work we exploit multi-grader annotation variability as a source of ‘groundtruth’ aleatoric uncertainty, which can be treated as a target in a supervised learning problem. We combine this groundtruth uncertainty with a Probabilistic U-Net and test on the LIDC-IDRI lung nodule CT dataset and MICCAI2012 prostate MRI dataset. We find that we are able to improve predictive uncertainty estimates. We also find that we can improve sample accuracy and sample diversity. In real-world applications, our method could inform doctors about the confidence of the segmentation results.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
We use the PyTorch implementation for the Probabilistic U-Net model from https://github.com/stefanknegt/Probabilistic-Unet-Pytorch.
References
Armato, S.G., McLennan, G., Bidaut, L., et al.: The lung image database consortium (LIDC) and image database resource initiative (IDRI): a completed reference database of lung nodules on ct scans. Med Phys. 38, 915–931 (2011)
Armato, S.G., et al.: Data from LIDC-IDRI. The Cancer Imaging Archive (2015)
Ayhan, M.S., Berens, P.: Test-time data augmentation for estimation of heteroscedastic aleatoric uncertainty in deep neural networks. In: MIDL (2018)
Blei, D.M., Kucukelbir, A., McAuliffe, J.D.: Variational inference: a review for statisticians. CoRR abs/1601.00670 (2016)
Bragman, F.J., et al.: Quality control in radiotherapy-treatment planning using multi-task learning and uncertainty estimation. In: MIDL (2018)
Causey, J., et al.: Highly accurate model for prediction of lung nodule malignancy with CT scans. CoRR abs/1802.01756 (2018)
Clark, K., et al.: The cancer imaging archive (TCIA): maintaining and operating a public information repository. J. Digit. Imaging 26, 1045–1057 (2013)
Gal, Y.: Uncertainty in deep learning. Ph.D. thesis, University of Cambridge (2016)
Gal, Y., Ghahramani, Z.: Dropout as a Bayesian approximation: representing model uncertainty in deep learning. In: ICML (2016)
Gruetzemacher, R., Gupta, A., Paradice, D.B.: 3D deep learning for detecting pulmonary nodules in CT scans. JAMIA 25, 1301–1310 (2018)
Gu, Y., et al.: Automatic lung nodule detection using A 3D deep convolutional neural network combined with a multi-scale prediction strategy in chest CTs. Comput. Biol. Med. 103, 220–231 (2018)
Jungo, A., Meier, R., Ermis, E., Herrmann, E., Reyes, M.: Uncertainty-driven sanity check: application to postoperative brain tumor cavity segmentation. In: MIDL (2018)
Kendall, A., Gal, Y.: What uncertainties do we need in Bayesian deep learning for computer vision? In: NIPS (2017)
Kingma, D.P., Salimans, T., Welling, M.: Variational dropout and the local reparameterization trick. In: NIPS (2015)
Kiureghian, A.D., Ditlevsen, O.: Aleatory or epistemic? Does it matter? Struct. Saf. 31, 105–112 (2009)
Kohl, S.A., et al.: A probabilistic u-net for segmentation of ambiguous images. In: NIPS (2018)
Lampert, T.A., Stumpf, A., Gancarski, P.: An empirical study of expert agreement and ground truth estimation. IEEE Trans. Image Process. 25, 2557–2572 (2016)
Litjens, G., Debats, O., van de Ven, W., Karssemeijer, N., Huisman, H.: A pattern recognition approach to zonal segmentation of the prostate on MRI. In: Ayache, N., Delingette, H., Golland, P., Mori, K. (eds.) MICCAI 2012. LNCS, vol. 7511, pp. 413–420. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-33418-4_51
MacKay, D.J.C.: Bayesian interpolation. Neural Comput. 4(3), 415–447 (1992)
Simard, P.Y., Steinkraus, D., Platt, J.C.: Best practices for convolutional neural networks applied to visual document analysis. In: International Conference on Document Analysis and Recognition (2003)
Tanno, R., et al.: Bayesian image quality transfer with CNNs: exploring uncertainty in dMRI super-resolution. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., Duchesne, S. (eds.) MICCAI 2017. LNCS, vol. 10433, pp. 611–619. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-66182-7_70
Wang, S., et al.: Central focused convolutional neural networks: developing a data-driven model for lung nodule segmentation. Med. Image Anal. 40, 172–183 (2017)
Acknowledgements
We thank Dimitrios Mavroeidis for helpful discussions and Arsenii Ashukha for the variational dropout code. This research was supported by NWO Perspective Grants DLMedIA and EDL, as well as the in-cash and in-kind contributions by Philips.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Hu, S., Worrall, D., Knegt, S., Veeling, B., Huisman, H., Welling, M. (2019). Supervised Uncertainty Quantification for Segmentation with Multiple Annotations. In: Shen, D., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2019. MICCAI 2019. Lecture Notes in Computer Science(), vol 11765. Springer, Cham. https://doi.org/10.1007/978-3-030-32245-8_16
Download citation
DOI: https://doi.org/10.1007/978-3-030-32245-8_16
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-32244-1
Online ISBN: 978-3-030-32245-8
eBook Packages: Computer ScienceComputer Science (R0)
