Skip to main content
SpringerLink
Log in

Uncertain-DeepSSM: From Images to Probabilistic Shape Models

  • Conference paper
  • First Online:
Book cover Shape in Medical Imaging (ShapeMI 2020)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 12474))

Included in the following conference series:

Abstract

Statistical shape modeling (SSM) has recently taken advantage of advances in deep learning to alleviate the need for a time-consuming and expert-driven workflow of anatomy segmentation, shape registration, and the optimization of population-level shape representations. DeepSSM is an end-to-end deep learning approach that extracts statistical shape representation directly from unsegmented images with little manual overhead. It performs comparably with state-of-the-art shape modeling methods for estimating morphologies that are viable for subsequent downstream tasks. Nonetheless, DeepSSM produces an overconfident estimate of shape that cannot be blindly assumed to be accurate. Hence, conveying what DeepSSM does not know, via quantifying granular estimates of uncertainty, is critical for its direct clinical application as an on-demand diagnostic tool to determine how trustworthy the model output is. Here, we propose Uncertain-DeepSSM as a unified model that quantifies both, data-dependent aleatoric uncertainty by adapting the network to predict intrinsic input variance, and model-dependent epistemic uncertainty via a Monte Carlo dropout sampling to approximate a variational distribution over the network parameters. Experiments show an accuracy improvement over DeepSSM while maintaining the same benefits of being end-to-end with little pre-processing.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Baccetti, T., Franchi, L., McNamara, J.: Thin-plate spline analysis of treatment effects of rapid maxillary expansion and face mask therapy in early class III malocclusions. Eur. J. Orthod. 21(3), 275–281 (1999)

    Article  Google Scholar 

  2. Bhalodia, R., Dvoracek, L.A., Ayyash, A.M., Kavan, L., Whitaker, R., Goldstein, J.A.: Quantifying the severity of metopic craniosynostosis: a pilot study application of machine learning in craniofacial surgery. J. Craniofac. Surg. 31, 697–701 (2020)

    Google Scholar 

  3. Bhalodia, R., Elhabian, S.Y., Kavan, L., Whitaker, R.T.: Deepssm: a deep learning framework for statistical shape modeling from raw images. CoRR abs/1810.00111 (2018). http://arxiv.org/abs/1810.00111

  4. Bhalodia, R., et al.: Deep learning for end-to-end atrial fibrillation recurrence estimation. In: Computing in Cardiology, CinC 2018, Maastricht, The Netherlands, 23–26 September 2018 (2018)

    Google Scholar 

  5. Bookstein, F.L.: Principal warps: thin-plate splines and the decomposition of deformations. IEEE Trans. Pattern Anal. Mach. Intell. 11(6), 567–585 (1989)

    Article  Google Scholar 

  6. Bryan, R., Nair, P.B., Taylor, M.: Use of a statistical model of the whole femur in a large scale, multi-model study of femoral neck fracture risk. J. Biomech. 42(13), 2171–2176 (2009)

    Article  Google Scholar 

  7. Cates, J., et al.: Computational shape models characterize shape change of the left atrium in atrial fibrillation. Clin. Med. Insights Cardiol. 8, CMC-S15710 (2014)

    Google Scholar 

  8. Cates, J., Elhabian, S., Whitaker, R.: Shapeworks: particle-based shape correspondence and visualization software. In: Statistical Shape and Deformation Analysis, pp. 257–298. Elsevier (2017)

    Google Scholar 

  9. Cates, J., Fletcher, P.T., Styner, M., Shenton, M., Whitaker, R.: Shape modeling and analysis with entropy-based particle systems. In: Karssemeijer, N., Lelieveldt, B. (eds.) IPMI 2007. LNCS, vol. 4584, pp. 333–345. Springer, Heidelberg (2007). https://doi.org/10.1007/978-3-540-73273-0_28

    Chapter  Google Scholar 

  10. Chang, J., Fisher, J.W.: Efficient MCMC sampling with implicit shape representations. In: CVPR 2011, pp. 2081–2088. IEEE (2011)

    Google Scholar 

  11. Davies, R.H., Twining, C.J., Cootes, T.F., Waterton, J.C., Taylor, C.J.: A minimum description length approach to statistical shape modeling. IEEE Trans. Med. Imaging 21(5), 525–537 (2002). https://doi.org/10.1109/TMI.2002.1009388

    Article  MATH  Google Scholar 

  12. Denker, J.S., LeCun, Y.: Transforming neural-net output levels to probability distributions. In: Advances in Neural Information Processing Systems, pp. 853–859 (1991)

    Google Scholar 

  13. Durrleman, S., et al.: Morphometry of anatomical shape complexes with dense deformations and sparse parameters. NeuroImage 101, 35–49 (2014)

    Article  Google Scholar 

  14. Gal, Y.: Uncertainty in deep learning. Univ. Camb. 1, 3 (2016)

    Google Scholar 

  15. Gal, Y., Ghahramani, Z.: Bayesian convolutional neural networks with bernoulli approximate variational inference. arXiv preprint arXiv:1506.02158 (2015)

  16. Gal, Y., Ghahramani, Z.: Dropout as a Bayesian approximation: representing model uncertainty in deep learning. In: International Conference on Machine Learning, pp. 1050–1059 (2016)

    Google Scholar 

  17. Galloway, F., et al.: A large scale finite element study of a cementless osseointegrated tibial tray. J. Biomech. 46(11), 1900–1906 (2013)

    Article  Google Scholar 

  18. Gardner, G., Morris, A., Higuchi, K., MacLeod, R., Cates, J.: A point-correspondence approach to describing the distribution of image features on anatomical surfaces, with application to atrial fibrillation. In: 2013 IEEE 10th International Symposium on Biomedical Imaging, pp. 226–229, April 2013. https://doi.org/10.1109/ISBI.2013.6556453

  19. Gerig, G., Styner, M., Jones, D., Weinberger, D., Lieberman, J.: Shape analysis of brain ventricles using spharm. In: Proceedings IEEE Workshop on Mathematical Methods in Biomedical Image Analysis (MMBIA 2001), pp. 171–178 (2001). https://doi.org/10.1109/MMBIA.2001.991731

  20. Gielis, J.: A generic transformation that unifies a wide range of natural and abstract shapes (2013)

    Google Scholar 

  21. Glorot, X., Bengio, Y.: Understanding the difficulty of training deep feedforward neural networks. In: Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics. Proceedings of Machine Learning Research, vol. 9, pp. 249–256. PMLR, 13–15 May 2010

    Google Scholar 

  22. Harris, M.D., Datar, M., Whitaker, R.T., Jurrus, E.R., Peters, C.L., Anderson, A.E.: Statistical shape modeling of cam femoroacetabular impingement. J. Orthop. Res. 31(10), 1620–1626 (2013). https://doi.org/10.1002/jor.22389

  23. He, K., Zhang, X., Ren, S., Sun, J.: Delving deep into rectifiers: surpassing human-level performance on imagenet classification. CoRR abs/1502.01852 (2015). http://arxiv.org/abs/1502.01852

  24. Ho, S.Y., Cabrera, J.A., Sanchez-Quintana, D.: Left atrial anatomy revisited. Circ. Arrhythmia Electrophysiol. 5(1), 220–228 (2012)

    Google Scholar 

  25. Huang, W., Bridge, C.P., Noble, J.A., Zisserman, A.: Temporal heartnet: towards human-level automatic analysis of fetal cardiac screening video. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D., Duchesne, S. (eds.) MICCAI 2017. LNCS, vol. 10434, pp. 341–349. Springer International Publishing, Cham (2017). https://doi.org/10.1007/978-3-319-66185-8_39

    Chapter  Google Scholar 

  26. Joshi, S.C., Miller, M.I.: Landmark matching via large deformation diffeomorphisms. IEEE Trans. Image Process. 9(8), 1357–1370 (2000)

    Article  MathSciNet  Google Scholar 

  27. Joshi, S., Davis, B., Jomier, M., Gerig, G.: Unbiased diffeomorphic atlas construction for computational anatomy. NeuroImage 23(Supplement1), S151–S160 (2004)

    Article  Google Scholar 

  28. Kendall, A., Gal, Y.: What uncertainties do we need in Bayesian deep learning for computer vision? CoRR abs/1703.04977 (2017). http://arxiv.org/abs/1703.04977

  29. Kingma, D., Ba, J.: Adam: a method for stochastic optimization. In: International Conference on Learning Representations (2014)

    Google Scholar 

  30. Kiureghian, A.D., Ditlevsen, O.D.: Aleatory or epistemic? Does it matter? (2009)

    Google Scholar 

  31. Kozic, N., et al.: Optimisation of orthopaedic implant design using statistical shape space analysis based on level sets. Med. Image Anal. 14(3), 265–275 (2010)

    Article  Google Scholar 

  32. Kwon, Y., Won, J.H., Kim, B.J., Paik, M.C.: Uncertainty quantification using Bayesian neural networks in classification: application to ischemic stroke lesion segmentation (2018)

    Google Scholar 

  33. Lamecker, H., Lange, T., Seebass, M.: A statistical shape model for the liver. In: Dohi, T., Kikinis, R. (eds.) MICCAI 2002. LNCS, vol. 2489, pp. 421–427. Springer, Heidelberg (2002). https://doi.org/10.1007/3-540-45787-9_53

    Chapter  Google Scholar 

  34. Lê, M., Unkelbach, J., Ayache, N., Delingette, H.: Sampling image segmentations for uncertainty quantification. Med. Image Anal. 34, 42–51 (2016)

    Article  Google Scholar 

  35. Le, Q.V., Smola, A.J., Canu, S.: Heteroscedastic Gaussian process regression. In: Proceedings of the 22nd International Conference on Machine Learning, pp. 489–496 (2005)

    Google Scholar 

  36. Li, X., Chen, S., Hu, X., Yang, J.: Understanding the disharmony between dropout and batch normalization by variance shift. CoRR abs/1801.05134 (2018). http://arxiv.org/abs/1801.05134

  37. MacKay, D.J.: A practical Bayesian framework for backpropagation networks. Neural Comput. 4(3), 448–472 (1992)

    Article  Google Scholar 

  38. Milletari, F., Rothberg, A., Jia, J., Sofka, M.: Integrating statistical prior knowledge into convolutional neural networks. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., Duchesne, S. (eds.) MICCAI 2017. LNCS, vol. 10433, pp. 161–168. Springer International Publishing, Cham (2017). https://doi.org/10.1007/978-3-319-66182-7_19

    Chapter  Google Scholar 

  39. Moghaddam, B., Pentland, A.: Probabilistic visual learning for object representation. IEEE Trans. Pattern Anal. Mach. Intell. 19(7), 696–710 (1997)

    Article  Google Scholar 

  40. Nix, D.A., Weigend, A.S.: Estimating the mean and variance of the target probability distribution. In: Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN 1994), vol. 1, pp. 55–60. IEEE (1994)

    Google Scholar 

  41. Oktay, O., et al.: Anatomically constrained neural networks (ACNN): application to cardiac image enhancement and segmentation. CoRR abs/1705.08302 (2017). http://arxiv.org/abs/1705.08302

  42. Reinhold, J.C., et al.: Validating uncertainty in medical image translation. In: 2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI), pp. 95–98. IEEE (2020)

    Google Scholar 

  43. Sarkalkan, N., Weinans, H., Zadpoor, A.A.: Statistical shape and appearance models of bones. Bone 60, 129–140 (2014)

    Article  Google Scholar 

  44. Selvan, R., Faye, F., Middleton, J., Pai, A.: Uncertainty quantification in medical image segmentation with normalizing flows. arXiv preprint arXiv:2006.02683 (2020)

  45. Styner, M., et al.: Framework for the statistical shape analysis of brain structures using SPHARM-PDM (2006)

    Google Scholar 

  46. Tóthová, K., et al.: Uncertainty quantification in cnn-based surface prediction using shape priors. CoRR abs/1807.11272 (2018). http://arxiv.org/abs/1807.11272

  47. Wang, D., Shi, L., Griffith, J.F., Qin, L., Yew, D.T., Riggs, C.M.: Comprehensive surface-based morphometry reveals the association of fracture risk and bone geometry. J. Orthop. Res. 30(8), 1277–1284 (2012)

    Article  Google Scholar 

  48. Xie, J., Dai, G., Zhu, F., Wong, E.K., Fang, Y.: Deepshape: deep-learned shape descriptor for 3d shape retrieval. IEEE Trans. Pattern Anal. Mach. Intell. 39(7), 1335–1345 (2017)

    Article  Google Scholar 

  49. Zhao, Z., et al.: Hippocampus shape analysis and late-life depression. PLoS One 3(3), e1837 (2008)

    Google Scholar 

  50. Zheng, Y., Liu, D., Georgescu, B., Nguyen, H., Comaniciu, D.: 3D deep learning for efficient and robust landmark detection in volumetric data. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9349, pp. 565–572. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24553-9_69

    Chapter  Google Scholar 

Download references

Acknowledgement

This work was supported by the National Institutes of Health under grant numbers NIBIB-U24EB029011, NIAMS-R01AR076120, NHLBI-R01HL135568, and NIGMS-P41GM103545. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. MRI scans and segmentation were obtained retrospectively from the AFib database at the University of Utah. The authors would like to thank the Division of Cardiovascular Medicine (data were collected under Nassir Marrouche, MD, oversight and currently managed by Brent Wilson, MD, PhD) at the University of Utah for providing the left atrium MRI scans and their corresponding segmentations.

Author information

Authors and Affiliations

  1. Scientific Computing and Imaging Institute, University of Utah, Salt Lake City, UT, USA

    Jadie Adams, Riddhish Bhalodia & Shireen Elhabian

  2. School of Computing, University of Utah, Salt Lake City, UT, USA

    Jadie Adams, Riddhish Bhalodia & Shireen Elhabian

Authors
  1. Jadie Adams
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Riddhish Bhalodia
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shireen Elhabian
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Jadie Adams , Riddhish Bhalodia or Shireen Elhabian .

Editor information

Editors and Affiliations

  1. Harvard Medical School, Boston, USA

    Martin Reuter

  2. Klinikum der Universität München, LMU München, München, Germany

    Christian Wachinger

  3. Inria Sophia-Antipolis, Valbonne, France

    Hervé Lombaert

  4. University of North Carolina at Chapel Hill, Kitware Inc., Carrboro, NC, USA

    Beatriz Paniagua

  5. ETH Zurich, Zürich, Switzerland

    Orcun Goksel

  6. Istanbul Technical University, Istanbul, Turkey

    Islem Rekik

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Adams, J., Bhalodia, R., Elhabian, S. (2020). Uncertain-DeepSSM: From Images to Probabilistic Shape Models. In: Reuter, M., Wachinger, C., Lombaert, H., Paniagua, B., Goksel, O., Rekik, I. (eds) Shape in Medical Imaging. ShapeMI 2020. Lecture Notes in Computer Science(), vol 12474. Springer, Cham. https://doi.org/10.1007/978-3-030-61056-2_5

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-61056-2_5

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-61055-5

  • Online ISBN: 978-3-030-61056-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Societies and partnerships

Search

Navigation