Integrating Statistical Prior Knowledge into Convolutional Neural Networks

  • Fausto MilletariEmail author
  • Alex Rothberg
  • Jimmy Jia
  • Michal Sofka
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10433)


In this work we show how to integrate prior statistical knowledge, obtained through principal components analysis (PCA), into a convolutional neural network in order to obtain robust predictions even when dealing with corrupted or noisy data. Our network architecture is trained end-to-end and includes a specifically designed layer which incorporates the dataset modes of variation discovered via PCA and produces predictions by linearly combining them. We also propose a mechanism to focus the attention of the CNN on specific regions of interest of the image in order to obtain refined predictions. We show that our method is effective in challenging segmentation and landmark localization tasks.


  1. 1.
    Ahmadi, S.-A., Baust, M., Karamalis, A., Plate, A., Boetzel, K., Klein, T., Navab, N.: Midbrain segmentation in transcranial 3D ultrasound for parkinson diagnosis. In: Fichtinger, G., Martel, A., Peters, T. (eds.) MICCAI 2011. LNCS, vol. 6893, pp. 362–369. Springer, Heidelberg (2011). doi: 10.1007/978-3-642-23626-6_45CrossRefGoogle Scholar
  2. 2.
    Çiçek, Ö., Abdulkadir, A., Lienkamp, S.S., Brox, T., Ronneberger, O.: 3D U-Net: learning dense volumetric segmentation from sparse annotation. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) MICCAI 2016. LNCS, vol. 9901, pp. 424–432. Springer, Cham (2016). doi: 10.1007/978-3-319-46723-8_49CrossRefGoogle Scholar
  3. 3.
    Cootes, T.F., Beeston, C., Edwards, G.J., Taylor, C.J.: A unified framework for atlas matching using active appearance models. In: Kuba, A., Šáamal, M., Todd-Pokropek, A. (eds.) IPMI 1999. LNCS, vol. 1613, pp. 322–333. Springer, Heidelberg (1999). doi: 10.1007/3-540-48714-X_24CrossRefGoogle Scholar
  4. 4.
    Cootes, T.F., Edwards, G.J., Taylor, C.J.: Active appearance models. In: Burkhardt, H., Neumann, B. (eds.) ECCV 1998. LNCS, vol. 1407, pp. 484–498. Springer, Heidelberg (1998). doi: 10.1007/BFb0054760CrossRefGoogle Scholar
  5. 5.
    Cootes, T.F., Taylor, C.J., Cooper, D.H., Graham, J.: Active shape models-their training and application. Comput. Vis. Image Underst. 61(1), 38–59 (1995)CrossRefGoogle Scholar
  6. 6.
    Jaderberg, M., Simonyan, K., Zisserman, A., et al.: Spatial transformer networks. In: Advances in Neural Information Processing Systems, pp. 2017–2025 (2015)Google Scholar
  7. 7.
    Kroll, C., Milletari, F., Navab, N., Ahmadi, S.-A.: Coupling convolutional neural networks and hough voting for robust segmentation of ultrasound volumes. In: Rosenhahn, B., Andres, B. (eds.) GCPR 2016. LNCS, vol. 9796, pp. 439–450. Springer, Cham (2016). doi: 10.1007/978-3-319-45886-1_36CrossRefGoogle Scholar
  8. 8.
    Milletari, F., Ahmadi, S.-A., Kroll, C., Hennersperger, C., Tombari, F., Shah, A., Plate, A., Boetzel, K., Navab, N.: Robust segmentation of various anatomies in 3d ultrasound using hough forests and learned data representations. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9350, pp. 111–118. Springer, Cham (2015). doi: 10.1007/978-3-319-24571-3_14CrossRefGoogle Scholar
  9. 9.
    Milletari, F., Ahmadi, S.A., Kroll, C., Plate, A., Rozanski, V., Maiostre, J., Levin, J., Dietrich, O., Ertl-Wagner, B., Boetzel, K., et al.: Hough-cnn: deep learning for segmentation of deep brain regions in mri and ultrasound. arXiv preprint (2016). arXiv:1601.07014
  10. 10.
    Milletari, F., Navab, N., Ahmadi, S.A.: V-net: Fully convolutional neural networks for volumetric medical image segmentation. arXiv preprint (2016). arXiv:1606.04797
  11. 11.
    Milletari, F., Yigitsoy, M., Navab, N.: Left ventricle segmentation in cardiac ultrasound using hough-forests with implicit shape and appearance priorsGoogle Scholar
  12. 12.
    Mitchell, S.C., Bosch, J.G., Lelieveldt, B.P., Van der Geest, R.J., Reiber, J.H., Sonka, M.: 3-d active appearance models: segmentation of cardiac mr and ultrasound images. IEEE Trans. Med. Imaging 21(9), 1167–1178 (2002)CrossRefGoogle Scholar
  13. 13.
    Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). doi: 10.1007/978-3-319-24574-4_28CrossRefGoogle Scholar
  14. 14.
    Sofka, M., Wetzl, J., Birkbeck, N., Zhang, J., Kohlberger, T., Kaftan, J., Declerck, J., Zhou, S.K.: Multi-stage learning for robust lung segmentation in challenging CT volumes. In: Fichtinger, G., Martel, A., Peters, T. (eds.) MICCAI 2011. LNCS, vol. 6893, pp. 667–674. Springer, Heidelberg (2011). doi: 10.1007/978-3-642-23626-6_82CrossRefGoogle Scholar
  15. 15.
    Wold, S., Esbensen, K., Geladi, P.: Principal component analysis. Chemometrics Intell. Lab. Syst. 2(1–3), 37–52 (1987)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Fausto Milletari
    • 1
    Email author
  • Alex Rothberg
    • 1
  • Jimmy Jia
    • 1
  • Michal Sofka
    • 1
  1. 1.4Catalyzer CorporationNew York CityUSA

Personalised recommendations