International Conference on Medical Image Computing and Computer-Assisted Intervention

MICCAI 2015: Medical Image Computing and Computer-Assisted Intervention -- MICCAI 2015 pp 710-718 | Cite as

Marginal Space Deep Learning: Efficient Architecture for Detection in Volumetric Image Data

  • Florin C. Ghesu
  • Bogdan Georgescu
  • Yefeng Zheng
  • Joachim Hornegger
  • Dorin Comaniciu
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9349)


Current state-of-the-art techniques for fast and robust parsing of volumetric medical image data exploit large annotated image databases and are typically based on machine learning methods. Two main challenges to be solved are the low efficiency in scanning large volumetric input images and the need for manual engineering of image features. This work proposes Marginal Space Deep Learning (MSDL) as an effective solution, that combines the strengths of efficient object parametrization in hierarchical marginal spaces with the automated feature design of Deep Learning (DL) network architectures. Representation learning through DL automatically identifies, disentangles and learns explanatory factors directly from low-level image data. However, the direct application of DL to volumetric data results in a very high complexity, due to the increased number of transformation parameters. For example, the number of parameters defining a similarity transformation increases to 9 in 3D (3 for location, 3 for orientation and 3 for scale). The mechanism of marginal space learning provides excellent run-time performance by learning classifiers in high probability regions in spaces of gradually increasing dimensionality, for example starting from location only (3D) to location and orientation (6D) and full parameter space (9D). In addition, for parametrized feature computation, we propose to simplify the network by replacing the standard, pre-determined feature sampling pattern with a sparse, adaptive, self-learned pattern. The MSDL framework is evaluated on detecting the aortic heart valve in 3D ultrasound data. The dataset contains 3795 volumes from 150 patients. Our method outperforms the state-of-the-art with an improvement of 36 than one second. To our knowledge this is the first successful demonstration of the DL potential to detection in full 3D data with parametrized representations.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Bengio, Y., Courville, A.C., Vincent, P.: Unsupervised Feature Learning and Deep Learning: A Review and New Perspectives. CoRR abs/1206.5538 (2012)Google Scholar
  2. 2.
    Lowe, D.G.: Object recognition from local scale-invariant features. In: ICCV, vol. 2, pp. 1150–1157 (1999)Google Scholar
  3. 3.
    Zheng, Y., Barbu, A., Georgescu, B., Scheuering, M., Comaniciu, D.: Four-Chamber Heart Modeling and Automatic Segmentation for 3-D Cardiac CT Volumes Using Marginal Space Learning and Steerable Features. IEEE TMI 27(11), 1668–1681 (2008)Google Scholar
  4. 4.
    Lecun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proceedings of the IEEE 86(11), 2278–2324 (1998)CrossRefGoogle Scholar
  5. 5.
    Hinton, G.E., Osindero, S., Teh, Y.W.: A Fast Learning Algorithm for Deep Belief Nets. NIPS 18(7), 1527–1554 (2006)MathSciNetMATHGoogle Scholar
  6. 6.
    Bengio, Y., Lamblin, P., Popovici, D., Larochelle, H., Montréal, U.D., Québec, M.: Greedy layer-wise training of deep networks. In: NIPS. MIT Press (2007)Google Scholar
  7. 7.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet Classification with Deep Convolutional Neural Networks. In: NIPS, pp. 1097–1105. Curran Associates, Inc. (2012)Google Scholar
  8. 8.
    Ciresan, D.C., Meier, U., Schmidhuber, J.: Multi-column Deep Neural Networks for Image Classification. CoRR abs/1202.2745 (2012)Google Scholar
  9. 9.
    Tu, Z.: Probabilistic Boosting-Tree: Learning Discriminative Models for Classification, Recognition, and Clustering. In: IEEE 10th ICCV, ICCV, pp. 1589–1596 (2005)Google Scholar
  10. 10.
    Shin, H.C., Orton, M., Collins, D.J., Doran, S.J., Leach, M.O.: Stacked Autoencoders for Unsupervised Feature Learning and Multiple Organ Detection in a Pilot Study Using 4D Patient Data. IEEE PAMI 35(8), 1930–1943 (2013)CrossRefGoogle Scholar
  11. 11.
    Ciresan, D., Giusti, A., Gambardella, L.M., Schmidhuber, J.: Deep Neural Networks Segment Neuronal Membranes in Electron Microscopy Images. In: Pereira, F., Burges, C., Bottou, L., Weinberger, K. (eds.) NIPS, pp. 2843–2851. Curran Associates, Inc. (2012)Google Scholar
  12. 12.
    Roth, H.R., et al.: A New 2.5D Representation for Lymph Node Detection Using Random Sets of Deep Convolutional Neural Network Observations. In: Golland, P., Hata, N., Barillot, C., Hornegger, J., Howe, R. (eds.) MICCAI 2014, Part I. LNCS, vol. 8673, pp. 520–527. Springer, Heidelberg (2014)Google Scholar
  13. 13.
    Rumelhart, D.E., Hinton, G.E., Williams, R.J.: Parallel Distributed Processing: Explorations in the Microstructure of Cognition, pp. 318–362. MIT Press (1986)Google Scholar
  14. 14.
    Ionasec, R.I., Voigt, I., Georgescu, B., Wang, Y., Houle, H., Vega-Higuera, F., Navab, N., Comaniciu, D.: Patient-specific modeling and quantification of the aortic and mitral valves from 4-D cardiac CT and TEE. IEEE Trans. Med. Imaging 29(9), 1636–1651 (2010)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Florin C. Ghesu
    • 1
    • 2
  • Bogdan Georgescu
    • 1
  • Yefeng Zheng
    • 1
  • Joachim Hornegger
    • 2
  • Dorin Comaniciu
    • 1
  1. 1.Imaging and Computer VisionSiemens Corporate TechnologyPrincetonUSA
  2. 2.Pattern Recognition LabFriedrich-Alexander-UniversitätErlangen-NürnbergGermany

Personalised recommendations