Learning Deep Features for Automated Placement of Correspondence Points on Ensembles of Complex Shapes

  • Praful AgrawalEmail author
  • Ross T. Whitaker
  • Shireen Y. Elhabian
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10433)


Correspondence-based shape models are an enabling technology for various medical imaging applications that rely on statistical analysis of populations of anatomical shape. One strategy for automatic correspondence placement is to simultaneously learn a compact representation of the underlying anatomical variation in the shape space while capturing the geometric characteristics of individual shapes. The inherent geometric complexity and population-level shape variation in anatomical structures introduce significant challenges in finding optimal shape correspondence models. Existing approaches adopt iterative optimization schemes with objective functions derived from probabilistic modeling of shape space, e.g. entropy of Gaussian-distributed shape space, to find useful sets of dense correspondence on shape ensembles. Nonetheless, anatomical shape distributions can be far more complex than this Gaussian assumption, which entails linear shape variation. Recent works address this limitation by adopting an application-specific notion of correspondence through lifting positional data to a higher-dimensional feature space (e.g. sulcal depth, brain connectivity, and geodesic distance to anatomical landmarks), with the goal of simplifying the optimization problem. However, this typically requires a careful selection of hand-crafted features and their success heavily rely on expertise in finding such features consistently. This paper proposes an automated feature learning approach using deep convolutional neural networks for optimization of dense point correspondence on shape ensembles. The proposed method endows anatomical shapes with learned features that enhance the shape correspondence objective function to deal with complex objects and populations. Results demonstrate that deep learning based features perform better than methods that rely on position and compete favorably with hand-crafted features.


Deep learning Correspondence models Statistical shape modeling 



Authors would like to thank Heath B. Henninger, PhD and Matthijs Jacxsens, MD for providing Scapula shapes with anatomical landmarks. This work was supported by NIH grants P41-GM103545-19 and R01-EB016701.


  1. 1.
    Balestra, S., Schumann, S., Heverhagen, J., Nolte, L., Zheng, G.: Articulated statistical shape model-based 2D-3D reconstruction of a hip joint. In: Stoyanov, D., Collins, D.L., Sakuma, I., Abolmaesumi, P., Jannin, P. (eds.) IPCAI 2014. LNCS, vol. 8498, pp. 128–137. Springer, Cham (2014). doi: 10.1007/978-3-319-07521-1_14CrossRefGoogle Scholar
  2. 2.
    Boscaini, D., Masci, J., Rodolà, E., Bronstein, M.: Learning shape correspondence with anisotropic convolutional neural networks. In: NIPS, pp. 3189–3197 (2016)Google Scholar
  3. 3.
    Bredbenner, T.L., Eliason, T.D., Potter, R.S., Mason, R.L., Havill, L.M., Nicolella, D.P.: Statistical shape modeling describes variation in tibia and femur surface geometry between control and incidence groups from the osteoarthritis initiative database. J. Biomech. 43(9), 1780–1786 (2010)CrossRefGoogle Scholar
  4. 4.
    Cates, J., Fletcher, P.T., Styner, M., Shenton, M., Whitaker, R.: Shape modeling and analysis with entropy-based particle systems. In: Karssemeijer, N., Lelieveldt, B. (eds.) IPMI 2007. LNCS, vol. 4584, pp. 333–345. Springer, Heidelberg (2007). doi: 10.1007/978-3-540-73273-0_28CrossRefGoogle Scholar
  5. 5.
    Chopra, S., Hadsell, R., LeCun, Y.: Learning a similarity metric discriminatively, with application to face verification. In: CVPR, vol. 1, pp. 539–546 (2005)Google Scholar
  6. 6.
    Datar, M., Lyu, I., Kim, S.H., Cates, J., Styner, M.A., Whitaker, R.: Geodesic distances to landmarks for dense correspondence on ensembles of complex shapes. In: Mori, K., Sakuma, I., Sato, Y., Barillot, C., Navab, N. (eds.) MICCAI 2013. LNCS, vol. 8150, pp. 19–26. Springer, Heidelberg (2013). doi: 10.1007/978-3-642-40763-5_3CrossRefGoogle Scholar
  7. 7.
    Davies, R.H., Twining, C.J., Cootes, T.F., Waterton, J.C., Taylor, C.J.: A minimum description length approach to statistical shape modeling. IEEE TMI 21(5), 525–537 (2002)zbMATHGoogle Scholar
  8. 8.
    Heimann, T., Meinzer, H.P.: Statistical shape models for 3D medical image segmentation: a review. MedIA 13(4), 543–563 (2009)Google Scholar
  9. 9.
    Meyer, M., Kirby, R.M., Whitaker, R.: Topology, accuracy, and quality of isosurface meshes using dynamic particles. IEEE TVCG 13(6), 1704–1711 (2007)Google Scholar
  10. 10.
    Oguz, I., Cates, J., Datar, M., Paniagua, B., Fletcher, T., Vachet, C., Styner, M., Whitaker, R.: Entropy-based particle correspondence for shape populations. IJCARS 11(7), 1221–1232 (2016)Google Scholar
  11. 11.
    Oguz, I., Cates, J., Fletcher, T., Whitaker, R., Cool, D., Aylward, S., Styner, M.: Cortical correspondence using entropy-based particle systems and local features. In: ISBI, pp. 1637–1640 (2008)Google Scholar
  12. 12.
    Oguz, I., Niethammer, M., Cates, J., Whitaker, R., Fletcher, T., Vachet, C., Styner, M.: Cortical correspondence with probabilistic fiber connectivity. In: Prince, J.L., Pham, D.L., Myers, K.J. (eds.) IPMI 2009. LNCS, vol. 5636, pp. 651–663. Springer, Heidelberg (2009). doi: 10.1007/978-3-642-02498-6_54CrossRefGoogle Scholar
  13. 13.
    Rusinkiewicz, S.: Estimating curvatures and their derivatives on triangle meshes. In: IEEE 3DPVT, pp. 486–493 (2004)Google Scholar
  14. 14.
    Sarkalkan, N., Weinans, H., Zadpoor, A.A.: Statistical shape and appearance models of bones. Bone 60, 129–140 (2014)CrossRefGoogle Scholar
  15. 15.
    Shen, K.K., Fripp, J., Mériaudeau, F., Chételat, G., Salvado, O., Bourgeat, P.: Detecting global and local hippocampal shape changes in Alzheimer’s disease using statistical shape models. Neuroimage 59(3), 2155–2166 (2012)CrossRefGoogle Scholar
  16. 16.
    Styner, M., Oguz, I., Xu, S., Brechbühler, C., Pantazis, D., Levitt, J.J., Shenton, M.E., Gerig, G.: Framework for the statistical shape analysis of brain structures using SPHARM-PDM. Insight J. 1071, 242 (2006)Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Praful Agrawal
    • 1
    Email author
  • Ross T. Whitaker
    • 1
  • Shireen Y. Elhabian
    • 1
  1. 1.Scientific Computing and Imaging InstituteUniversity of UtahSalt Lake CityUSA

Personalised recommendations