Skip to main content

Error-Controlled Model Approximation for Gaussian Process Morphable Models


Gaussian Process Morphable Models (GPMMs) unify a variety of non-rigid deformation models for surface and image registration. Deformation models, such as B-splines, radial basis functions, and PCA models are defined as a probability distribution using a Gaussian process. The method depends heavily on the low-rank approximation of the Gaussian process, which is mandatory to obtain a parametric representation of the model. In this article, we propose the use of the pivoted Cholesky decomposition for this task, which has the following advantages: (1) Compared to the current state of the art used in GPMMs, it provides a fully controllable approximation error. The algorithm greedily computes new basis functions until the user-defined approximation accuracy is reached. (2) Unlike the currently used approach, this method can be used in a black-box-like scenario, whereas the method automatically chooses the amount of basis functions for a given model and accuracy. (3) We propose the Newton basis as an alternative basis for GPMMs. The proposed basis does not need an SVD computation and can be iteratively refined. We show that the proposed basis functions achieve competitive registration results while providing the mentioned advantages for its computation.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9


  1. Albrecht, T., Lüthi, M., Gerig, T., Vetter, T.: Posterior shape models. Med. Image Anal. 17(8), 959–973 (2013)

    Article  Google Scholar 

  2. Amberg, B., Romdhani, S., Vetter, T.: Optimal step nonrigid icp algorithms for surface registration. In: 2007 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8. IEEE (2007)

  3. Amit, Y., Grenander, U., Piccioni, M.: Structural image restoration through deformable templates. J. Am. Stat. Assoc. 86(414), 376–387 (1991)

    Article  Google Scholar 

  4. Beebe, N.H.F., Linderberg, J.: Simplifications in the generation and transformation of two-electron integrals in molecular calculations. Int. J. Quantum Chem. 12(4), 683–705 (1977)

    Article  Google Scholar 

  5. Berlinet, A., Thomas-Agnan, C.: Reproducing Kernel Hilbert Spaces in Probability and Statistics, vol. 3. Springer, Berlin (2004)

    Book  MATH  Google Scholar 

  6. Blanz, V., Vetter, T.: A morphable model for the synthesis of 3d faces. In: SIGGRAPH ’99: Proceedings of the 26th Annual Conference on Computer Graphics and Interactive Techniques, pp. 187–194. ACM Press (1999)

  7. Cootes, T.F., Beeston, C., Edwards, G.J., Taylor, C.J .: A unified framework for atlas matching using active appearance models. In: Biennial International Conference on Information Processing in Medical Imaging, pp. 322–333. Springer (1999)

  8. Cuadra, M.B., Duay, V., Thiran, J.-P.: Atlas-based segmentation. In: Handbook of Biomedical Imaging, pp. 221–244. Springer (2015)

  9. Foster, L., Waagen, A., Aijaz, N., Hurley, M., Luis, A., Rinsky, J., Satyavolu, C., Way, M.J., Gazis, P., Srivastava, A.: Stable and efficient Gaussian process calculations. J. Mach. Learn. Res. 10, 857–882 (2009)

    MathSciNet  MATH  Google Scholar 

  10. Gerig, T., Morel-Forster, A., Blumer, C., Egger, B., Luthi, M., Schoenborn, S., Vetter, T.: Morphable face models—an open framework. In: 2018 13th IEEE International Conference on Automatic Face Gesture Recognition (FG 2018), pp. 75–82 (2018)

  11. Gerig, T., Shahim, K., Reyes, M., Vetter, T., Lüthi, M.: Spatially varying registration using Gaussian processes. In: Medical Image Computing and Computer-Assisted Intervention—MICCAI 2014, pp. 413–420. Springer (2014)

  12. Golub, G.H., Van Loan, C.F.: Matrix Computations, 4th edn. Johns Hopkins University Press, Baltimore (2012)

    MATH  Google Scholar 

  13. Grenander, U., Miller, M.I.: Computational anatomy: an emerging discipline. Q. Appl. Math. 56(4), 617–694 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  14. Griebel, M., Harbrecht, H.: Approximation of bi-variate functions: singular value decomposition versus sparse grids. IMA J. Numer. Anal. 34(1), 28–54 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  15. Hackbusch, W.: Integral Equations: Theory and Numerical Treatment, vol. 4. Birkhäuser, Basel (1995)

    Book  MATH  Google Scholar 

  16. Halko, N., Martinsson, P.-G., Tropp, J.A.: Finding structure with randomness: probabilistic algorithms for constructing approximate matrix decompositions. SIAM Rev. 53(2), 217–288 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  17. Harbrecht, H., Peters, M., Siebenmorgen, M.: Efficient approximation of random fields for numerical applications. Numer. Linear Algebra Appl. 22(4), 596–617 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  18. Harbrecht, H., Peters, M., Schneider, R.: On the low-rank approximation by the pivoted cholesky decomposition. Appl. Numer. Math. 62(4), 428–440 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  19. Heimann, T., Van Ginneken, B., Styner, M., Arzhaeva, Y., Aurich, V., Bauer, C., Beck, A., Becker, C., Beichel, R., Bekes, G.: Comparison and evaluation of methods for liver segmentation from ct datasets. IEEE Trans. Med. Imaging 28(8), 1251–1265 (2009)

    Article  Google Scholar 

  20. Iglesias, J.E., Sabuncu, M.R.: Multi-atlas segmentation of biomedical images: a survey. Med. Image Anal. 24(1), 205–219 (2015)

    Article  Google Scholar 

  21. Joshi, S.C., Banerjee, A., Christensen, G.E., Csernansky, J.G., Haller, J.W., Miller, M.I., Wang, L.: Gaussian random fields on sub-manifolds for characterizing brain surfaces. In: Information Processing in Medical Imaging, pp. 381–386. Springer (1997)

  22. Jud, C., Mori, N., Cattin, P.C.: Sparse kernel machines for discontinuous registration and nonstationary regularization. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 9–16 (2016)

  23. Klein, S., Staring, M., Murphy, K., Viergever, M.A., Pluim, J.P.W.: Elastix: a toolbox for intensity-based medical image registration. IEEE Trans. Med. Imaging 29(1), 196–205 (2010)

    Article  Google Scholar 

  24. Lüthi, M., Jud, C., Vetter, T.: Using landmarks as a deformation prior for hybrid image registration. In: Pattern Recognition, pp. 196–205 (2011)

  25. Lüthi, M., Forster, A., Gerig, T., Vetter, T.: Gaussian process morphable models. In: Zheng, G., Li, S., Szekely, G. (eds.) Statistical Shape and Deformation Analysis—Methods, Implementation and Applications. Academic Press (2017)

  26. Lüthi, M., Gerig, T., Jud, C., Vetter, T.: Gaussian process morphable models. IEEE Trans. Pattern Anal. Mach. Intell. 40(8), 1860–1873 (2018)

    Article  Google Scholar 

  27. Lüthi, M., Jud, C., Vetter, T.: A unified approach to shape model fitting and non-rigid registration. In: Machine Learning in Medical Imaging, pp. 66–73. Springer (2013)

  28. Ma, J., Zhao, J., Tian, J., Tu, Z., Yuille, A.L.: Robust estimation of nonrigid transformation for point set registration. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June (2013)

  29. Opfer, R.: Multiscale kernels. Adv. Comput. Math. 25(4), 357–380 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  30. Pazouki, M., Schaback, R.: Bases for kernel-based spaces. J. Comput. Appl. Math. 236(4), 575–588 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  31. Rasmussen, C.E., Williams, C.K.I.: Gaussian Processes for Machine Learning. Springer, Berlin (2006)

    MATH  Google Scholar 

  32. Rosasco, L., Belkin, M., De Vito, E.: On learning with integral operators. J. Mach. Learn. Res. 11, 905–934 (2010)

    MathSciNet  MATH  Google Scholar 

  33. Rueckert, D., Frangi, A.F., Schnabel, J.A.: Automatic construction of 3d statistical deformation models using non-rigid registration. In: MICCAI’01: Medical Image Computing and Computer-Assisted Intervention, pp. 77–84 (2001)

  34. Santin, G., Schaback, R.: Approximation of eigenfunctions in kernel-based spaces. Adv. Comput. Math. 42(4), 973–993 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  35. Scalismo-scalable image analysis and shape modelling. Accessed 10 Oct 2018

  36. Yuille, A., Kersten, D.: Vision as Bayesian inference: analysis by synthesis? Trends Cogn. Sci. 10(7), 301–308 (2006)

    Article  Google Scholar 

Download references


This work has been funded as part of two Swiss National Science foundation projects in the context of the Projects SNF153297 and SNF156101. We thank Andreas Morel-Forster and Volker Roth for interesting and enlightening discussions. A special thanks goes to Ghazi Bouabene and Christoph Langguth for their work on the Scalismo software, in which all the methods are implemented.

Author information

Authors and Affiliations


Corresponding author

Correspondence to Thomas Gerig.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Jürgen Dölz and Thomas Gerig share first authorship. Helmut Harbrecht and Thomas Vetter share last authorship. Author names in alphabetical order .

A Appendix

A Appendix

1.1 A.1 Advanced Nyström Schemes

Nyström schemes are suitable if the eigenfunctions of the Karhunen–Loève expansion are only required in certain predetermined points \(\mathbf {x}_1,\ldots ,\mathbf {x}_N\). For this purpose, the integral operator (2) is approximated by a quadrature formula

$$\begin{aligned} \int _\varOmega \mathbf {K}(\cdot ,\mathbf {x}) \mathbf {f}(\mathbf {x}) {\text {d}}\!\rho (\mathbf {x})\approx \sum _{i=1}^N\omega _i \mathbf {K}(\cdot ,\varvec{\xi }_i)\mathbf {f}(\varvec{\xi }_i) \end{aligned}$$

with quadrature points \(\varvec{\xi }_i\) and weights \(\omega _i\). The discrete eigenvalue problem then reads

$$\begin{aligned} \mathbf {C}_{\text {Nystr}}\hat{\varvec{\phi }}_{m,N}=\lambda _{m,N}\hat{\varvec{\phi }}_{m,N} \end{aligned}$$

with the system matrix

$$\begin{aligned} \mathbf {C}_{\text {Nystr}}=\big [\omega _j\mathbf {K}(\mathbf {x}_i,\mathbf {x}_j)\big ]_{i,j=1}^N \end{aligned}$$

and the point values

$$\begin{aligned} \hat{\varvec{\phi }}_{m,N}\approx \big [\varvec{\phi }_m(\mathbf {x}_i)\big ]_i,\quad i=1,\ldots ,N. \end{aligned}$$

Note that the system matrix \(\mathbf {C}_{\text {Nystr}}\) is not symmetric in general. Assuming positive quadrature weights, i.e. \(\omega _i>0\), defining

$$\begin{aligned} \mathbf {M}_{\text {Nystr}}={\text {diag}}(\sqrt{\omega _1},\ldots ,\sqrt{\omega _N}) \end{aligned}$$

and setting \(\varvec{\phi }_{m,N}=\mathbf {M}_{\text {Nystr}}\hat{\varvec{\phi }}_{m,N}\) yields a symmetric, generalized eigenvalue problem

$$\begin{aligned} \mathbf {M}_{\text {Nystr}}\mathbf {C}\mathbf {M}_{\text {Nystr}}^{\intercal }\varvec{\phi }_{m,N}=\lambda _{m,N}\mathbf {M}_{\text {Nystr}}\varvec{\phi }_{m,N} \end{aligned}$$

with the matrix

$$\begin{aligned} \mathbf {C}=\big [\mathbf {K}(\mathbf {x}_i,\mathbf {x}_j)\big ]_{i,j=1}^N, \end{aligned}$$

see also [15]. As it turns out, the finite element scheme yields an eigenvalue problem with a similar structure.

1.2 A.2 Finite Element Scheme on a Rectangular Grid

Finite element schemes for functions with values in three dimensions rely on a finite dimensional subspace \(\mathbf {V}_N\subset \big [L^2(\varOmega )\big ]^3\) with basis \(\big \{\varvec{\varphi }_1,\ldots ,\varvec{\varphi }_N\big \}\) to represent the eigenfunctions of the Karhunen–Loève expansion. To construct such a finite dimensional space, we consider a uniform rectangular grid \({\mathcal {Q}}_h\) on \(\varOmega \) where each cell has a size of \(h_1\times h_2\times h_3\). To each vertex \(\mathbf {x}_1,\ldots ,\mathbf {x}_N\) we assign a function \(\varphi _i\) with the property

$$\begin{aligned} \varphi _i(\mathbf {x}_j)= {\left\{ \begin{array}{ll} 1,&{}i=j,\\ 0,&{}i\ne j, \end{array}\right. } \quad i,j=1,\ldots ,N, \end{aligned}$$

where on each cell \(Q_h\in \mathcal {Q}_h\), the basis function \(\varphi _i\) is a trilinear polynomial, i.e.

$$\begin{aligned} \varphi _i(\mathbf {y})\big |_{Q_h}&= a_1\,+a_2y_1+a_3y_2+a_4y_1y_2\\&\quad +\,a_5y_3+a_6y_1y_3+a_7y_2y_3+a_8y_1y_2y_3. \end{aligned}$$

Here, the coefficients are uniquely determined such that (18) holds. This means especially that the \(\varphi _i\) are only non-zero in the eight cells with vertex \(\mathbf {x}_i\). Note especially that all \(\varphi _i\) are linearly independent, so we can define \(V_h\subset L^2(\varOmega )\) as the vector space spanned by the basis \(\varphi _1,\ldots ,\varphi _N\). A finite dimensional subspace of \(\big [L^2(\varOmega )\big ]^3\) is then given by \(\mathbf {V}_h=V_h\times V_h\times V_h\).

1.3 A.3 Advanced Finite Element Schemes

Having a finite dimensional subspace at hand yields, cf., e.g., [15], the generalized eigenvalue problem

$$\begin{aligned} \mathbf {C}_{\text {FEM}}\varvec{\phi }_{m,N}=\lambda _{m,N}\mathbf {M}_{\text {FEM}}\varvec{\phi }_{m,N} \end{aligned}$$

with system matrices

$$\begin{aligned} \mathbf {C}_{\text {FEM}}={}&\big [\big ({\mathcal {T}}_{\mathbf {K}}\varvec{\varphi }_j,\varvec{\varphi }_i\big )_{[L^2_{\rho }(D)]^3}\big ]_{i,j=1}^N,\\ \mathbf {M}_{\text {FEM}}={}&\big [\big (\varvec{\varphi }_j,\varvec{\varphi }_i\big )_{[L^2_{\rho }(D)]^3}\big ]_{i,j=1}^N, \end{aligned}$$

\(\mathcal {T}_\mathbf {K}\) denoting the integral operator from (2), and the approximate eigenfunctions

$$\begin{aligned} \varvec{\phi }_m(\mathbf {x}) \approx \varvec{\phi }_{m,N}(\mathbf {x}) =\sum _{i=1}^N\big (\varvec{\phi }_{m,N}\big )_i\varvec{\varphi }_i(\mathbf {x}). \end{aligned}$$

It thus remains to explain how to assemble these matrices.

Since the basis functions \(\varvec{\varphi }_i\) are non-zero only on a few elements, the mass matrix \(\mathbf {M}_{\text {FEM}}\) is sparse. Inserting the definition of \({\mathcal {T}}_\mathbf {K}\) into the definition of \(\mathbf {C}_{\text {FEM}}\), we obtain

$$\begin{aligned} \mathbf {C}_{\text {FEM}}=\bigg [\int _D\int _D\mathbf {K}(\mathbf {x},\mathbf {y}) \varvec{\varphi }_j(\mathbf {y})\varvec{\varphi }_i^\intercal (\mathbf {x}){\text {d}}\!{\rho }(\mathbf {y}){\text {d}}\!{\rho }(\mathbf {x})\bigg ]_{i,j=1}^N. \end{aligned}$$

In order to compute this integral, it is very common in finite element methods to replace \(\mathbf {K}\) by its interpolation \(\mathbf {K}_h\) in the finite element space, i.e. we approximate

$$\begin{aligned} \mathbf {K}(\mathbf {x},\mathbf {y})\approx \sum _{i,j=1}^N\mathbf {K}(\mathbf {x}_i,\mathbf {x}_j) \varvec{\varphi }_i(\mathbf {x})\varvec{\varphi }_j^\intercal (\mathbf {y}). \end{aligned}$$

Inserting this approximation into the definition of \(\mathbf {C}_{\text {FEM}}\) yields

$$\begin{aligned} \mathbf {C}_{\text {FEM}}\approx \mathbf {M}_{\text {FEM}}\mathbf {C}\mathbf {M}_{\text {FEM}}^{\intercal }, \end{aligned}$$

with the matrix \(\mathbf {C}\) defined as for the Nyström scheme in (17). The eigenvalue problem (19) thus turns into

$$\begin{aligned} \mathbf {M}_{\text {FEM}}\mathbf {C}\mathbf {M}_{\text {FEM}}^{\intercal }\varvec{\phi }_{m,N}= \lambda _{m,N}\mathbf {M}_{\text {FEM}}\varvec{\phi }_{m,N}. \end{aligned}$$

1.4 A.4 Connection Between the Two Schemes

The two schemes can lead to the very same eigenvalue problem. In implementations of finite element schemes, there are almost always quadrature formulas involved. Using piecewise linear ansatz functions and replacing the integrals by a trapezoidal rule yields a diagonal matrix \(\mathbf {M}_{\text {FEM}}\) (this is also referred to as “mass lumping”). The definition of \(\mathbf {M}_{\text {Nystr}}\) then amounts to quadrature weights to a quadrature formula with the vertices of the finite element mesh as evaluation points. The two schemes are thus equivalent in this specific case.

1.5 A.5 Computing Karhunen–Loève Expansions using Low-rank Approximations

Again, having a low-rank factorization \(\mathbf {C}\approx \mathbf {L}_M \mathbf {L}_M^{\intercal }\) of rank M at hand, one can reduce the dimension of the eigenvalue problems (20). For ease of notation, we do not distinguish between \(\mathbf {M}_{\text {FEM}}\) and \(\mathbf {M}_{\text {Nystr}}\) and consider the eigenvalue problem

$$\begin{aligned} \mathbf {M}\mathbf {C}\mathbf {M}^{\intercal }\varvec{\phi }_{m,N}= \lambda _{m,N}\mathbf {M}\varvec{\phi }_{m,N}. \end{aligned}$$

By substituting the low-rank approximation \(\mathbf {C}\approx \mathbf {L}_M \mathbf {L}_M^{\intercal }\) and \(\mathbf {v}_{m,N}=\mathbf {M}^{1/2}\varvec{\phi }_{m,N}\) into (21), the eigenvalue problem becomes

$$\begin{aligned} \mathbf {M}^{1/2}\mathbf {L}_M\mathbf {L}_M^{\intercal }(\mathbf {M}^{1/2})^{\intercal }\mathbf {v}_{m,N}=\lambda _{m,N}\mathbf {v}_{m,N}. \end{aligned}$$

Exploiting the fact that \(\mathbf {M}^{1/2}\mathbf {L}_M\mathbf {L}_M^{\intercal } (\mathbf {M}^{1/2})^{\intercal }\) has the same eigenvalues as \(\mathbf {L}_M^{\intercal } (\mathbf {M}^{1/2})^{\intercal }\mathbf {M}^{1/2}\mathbf {L}_M=\mathbf {L}_M^{\intercal } \mathbf {M}\mathbf {L}_M\), we obtain an equivalent eigenvalue problem

$$\begin{aligned} \mathbf {L}_M^{\intercal } \mathbf {M}\mathbf {L}_M\tilde{\mathbf {v}}_{m,N}=\lambda _{m,N}\tilde{\mathbf {v}}_{m,N}. \end{aligned}$$

This modified eigenvalue problem has again dimension \(M\ll N\) and can thus be solved by standard eigensolvers for dense matrices.

Rights and permissions

Reprints and Permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Dölz, J., Gerig, T., Lüthi, M. et al. Error-Controlled Model Approximation for Gaussian Process Morphable Models. J Math Imaging Vis 61, 443–457 (2019).

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: