Gaussian mixture models based 2D–3D registration of bone shapes for orthopedic surgery planning

Abstract

In orthopedic surgery, precise kinematics assessment helps the diagnosis and the planning of the intervention. The correct placement of the prosthetic component in the case of knee replacement is necessary to ensure a correct load distribution and to avoid revision of the implant. 3D reconstruction of the knee kinematics under weight-bearing conditions becomes fundamental to understand existing in vivo loads and improve the joint motion tracking. Existing methods rely on the semiautomatic positioning of a shape previously segmented from a CT or MRI on a sequence of fluoroscopic images acquired during knee flexion. We propose a method based on statistical shape models (SSM) automatically superimposed on a sequence of fluoroscopic datasets. Our method is based on Gaussian mixture models, and the core of the algorithm is the maximization of the likelihood of the association between the projected silhouette and the extracted contour from the fluoroscopy image. We evaluated the algorithm using digitally reconstructed radiographies of both healthy and diseased subjects, with a CT-extracted shape and a SSM as the 3D model. In vivo tests were done with fluoroscopically acquired images and subject-specific CT shapes. The results obtained are in line with the literature, but the computational time is substantially reduced.

Appendices

Appendix A: statistical shape model

A SSM is a set of shapes on which the location of the landmarks is correspondent. From this set of shapes, we can extract the mean model ${\bar{\mathbf{M}}}$ and the covariance matrix, from which we can compute the eigenvectors (modes of variation) and the eigenvalues.

$$\begin{aligned} \begin{aligned}&\mathbf {D} = \frac{1}{K-1}\sum \limits _{k = 1}^{K}(\mathbf {M}_k - {\bar{\mathbf{M}}})(\mathbf {M}_k - {\bar{\mathbf{M}}})^T \\&\mathbf {D}\cdot \overrightarrow{\mathbf {M}}_k = \lambda _k^2 \cdot \overrightarrow{\mathbf {M}}_k\\&\sigma _1^2\ge \lambda _2^2\ge \dots \ge \lambda _{K-1}^2 \end{aligned} \end{aligned}$$

(8)

where K is the number of shapes, $\lambda _k^2$ are the descending-order eigenvalues of the covariance matrix $\mathbf {D},$ and $\overrightarrow{\mathbf {M}}_k$ are the corresponding eigenvectors. To deform a SSM, we can multiply specific weights to the modes and add them to the mean model.

$$\begin{aligned} \mathbf {M}_{SSM} = {\bar{\mathbf{M}}} + \sum \limits _{k = 1}^{K'}\beta _k\overrightarrow{\mathbf {M}}_k \end{aligned}$$

(9)

Appendix B: expectation conditional maximization algorithm for GMMs

A femur model is represented by a set of 3D points $\mathbf {X}_s, s=1, \dots , S$. A set of fluoroscopic images $I_j, j=1, \dots , J$ are simultaneously acquired with different sources ${}^j\mathbf {S}$ and image planes. On each fluoroscopic image $I_j$, the contour of the femur ${}^j\mathbf {y}_n$ is semiautomatically segmented. The femur silhouette is defined by points $\mathbf {X}_m$, and their projection leads to ${}^j\mathbf {x}_m$ where j indicates the image on which the points are projected. We also define a set of virtual observations ${}^j\mathbf {o}_m$ that have a correspondent point in the 3D space $\mathbf {O}_m$. The registration problem is the estimation of the homogeneous matrix (expressed by the transformation parameters $\mathbf {\theta }$) which minimizes the distance between the virtual observation $\mathbf {O}_m$ and the silhouette point $\mathbf {X}_m$.

The variables used in this description are:

• ${}^j{\mathcal {Y}}$ is the contour extracted from each image, whose pixels are ${}^j\mathbf {y}_n, n = 1, \dots , N$ (also called observations)

• $\mathbf {X}_s, s=1, \dots , S$ are the points of the 3D shape

• ${}^j{\mathcal {X}}$ is the set of points of the silhouette ${}^j\mathbf {X}_m, m=1, \dots , M<S$

• ${}^j\mathbf {x}_m, m=1, \dots , M$ are the pixel of the shape’s silhouette projected on image j

• ${}^j\mathbf {o}_m, m=1, \dots , M$ are the virtual observations on the image j

• $\mathbf {O}_m, m=1, \dots , M$ are the virtual points backprojected in the 3D space (Fig. 8).

Fig. 8

In the figure are represented the shape with the points $\mathbf {X}_s$, the extracted silhouette on the shape $\mathbf {X}_m$ and their projection $\mathbf {x}_m$ on the image plane. It also represented the source of the x-ray beam ($\mathbf {S}$) and the points extracted from the contour of the image $\mathbf {y}_n$, from which we can calculate the virtual observation $\mathbf {o}_m$ and its backprojection $\mathbf {O}_m$. The white arrow between the backprojected virtual observation $\mathbf {O}_m$ and its associated silhouette point $\mathbf {X}_m$ is the minimized distance at each iteration

1.1 Gaussian model and likelihood

Each $\mathbf {X}_s$ point of the model is defined as the centroid of a 3D Gaussian distribution with mean $\mathbf {X}_s$ and covariance matrix $\mathbf {\Sigma }_s$, identifying in this way a Gaussian mixture model (GMM). Considering isotropic covariances, each $\mathbf {\Sigma }_s, s=1, \dots , S$ is defined as

$$\begin{aligned} \mathbf {\Sigma }_s = \sigma _s \mathbf {I}_3 \end{aligned}$$

(10)

where $\mathbf {I}_3$ is the $3\times 3$ identity matrix, and $\sigma _s$ is the scalar value of the covariance that varies for each $\mathbf {X}_s$ point. The operator $\mu :{\mathbb {R}}^3\rightarrow {\mathbb {R}}^3$ transforms a point $\mathbf {X}_s$ in another point $\mu (\mathbf {X}_s, \mathbf {\theta })$ where $\mathbf {\theta }$ is the parametrization of the transformation.

The likelihood (${\mathcal {L}}$) that expresses the probability that the contour is coincident with the silhouette projection is a function of both the registration parameters $\mathbf {\theta }$ and the covariances.

$$\begin{aligned} {\mathcal {L}}(\mathbf {\theta }, \sigma _1, \dots , \sigma _S|{{\mathcal {Y}}}) = \log {{\mathcal {P}}}({{\mathcal {Y}}; \mathbf {\theta }, \sigma _1, \dots , \sigma _S}) \end{aligned}$$

(11)

where ${{\mathcal {P}}}()$ is the probability that the set of observations ${\mathcal {Y}}$ is extracted from the GMM with parameters ($\mathbf {\theta }, \sigma$) and the likelihood indicates the probability that the set of observations ${\mathcal {Y}}$ is coincident with the projection of the shape’s silhouette ${\mathcal {X}}$.

This maximization cannot be performed due to the presence of missing data, as the assignment of each observation to one of the Gaussian of the GMM is unknown. The operator $\{Z:\mathbf {y}_n\rightarrow \mathbf {x}_m\}, n=1, \dots , N$ assigns an observation $\mathbf {y}_n$ either to a silhouette model point $\mathbf {x_m}$ or to an outlier class. If ($Z:\mathbf {y}_n\rightarrow \mathbf {x}_m$), then the observation $\mathbf {y}_n$ is associated with the point $\mathbf {x}_m$, otherwise, if ($Z:\mathbf {y}_n\rightarrow \mathbf {x}_{M+1}$), then the observation $\mathbf {y}_n$ is an outlier.

The likelihood is replaced by the expected complete-data log-likelihood ${\mathcal {E}}$ conditioned by the observed data, as suggested by Dempster [17].

$$\begin{aligned} {\mathcal {E}}(\mathbf {\theta }, \sigma _1, \dots , \sigma _S|{\mathcal {Y}},{Z}) = E_{Z}[\log {{\mathcal {P}}}({{\mathcal {Y}}, {Z}; \mathbf {\theta }, \sigma _1, \dots , \sigma _S})|{\mathcal {Y}}] \end{aligned}$$

(12)

To evaluate Eq. (12), the probabilities of the observations must be expressed as a set of probability density functions (PDFs). $p_m={{\mathcal {P}}}({Z:\mathbf {y}_n\rightarrow \mathbf {x}_m})$ is the prior probability that the observation $\mathbf {y}_n$ belongs to the cluster m with center $\mu (\mathbf {x}_m;\mathbf {\theta })$ while $p_{M+1}={{\mathcal {P}}}({Z:\mathbf {y}_n\rightarrow \mathbf {x}_{M+1}})$ expresses the prior probability of $\mathbf {y}_n$ to be an outlier.

$$\begin{aligned} p_m = {\left\{ \begin{array}{ll} {{\mathcal {P}}}({Z:\mathbf {y}_n\rightarrow \mathbf {x}_m}) = \frac{a}{A} &{} \quad \text {if } 1\le m \le M\\ {{\mathcal {P}}}({Z:\mathbf {y}_n\rightarrow \mathbf {x}_{M+1}}) = \frac{A-Ma}{A} &{} \quad \text {if } m = M+1\\ \end{array}\right. } \end{aligned}$$

(13)

In Eq. (13), the variable a indicates a small circular area $\left( a =\pi r^2\right)$ around the center of the projected GMM $\mu (\mathbf {x}_m,\mathbf {\theta })$, whereas A indicates the whole volume of work, so that $a\ll A$. The likelihood of an observation $\mathbf {y}_n$ given its assignment to cluster m is drawn from a normal distribution:

$$\begin{aligned} {{\mathcal {P}}}({\mathbf {y}_n| Z:\mathbf {y}_n\rightarrow \mathbf {x}_m}) = {\mathcal {N}}(\mathbf {y}_n| \mu (\mathbf {x}_m;\mathbf {\theta }), \sigma _m)= \frac{1}{\sigma _m\sqrt{2\pi } } e^{ -\frac{||\mathbf {y}_n-\mathbf {x}_m||^2}{2\sigma _m^2} } \end{aligned}$$

(14)

and the same likelihood of the observation given its assignment to the outlier class is a uniform distribution over the area A

$$\begin{aligned} {{\mathcal {P}}}({\mathbf {y}_n| Z:\mathbf {y}_n\rightarrow \mathbf {x}_{M+1}}) = {\mathcal {U}}(\mathbf {y}_n| A,0) = \frac{1}{A} \end{aligned}$$

(15)

The marginal distribution of an observation is:

$$\begin{aligned} {{\mathcal {P}}}({\mathbf {y}_n})=\sum \limits _{m=1}^{M+1}p_m{{\mathcal {P}}}({\mathbf {y}_n| Z:\mathbf {y}_n\rightarrow \mathbf {x}_{m} }) \end{aligned}$$

(16)

Equation (11) then becomes

$$\begin{aligned} \log {{\mathcal {P}}}({{\mathcal {Y}}}) = \sum \limits _{m=1}^{M}\log \left( \sum \limits _{n= 1}^{N} p_n {\mathcal {N}}(\mathbf {y}_m|\mathbf {\mu }(\mathbf {x}_n;\mathbf {\theta }),\sigma _n)+\frac{p_{n+1}}{A}\right) ,\end{aligned}$$

(17)

and Eq. (12) becomes

$$\begin{aligned} {\mathcal {E}}(\mathbf {\theta }, \sigma _1, \dots , \sigma _S|{\mathcal {Y}},{\mathcal {Z}}) = \sum \limits _{{\mathcal {Z}}} {{\mathcal {P}}}({{\mathcal {Z}}|{\mathcal {Y}},\mathbf {\theta }, \sigma _1, \dots , \sigma _S}) \log {{\mathcal {P}}}({{\mathcal {Y}},{\mathcal {Z}};\mathbf {\theta }, \sigma _1, \dots , \sigma _S}) \end{aligned}$$

(18)

1.2 Expectation Maximization

The expectation conditional maximization method is an iterative way to solve the maximum likelihood problem of Eq. (12). Starting from an initial estimate of the parameters, the method computes the posterior probabilities given the current parameters and covariances and then maximizes the expectation in (12) with respect to the registration parameters (given the current covariances) and the covariances (given the newly estimated parameters).

1.2.1 Expectation step

The expectation step is defined as the computation of the posterior probabilities given the current estimate of the registration parameters and the covariance matrix. In this case, the posterior probability is computed between the contour points (${}^j\mathbf {y}_n$) and the projection of the silhouette on the 2D images (${}^j\mathbf {x}_m$). Recovering the Eqs. (13), (14), (15), and (16) and using the Bayes’ rule, the expression for the posterior probability becomes:

$$\begin{aligned} \begin{aligned} p^q_{mn}&= {{\mathcal {P}}}({Z:\mathbf {y}_n\rightarrow \mathbf {x}_m| \mathbf {y}_n; \mathbf {\theta }^q,\sigma ^q}) \\&=\frac{{{\mathcal {P}}}({\mathbf {y}_n| Z:\mathbf {y}_n\rightarrow \mathbf {x}_m}){{\mathcal {P}}}({Z:\mathbf {y}_n\rightarrow \mathbf {x}_m})}{{{\mathcal {P}}}({\mathbf {y}_n})} \\&= \frac{\sigma _m^{-2}e^{\frac{-||\mathbf {y}_n-\mathbf {x}_m||^2}{2\sigma _m^2}}}{\sum \nolimits _{i=1}^{M}\sigma _i^{-2}e^{\frac{-||\mathbf {y}_n-\mathbf {x}_i||^2}{2\sigma _i^2}}+c} \end{aligned} \end{aligned}$$

(19)

with c that is the outlier component:

$$\begin{aligned} c = 2r^{-2} \end{aligned}$$

(20)

1.2.2 Conditional maximization step

The conditional maximization step aims at maximizing the likelihood described in Eqs. (11) and (12). It uses the definition of virtual observation, that is, a normalized sum over all the observations weighted by their posterior probability [21]. The virtual observation O and its weight $\lambda$ are obtained for each model point $x_n$ using the posterior probabilities $p^q_{mn}$ and the observations $y_m$:

$$\begin{aligned} \begin{aligned} \nu _n&= \sum \limits _{m=1}^{M} p_{mn}\\ \mathbf {o}_n&= \frac{1}{\nu _n}\sum \limits _{m=1}^{M}p_{mn}\mathbf {y}_m \end{aligned} \end{aligned}$$

(21)

Equation (12) can be rewritten replacing the conditional probabilities with the normal and uniform distribution as expressed in Eq. (22) (for the complete steps, the reader can refer to [21])

$$\begin{aligned} {\mathcal {E}} = -\frac{1}{2} \sum \limits _{m=1}^{M} \sum \limits _{n=1}^{N} \frac{p_{mn}}{\sigma _n^2}(||\mathbf {y}_m-\mathbf {\mu }(\mathbf {X}_n,\mathbf {\theta })||^2+\log (\sigma _n^2)) \end{aligned}$$

(22)

The minimization of Eq. (22) over $\mathbf {\theta }$ keeping constant the covariances $\sigma$ leads to:

$$\begin{aligned} \mathbf {\theta }^{q+1} = \arg \min _{\mathbf {\theta }} \frac{p_{mn}}{\sigma _n^2}||\mathbf {y}_m-\mathbf {\mu }(\mathbf {X}_n,\mathbf {\theta })||^2 + \frac{\rho }{2}||L(\mathbf {\mu })||^2 \end{aligned}$$

(23)

where $||L(\mathbf {\mu })||^2$ is a regularization term over the parameters. Equation (23) can be simplified using the definitions of Eq. (21):

$$\begin{aligned} \mathbf {\theta }^{q+1} = \arg \min _{\mathbf {\theta }} \nu _n||\mathbf {O}_n-\mathbf {\mu }(\mathbf {X}_n,\mathbf {\theta })||^2 + \frac{\rho }{2}||L(\mathbf {\mu })||^2 \end{aligned}$$

(24)

where $\mathbf {O}_n$ is the 3D point nearest to $\mathbf {X}_n$ on the ray backprojected from $\mathbf {o}_n$. A 2D/3D registration problem is now cast into a 3D/3D registration that can be solved using already addressed solutions [4, 10, 21, 37].

The second step of the conditional maximization is the update of the covariances, using the registration parameters newly computed:

$$\begin{aligned} \sigma _n^2 = \frac{\sum \nolimits _{m=1}^{M}p_{mn}||\mathbf {y}_m-\mathbf {\mu }(\mathbf {x}_n,\mathbf {\theta })||^2}{2\sum \nolimits _{m=1}^{M}p_{mn}} \end{aligned}$$

(25)

In Eq. (25), the value $\mathbf {\mu }(\mathbf {x}_n, \mathbf {\theta })$ is the projection of the 3D point $\mathbf {X}_n$ updated with the parameters $\mathbf {\theta }$.