Abstract
In orthopedic surgery, precise kinematics assessment helps the diagnosis and the planning of the intervention. The correct placement of the prosthetic component in the case of knee replacement is necessary to ensure a correct load distribution and to avoid revision of the implant. 3D reconstruction of the knee kinematics under weight-bearing conditions becomes fundamental to understand existing in vivo loads and improve the joint motion tracking. Existing methods rely on the semiautomatic positioning of a shape previously segmented from a CT or MRI on a sequence of fluoroscopic images acquired during knee flexion. We propose a method based on statistical shape models (SSM) automatically superimposed on a sequence of fluoroscopic datasets. Our method is based on Gaussian mixture models, and the core of the algorithm is the maximization of the likelihood of the association between the projected silhouette and the extracted contour from the fluoroscopy image. We evaluated the algorithm using digitally reconstructed radiographies of both healthy and diseased subjects, with a CT-extracted shape and a SSM as the 3D model. In vivo tests were done with fluoroscopically acquired images and subject-specific CT shapes. The results obtained are in line with the literature, but the computational time is substantially reduced.
Similar content being viewed by others
References
Acker S, Li R, Murray H, John P, Banks S, Mu S, Wyss U, Deluzio K (2011) Accuracy of single-plane fluoroscopy in determining relative position and orientation of total knee replacement components. J Biomech 44(4):784–787
Akbari Shandiz M (2015) Component placement in hip and knee replacement surgery: device development, imaging and biomechanics. PhD thesis, Biomedical Engineering
Anderst W, Zauel R, Bishop J, Demps E, Tashman S (2009) Validation of three-dimensional model-based tibio-femoral tracking during running. Med Eng Phys 31(1):10–16
Arun KS, Huang TS, Blostein SD (1987) Least-squares fitting of two 3-d point sets. Pattern Anal Mach Intell IEEE Trans (5):698–700
Baka N, Kaptein B, de Bruijne M, van Walsum T, Giphart J, Niessen WJ, Lelieveldt BP (2011) 2d-3d shape reconstruction of the distal femur from stereo x-ray imaging using statistical shape models. Med Image Anal 15(6):840–850
Baka N, de Bruijne M, van Walsum T, Kaptein B, Giphart J, Schaap M, Niessen WJ, Lelieveldt BP (2012) Statistical shape model-based femur kinematics from biplane fluoroscopy. Med Imaging IEEE Trans 31(8):1573–1583
Baka N, Kaptein B, Giphart J, Staring M, de Bruijne M, Lelieveldt B, Valstar E (2014) Evaluation of automated statistical shape model based knee kinematics from biplane fluoroscopy. J Biomech 47(1):122–129
Beretta E, De Momi E, Camomilla V, Cereatti A, Cappozzo A, Ferrigno G (2014) Hip joint centre position estimation using a dual unscented kalman filter for computer-assisted orthopaedic surgery. Proc Inst Mech Eng H J Eng Med 228(9):971–982
Beretta E, Valenti M, De Momi E, Ferrigno G (2012) Hip joint center location with kalman filter. J Bone Joint Surg, British Volume, 94-B(SUPP XLIV):68
Besl PJ, McKay ND (1992) Method for registration of 3-d shapes. In: Robotics-DL tentative, pp 586–606, International society for optics and photonics
Bijlsma J, Knahr K (2007) Strategies for the prevention and management of osteoarthritis of the hip and knee. Best Pract Res Clin Rheumatol 21(1):59–76
Canny J (1986) A computational approach to edge detection. Pattern Anal Mach Intell IEEE Trans 6:679–698
Chui H, Rangarajan A (2003) A new point matching algorithm for non-rigid registration. Comput Vis Image Underst 89(2):114–141
Cootes TF, Cooper DH, Taylor CJ, Graham J (1991) A trainable method of parametric shape description. In BMVC91, pp 54–61, Springer
Costa Filho IG (2008) Mixture models for the analysis of gene expression: integration of multiple experiments and cluster validation. PhD thesis, Freie Universität Berlin
De Momi E, Beretta E, Ferrigno G (2013) Hip joint centre localisation with an unscented kalman filter. Comput Methods Biomech Biomed Eng 16(12):1319–1329
Dempster AP, Laird NM, Rubin DB (1977) Maximum likelihood from incomplete data via the em algorithm. J R Stat Soc. Series B (methodological), 1–38
Dennis D, Mahfouz M, Komistek R, Hoff W (2005) In vivo determination of normal and anterior cruciate ligament-deficient knee kinematics. J Biomech 38:241-53
Ferrigno G, Pedotti A (1995) Opto-electronics based systems. In: Allard P, Stokes IA, Bianchi JP (eds) Three-dimensional analysis of human movement, human kinetics 1st ed, pp 57–78, Human Kinetics Publishers
Ho K, Saevarsson S, Ramm H, Lieck R, Zachow S, Sharma G, Rex E, Amiri S, Wu B, Leumann A et al (2012) Computed tomography analysis of knee pose and geometry before and after total knee arthroplasty. J Biomech 45(13):2215–2221
Horaud R, Forbes F, Yguel M, Dewaele G, Zhang J (2011) Rigid and articulated point registration with expectation conditional maximization. Pattern Anal Mach Intell IEEE Trans 33(3):587–602
Juneja M, Sandhu PS (2009) Performance evaluation of edge detection techniques for images in spatial domain. Int J Comput Theory Eng 1(5):614–621
Kittler J (1983) On the accuracy of the sobel edge detector. Image Vis Comput 1(1):37–42
Li J-S, Tsai T-Y, Wang S, Li P, Kwon Y-M, Freiberg A, Rubash HE, Li G (2014) Prediction of in vivo knee joint kinematics using a combined dual fluoroscopy imaging and statistical shape modeling technique. J Biomech Eng 136(12):124503
Litwic A, Edwards MH, Dennison EM, Cooper C (2013) Epidemiology and burden of osteoarthritis. British Medical Bulletin, p lds038
Lorensen W, Cline H (1987) Marching cubes: A high resolution 3d surface construction algorithm. In: ACM Siggraph Computer Graphics, vol 21, pp 163–169, ACM
Metz C (2005) Digitally reconstructed radiographs. Utrecht: Utrecht University, p 79
Miranda D, Rainbow M, Leventhal E, Crisco J, Fleming B (2010) Automatic determination of anatomical coordinate systems for three-dimensional bone models of the isolated human knee. J Biomech 43(8):1623–1626
Sharma G, Saevarsson S, Amiri S, Montgomery S, Ramm H, Lichti D, Lieck R, Zachow S, Anglin C (2012) Radiological method for measuring patellofemoral tracking and tibiofemoral kinematics before and after total knee replacement. Bone Joint Res 1(10):263–271
Tashman S, Anderst W (2003) In-vivo measurement of dynamic joint motion using high speed biplane radiography and CT: Application to canine ACL deficiency. J Biomech Eng 125(2):238
Tersi L, Barré A, Fantozzi S, Stagni R (2013) In vitro quantification of the performance of model-based mono-planar and bi-planar fluoroscopy for 3d joint kinematics estimation. Med Biol Eng Comput 51(3):257–265
Tsai T-Y, Lu T-W, Chen C-M, Kuo M-Y, Hsu H-C (2010) A volumetric model-based 2d to 3d registration method for measuring kinematics of natural knees with single-plane fluoroscopy. Med Phys 37(3):1273–1284
Valenti M, De Momi E, Yu W, Ferrigno G, Zheng G (2014) 2d/3d ssm reconstruction method based on robust point matching. Bone Joint J Orthop Proc Suppl 96(SUPP 16):1–1
Valenti M, De Momi E, Yu W, Ferrigno G, Akbari Shandiz M, Anglin C, Zheng G (2015) Fluoroscopy-based tracking of femoral kinematics with statistical shape models. Int J Comput Assist Radiol Surgery, 1–9
Yu W, Zheng G (2014) Personalized x-ray reconstruction of the proximal femur via a new control point-based 2d-3d registration and residual complexity minimization. In: Eurographics workshop on visual computing for biology and medicine, VCBM 2014, Vienna, Austria, 2014. Proceedings, pp 155–162
Zhang W, Moskowitz R, Nuki G, Abramson S, Altman R, Arden N, Bierma-Zeinstra S, Brandt K, Croft P, Doherty M et al (2007) OARSI recommendations for the management of hip and knee osteoarthritis, part i: critical appraisal of existing treatment guidelines and systematic review of current research evidence. Osteoarthritis Cartilage 15(9):981–1000
Zheng G (2013) Expectation conditional maximization-based deformable shape registration. In: Computer analysis of images and patterns, pp 548–555, Springer
Zheng G, Dong X, Rajamani K, Zhang X, Styner M, Thoranaghatte R, Nolte L, Ballester M (2007) Accurate and robust reconstruction of a surface model of the proximal femur from sparse-point data and a dense-point distribution model for surgical navigation. IEEE Trans Bio-Med Eng 54:2109-22
Zhu Z, Li G (2012) An automatic 2d–3d image matching method for reproducing spatial knee joint positions using single or dual fluoroscopic images. Comput Methods Biomech Biomed Eng 15(11):1245–1256
Acknowledgments
The authors want to acknowledge the Natural Sciences and Engineering Research Council of Canada (NSERC) and the Canadian Institutes of Health Research (CIHR) for funding the acquisition of the validation images.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Appendices
Appendix A: statistical shape model
A SSM is a set of shapes on which the location of the landmarks is correspondent. From this set of shapes, we can extract the mean model \({\bar{\mathbf{M}}}\) and the covariance matrix, from which we can compute the eigenvectors (modes of variation) and the eigenvalues.
where K is the number of shapes, \(\lambda _k^2\) are the descending-order eigenvalues of the covariance matrix \(\mathbf {D},\) and \(\overrightarrow{\mathbf {M}}_k\) are the corresponding eigenvectors. To deform a SSM, we can multiply specific weights to the modes and add them to the mean model.
Appendix B: expectation conditional maximization algorithm for GMMs
A femur model is represented by a set of 3D points \(\mathbf {X}_s, s=1, \dots , S\). A set of fluoroscopic images \(I_j, j=1, \dots , J\) are simultaneously acquired with different sources \({}^j\mathbf {S}\) and image planes. On each fluoroscopic image \(I_j\), the contour of the femur \({}^j\mathbf {y}_n\) is semiautomatically segmented. The femur silhouette is defined by points \(\mathbf {X}_m\), and their projection leads to \({}^j\mathbf {x}_m\) where j indicates the image on which the points are projected. We also define a set of virtual observations \({}^j\mathbf {o}_m\) that have a correspondent point in the 3D space \(\mathbf {O}_m\). The registration problem is the estimation of the homogeneous matrix (expressed by the transformation parameters \(\mathbf {\theta }\)) which minimizes the distance between the virtual observation \(\mathbf {O}_m\) and the silhouette point \(\mathbf {X}_m\).
The variables used in this description are:
-
\({}^j{\mathcal {Y}}\) is the contour extracted from each image, whose pixels are \({}^j\mathbf {y}_n, n = 1, \dots , N\) (also called observations)
-
\(\mathbf {X}_s, s=1, \dots , S\) are the points of the 3D shape
-
\({}^j{\mathcal {X}}\) is the set of points of the silhouette \({}^j\mathbf {X}_m, m=1, \dots , M<S\)
-
\({}^j\mathbf {x}_m, m=1, \dots , M\) are the pixel of the shape’s silhouette projected on image j
-
\({}^j\mathbf {o}_m, m=1, \dots , M\) are the virtual observations on the image j
-
\(\mathbf {O}_m, m=1, \dots , M\) are the virtual points backprojected in the 3D space (Fig. 8).
1.1 Gaussian model and likelihood
Each \(\mathbf {X}_s\) point of the model is defined as the centroid of a 3D Gaussian distribution with mean \(\mathbf {X}_s\) and covariance matrix \(\mathbf {\Sigma }_s\), identifying in this way a Gaussian mixture model (GMM). Considering isotropic covariances, each \(\mathbf {\Sigma }_s, s=1, \dots , S\) is defined as
where \(\mathbf {I}_3\) is the \(3\times 3\) identity matrix, and \(\sigma _s\) is the scalar value of the covariance that varies for each \(\mathbf {X}_s\) point. The operator \(\mu :{\mathbb {R}}^3\rightarrow {\mathbb {R}}^3\) transforms a point \(\mathbf {X}_s\) in another point \(\mu (\mathbf {X}_s, \mathbf {\theta })\) where \(\mathbf {\theta }\) is the parametrization of the transformation.
The likelihood (\({\mathcal {L}}\)) that expresses the probability that the contour is coincident with the silhouette projection is a function of both the registration parameters \(\mathbf {\theta }\) and the covariances.
where \({{\mathcal {P}}}()\) is the probability that the set of observations \({\mathcal {Y}}\) is extracted from the GMM with parameters (\(\mathbf {\theta }, \sigma\)) and the likelihood indicates the probability that the set of observations \({\mathcal {Y}}\) is coincident with the projection of the shape’s silhouette \({\mathcal {X}}\).
This maximization cannot be performed due to the presence of missing data, as the assignment of each observation to one of the Gaussian of the GMM is unknown. The operator \(\{Z:\mathbf {y}_n\rightarrow \mathbf {x}_m\}, n=1, \dots , N\) assigns an observation \(\mathbf {y}_n\) either to a silhouette model point \(\mathbf {x_m}\) or to an outlier class. If (\(Z:\mathbf {y}_n\rightarrow \mathbf {x}_m\)), then the observation \(\mathbf {y}_n\) is associated with the point \(\mathbf {x}_m\), otherwise, if (\(Z:\mathbf {y}_n\rightarrow \mathbf {x}_{M+1}\)), then the observation \(\mathbf {y}_n\) is an outlier.
The likelihood is replaced by the expected complete-data log-likelihood \({\mathcal {E}}\) conditioned by the observed data, as suggested by Dempster [17].
To evaluate Eq. (12), the probabilities of the observations must be expressed as a set of probability density functions (PDFs). \(p_m={{\mathcal {P}}}({Z:\mathbf {y}_n\rightarrow \mathbf {x}_m})\) is the prior probability that the observation \(\mathbf {y}_n\) belongs to the cluster m with center \(\mu (\mathbf {x}_m;\mathbf {\theta })\) while \(p_{M+1}={{\mathcal {P}}}({Z:\mathbf {y}_n\rightarrow \mathbf {x}_{M+1}})\) expresses the prior probability of \(\mathbf {y}_n\) to be an outlier.
In Eq. (13), the variable a indicates a small circular area \(\left( a =\pi r^2\right)\) around the center of the projected GMM \(\mu (\mathbf {x}_m,\mathbf {\theta })\), whereas A indicates the whole volume of work, so that \(a\ll A\). The likelihood of an observation \(\mathbf {y}_n\) given its assignment to cluster m is drawn from a normal distribution:
and the same likelihood of the observation given its assignment to the outlier class is a uniform distribution over the area A
The marginal distribution of an observation is:
Equation (11) then becomes
and Eq. (12) becomes
1.2 Expectation Maximization
The expectation conditional maximization method is an iterative way to solve the maximum likelihood problem of Eq. (12). Starting from an initial estimate of the parameters, the method computes the posterior probabilities given the current parameters and covariances and then maximizes the expectation in (12) with respect to the registration parameters (given the current covariances) and the covariances (given the newly estimated parameters).
1.2.1 Expectation step
The expectation step is defined as the computation of the posterior probabilities given the current estimate of the registration parameters and the covariance matrix. In this case, the posterior probability is computed between the contour points (\({}^j\mathbf {y}_n\)) and the projection of the silhouette on the 2D images (\({}^j\mathbf {x}_m\)). Recovering the Eqs. (13), (14), (15), and (16) and using the Bayes’ rule, the expression for the posterior probability becomes:
with c that is the outlier component:
1.2.2 Conditional maximization step
The conditional maximization step aims at maximizing the likelihood described in Eqs. (11) and (12). It uses the definition of virtual observation, that is, a normalized sum over all the observations weighted by their posterior probability [21]. The virtual observation O and its weight \(\lambda\) are obtained for each model point \(x_n\) using the posterior probabilities \(p^q_{mn}\) and the observations \(y_m\):
Equation (12) can be rewritten replacing the conditional probabilities with the normal and uniform distribution as expressed in Eq. (22) (for the complete steps, the reader can refer to [21])
The minimization of Eq. (22) over \(\mathbf {\theta }\) keeping constant the covariances \(\sigma\) leads to:
where \(||L(\mathbf {\mu })||^2\) is a regularization term over the parameters. Equation (23) can be simplified using the definitions of Eq. (21):
where \(\mathbf {O}_n\) is the 3D point nearest to \(\mathbf {X}_n\) on the ray backprojected from \(\mathbf {o}_n\). A 2D/3D registration problem is now cast into a 3D/3D registration that can be solved using already addressed solutions [4, 10, 21, 37].
The second step of the conditional maximization is the update of the covariances, using the registration parameters newly computed:
In Eq. (25), the value \(\mathbf {\mu }(\mathbf {x}_n, \mathbf {\theta })\) is the projection of the 3D point \(\mathbf {X}_n\) updated with the parameters \(\mathbf {\theta }\).
Rights and permissions
About this article
Cite this article
Valenti, M., Ferrigno, G., Martina, D. et al. Gaussian mixture models based 2D–3D registration of bone shapes for orthopedic surgery planning. Med Biol Eng Comput 54, 1727–1740 (2016). https://doi.org/10.1007/s11517-016-1460-6
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11517-016-1460-6