Keywords

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

1 Introduction

Assessing variability of the myocardium is essential in diagnosis and treatment management of patients who suffer from myocardial infarction [1]. Cardiac MRI sequences are widely used in clinics, in particular the LE sequence which visualizes the infarcts, the T2-weighted MRI which provides imaging of the acute injury and ischemic regions, and the balanced-Steady State Free Precession (bSSFP) cine sequence which captures cardiac motions and presents clear boundaries. Cardiac segmentation, mainly on the endocardium and epicardium of the left ventricle (LV), is a common prerequisite in quantifying and analyzing pathological conditions of the heart.

However, while manual segmentation is time-consuming and subject to inter-observer variations, developing a fully automatic method is still arduous, especially for the LE sequence. Besides the great variations of the heart shape across different subjects, there are three issues related to the intensity distributions of the images, which challenge the conventional Gaussian mixture model (GMM)-based classification methods. Firstly, the intensity of the myocardium in LE or T2 MRI is heterogeneous due to the existence of the enhanced infarcts (in LE) or injuries (in T2). Similarly, the background of cardiac images generally includes the lung, chest skin, liver and stomach. These regions however have different intensity distributions and cannot be described by one simple intensity distribution model. Secondly, the patterns of intensity heterogeneity are complex. For example, the location and size of scars can differ greatly across different patients. Hence, it is difficult to predict or make assumption of the location and geometry of them. Finally, the intensity range of one tissue can fully overlap that of its surroundings. For example, in LE images, the myocardial infarcts have similar intensity range to that of the blood pool, leading to indistinguishable boundaries between the infarcts and blood pools. Figure 1 provides the images of the three sequences superimposed on with myocardial contours.

Previous works are mainly manual or semi-automatic to achieve myocardium segmentation from LE MRI [2], or to propagate the segmented myocardium from the bSSFP sequence as a hard constraint for the segmentation of scars [3, 4]. In this work, we propose to combine the complementary information from MS cardiac images and perform the segmentation of them simultaneously within a unified framework. We develop the MvMM for text classification, which considers the MS MRI as a segmentation problem of multivariate images.

Furthermore, the slices from the same or different MRI sequences of a patient can be misaligned due to body motions, referred to as motion shifts. An atlas is used to provide prior knowledge for segmentation and the atlas can also be misregistered to the MRI images. To correct them, we formulate the MvMM with transformations, assigned to each slice of the MRI images as well as to the atlas, for accurate registration to a common 3D space. All the transformations are embedded into the LL maximization framework of the MvMM, and the two groups of parameters can be optimized by the iterative conditional modes (ICM) method.

In this framework, the indistinct boundaries such as these between scars and blood pools in LE can be delineated under the guidance of the T2 and bSSFP, and vice versa. Also, the misalignments can be corrected by the combination of the multivariate images.

Fig. 1.
figure 1

Segmentation results of the three cardiac MRI sequences, (a) bSSFP, (b) T2-weight, (c) LE, by the proposed MvMM method; the LE segmentation results by atlas propagation (d), conventional GMM (e) and the Mvmm\(^\ominus \) without registration correction. Note that the original MRI images have motion shifts and the corrected images are presented in the green boxes of (a)-(c).

Full size image

2 Method

The goal of the method is to segment the endocardium and epicardium of the LV from the MS MRI images, particularly the LE MRI.

Let \(\hat{I}=\{I_i|_{i=1...N_I}\}\) be the set of \(N_I\) MS MRI images which are acquired from the same subject. We denote the spatial domain of the region of interest (ROI) of the subject as \(\varOmega \), referred to the common space which is defined to the coordinate of the patient and thus defined by the combination of the MS images in this formulation. For a location \(x\in \varOmega \), the tissue class of x, i.e. LV, right ventricle (RV) or background, is determined regardless the appearance of the MRI images. We denote this tissue type using a label, namely \(s(x)\!\!=\!\!k, k\in K\). Provided the MS images are all aligned to the common space, the label information for each image is the same, but the intensity appearance and classification of subtypes can be different. For example, the myocardial scars have different intensity values to the normal myocardial tissue in the LE sequence, while their intensity values are similar in the bSSFP sequence. We denote this subtype of a tissue k in image \(I_i\) as \(z_i(x)=c, c\in C_{ik}\).

2.1 MvMM and LL for Multivariate Image Segmentation

For a single image, one can use the mixture of Gaussian to model the intensity distributions, namely the GMM method, where the intensity probability density function (PDF) of one tissue is given by a Gaussian function. For a tissue with multiple subtypes, the multi-component GMM can be used [5]. To segment the MS images, referred to as multivariate image segmentation, we formulate a MvMM, which can also assign multiple components to a tissue for specific MRI images.

The likelihood (LH) of the model parameters \(\theta \) in multivariate image segmentation is given by \(LH(\theta ;\hat{I})=p(\hat{I}|\theta )\), similar to the GMM for tissue classification, where \(\hat{I}=\{I_1,\ldots I_{N_I}\}\) is the multivariate image vector. Assuming independence of each location (pixel), one gets \(LH(\theta ;\hat{I})=\prod _{x\in \varOmega }p(\hat{I}(x)|\theta )\). In the EM framework, the label and component information are considered as hidden data. Let \(\varTheta \) denotes the set of both hidden data and model parameters. Hence, the likelihood of the complete data is given by,

$$\begin{aligned} p(\hat{I}(x)|\varTheta )=\sum _{k\in K} \pi _{kx} p(\hat{I}(x)| s(x)\!\!=\!\!k,\varTheta ) \ , \end{aligned}$$
(1)

where, \(\pi _{kx}=p(s(x)\!\!=\!\!k|\varTheta )=p_A(s(x)\!\!=\!\!k) \pi _k /N\!\!F\), \(p_A(s(x)\!\!=\!\!k)\) is the atlas prior probability, \(\pi _k\) is the label proportion, and \(N\!\!F\) is the normalization factor.

When the tissue type of a position is known, the intensity values from different images then become independent,

$$\begin{aligned} p(\hat{I}(x)|s(x)\!\!=\!\!k,\varTheta )=\prod _{i=1,\ldots N_I} p(I_i(x)| s(x)\!\!=\!\!k,\varTheta ) \ . \end{aligned}$$
(2)

The intensity PDF of an image, \(p(I_i(x)| s(x)\!\!=\!\!k,\varTheta )\), is given by the conventional multi-component GMM, as follows,

$$\begin{aligned} p(I_i(x)| s(x)\!\!=\!\!k,\varTheta ) =\sum _{c\in C_{ik}} \tau _{ikc} \varPhi _{ikc}(\mu _{ikc},\sigma _{ikc},I_i(x)) \ , \end{aligned}$$
(3)

where, \(\tau _{ikc}=p(z_i(x)\!\!=\!\!c,s(x)\!\!=\!\!k|\varTheta )\), s.t. \(\sum _c \tau _{ikc}=1\), is the component proportion, and \(\varPhi _{ikc}(\cdot )=p(I_i(x)|z_i(x)=c_{},\varTheta )\) is the Gaussian function to model the intensity PDF of a tissue subtype c belonging to a tissue k in the image \(I_i\).

To estimate the Gaussian model parameters and then segmentation varaibles, we employ the EM to solve the LL and rewrite it as follows,

$$\begin{aligned} LL=\sum _x\sum _k \delta _{s(x),k}\left( \log \pi _{kx} +\sum _i\sum _{c_{ik}} \delta _{z_i(x),c_{ik}}(\log \tau _{ikc}+\log \varPhi _{ikc}(I_i(x))) \right) , \end{aligned}$$
(4)

where \(\delta _{a,b}\) is the Kronecker delta function.

E-Step: We obtain the expectation of LL by computing the expectation of \(\delta _{s(x),k}\) and \(\delta _{s(x),k}\delta _{z_i(x),c_{ik}}\), as follows,

$$\begin{aligned} \begin{array}{l l} P^{[m+1]}_{kx}&{}=E_{(\hat{I},\varTheta ^{[m]})}\left( \delta _{s(x),k}\right) =p(s(x)\!\!=\!\!k|\hat{I},\varTheta ^{[m]})\\ &{}=\displaystyle \frac{p(\hat{I}(x)|s(x)\!\!=\!\!k,\varTheta ^{[m]})\pi ^{m}_{kx}}{\sum _{l\in {1\ldots K}}p(\hat{I}(x)|l,\varTheta ^{[m]}) \pi ^{m}_{lx} } \\ P^{[m+1]}_{ikcx}&{}=E_{(I_i,\varTheta ^{[m]})}\left( \delta _{s(x),k}\delta _{z_i(x),c_{ik}}\right) = p\left( z_i (x)=c_{ik}|\hat{I},\varTheta ^{[m]} \right) \\ &{}=\displaystyle \frac{ \varPhi (\mu _{ikc},\sigma _{ikc},I_i (x)) \tau _{ikc} P^{[m+1]}_{kx}}{p(I_i (x)| k,\varTheta ^{[m]})} \end{array} \end{aligned}$$
(5)

M-Step: We compute the model parameters by analytically solving the maximization problem of LL given the posteriors from the E-step,

$$\begin{aligned} \begin{array}{r l} \tau _{ikc}^{[m+1]}&{}=\frac{\sum _{x\in \varOmega } P_{ikcx}^{[m+1]}}{\sum _{d\in C_{ik}} \sum _{x\in \varOmega } P_{ikdx}^{[m+1]}} \\ \mu _{ikc}^{[m+1]}&{}=\frac{\sum _{x\in \varOmega } I_i(x) P_{ikcx}^{[m+1]}}{\sum _{x\in \varOmega } P_{ikcx}^{[m+1]} }\\ (\sigma _{ikc}^{[m+1]})^2 &{}= \frac{\sum _{x\in \varOmega } (I_i(x)-\mu _{ikc}^{[m+1]} )^2 P_{ikcx}^{[m+1]}}{\sum _{x\in \varOmega } P_{ikcx}^{[m+1]}} \ . \end{array} \end{aligned}$$
(6)

For \(\pi ^{[m+1]}_k \), we compute

$$\begin{aligned} \frac{\partial LL}{\partial \pi _k}= \sum _x \frac{P^{[m+1]}_{kx}}{\pi _k} - \sum _x\sum _{j\in K}\frac{P^{[m+1]}_{jx}p_A(s(x)\!\!=\!\!k)}{\sum _{l\in K}p_A(s(x)\!\!=\!\!l)\pi _l} =0 . \end{aligned}$$
(7)

Since there is no close form for the solution, for simplicity we regard \(\mathcal {C}_x^{[m]}=\sum _{l\in K}p_A(s(x)\!\!=\!\!l)\pi _l\) as a constant using \(\pi _l=\pi ^{[m]}_l\) in the [m+1]th iteration, which results in

$$\begin{aligned} \pi ^{[m+1]}_k=\frac{\sum _x P^{[m+1]}_{kx}}{ \sum _x \left( {p_A(s(x)\!\!=\!\!k)/ \mathcal {C}_x^{[m]}} \right) } . \end{aligned}$$
(8)

This also guarantees to improve the LL [6].

Initialization of \(\pi _{k}^{[0]}\), \(\tau _{ikc}^{[0]}\), \(\mu _{ikc}^{[0]}\) and \(\sigma _{ikc}^{[0]}\) are computed based on the propagated probabilities from an atlas to each image of the MS MRI [5].

2.2 Multivariate Image Segmentation with Registration Correction

There are two kinds of misalignments in the MS MRI segmentation framework. Firstly, the motion shift of each slice is commonly seen in the multi-slice cardiac MRI, besides the misalignment of the whole image volume to the common space of the subject. Secondly, the atlas, providing the prior probabilities, can be mis-registered to the common space.

The motion shift of a slice can be modeled by an affine transformation \(\{G_{i,s}\}\), leading to reformulation of the intensity likelihood of a subtype tissue as follows,

$$\begin{aligned} p(I_i(x)|c_{ik},\varTheta , \{G_{i,s}\})=\varPhi _{ikc}(I_i(G_{i,s}(x))) . \end{aligned}$$
(9)

The atlas deformation, D, can be embedded into the prior probabilities,

$$\begin{aligned} p_A(s(x)\!\!=\!\!k|D)=p_A(s(D(x))=k)=A_k(D(x)), k=1\ldots K \ , \end{aligned}$$
(10)

which are the probabilistic atlas images. The original LL then becomes,

$$\begin{aligned} \begin{array}{ll} LL(\varTheta ,D,\{G_{i,s}\})&{}= \displaystyle \sum _{x\in {\varOmega }} \log \sum _k p(s(x)\!\!=\!\!k|D)\prod _i \sum _{c_{ik}} \tau _{ikc}\varPhi _{ikc}(I_i(G_{i,s}(x))) \\ &{}= \displaystyle \sum _{x\in {\varOmega }} \log LH(x) . \end{array} \end{aligned}$$
(11)

Here, we define the prior to \(p(s(x)\!\!=\!\!k|D)=A_k(D(x)) \pi _{kx}/\mathcal {N}\), where \(\mathcal {N}\) is the normalization factor [7].

2.3 Optimization

There is no close form solution for minimization of (11). Since the Gaussian parameters are dependent on the values of the transformation parameters, and vice versa, one can use the ICM approach to solve this optimization problem, which is eventual a coordinate ascent method in this formulation [8]. The ICM scheme optimizes one group of parameters while keeping the others static at each iteration. The different groups of parameters are alternately optimized and this alternation process iterates until a local optimum is found. In this work, the mixture model parameters are updated using the EM algorithm (Sect. 2.1) and the transformation parameters are optimized using the gradient ascent method. The derivatives of LL with respect to the shift transformation and atlas deformation are respectively given by,

$$\begin{aligned}\begin{array}{r r} \displaystyle \frac{\partial LL}{\partial G_{i,s}} &{}= \displaystyle \sum _x \frac{1}{LH(x)}\sum _k p(s(x) = k|D) \prod _{j\ne i} \{ p(I_j(x)|s(x)\!\!=\!\!k)|\\ &{} \cdot \sum \nolimits _c(\tau _{ikc}\varPhi '_{ikc}\nabla I_i(y)\nabla G_{i,s}(x)) \} \end{array} \end{aligned}$$

where \(y=G_{i,s}(x)\), and

$$\begin{aligned} \displaystyle \frac{\partial LL}{\partial D} = \displaystyle \sum _x \frac{1}{LH(x)}\sum _k \frac{\partial p(s(x) = k|D)}{\partial D} p(\hat{I}(x)|s(x)\!\!=\!\!k), \end{aligned}$$
(12)

where the computation of \(\frac{\partial p(s(x) = k|D)}{\partial D}\) is related to \(\frac{\partial A_k(D(x))}{\partial D}\),

$$\begin{aligned} \frac{\partial A_k(D(x))}{\partial D} = \nabla A_k|_{y=[y_1,y_2,y_3]} \times \left[ \frac{\partial y_1}{\phi _d},\frac{\partial y_2}{\phi _d},\frac{\partial y_3}{\phi _d} \right] ^{\texttt {T}}. \end{aligned}$$
(13)

Here, \(y=D(x)\) and \(\{\phi _d\}\) are the free-form deformation parameters [9].

3 Results

We applied the proposed method to cardiac segmentation of the three MRI sequences, namely LE, bSSFP and T2 MRI, from 18 patients who underwent cardiomyopathy. The images were acquired and reconstructed into resolution about \(0.75\times 0.75\) mm in-plane and 5 mm slice thickness for LE, \(1.25\times 1.25\) mm in-plane and 8 to 12 mm thickness for bSSFP, and \(1.35\times 1.35\) mm in-plane and 12 to 20 mm thickness for T2. The bSSFP images fully covered the ventricles from the apex to the basal plane of the mitral valve, with a number of cases having one to three more slices beyond the apex and the base. The coverage of the LE and T2 images was only limited to the main middle body of the ventricle. Figure 1 provides an example.

Table 1. This table provides the Dice scores of the atlas-based segmentation (Atlas), conventional GMM segmentation, the Mvmm\(^\ominus \) without registration correction, and the proposed MvMM. Endo: endocardium, Epi: epicardium, Myo: myocardium. The asterisk (\(^{*}\)) indicates the best result for the sequence.
Full size table

For comparisons, we included the results using single atlas-based segmentation [10], the conventional GMM segmentation [5], and the MvMM without registration correction (referred to as Mvmm \(^\ominus \)). For the atlas-to-image registration, we used the algorithm specifically designed for cardiac images, which consists of a three-level hierarchical scheme (affine, locally affine, and free-form deformation) [10]. For the atlas-to-multivariate image registration, we registered the atlas to the bSSFP image of each subject using the hierarchical registration scheme, and aligned the corresponding LE and T2 images to the bSSFP using the global affine registration. The resulting transformations were used for initialization of the MvMM registration correction. In both GMM and MvMM segmentation, we assigned two components to the myocardium of LE and T2-weight images. Furthermore, since T2 and bSSFP have distinct intensity distribution of myocardium we applied a GMM step to them after the simultaneous multi-sequence segmentation in MvMM.

Figure 1 provides an example for illustration. The proposed MvMM can accurately segment the myocardium of the images even though the motion shifts are presented among certain slices, as the red arrows points in (a)-(c). The MvMM can also generate the shift corrected MS images, which are presented in the green boxes. Figure 1(d)-(f) are the segmentation results of the LE image by the atlas-based method, the GMM method, and the Mvmm\(^\ominus \). The atlas-based method does not perform well at certain areas. GMM can improve the segmentation, but still misclassifies parts of the myocardium due to the poor initialization and enhancement. With the guidance and constraints from the other two sequences, Mvmm\(^\ominus \) performs much better, but it still has erroneous delineation due to the motion shifts, including one slice (the red arrow in (f)) which corresponds to the shifted slice in bSSFP (red arrow in (a)).

Table 1 presents the Dice scores of the three sequences from the four compared methods. For the segmentation of LE and T2, MvMM methods performed significantly better than the two conventional methods (\(p<0.01\)) and the improvements were evident. For bSSFP, GMM achieved better results. This was probably due to the fact that the three sequences were not perfectly registered even after the correction and the segmentation of bSSFP was affected by the constraints from T2 and LE sequences, which contributes more errors than improvement to the segmentation of bSSFP in MvMM.

4 Discussion and Conclusion

We have presented a new method, i.e. MvMM, for cardiac segmentation combining the complementary information of MS MRI. The MS images of the same subject are aligned to a common space and the segmentation of them is performed simultaneously. To correct the misalignments of slices due to motion shift and the mis-registered atlases, we formulate the LL with transformations and propose to use ICM to update the different groups of parameters, where the classification parameters are optimized using the EM algorithm and the transformations are updated by the gradient ascent method. We evaluated the proposed techniques using eighteen pathological datasets. The MvMM myocardium segmentation yielded significantly higher Dice scores, \(p<0.01\), on the LE and T2 MRI sequences, compared either to the atlas-based segmentation or the conventional GMM method. On the bSSFP sequence, GMM achieved the best Dice scores. This is because intensity distribution of the myocardium in bSSFP is generally uniform and distinct, and the registration provides a good initialization. Finally, we demonstrated that the proposed MvMM achieved mean Dice scores of \(.740\pm .051\), \(.769\pm .107\), and \(.770\pm .057\) for the myocardium segmentation of LE, T2 and bSSFP, respectively. In conclusion, MvMM is a generic, novel and useful model for multivariate image analysis. It has the potential of achieving good performance in the fully automatic myocardium segmentation from the cardiac LE and T2 MRI sequences.