Local Features Based Level Set Method for Segmentation of Images with Intensity Inhomogeneity

Abstract

Local region-based level set models have recently been recognized as promising methods to segment images with intensity inhomogeneity. In these models, local intensity information in a neighborhood of predetermined size is extracted and then embedded into the energy function, where the local neighborhood intensities are assumed to be rather constant. Complex image characteristics, such as variation in degree of intensity inhomogeneity and noise levels, can lead to severe challenges for accurate image segmentation when using only a fixed scale parameter for local regions. In this paper, we propose a new multi-scale local feature-based level set method based on previous studies of multi-scale image filtering methods. Our novel method can adaptively determine the optimal scale parameter for each pixel during contour evolution, alleviating the challenges caused by severe intensity inhomogeneity. Our experimental results illustrate the good performance of the proposed level set method.

1 Introduction

Local region-based level set models [1, 14, 20], such as local region descriptors (LRDs) method [2], localizing region-based (LRB) active contours [4], local binary fitting (LBF) model [7, 8], local image fitting (LIF) model [19], and local Chan-Vese (LCV) model [16], have been recognized as effective methods to segment images with intensity inhomogeneity. In these models, local intensity information in a neighborhood of a predetermined size is extracted and then embedded into the energy functional with CV-like structure, thus guiding the evolution of deformable contour used to identify object boundaries. Though they do not assume homogeneous intensity in the whole object being segmented, these methods are limited by the assumption that the intensities in each local region are rather constant. Therefore, selection of an optimal value of the scale parameter is a critical factor for segmenting images with intensity inhomogeneity. To determine the optimal size of local region, a trial-and-error solution, along with visual assessment of segmentation accuracy, is usually employed in the traditional procedure. Complex image characteristics, such as variations in the degree of intensity inhomogeneity and noise levels for different regions or images, lead to severe challenges for accurate image segmentation using only a fixed scale parameter for all local regions.

Fig. 1.

The schematic diagram of the proposed multi-scale local region-based level set method.

In recent years, multi-scale level set methods have been explored to overcome the difficulties caused by single scale methods. For example, Lin et al. [9] presented a multi-scale level set framework to segment echocardiographic images, where a coarse scale was first used to extract image boundaries and fine scale was then adopted to refine the results. A similar scheme was used by Kim et al. [3] to track non-rigid object boundaries. However, these methods are essentially traditional multi-scale image processing approaches with a predetermined scale parameter in each step; rather than an adaptive approach for scale parameter selection.

Motivated by previous studies on multi-scale image processing [5, 10, 12, 13], we propose a new multi-scale local feature-based level set method for image segmentation. Our new method can adaptively determine the optimal scale parameter for each pixel during contour evolution. Figure 1 is a schematic diagram of the proposed method. First, by using the multi-scale low pass filters, we construct multi-scale local region descriptors. Based on the descriptors, a local maximum description difference feature (LMDD) is defined, which is associated with the maximum response of multi-scale high-pass filters. Since intensity inhomogeneity is believed to be primarily located in the low-frequency band [11, 15], the LMDD feature is expected to significantly reduce the influence of intensity inhomogeneity on image segmentation. Meanwhile, the optimal scale value is determined automatically. The LMDD feature is then incorporated into one typical local-region based level set model with Chan-Vese (CV)-like structures, namely LBF model [8], to construct the energy function. Finally, minimization of this energy completes the segmentation. It should be noted that the proposed method can easily be incorporated into other typical local region based level set models, such as LIF, LCV etc.

The rest of the paper is organized as follows: In Sect. 2, we will introduce the related works and the way to construct the novel multi-scale local region-based level set model. Experimental results and associated performance analysis are illustrated in Sect. 3.

2 Method

2.1 Local Binary Fitting (LBF) Model

In order to segment images with intensity inhomogeneity, Li et al. [8] proposed the LBF model, which draws upon the intensity information in local regions by using the kernel function with one fixed scale parameter. Let $\varOmega \in R^2$ be the image domain, and $I:\varOmega \rightarrow R$ the given image. The energy functional of the LBF model is defined as:

$$\begin{aligned} \begin{aligned} E&= \lambda _1 \int _{\varOmega } \int _{in(c)} K_{\sigma }(x-y)(I(y)-f_1(x))^2 \mathrm {d}y \mathrm {d}x \\&\quad + \lambda _2 \int _{\varOmega } \int _{out(c)} K_{\sigma }(x-y)(I(y)-f_2(x))^2 \mathrm {d}y \mathrm {d}x \\&\quad + \mu l + \upsilon p (\phi ) \end{aligned} \end{aligned}$$

(1)

where I(y) denotes the intensity configuration of point $y \in \varOmega$. The segmenting curve c is represented by the zero level set, i.e. $c=\{x \in \varOmega |\phi (x)=0\}$. in(c) and out(c) represent the inside and outside region of evolving contour c, respectively. l denotes the length of c. $K_{\sigma }$ is the Gaussian kernel with standard deviation $\sigma$. $\lambda _1, \lambda _2, \mu$ and $\upsilon$ are fixed parameters. $p(\phi )$ is used to avoid the re-initialization step. $f_1$ and $f_2$ are smooth functions approximating the local image intensities inside and outside the contour c, respectively. Obviously, the energy functional (1) is region-scalable, and $\sigma$ plays a key role to control the size of local regions [8]. However, in the classical LBF model, only one fixed scale parameter $\sigma$ is applied for each image and there is no general guideline for LBF model to choose suitable scale parameters for different images.

2.2 Local Maximum Description Difference Feature (LMDD)

In this section, we will introduce how to construct the multi-scale local region descriptor and the LMDD feature.

Multi-scale Local Region Descriptor. The most common model to describe intensity inhomogeneity [6, 18] can be written as:

$$\begin{aligned} I=bJ+n \end{aligned}$$

(2)

where $J:\varOmega \rightarrow R$ is the true image to be restored, $b:\varOmega \rightarrow R$ denotes the intensity inhomogeneity field, and $n:\varOmega \rightarrow R$ is the noise.

Based on the assumption that the spectrum of intensity inhomogeneity is mainly concentrated in the lower frequency band, the local region descriptors can be constructed by using multi-scale low-pass filters, e.g. the Gaussian filter, the mean filter or median filter, etc. We take Gaussian filter as an example to elucidate how the multi-scale low-pass filters are used and embedded into the local region-based level set model. The multi-scale Gaussian filter is given by:

$$\begin{aligned} K_{{\sigma }_k}(x-y)=\frac{1}{\sqrt{2{\pi }}{\sigma }_k}e^{\frac{-{|x-y|}^2}{2{\sigma }_k^2}},~~k=1,2...m, \end{aligned}$$

(3)

where x is the center pixel and y denotes the pixel in the neighborhood. Neighborhood scale is controlled by $\sigma _k=2k+1$. After determining the filters, multi-scale local region descriptors $LI_k^{LBF}$ for LBF model is given by:

$$\begin{aligned} LI_k^{LBF}=\frac{\int _{\varOmega }K_{\sigma _k}(x-y)I(y)\mathrm {d}y}{\int _{\varOmega }K_{\sigma _k}(x-y)\mathrm {d}y} \end{aligned}$$

(4)

It can be seen that $LI_k^{LBF}$ denotes the Gaussian weighting mean in local regions with different scale.

LMDD and Optimal Scale Value. After obtaining the multi-scale local region descriptor, we can calculate the LMDD feature. First, the multi-scale local region description difference $d_k$ is defined as:

$$\begin{aligned} d_k(x)=(I(x)-LI_k^{LBF}(x))^2 \end{aligned}$$

(5)

Then, the LMDD feature M(x) is given by:

$$\begin{aligned} M(x)=max_k(d_k(x)) \end{aligned}$$

(6)

The optimal scale value for local region is obtained as follows:

$$\begin{aligned} s(x)=\arg max_k(d_k(x)) \end{aligned}$$

(7)

It can be seen that, $d_k$ indicates the approximation degree between $LI_k$ and the original image I. Since $LI_k^{LBF}$ is constructed by using the low-pass filters, $d_k$ is actually a high-pass filtering operator. The LMDD feature, which will be embedded into the level set energy functional, is the maximum response of multi-scale high-pass filters.

The advantages to extract the LMDD feature are as follows: First, through LMDD feature, i.e. the maximum response of multiple high-pass filters with different scale, the intensity inhomogeneity located in low frequency band can be greatly restrained. Meanwhile, image details, as well as object boundaries, in high frequency are well preserved. Second, with the segmenting contour evolving, the LMDD feature for a pixel near the boundary and the corresponding optimal scale of local region will also be adjusted. Figure 2 shows an example of the optimal scale update.

Fig. 2.

Illustrates the evolution of the segmenting contour (red boundary) at different iteration and the corresponding optimal scale (denoted by the diameter of green circle and blue circle) for local regions centered at $x_1$ and $x_2$. (Color figure online)

Feature Incorporation and Multi-scale Local LBF Models (MS-LBF). In this section, we will introduce how to incorporate the LMDD feature into the traditional level set energy functional with CV-like structure:

$$\begin{aligned} E=E_D+E_R=E_{in}+E_{out}+E_R \end{aligned}$$

(8)

where $E_D$ is the data term, which consists of two items, $E_{in}$ and $E_{out}$, corresponding to the inside and outside regions of the evolving contour, respectively. $E_R$ is the regularization term with the aim to smooth the evolving contour and avoid the reinitialization step. In this paper, $E_R$ includes the arc length penalty term [18] and the re-initialization penalty term:

$$\begin{aligned} \begin{aligned} E_R&=\mu l+\upsilon p(\phi ) \\&=\mu \int _{\varOmega }|\nabla H(\phi )|\mathrm {d}x +\upsilon \int _{\varOmega }(\nabla \phi -1)^2\mathrm {d}x\ \end{aligned} \end{aligned}$$

(9)

Formula (5) can be divided into two parts $d_{k,in}^{MS-LBF}$and $d_{out,k}^{MS-LBF}$, corresponding to the inside and outside region of evolving contour:

$$\begin{aligned} d_{k,in}^{MS-LBF}=(I-LI_{k,in}^{MS-LBF})^2 \end{aligned}$$

(10)

$$\begin{aligned} d_{out,k}^{MS-LBF}=(I-LI_{k,out}^{MS-LBF})^2 \end{aligned}$$

(11)

where

$$\begin{aligned} LI_{k,in}^{MS-LBF}=\frac{\int _{\varOmega }K(x-y,\sigma _k)I(y)H(\phi )\mathrm {d}y}{\int _{\varOmega }K(x-y,\sigma _k)H(\phi )\mathrm {d}y} \end{aligned}$$

(12)

$$\begin{aligned} LI_{k,out}^{MS-LBF}=\frac{\int _{\varOmega }K(x-y,\sigma _k)I(y)(1-H(\phi ))\mathrm {d}y}{\int _{\varOmega }K(x-y,\sigma _k)(1-H(\phi ))\mathrm {d}y} \end{aligned}$$

(13)

$LI_{k,in}^{MS-LBF}$and $LI_{k,out}^{MS-LBF}$ are the multi-scale local region descriptors with scale $\sigma _k$ of inside and outside regions of evolving contour, respectively. $d_{k,in}^{MS-LBF}$ and $d_{out,k}^{MS-LBF}$, represent the multi-scale local region description difference with scale $\sigma _k$ of inside and outside regions of evolving contour. Then, the LMDD feature inside and outside of contour at image pixel I(x) can be computed as:

$$\begin{aligned} M_{in}^{MS-LBF}(x)=max_k(d_{k,in,i,j}^{MS-LBF}(x)) \end{aligned}$$

(14)

$$\begin{aligned} M_{out}^{MS-LBF}(x)=max_k(d_{k,out,i,j}^{MS-LBF}(x)) \end{aligned}$$

(15)

According to (8), the multi-scale data term $E_D^{MS-LBF}$ of MS-LBF model is obtained by:

$$\begin{aligned} \begin{aligned} E_D^{MS-LBF}&=E_{in}^{MS-LBF}+E_{out}^{MS-LBF}\\&=\int _{\varOmega }M_{in}^{MS-LBF}H(\phi )\mathrm {d}x+\int _{\varOmega }M_{out}^{MS-LBF}(1-H(\phi ))\mathrm {d}x \end{aligned} \end{aligned}$$

(16)

The energy functional of MS-LBF model is obtained by:

(17)

Finally, the energy functional (17) is minimized by gradient descend method. Keeping $LI_{k,in}^{MS-LBF}$ and $LI_{k,out}^{MS-LBF}$ fixed and minimizing the energy functional with respect to $\phi$, the Euler-Lagrange equation for $\phi$ can be deduced. Parameterizing the descent direction with an artificial time t, the evolution equation of MS-LBF model can be written as:

(18)

3 Experimental Results

In this section, experiments on real and simulated data are carried out to evaluate the performance of the proposed method. We also compare with traditional local region-based level set methods, i.e. LBF model. The parameters are set as follows: $\upsilon =1, \Delta t=0.1$ (the time step), $\sigma _k=2k+1, k\in [1,m]$. Here, m determines the range of local region scale. If m is too big, the computational burden at each iteration will be greatly increased since much statistical information needs to be calculated. If m is too small, the local region will be too narrow to cover adequate object and background pixels. Generally, m can be defined in the interval [8, 32]. In this paper, m is set as 16 for MS-LBF model.

Meanwhile, Jaccard similarity coefficient (JSC) is used as a quantitative measure to evaluate the segmentation results [17]. JSC is defined as:

$$\begin{aligned} J(O_m,O_t)=\frac{A(O_m\cap O_t)}{A(O_m\cup O_t)} \ \end{aligned}$$

(19)

where $O_m$ denotes the derived object region by the algorithm and $O_t$ denotes its corresponding object region in the ground truth image. $A(*)$ represents the area of region. The Jaccard similarity coefficient is bounded in [0, 1], and the larger value implies better segmentation result.

3.1 Evolving Process and Visual Evaluation

In this experiment, the proposed MS-LBF models were applied to real images with intensity inhomogeneity. The evolving process and the segmentation results are shown in Fig. 3. We can see that the proposed method yields reasonable segmentation results. Meanwhile, the last column displays the final level set functions, which are smooth and steady, demonstrating the capability of the proposed methods to keep the level set function regular during the curve evolving.

Fig. 3.

The evolving process and the final level sets for MS-LBF model.

3.2 Robustness to Contour Initialization

We evaluated the influence of contour initialization on the final results with the MS-LBF. Real images with intensity inhomogeneity were used, and the initial contours (green polygons in Fig. 4.) were placed at different parts of the images. We also show the segmentation results with the LBF on the same image with same initial contours for comparison. The blue and red contours in Fig. 4 denote the final segmentation results of the LBF model and our method. It can be seen that the multi-scale model is robust to the contour initialization, and can obtain reasonable and almost same results, despite totally different initial contours. On the contrary, the LBF model cannot segment the image accurately by using the four different initial contours. This is because that the LMDD method can capture more boundary information, rather than be constrained by local region.

Fig. 4.

Influence of the contour initialization on the final segmentation results for (a) LBF model and (b) MS-LBF model. (Color figure online)

Fig. 5.

(a) Segmentation results of LBF model with different scale parameter ranging from 1 to 16; (b) Final scale map obtained from LMDD feature; (c) Segmentation result of MS-LBF model. Yellow arrows in (b) and (c) point out boundary of the segmented object, where the scale values tend to be small. (Color figure online)

3.3 Comparison with LBF Model

In this experiment, the proposed MS-LBF model was compared with the traditional LBF model. When applying the LBF model, because there is no general guideline to choose suitable scale parameters, different scale values ranging from 1 to 16 were tested one by one. The segmentation results of LBF model are shown in Fig. 5(a). Among all the segmentation results of LBF model, the one with $\sigma =3$ is best (enclosed by red rectangle in Fig. 5(a)). However, it still fails to segment the object accurately, especially in regions with severe intensity inhomogeneity (denoted by dotted blue circle in Fig. 5(a)). By comparison, Fig. 5(b) and (c) show the final scale map and the corresponding segmentation result generated by the proposed MS-LBF model. The optimal scale values in the image vary largely with different locations. Specifically, for pixels around the object boundary (denoted by yellow arrow in Fig. 5(b)) or regions with severe intensity inhomogeneity, the optimal scales for local regions tend to be small, so that the detailed information of the image can be captured. Whereas, for smooth regions, the optimal scale is big, and global information about intensity contrast is captured. In this way, the scale value can be adaptively determined to promote suitable local region descriptors to model the piecewise constant image, thus guiding the evolving contour toward desired boundary (Fig. 5(c)). Here, the parameter $\mu$ are set as $0.0001\times 255^2$.

Fig. 6.

Comparison of LBF model with MS-LBF model on two real medical images. (a) and (d) are the original images with initialized contours; (b) and (e) are the segmentation results of LBF model, where yellow arrows point to regions with segmentation obstacles; (e) and (f) show the segmentation results of MS-LBF model. (Color figure online)

To further demonstrate the power of the proposed multi-scale method, two more experiments on real images are conducted and the results are shown in Fig. 6. It can been seen that MS-LBF model can generate reasonable segmentation results, while the LBF model fails in regions with severe intensity inhomogeneity (denoted by yellow arrows in Fig. 6), even various scale parameters are tried out. Here, the parameter $\mu$ are set as $0.001\times 255^2$ and $0.01\times 255^2$ for Fig. 6 (a) and (d), respectively.

To quantitatively evaluate the performance of the proposed method, the Jaccard similarity coefficient (JSC) between the ground truth and the segmentation results obtained by MS-LBF model and LBF model were calculated. The results are shown in Table 1. It is apparent that the JSC values of MS-LBF model are quite higher than that of LBF model, demonstrating better performance of the proposed MS-LBF model in comparison with LBF model for image with severe intensity inhomogeneity.

Table 1. Jaccard similarity coefficients of LBF model and MS-LBF model for images in Figs. 5 and 6.

4 Discussion and Conclusion

Motivated by previous studies on multi-scale image processing and local region-based level set method, we propose a novel multi-scale local region-based level set method for segmentation of images with severe intensity inhomogeneity. By using the proposed LMDD feature, the optimal scale value of the local region for each image pixel is determined in an automatic, adaptive, and dynamic way. Then, the LMDD feature is incorporated into three classical local region-based level set model, such as LBF model, to complete the image segmentation. Experiments on synthetic and real images demonstrate better performance compared with the traditional local region-based level set models. It should be noted that since multi-scale or multi-layer structure is adopted in the proposed image segmentation method, the computational efficiency is suboptimal. Our future work will consider combining semantic information into the method, aiming to promote the computational efficiency of the proposed adaptive scale method.