Image segmentation algorithm by piecewise smooth approximation

Abstract

We propose a novel image segmentation algorithm using piecewise smooth (PS) approximation to image. The proposed algorithm is inspired by four well-known active contour models, i.e., Chan and Vese’ piecewise constant (PC)/smooth models, the region-scalable fitting model, and the local image fitting model. The four models share the same algorithm structure to find a PC/smooth approximation to the original image; the main difference is how to define the energy functional to be minimized and the PC/smooth function. In this article, pursuing the same idea we introduce different energy functional and PS function to search for the optimal PS approximation of the original image. The initial function with our model can be chosen as a constant function, which implies that the proposed algorithm is robust to initialization or even free of manual initialization. Experiments show that the proposed algorithm is very appropriate for a wider range of images, including images with intensity inhomogeneity and infrared ship images with low contrast and complex background.

Introduction

Image segmentation is a popular problem in image processing and computer vision, which has been studied extensively in past decades. The segmentation goal is to separate image domain into a collection of distinct regions, upon which other high-level tasks such as objects recognition and tracking can be further performed. Due to the presence of noise, complex background, low intensity contrast with weak edges, and intensity inhomogeneity [1], image segmentation is still a difficult problem in practical applications, especially for traditional segmentation methods [2–8]. Traditional methods, like Canny edge detection [7], are simple and fast, but they always need further edge linking operation to produce continuous object boundaries [9]. To address these issues, more recent methods including implicit active contours [9–17] have been developed for image segmentation.

Implicit active contours are active contour models implemented via the level set method [18]. One of the remarkable advantages of active contour models is that it can provide smooth and closed contours, which is generally impossible in traditional segmentation methods. Existing implicit models can roughly be categorized into two classes: edge-based models [10, 11, 13] and region-based models [12, 14–16]. Edge-based models use local edge information (image gradient) to perform contour extraction, which are usually to noise and weak edges. Region-based models utilize the global and/or local image statistics inside and outside the active contour (evolving curve) to find a partition of image domain. They generally have better performance in the presence of weak or discontinuous boundaries and less sensitive to initialization.

A major category of the region-based level set methods is proposed to minimize the well-known Mumford and Shah (MS) functional [19]. Due to the difficulty of directly minimizing the MS functional, different approximation methods have been proposed to allow more efficient energy minimization. For example, the piecewise constant (PC) models [12, 20] approximate image domain by a set of homogenous regions, but which is not true for images with intensity inhomogeneity. In [21, 22], more advanced piecewise smooth (PS) models have been proposed to improve the PC model performance in terms of intensity inhomogeneity. However, due to rather complicated algorithms, these models are usually computationally expensive. As indicated in [23], the models using global image statistics usually have difficulty to segment many real-world images with intensity inhomogeneity.

To address this issue, recently localized region-based models [14, 16, 23–25] have been proposed. For example, Li et al. [14] proposed a region-scalable fitting (RSF) model (originally termed as local binary fitting (LBF) model [26]), which draws upon spatially varying local region information and thus is able to deal with intensity inhomogeneity. The RSF model has better performance than PC and PS models in segmentation of accuracy and computational efficiency. However, since there are totally four convolutions to be computed at each iteration in the implementation, the RSF model is still computationally expensive. For more efficient segmentation, Zhang et al. [16] proposed a novel active contour model driven by local image fitting (LIF) energy. The LIF energy is constructed by the local image information, which can be viewed as a constraint of the differences between the fitting image and the original image. The complexity analysis and experimental results show that this model is more efficient than the LBF model, while yielding similar results.

These localized methods [14, 16, 25, 26] are, however, to some extent sensitive to contour initialization. Since the segmentation results typically depend on the selection of initial contours, these methods need user intervention to define the initial contours professionally. This means that they may be fraught with the problems of how and where to define the initial contours. The multiple-seed initialization used by Vese and Chan [21] also cannot always produce better results than the single-seed initialization [27]. Therefore, it is still a great challenge to find an efficient way to tackle the initialization problem.

In this article, inspired by the above-mentioned four models [Chan–Vese (CV), PS, RSF, and LIF models], we propose a novel image segmentation algorithm without initial contours using PS approximation. The four models share the same algorithm structure to find a PC/smooth approximation to the original image; the main difference is how to define the energy functional to be minimized and the PC/smooth function. Pursuing the same idea, in this study, we introduce different energy functional and PS functions to search for the optimal PS approximation of the original image. In the proposed algorithm, the initial function can be chosen as a constant function, which implies that the proposed algorithm allows for robustness to initialization or even free of manual initialization. Experiments show that the proposed algorithm works well for images with intensity inhomogeneity. Moreover, it is very appropriate for infrared images with low contrast and complex background.

The remainder of this article is organized as follows. In “Related studies” section, we briefly review several classical region-based models and indicate their limitations. The proposed algorithm is introduced in “The LIF model results in a local fitted image” section. The proposed model presents experimental results using a set of synthetic and real images. This article is summarized in “Experimental results” section.

Related studies

Chan–Vese (CV) PC models

Chan and Vese [12] restricted the MS minimal partition problem [19] to PC functions and proposed a technique that implements efficiently the PC MS model via level set methods [18]. In level set methods, a contour C is represented implicitly by the zero-level set of a Lipschitz function $\varphi :\Omega \to$, which is called a level set function. In what follows, we let the level set function $\varphi$ take positive and negative values inside and outside the contour C, respectively.

Let $I:\Omega \subset 2\to$ be an input image and H be the Heaviside function, the energy functional of the CV model is defined as:

$$\begin{array}{c}{E}^{\mathit{CV}}(c_1,c_2,\varphi )={\lambda }_1{{\displaystyle {\int }_{\Omega }\left|I-c_1\right|}}^{2}H(\varphi )dx+{\lambda }_1{{\displaystyle {\int }_{\Omega }\left|I-c_2\right|}}^2(1-H(\varphi ))dx\\ +\nu {\displaystyle {\int }_{\Omega }\left|\nabla H\left(\varphi \right)\right|d}x\end{array}$$

(1)

where ${\lambda }_1,{\lambda }_2>0$, v > 0 are constants. c₁ and c₂ are the global averages of the image intensities in the region $\left\{x:\varphi \left(x\right)>0\right\}$ and$\left\{x:\varphi \left(x\right)<0\right\}$, respectively, which are defined as:

$$c_1\left(\varphi \right)=\frac{{\displaystyle {\int }_{\Omega }I\left(x\right)H\left(\varphi \left(x\right)\right)dx}}{{\displaystyle {\int }_{\Omega }H\left(\varphi \left(x\right)\right)dx}},c_2\left(\varphi \right)=\frac{{\displaystyle {\int }_{\Omega }I\left(x\right)\left(1-H\left(\varphi \left(x\right)\right)\right)dx}}{{\displaystyle {\int }_{\Omega }\left(1-H\left(\varphi \left(x\right)\right)\right)dx}}$$

(2)

The CV model is implemented by an alternative procedure: for each iteration and the corresponding level set function ${\varphi }^n$, we first compute the optimal constants $c_1\left({\varphi }^n\right)$ and $c_2\left({\varphi }^n\right)$, then obtain ${\varphi }^{n+1}$ by minimizing $E^{\mathit{CV}}\left(c_1\left({\varphi }^n\right),c_2\left({\varphi }^n\right),\varphi \right)$ with respect to $\varphi$. This process is repeated until the zero-level set of ${\varphi }^{n+1}$ is exactly on the object boundary.

The solution of the CV model in fact leads to a PC segmentation of the original image I (x):

$$u\left(x\right)=c_1{H}_{\epsilon }\left(\varphi \left(x\right)\right)+c_2\left(1-H_{\epsilon }\left(\varphi \left(x\right)\right)\right)$$

(3)

where c₁ and c₂ are the optimal constants c₁ and c₂ that minimize the global fitting energy (1) are the averages of the image intensities in the region $\left\{x:\varphi \left(x\right)>0\right\}$ and $\left\{x:\varphi \left(x\right)<0\right\}$, respectively. Such optimal constants can be far away from the original image data, if the intensities outside or inside the contour $C=\left\{x:\varphi \left(x\right)=0\right\}$ are not homogeneous. As a result, the CV model generally fails to segment images with intensity inhomogeneity.

(PS) model

Intensity inhomogeneity can be addressed by more sophisticated models than PC models. Vese and Chan [21] and Tsai et al. [22] independently propose two similar region-based models for general images. These models, widely known as (PS) models, aims at expressing the intensities inside and outside the contour as (PS) functions instead of constants. The following energy functional was defined:

$$\begin{array}{c}{E}^{\mathit{PS}}(u^{+},u^{-},\varphi )={{\displaystyle {\int }_{\Omega }\left|u^{+}-I\right|}}^{2}H(\varphi )dx+\mu {{\displaystyle {\int }_{\Omega }\left|\nabla u^{+}\right|}}^{2}H(\varphi )dx\\ +{{\displaystyle {\int }_{\Omega }\left|u^{-}-I\right|}}^2\left(1-H\left(\varphi \right)\right)dx+\mu {{\displaystyle {\int }_{\Omega }\left|\nabla u^{-}\right|}}^2\left(1-H\left(\varphi \right)\right)dx\\ +\nu {\displaystyle {\int }_{\Omega }\left|\nabla H\left(\varphi \right)\right|dx}\end{array}$$

(4)

where u⁺and u^- are smooth functions approximating the image I inside and outside the contour, respectively, which are obtained by solving the following two damped Poisson equations:

$$u^{+}-I=\mu \Delta u^{+}in\left\{x:\varphi \left(x\right)>0\right\},\frac{\partial u^{+}}{\partial n}=0on\left\{x:\varphi \left(x\right)=0\right\}$$

(5)

$$u^{-}-I=\mu \Delta u^{-}in\left\{x:\varphi \left(x\right)<0\right\},\frac{\partial u^{-}}{\partial n}=0on\left\{x:\varphi \left(x\right)=0\right\}$$

(6)

The PS model is also implemented by an alternative procedure: for each iteration and the corresponding level set function ${\varphi }^n$, we first obtain $u^{+}\left({\varphi }^n\right)$and $u^{-}\left({\varphi }^n\right)$ by solving Equations (5) and (6), then obtain ${\varphi }^{n+1}$by minimizing the functional $E^{\mathit{PS}}\left(u^{+}\left({\varphi }^n\right),u^{-}\left({\varphi }^n\right),\varphi \right)$ with respect to $\varphi$. Repeat the process until the zero-level set of ${\varphi }^{n+1}$ is exactly on the object boundary.

The solution of the PS model lead to a PS approximation of the original image I(x):

$$u\left(x\right)=u^{+}H\left(\varphi \left(x\right)\right)+u^{-}\left(1-H\left(\varphi \left(x\right)\right)\right)$$

(7)

where u⁺ and u^- are obtained by solving Equations (5) and (6). In the PS model, two coupled equations must be solved to obtain u⁺ and u^- before each iteration, and the computational cost is very expensive. Moreover, in the implementation of PS model, u⁺ and u^- must be extended to the whole image domain, which is difficult to implement and also increases the computational cost. In summary, the high complexity limits the application of PS model in practice.

RSF model

In order to improve the performance of the global CV [12] and PS models on [21, 22] images with inhomogeneity, Li et al. [14, 26] recently proposed a novel region-based active contour model in a variational level set formulation. They introduced a kernel function and defined the following energy functional:

$$\begin{array}{c}{E}^{\mathit{RSF}}(f_1,f_2,\varphi )={\lambda }_1{\displaystyle \int \left[{\displaystyle \int K_{\sigma }\left(x-y\right){\left|I\left(y\right)-f_1\left(x\right)\right|}^{2}H\left(\varphi \left(y\right)\right)dy}\right]dx}\\ +{\lambda }_2{\displaystyle \int \left[{\displaystyle \int K_{\sigma }\left(x-y\right){\left|I\left(y\right)-f_1\left(x\right)\right|}^2\left(1-H\left(\varphi \left(y\right)\right)\right)dy}\right]dx}\\ +\nu {\displaystyle {\int }_{\Omega }\left|\nabla H\left(\varphi \left(x\right)\right)\right|}dx+\mu {\displaystyle {\int }_{\Omega }\frac{1}{2}{\left(\left|\nabla \varphi \left(x\right)\right|-1\right)}^{2}dx}\end{array}$$

(8)

where K _σ is a Gaussian kernel with standard deviation σ, and f₁ (x) and f₂ (x) are two smooth functions that approximate the local image intensities inside and outside the contour, respectively. They are computed as:

$$f_1\left(x\right)=\frac{{\displaystyle {\int }_{\Omega }{K}_{\sigma }\left(x-y\right)I\left(y\right)H\left(\varphi \left(y\right)\right)dy}}{{\displaystyle {\int }_{\Omega }{K}_{\sigma }\left(x-y\right)H\left(\varphi \left(y\right)\right)dy}},f_2\left(x\right)=\frac{{\displaystyle {\int }_{\Omega }{K}_{\sigma }\left(x-y\right)I\left(y\right)\left(1-H\left(\varphi \left(y\right)\right)\right)dy}}{{\displaystyle {\int }_{\Omega }{K}_{\sigma }\left(x-y\right)\left(1-H\left(\varphi \left(y\right)\right)\right)dy}}$$

(9)

The RSF model is implemented via an alternative procedure: for each iteration and the corresponding level set function ${\varphi }^n$, we first compute the fitting values $f_1\left({\varphi }^n\right)$ and $f_2\left({\varphi }^n\right)$, then obtain ${\varphi }^{n+1}$ by minimizing $E^{\mathit{RSF}}\left(f_1\left({\varphi }^n\right),f_2\left({\varphi }^n\right),\varphi \right)$ with respect to $\varphi$. This process is repeated until the zero-level set of ${\varphi }^{n+1}$ is exactly on the object boundary.

Like the PS model, the solution of the RSF model also lead to a piecewise smooth approximation of the original image I (x):

$$u\left(\mathbf{x}\right)=f_1{H}_{\epsilon }\left(\varphi \left(\mathbf{x}\right)\right)+f_2\left(1-H_{\epsilon }\left(\varphi \left(\mathbf{x}\right)\right)\right)$$

(10)

However, the smooth functions f₁ and f₂ are computed directly from (9) but no longer obtained by solving equations.

The RSF model improves the global PC and PS models on intensity inhomogeneity and is more computationally efficient than the PS models. Nevertheless, four convolutions to be computed at each iteration still demand a high computational cost. Besides, the RSF model is to some extent sensitive to contour initialization (initial locations, sizes and shapes); it obtained different segmentation with different initial contours, even with the same parameter settings.

This can be seen from a simple experiment for an X-ray vessel image that used in [14]. Figure 1 shows the segmentation results of the RSF model for the vessel image with five different initial contour locations (circles with the same size but different positions). The initial contours over the original image are shown in the upper row of Figure 1. From the second row of Figure 1, we observe that the RSF model fails to extract the vessel for the first initial location (Figure 1(f)), although obtain satisfactory segmentation results for the other four locations (Figure 1(g)-(j)). Figure 2 displays the segmentation results of the RSF model for the vessel image with different initial contour sizes and shapes. The initial contours are chosen as circles and squares at the center of image but with different sizes, as shown in the upper row of Figure 2. The segmentation results in the second row of Figure 2 show that the RSF model can only obtain accurate segmentation for the second initial contour (Figure 2(g)).

Figure 1

Results of the RSF and LIF models for a vessel image with five distinct initial contour locations. Upper row: original image and initial contours. Middle row: segmentation results of the RSF model (2000, 120, 200, 120, 230 iterations). Lower row: segmentation results of the LIF model (2000, 500, 200, 500, 2000 iterations).

Figure 2

Results of the RSF and LIF models for a vessel image with five distinct initial contour sizes and shapes. Upper row: original image and initial contours. Middle row: segmentation results of the RSF model (2000, 350, 2000, 2000, 2000 iterations). Lower row: segmentation results of the LIF model (2000, 500, 200, 500, 2000 iterations).

LIF model

For efficient segmentation, Zhang et al. [16] utilized the local image information and proposed a LIF energy functional by minimizing the difference between the fitted image and the original image. The formulation is expressed as:

$$E^{\mathit{LIF}}\left(\varphi \right)=\frac{1}{2}{{\displaystyle {\int }_{\Omega }\left|I\left(x\right)-\left(m_{1}H\left(\varphi \right)+m_2\left(1-H\left(\varphi \right)\right)\right)\right|}}^{2}dx$$

(11)

where I (x) is the image to be segmented, while m₁ and m₂ are two fitting functions defined as

$$\{\begin{array}{l}{m}_1=mean(I\in (\{x\in \Omega |\varphi \left(x\right)<0\}\cap W_k(x)))\\ m_2=mean(I\in (\{x\in \Omega |\varphi \left(x\right)\ge 0\}\cap W_k(x)))\end{array}$$

(12)

Here, W _k (x) is a rectangular window, which is generally chosen as a truncated Gaussian window K _σ (x) with standard deviation σ and of size 4 k × 4 k (k is the greatest integer smaller than σ).

Similar as the above three models, the LIF is also implemented by an alternative procedure: for each iteration and the corresponding level set function ${\varphi }^n$, we first compute the fitting values $m_1\left({\varphi }^n\right)$ and $m_2\left({\varphi }^n\right)$, then obtain ${\varphi }^{n+1}$ by minimizing $E^{\mathit{LIF}}\left(\varphi \right)$ with respect to $\varphi$. This process is repeated until the zero-level set of ${\varphi }^{n+1}$ is exactly on the object boundary.

The LIF model results in a local fitted image

$$I^{\mathit{LFI}}\left(\mathbf{x}\right)=m_{1}H\left(\varphi \left(\mathbf{x}\right)\right)+m_2\left(1-H\left(\varphi \left(\left(\mathbf{x}\right)\right)\right)\right)\text{.}$$

(13)

It is also a PS approximation of the original image but with different fitting values m₁ and m₂.

By using local region information, the LIF model is able to provide desirable segmentation results in the presence of intensity inhomogeneity. Although it is more efficient than the RSF model, it is still sensitive to contour initialization, like most existing localized active contours [14, 16, 25, 26]. In practice, the LIF model initial contour has to carefully be selected for a satisfactory segmentation.

This can be seen from a similar experiment as the RSF model in Figures 1 and 2. Figure 1(k)-(o) shows the segmentation results of the LIF model for the vessel image with five distinct initial locations, which reveal that the LIF model extracts accurately the object only for the second and fourth initial contours (Figure 1 (l, n)). In Figure 2(k)-(o), we demonstrate the segmentation results of the LIF model for the vessel image with five initial contours of different sizes and shapes. From the results, we observe that the LIF model fails to segment the vessel image for the third and fifth initial contours (Figure 2 (m, o)), although it better captures the object for the other three initial contours (Figure 2 (k, l, n)).

The proposed model

The above-mentioned four models share the similar algorithm structure to find a PS (constant) approximation of the original image. The main difference is how to define both the energy functional to be minimized and the PS function approximating the original image. In this section, pursuing the same idea we introduce different energy functional and PS function to search for the optimal PS approximation of the original image.

For a given image $I:\Omega \subset \mathbf{2}\to$ and a level set function $\psi$, we first define two smooth functions as follows:

$$\begin{array}{l}{h}_1(\mathbf{x},\psi )=\frac{{\displaystyle {\int }_{\Omega }{K}_{\sigma }\left(\mathbf{x}-\mathbf{y}\right)I\left(\mathbf{y}\right)H\left(\psi \left(\mathbf{y}\right)\right)d\mathbf{y}}}{{\displaystyle {\int }_{\Omega }{K}_{\sigma }\left(\mathbf{x}-\mathbf{y}\right)H\left(\psi \left(\mathbf{y}\right)\right)d\mathbf{y}}},\mathbf{x}\in \Omega \\ h_2(\mathbf{x},\psi )=\frac{{\displaystyle {\int }_{\Omega }{K}_{\sigma }\left(\mathbf{x}-\mathbf{y}\right)I\left(\mathbf{y}\right)\left(1-H\left(\psi \left(\mathbf{y}\right)\right)\right)d\mathbf{y}}}{{\displaystyle {\int }_{\Omega }{K}_{\sigma }\left(\mathbf{x}-\mathbf{y}\right)\left(1-H\left(\psi \left(\mathbf{y}\right)\right)\right)d\mathbf{y}}}\end{array}$$

(14)

where H is the Heaviside function, and K _σ is a Gaussian kernel function:

$$K_{\sigma }\left(\mathbf{u}\right)=\frac{1}{{\left(2\pi \right)}^{1/2}\sigma }{e}^{-\left|\mathbf{u}\right|/2{\sigma }^2}$$

(15)

with a scale parameter σ > 0. The functions $h_1\left(\mathbf{x},\psi \right)$ and $h_2\left(\mathbf{x},\psi \right)$ are really the weighted averages of the image intensities in the regions $\left\{\mathbf{x}:\psi \left(\mathbf{x}\right)>0\right\}$ and $\left\{\mathbf{x}:\psi \left(\mathbf{x}\right)\le 0\right\}$ respectively, with $K_{\sigma }\left(\mathbf{x}-\mathbf{y}\right)$ as the weight assigned to the intensity I(y). The smoothness of $h_1\left(\mathbf{x},\psi \right)$ and $h_2\left(\mathbf{x},\psi \right)$ can be confirmed by the Gaussian convolutions in (14).

Next, we introduce another level set function $\varphi$ to alternate with $\psi$, and finally find a level set function ${\varphi }^{\ast }$ so that the following PS function:

$$\begin{array}{l}{I}_{\psi }^{\mathit{PS}}(\mathbf{x},{\varphi }^{\ast })=h_1(\mathbf{x},\psi )H({\varphi }^{\ast }(\mathbf{x}))+h_2(\mathbf{x},\psi )(1-H({\varphi }^{\ast }(\mathbf{x})))\\ =\{\begin{array}{l}{h}_1(\mathbf{x},\psi ),\mathbf{x}\in \{{\varphi }^{\ast }>0\}\\ h_2(\mathbf{x},\psi ),\mathbf{x}\in \{{\varphi }^{\ast }\le 0\}\end{array}\end{array}$$

(16)

approximates optimally the original image I(x), in the sense of

$$E_{\psi }^{\mathit{PSF}}\left({\varphi }^{\ast }\right)=\underset{\varphi }{min}{E}_{\psi }^{\mathit{PSF}}\left(\varphi \right)$$

(17)

with

$$E_{\psi }^{\mathit{PSF}}\left(\varphi \right)={\displaystyle {\int }_{\Omega }{\left|I\left(\mathbf{x}\right)-h_1\left(\mathbf{x},\psi \right)\right|}^{2}H\left(\varphi \left(\mathbf{x}\right)\right)}d\mathbf{x}+{\displaystyle {\int }_{\Omega }{\left|I\left(\mathbf{x}\right)-h_2\left(\mathbf{x},\psi \right)\right|}^2\left(1-H\left(\varphi \left(\mathbf{x}\right)\right)\right)d\mathbf{x}}$$

(18)

where $\varphi$ is a level set function over the image domain Ω. The energy functional $E_{\psi }^{\mathit{PSF}}\left(\varphi \right)$ can clearly be rewritten as

$$E_{\psi }^{\mathit{PSF}}\left(\varphi \right)={\displaystyle {\int }_{\Omega }{\left|I\left(\mathbf{x}\right)-I_{\psi }^{\mathit{PS}}\left(\mathbf{x},\varphi \right)\right|}^2}d\mathbf{x}$$

(19)

The energy functional $E_{\psi }^{\mathit{PSF}}\left(\varphi \right)$ in (19) is in fact a square error of the approximation of the image I (x) by the PS function $I_{\psi }^{\mathit{PS}}\left(\mathbf{x},\varphi \right)$; therefore, we call $E_{\psi }^{\mathit{PSF}}\left(\varphi \right)$ the piecewise smooth fitting (PSF) energy.

As in typical level set methods [12, 14, 15, 21], it is necessary to smooth the zero-level set by penalizing its length:

$$L\left(\varphi \right)={\displaystyle {\int }_{\Omega }\left|\nabla H\left(\varphi \left(\mathbf{x}\right)\right)\right|}d\mathbf{x}$$

(20)

In addition, for more accurate computation and stable level set evolution, we need to regularize the level set function by penalizing its deviation from a signed distance function [13, 28], which can be characterized by the following energy functional

$$P\left(\varphi \right)={\displaystyle {\int }_{\Omega }\frac{1}{2}{\left(\left|\nabla \varphi \left(\mathbf{x}\right)\right|-1\right)}^{2}d\mathbf{x}}$$

(21)

The regularizing term $P\left(\varphi \right)$ intrinsically maintains the regularity of the level set function without the need for extra re-initialization procedures.

Therefore, the total energy functional of the proposed model is given by

$$\begin{array}{c}{E}_{\psi }(\varphi )=E_{\psi }^{\mathit{PSF}}(\varphi )+vL(\varphi )+\mu P(\varphi )\\ ={\displaystyle {\int }_{\Omega }{\left|I-h_1\right|}^{2}H\left(\varphi \right)}d\mathbf{x}+{\displaystyle {\int }_{\Omega }{\left|I-h_2\right|}^2\left(1-H\left(\varphi \right)\right)d\mathbf{x}}\\ +\nu {\displaystyle {\int }_{\Omega }\left|\nabla H\left(\varphi \right)\right|}d\mathbf{x}+\mu {\displaystyle {\int }_{\Omega }\frac{1}{2}{\left(\left|\nabla \varphi \right|-1\right)}^{2}d\mathbf{x}}\end{array}$$

(22)

where v, μ > 0 are constants to balance the terms.

In practice, the Heaviside function H(z) needs to be approximated by a smooth function $H_{\epsilon }\left(z\right)$, which is typically given by

$$H_{\epsilon }\left(z\right)=\frac{1}{2}\left[1+\frac{2}{\pi }arctan\left(\frac{z}{\epsilon }\right)\right]$$

(23)

Following the same algorithm structure as the four models mentioned in “Related studies” section, the proposed algorithm can simply be described as follows:

• 1 Initialize the level set function${\psi }^0\left(\mathbf{x}\right)$, and set n = 0.

• 2. Compute the smooth functions $h_1\left(\mathbf{x},{\psi }^n\right)$ and$h_2\left(\mathbf{x},{\psi }^n\right)$.

• 3. Obtain ${\varphi }^{n+1}\left(\mathbf{x}\right)$ by minimizing the energy functional$E_{{\psi }^n}\left(\varphi \right)$.

• 4. If the zero-level set of ${\varphi }^{n+1}\left(\mathbf{x}\right)$ is exactly on the object boundary, then stop; otherwise, let n = n + 1 and ${\psi }^n\left(\mathbf{x}\right)={\varphi }^n\left(\mathbf{x}\right)$, then return to Step 2.

Note that the minimization problem in step 3 can be solved by the standard gradient descent method [29]. In detail, for ${\psi }^n$, we minimize $E_{{\psi }^n}\left(\varphi \right)$ with respect to $\varphi$ by solving the following gradient flow equation:

$$\begin{array}{l}\frac{\partial \varphi }{\partial t}=-{\delta }_{\epsilon }(\varphi )\left({\left(I-h_1\left(\mathbf{x},{\psi }^n\right)\right)}^2-{\left(I-h_2\left(\mathbf{x},{\psi }^n\right)\right)}^2\right)\\ +\nu {\delta }_{\epsilon }(\varphi )div\left(\frac{\nabla \varphi }{\left|\nabla \varphi \right|}\right)+\mu \left(\Delta \varphi -div\left(\frac{\nabla \varphi }{\left|\nabla \varphi \right|}\right)\right)\end{array}$$

(24)

with the initial condition $\varphi \left(0,\mathbf{x}\right)={\psi }^n\left(\mathbf{x}\right)$ and Neumann boundary condition. In Equation (24), ${\delta }_{\epsilon }\left(z\right)$ is the derivative of the function$H_{\epsilon }\left(z\right)$:

$${\delta }_{\epsilon }\left(z\right)={H^{\prime }}_{\epsilon }\left(z\right)=\frac{1}{\pi }\frac{\epsilon }{{\epsilon }^2+z^2}$$

(25)

Remarks:

The CV PC model can be considered as an extreme case of the proposed model for$\sigma \to \infty$. This can be seen from the fact that ${lim}_{\sigma \to \infty }{h}_i\left(\mathbf{x},\psi \right)=c_i\left(i=1,2\right)$[14].

For the regularized Heaviside function$H_{\epsilon }$, the functions $h_1\left(\mathbf{x},\psi \right)$ and $h_2\left(\mathbf{x},\psi \right)$ in (14) are well defined for any $\psi$. In applications, we suggest that the function $\psi$ be simply initialized to a constant function:

$${\psi }_0\left(\mathbf{x}\right)=\rho ,\mathbf{x}\in \Omega$$

(26)

where ρ is a constant. Such constant initialization completely eliminates the need of initial contours. We do not need to consider the problems such as the determination of where and how to define the initial contours.

Experimental results

Equation (24) is numerically implemented using a simple finite differencing (forward-time central-space finite difference scheme). All the spatial partial derivatives $\partial \varphi /\partial x$ and $\partial \varphi /\partial y$are approximated by the central difference, and the temporal partial derivative $\partial \varphi /\partial t$is discretized as the forward difference. The approximation of Equation (24) can simply be written as

$${\varphi }_{i.j}^{k+1}={\varphi }_{i.j}^k+\Delta t.L\left({\varphi }_{i.j}^k\right)$$

(27)

where $L\left({\varphi }_{i.j}^k\right)$is the approximation of the right-hand side of Equation (24) by the above spatial difference scheme.

We next present the experiments on both synthetic and real images with intensity inhomogeneity, noise and complex background. The function $\psi$ is simply initialized to a constant function: ${\psi }_0=2$. Unless otherwise specified, we use the following default setting of the parameters which are determined by experiments: $\sigma =18$, $\epsilon =1.0$, $\mu =1$, $\nu =0.001\times 255\times 255$, space step $h=1$(implied pixel spacing) and time step $\Delta t=0.1$. All experiments were run under Matlab R2007a on a PC with Dual 2.7 GHz processor.

Figure 3 shows the segmentation process of our model for four typical images with intensity inhomogeneity taken from [14]. The four images, which are plotted in Figure 3a,f,k,p, are a synthetic image (79 × 75) with Gaussian noise, a real image of a T-shaped object (127 × 90), an X-ray image of blood vessel (103 × 131), and a brain MR image (119 × 78). The contours (zero-level set) evolution processes are shown in second to forth columns. Although there are no initial contours for such non-zero constant function, we see from the second column that the contours emerge automatically after a few iterations. For the sake of clarity, we also list the segmentation results of the RSF model as well as initial contours (in red) in the last column of Figure 3. It can be seen from Figure 3 that our model obtains the similar results with the classical RSF model on the segmentation of images with intensity inhomogeneity.

Figure 3

The curve evolution process of our model from the automatically generated contours to the final contours, starting with a constant function${\mathbf{\psi }}_{\mathbf{0}}\mathbf{=}\mathbf{2}$. First column: original images. Second and third columns: middle evolution process. Fourth column: final contours.

The second example (see Figures 4 and 5) shows that our model can work well for images with noise or non-uniform background and images with multiple objects of inhomogeneous intensities, starting with a constant function. Meanwhile, to further demonstrate the advantages of our model, we give the segmentation results of one traditional method (Canny edge detector [7]) and two famous active contour models (the CV model [12] and the RSF model [14]) for comparison. Note that, for the Canny edge detector, we implement the fix threshold version and optimize the method with respect to σ (the scale parameter in the Gaussian filtering function K _σ ) and to the gradient thresholds $T=\left[T_{\mathit{low}},T_{\mathit{high}}\right]$; for the CV and RSF models, the initial contours are necessary (we choose a square centered at the image with the size of 40 × 40 pixels for all the images in Figures 4, 5 and 6). Besides, we choose the best parameters for them (given in the Figure Caption) and list the best results we have obtained.

Figure 4

Segmentations of four models for four real images with non-uniform backgrounds, starting with a constant function${\mathbf{\psi }}_{\mathbf{0}}\mathbf{=}\mathbf{2}$. (a)-(d): original images. (e)-(h): results of the Canny algorithm (left to right: $\sigma =1,T=\left[0\text{.02,}0.2\right]$; $\sigma =1,T=\left[0\text{.2,}0.3\right]$; $\sigma =2.2,T=\left[0\text{.09,}0.7\right]$; $\sigma =2.2,T=\left[0\text{.09,}0.4\right]$). (i)-(l): results of the CV model (5, 5, 300, 500 iterations). (m)-(p): results of the RSF model (left to right: $\nu =0.01\times 255\times 255$with 300 iterations, $\nu =0.008\times 255\times 255$with 560 iterations, ${\lambda }_2=1.2,\nu =0.008\times 255\times 255$ with 300 iterations, $\nu =0.004\times 255\times 255$ with 300 iterations). (q)-(t): results of our model (12, 24, 45, 87 iterations). (u) and (x): 3D plots of final level set functions.

Figure 5

Segmentations of four models for four images with multiple objects of inhomogeneous intensities or inside holes, starting with a constant function${\mathbf{\psi }}_{\mathbf{0}}\mathbf{=}\mathbf{2}$. (a)-(d): original images. (e)-(h): results of the Canny algorithm (left to right: $\sigma =1,T=\left[0\text{.06,}0.17\right]$; $\sigma =2,T=\left[0\text{.05,}0.4\right]$; $\sigma =1,T=\left[0\text{.006,}0.3\right]$; $\sigma =1,T=\left[0\text{.006,}0.02\right]$). (i)-(l): results of the CV model (left to right: ${\lambda }_1=3.5$with 500 iterations, $\nu =0.015\times 255\times 255$ with 500 iterations, 10, 300 iterations). (m)-(p): results of the RSF model (left to right: ${\lambda }_1=1.2$with 600 iterations, $\nu =0.01\times 255\times 255$with 500 iterations, 500 iterations, ${\lambda }_1=1.5,\nu =0.002\times 255\times 255$ with 240 iterations). (q)-(t): results of our model (left to right: 21, 15, 10, 45 iterations). (u) and (x): 3D plots of final level set functions.

Figure 6

Segmentations of four models for four real infrared images with blurry edges and complex backgrounds, starting with a constant function${\mathbf{\psi }}_{\mathbf{0}}\mathbf{=}\mathbf{2}$. (a)-(d): original images. (e)-(h): results of the Canny algorithm (left to right: $\sigma =1,T=\left[0\text{.01,}0.3\right]$; $\sigma =1.3,T=\left[0\text{.03,}0.4\right]$; $\sigma =2,T=\left[0\text{.03,}0.6\right]$; $\sigma =1.5,T=\left[0\text{.03,}0.9\right]$). (i)-(l): results of the CV model (left to right: 500 iterations, 140 iterations, $\nu =0.015\times 255\times 255$with 500 iterations, ${\lambda }_1=1.2,\nu =0.02\times 255\times 255$ with 500 iterations). (m)-(p): results of the RSF model (left to right: 40 iterations, ${\lambda }_1=1.2,\nu =0.005\times 255\times 255$with 500 iterations, ${\lambda }_1=1.2,\nu =0.007\times 255\times 255$ with 1000 iterations, $\nu =0.06\times 255\times 255$ with 280 iterations). (q)-(t): results of our model (15 iterations). (u) and (x): 3D plots of final level set functions.

Figure 4 shows the segmentation results of four methods for four real images: a leaf image (122×120) with texture background, a chip image (182 × 180), a breast cyst image (91 × 92), and a CT medical image (117 × 123). Because of the influence of the texture, material properties, breast tissue, and the technical limitations, the backgrounds in these images are non-uniform, noise or even complex. Figure 4(e)-(h) give the contours detected by Canny algorithm; we see that some contours detected by this algorithm are not continuous (Figure 4(e)). From the results of the CV model shown in Figure 4(i)-(l), we observe that, the CV model works well for images with less non-uniform (Figure 4(i) and (j)), however, if the level of non-uniform is higher, it fails to obtain exact segmentation results and identifies incorrectly some part of the background/foreground as the background/foreground (Figure 4(k) and (l)). Figure 4(m)-(p) shows the results of the RSF model; with fine-tuning parameters (given in the Figure Caption), the RSF model obtain satisfactory segmentation results expect for a little deficit in the lower left corner of the object in Figure 4(p). The results of our model and the corresponding 3D plots of final level set functions are shown in the last two rows of Figure 4. As can be seen from Figure 4(q)-(t), our model has effectively extracted object boundaries from the non-uniform and noisy background. For these four images, we chose large values for v to enhance the model’s robustness to noise: σ = 5 for Figure 4(a), $\sigma =20,\nu =0.003\times 255\times 255$ for Figure 4(b), $\nu =0.0085\times 255\times 255$ for Figure 4(c) and $\sigma =12,\nu =0.003\times 255\times 255$ for Figure 4(d).

Figure 5 show experiments on multiple objects and inside holes segmentation. Figure 5(a)-(d) is: a potato image (157 × 157) with inhomogeneous intensities, a cell image (195 × 183) with one cell crossing the image boundary, a wrench image (200 × 200) with interior holes in each object, and a synthetic image (118 × 134) with seven different intensity values, respectively. Figure 5(e)-(h) give the contours detected by Canny algorithm; we see that it extracts the satisfactory object contours for images with obvious intensity gradient (Figure 5(f)-(h)), but the contour detected for another image is disturbed by non-uniform intensity (Figure 5(e)). From the results of the CV model shown in Figure 5(i)-(l), we observe that, the CV model can deal with homogeneous images with multiple objects or inside holes (Figure 5(f) and (g)), but if the intensity in such images are inhomogeneous, it fails to obtain the whole boundaries for all of the objects (Figure 5(i) and (l)). Figure 5(m)-(p) shows the best segmentation results of the RSF model we obtained by tuning parameters; it obtain the exact result only for the last image (Figure 5(p)). The results of our model and the corresponding 3D plots of final level set functions are shown in the last two rows of Figure 5. As can be seen from Figure 5(q)-(t), our method detects all the objects, even though the object intensity is inhomogeneous and some edges are blurry.

The third example presented in Figure 6 demonstrates that our model is especially suitable for segmentation of infrared ship images with low contrast, blurry edges and complex (noisy) background. Because of the limitation of thermal imaging and the actual water-mountain-sky conflicts, infrared images always suffers from low contrast and complex background; thus, it is not a trivial task for most existing models to extract the objects (ships) in such images. As can be seen from Figure 6(e)-(h), Canny algorithm has difficulty dealing with such infrared ship images, due to the presence of noise, low contrast, complex background and so on. To suppress noise effectively, a larger σ or a higher threshold T is needed, but a larger σ can result in edge shifting or inaccurate edge location (Figure 6(s)) and a higher T always causes edge leaking (Figure 6(r)). From the results of the CV model shown in Figure 6(i)-(l), we observe that, the CV model obtain acceptable results in Figure 6(j) and (k), but due to the intensity inhomogeneity and noise, it can’t obtain the exact object boundaries in Figure 6(i) and (l). Figure 6(m)-(p) shows the segmentation results of the RSF model; it obtain the accurate results only for the first and the last image (Figure 6(m) and (p)). In contrast, our model, starting with a nonzero constant function, has successfully extracted the ships from the complex backgrounds, as shown in Figure 6(q)-(t). For noisy images, we increase the value of v to improve the model’s performance: $\nu =0.007\times 255\times 255$ for Figure 6(c) and $\nu =0.02\times 255\times 255$ for Figure 6(d).

Conclusion

In this article, we propose a novel image segmentation algorithm by PS approximation. The proposed model, which pursues the basic ideas behind four well-known region-based models, introduces different energy functional and PS functions to find the optimal PS approximation of the original image. The initial function can be chosen as a constant function, which completely eliminates the need of initial contours. The proposed model is very appropriate for a wider range of images, including images with intensity inhomogeneity and infrared images with low contrast and complex background.