Keywords

1 Introduction

Image segmentation means that classifies all pixels to form sub regions based on certain classification rules. So far it has played an important role in the field of pattern recognition and computer vision. Image segmentation usually is the first step of the image processing, which the segmentation results will influence the final results of the whole image processing. Now image segmentation has been widely used in many fields, such as intelligent transportation, remote sensing image and medical image processing and so on. At present, there are plenty of common image segmentation algorithms, including threshold segmentation method [1], regional growth or Split Merge segmentation method, common edge detection operator segmentation method [2], morphological segmentation method [3] and clustering segmentation algorithm. In addition, there are also genetic algorithm segmentation method and neural network segmentation method which are combined with bioinformatics. The image segmentation method based on clustering is very suitable to be extended to color space, which has been attracted many scholars.

Graph cut theory is one of the most widely used image segmentation methods. It is mainly form an energy function, and then optimize the energy function to minimize. Ncut algorithm is a mature technology based on the graph cut theory in the field of image segmentation. Its basic idea is that an image is indicated as an undirected weighted and the pixels are regarded as nodes, following the segmentation of graph-cut theory. However, there are two drawbacks in the traditional Ncut algorithm: the first one is the expensive calculation due to the eigenvector and the eigenmatrix; the second one is the inadequate ability of resisting noise. Many scholars have improved the Ncut algorithm in the application process. For example, Cui [4] presented a method that combined Ncut segmentation with two watershed algorithm to image pre-processing twice, which generated the area block instead of raw image pixels. This method reduced the following computation of Ncut segmentation as well as improved the efficiency of segmentation, but the results were easily influenced by human interference and the segmentation was limited. Huang et al. [5] utilized classical Merge-Split method to improve the segmentation algorithm based on the Ncut segmentation, which compared the similarity of texture and color in the adjacent regions of the image. This method eliminated most situations of over segmentation and under segmentation. However, the selection of value K and the setting of some thresholds will affect the results in different degrees. Moreover, the segmentation results and effects are not yet solved well.

For solving the disadvantages of traditional Ncut algorithm, in this paper, we present an improved Ncut algorithm based on the Mean Shift pre-segmentation. Firstly, Mean Shift pre-segmentation is used on raw image, which helps construct some regional block diagrams. Then, Ncut algorithm is applied to segment image to get the final segmentation results. The experimental results indicate that the improved Ncut algorithm makes the segmentation more efficiently since the number of pixel blocks is much less than the raw image pixels. Furthermore, the improved method has excellent ability to resist noise interference because of the Mean Shift algorithm.

2 Related Work

There are some drawbacks in traditional Ncut algorithm. For example, the algorithm is a kind of NP-hard problems, so it needs expensive computation, which lead to the limitations of the application of real-time image segmentation. Besides, Ncut is chosen as K-means clustering algorithm. In this situation, the selection of parameter K will have pretty more influence on the final segmentation results, so some rules of thumb are needed to ensure how the parameter K should be set. In addition, the algorithm is very sensitive to noise.

In this paper, an improved Ncut algorithm based on Mean Shift pre-segmentation is presented to reduce the computational complexity of the algorithm. The core idea is to pre-segment the original image through Mean Shift, and then use the Ncut algorithm to segment the undirected weighted graph formed by the pixel blocks.

Mean Shift is a spatial analysis method based on the feature offset vectors. One of its great advantages is that it has the property of clustering segmentation. Moreover, the Ncut can be regarded as a classification algorithm. It can satisfy most image segmentation applications. However, it will need expensive calculations when there is a large amount of raw image pixels. Instead, our improved algorithm is based on the pixels block formed by the image block, which can greatly reduce the amount of calculations. Our method has two advantages, which are as follows:

  1. ①

    The process greatly reduces the amount of computation. As the number of regional blocks formed by Mean Shift pre-segmentation is less than the number of raw image pixels, the dimension of eigenmatrix will be reduced greatly. At the same time, the vertices and edges of the graph are also reduced a lot after transforming into a graph.

  2. ②

    The effect of image segmentation will be better. Because Ncut algorithm is pretty sensitive to noise, even a little amount of noise will have great influence on the final image segmentation results. The Mean Shift algorithm has a fixed inhibitory effect on the noise, and reduces the edge of the mutation to some degree. In addition, due to the Ncut is a global clustering algorithm [6], it can effectively eliminate the over-segmentation phenomenon of Mean Shift [7]. Furthermore, the pre-processing of Mean Shift can make up for the disadvantages of the Ncut algorithm which is overly dependent on the classification parameter K.

Mean Shift divides the pixels into different classes through the distance. Therefore, it is necessary to transform the RGB to the LUV before Mean Shift pre-segmentation. In order to eliminate the interference of noise and get more smoothing segmentation results, the median filter is needed to pre-process the image, and then an improved adaptive Mean Shift algorithm is applied to cluster the processed image.

3 The Proposed Algorithm

In general, segmenting image is an ill-conditioned problem. In order to achieve better results, we need to preprocess the image before segmentation. In this section, we first operate space transformation and smoothing on the image, and then present our improved method. Figure 1 shows the schematic of our approach.

Fig. 1.
figure 1

System design.

Full size image

3.1 Color Space Transformation

The full name of LUV is CIE 1976 (L*, u*, v*), where L* represents object luminance, u* and v* are chroma. In general, u* and v* are range from −100 to +100, and luminance L* is range from 0 to 100.

It is relatively simple to transform RGB to LUV. That is, after transforming RGB to XYZ, XYZ is then transformed to LUV. Lena picture in Fig. 2 is selected to show the results of the space transformation.

Fig. 2.
figure 2

Space transformation from RGB to LUV.

Full size image

RGB to CIE XYZ:

$$ \left[ {\begin{array}{*{20}c} X \\ Y \\ Z \\ \end{array} } \right]\, = \,\frac{1}{{b_{21} }}\left[ {\begin{array}{*{20}c} {b_{11} } & {b_{12} } & {b_{13} } \\ {b_{21} } & {b_{22} } & {b_{23} } \\ {b_{31} } & {b_{32} } & {b_{33} } \\ \end{array} } \right]\left[ {\begin{array}{*{20}c} R \\ G \\ B \\ \end{array} } \right]\, = \,\frac{1}{0.17697}\left[ {\begin{array}{*{20}c} {0.49} & {0.31} & {0.20} \\ {0.17697} & {0.81240} & {0.01063} \\ {0.00} & {0.01} & {0.99} \\ \end{array} } \right]\left[ {\begin{array}{*{20}c} R \\ G \\ B \\ \end{array} } \right] $$
(1)

CIE XYZ to LUV:

$$ \begin{aligned}\begin{array}{cc} L^{*} = \left\{ {\begin{array}{*{20}l} {116 \bullet \left( {\frac{Y}{{Y_{n} }}} \right)^{{\frac{1}{3}}} - 16,} \hfill & {\frac{Y}{{Y_{n} }} > \left( {\frac{6}{29}} \right)^{3} } \hfill \\ {\left( {\frac{29}{3}} \right)^{3} \frac{Y}{{Y_{n} }},} \hfill & {\frac{Y}{{Y_{n} }} \le \left( {\frac{6}{29}} \right)^{3} } \hfill \\ \end{array} } \right. \hfill \\ u^{*} = 13L^{*} \, \bullet \,\left( {u^{\prime } - u_{n}^{\prime } } \right) \hfill \\ v^{*} = 13L^{*} \, \bullet \left( {v^{\prime } - v_{n}^{\prime } } \right) \hfill \\ \end{array}\end{aligned} $$
(2)

3.2 Median Filter

The median filter is a typical nonlinear smoothing technique. In some situation, it can overcome the problem of blurred images brought by the minimum mean square filter and mean filter. Moreover, it can efficiently suppress the interference of filter pulse and the noise of image scanning.

The equation of 2D median filter is as follow:

$$ G\left( {x,y} \right) = med\left\{ {f\left( {x - k,y - l} \right),\left( {k,l \in W} \right)} \right\} $$
(3)

Where “med” indicates the median operation, F (x, y) and G (x, y) are the original and processed images respectively. W is a two-dimensional sliding template, which the template size is generally selected as 3.

The median filtering method is to sort the size of the pixels in the sliding window, and then the output pixel values of the filtering result are defined as the median value of the sequence. An example is given in Fig. 3.

Fig. 3.
figure 3

An example of median filter

Full size image

3.3 Mean Shift Pre-segmentation

According to the paper presented by Li et al. [8], an improved adaptive Mean Shift is used to segment the image. In the Mean Shift algorithm, the bandwidth of H is the only parameter which affects the final clustering result. The method is to combine the advantages of the fixed bandwidth selection and the adaptive bandwidth selection. And the bandwidth selection is in formula (4):

$$ h_{i} = \left\{ {\begin{array}{*{20}l} {H_{\hbox{min} } ,} \hfill & {h_{i} \, < \,H_{\hbox{min} } } \hfill \\ {h_{i} ,} \hfill & {H_{\hbox{min} } \, \le \,h_{i} \, \le \,H_{\hbox{max} } ,\,h_{i} \, = \,\left\| {x_{i} \, - \,x_{i,\,k} } \right\|} \hfill \\ {H_{\hbox{max} } ,} \hfill & {{\text{h}}_{\text{i}} \, > \,{\text{H}}_{\hbox{max} } } \hfill \\ \end{array} } \right. $$
(4)

We choose fixed bandwidth \( {\text{H}}_{ \hbox{min} } \) and \( {\text{H}}_{\hbox{max} } \) as the upper and lower limits of bandwidth. When the value of the adaptive bandwidth is in the interval \( \left[ {H_{\hbox{min} } ,H_{\hbox{max} } } \right] \), it will choose the adaptive bandwidth value, otherwise choose the fixed bandwidth value. The method of adaptively determining the bandwidth, that is \( h_{i} = \left\| {x_{i} - x_{i,k} } \right\| \), is presented by Wang et al. [9], where \( x_{i} \) represents the sample, and \( x_{i,k} \) is the k nearest neighbor point. This method is simple in calculation and has little dependence on value K.

3.4 Ncut Clustering Segmentation

After the pre-segmentation of Mean Shift, the pixels of the image will form several regional blocks in which pixels are regarded as pixel blocks.

We construct an undirected weighted graph with nodes formed by pixel blocks. After Ncut is used to segment the resulting undirected weighted graph. The corresponding weight matrix is formula (5):

$$ W_{ij} \, = \,e^{{\frac{{ - \left\| {F_{(i)} - F_{(j)} } \right\|_{2}^{2} }}{{\sigma_{I}^{2} }}}} * \left\{ {\begin{array}{*{20}l} {e^{{\frac{{ - \left\| {X_{(i)} - X_{(j)} } \right\|_{2}^{2} }}{{\sigma_{I}^{2} }}}} ,} \hfill & {if{\text{ i is neighbor j}}} \hfill \\ {0,} \hfill & {\text{else}} \hfill \\ \end{array} } \right. $$
(5)

The weight matrix considers the information of spatial location and eigenvector. In the upper formula, \( F_{(i)} = \{ R_{(i)} ,G_{(i)} ,B_{(i)} \} \) indicates the information of spatial. That is, if the original image is gray image instead of color image, then \( F_{(i)} \) represents the gray value of pixel blocks. \( X_{\left( i \right)} \) represents the spatial location information of the pixel block. As mentioned in the previous section, the definition of the pixel block is as follows: the spatial information of eigenvector is represented by the average RGB of pixel blocks or gray values, while the spatial coordinate information is represented by the central point of the blocks. At the same time, the boundary of the image pretreated by Mean Shift has been saved, which can be used to judge whether the pixel block \( i \) and \( j \) are adjacent.

4 Experiments

4.1 Simulation Platform

The goal of Simulation platform is to produce better experimental results. In this paper, the experimental simulation platform of all algorithms we use is Matlab R2014b. We analyze the performance of the algorithm from two aspects: the segmentation effect and efficiency. To verify the effectiveness of our improved algorithm, we also compare our algorithm with the traditional Ncut algorithm.

4.2 Experimental Treatment

Firstly, an original image of 256 × 384 pixels in the (a) of Fig. 4 is imported. The segmentation results of the Mean Shift, traditional Nut and our improved algorithm are showed in Fig. 4 from (b) to (e) respectively, in which the corresponding parameters of Mean Shift is \( h_{r} = 6 \), \( h_{s} = 8 \), \( M = 2000 \) and the parameter of traditional Nut is \( K = 4\left( {\sigma_{I} ,\sigma_{X} } \right) = \left( {90,110} \right) \). In our experiment, Mean Shift segments the image into 12 regional blocks. The running time of Mean Shift pre-segmentation is 2.59 s. The Ncut segments the combined pixel blocks, which runs 0.61 s, so the total time of improved Ncut is 3.20 s. Since the running time of the classical Ncut segmentation algorithm is 7.86 s, the total time of improved Ncut algorithm is half less than that of the classical Ncut.

Fig. 4.
figure 4

Comparison between the improved algorithm and traditional Ncut algorithm. (a) Original image. (b) Mean Shift. (c) Ncut clustering segmentation. (d) Improved algorithm for grayscale image segmentation. (e) The effect of improved algorithm segmentation.

Full size image

Follow the same operation as Fig. 2, we import an different original image of 277 × 384 pixels in the (a) of Fig. 5, thus we can also get the segmentation results of the Mean Shift, traditional Nut and our improved algorithm respectively, in which the corresponding parameters of Mean Shift is \( h_{r} = 8,\,h_{s} = 8,\,M\, = \,2000 \) and the parameter of traditional Nut is \( K = 2\left( {\sigma_{I} ,\sigma_{X} } \right) = \left( {90,110} \right) \). Here, Mean Shift segments the image into 8 regional blocks, and the running time of Mean Shift pre-segmentation is 3.69 s. The Ncut segments the combined pixels blocks with the running time of 0.75 s, so the total time of improved Ncut is 4.44 s. Because the total running time of classical Ncut segmentation algorithm is 10.56 s, we can see that the total time of improved Ncut algorithm is half less than that of the classical Ncut, too.

Fig. 5.
figure 5

Comparison between the improved algorithm and traditional Ncut algorithm. (a) Original image. (b) Mean Shift. (c) Ncut clustering segmentation. (d) Improved algorithm for grayscale image segmentation. (e) The effect of improved algorithm segmentation.

Full size image

4.3 Effect Analysis

It can be seen from the segmentation results that Mean Shift is a good segmentation of the target, but there will be an obvious situation of over-segmentation. Furthermore, the segmentation effect of classical Ncut algorithm is not so ideal. Instead, our improved algorithm is proved to be relative perfect to image segmentation. With the increase of the number of pixels in the original image, the time of the improved algorithm is much less than that of the classical Ncut algorithm. A comparable table is showed in the following Table 1:

Table 1. Time comparison of the improved Ncut and traditional Ncut.
Full size table

5 Conclusions

Image segmentation is an essential step in image recognition and understanding. Because of the drawbacks of traditional Ncut, such as the expensive calculation and the insufficient ability of resisting noise, an improved Ncut is proposed in this paper to segment color images. First of all, we apply LUV space transform and median filter to pre-process the original image. Then, we adapt Mean Shift to segment the processed image to construct the regional block. Here, the segmentation efficiency will be greatly improved due to the reduction of the number of the pixels in original image. After that the Ncut algorithm is used to get the final segmentation result. Several experimental results show that the accuracy and efficiency of our proposed algorithm are all good. Furthermore, and it can implement the effective segmentation of the color images. However, the segmentation effect is largely affected by the pre-segmentation of Mean Shift. The final segmentation effect of the improved Ncut algorithm will be disturbed when the effect of the pre-segmentation of Mean Shift is not ideal, which is what we need to solve in our future work.