Hard versus fuzzy c-means clustering for color quantization

Abstract

Color quantization is an important operation with many applications in graphics and image processing. Most quantization methods are essentially based on data clustering algorithms. Recent studies have demonstrated the effectiveness of hard c-means (k-means) clustering algorithm in this domain. Other studies reported similar findings pertaining to the fuzzy c-means algorithm. Interestingly, none of these studies directly compared the two types of c-means algorithms. In this study, we implement fast and exact variants of the hard and fuzzy c-means algorithms with several initialization schemes and then compare the resulting quantizers on a diverse set of images. The results demonstrate that fuzzy c-means is significantly slower than hard c-means, and that with respect to output quality, the former algorithm is neither objectively nor subjectively superior to the latter.

1 Introduction

True-color images typically contain thousands of colors, which makes their display, storage, transmission, and processing problematic. For this reason, color quantization (reduction) is commonly used as a preprocessing step for various graphics and image processing tasks. In the past, color quantization was a necessity due to the limitations of the display hardware, which could not handle over 16 million possible colors in 24-bit images. Although 24-bit display hardware has become more common, color quantization still maintains its practical value [1]. Modern applications of color quantization in graphics and image processing include: (i) compression [2], (ii) segmentation [3], (iii) text localization/detection [4], (iv) color-texture analysis [5], (v) watermarking [6], (vi) non-photorealistic rendering [7], (vii) and content-based retrieval [8].

The process of color quantization is mainly comprised of two phases: palette design (the selection of a small set of colors that represents the original image colors) and pixel mapping (the assignment of each input pixel to one of the palette colors). The primary objective is to reduce the number of unique colors, N', in an image to C, C ≪ N', with minimal distortion. In most applications, 24-bit pixels in the original image are reduced to 8 bits or fewer. Since natural images often contain a large number of colors, faithful representation of these images with a limited size palette is a difficult problem.

Color quantization methods can be broadly classified into two categories [9]: image-independent methods that determine a universal (fixed) palette without regard to any specific image [10] and image-dependent methods that determine a custom (adaptive) palette based on the color distribution of the images. Despite being very fast, image-independent methods usually give poor results since they do not take into account the image contents. Therefore, most of the studies in the literature consider only image-dependent methods, which strive to achieve a better balance between computational efficiency and visual quality of the quantization output.

Numerous image-dependent color quantization methods have been developed in the past three decades. These can be categorized into two families: preclustering methods and postclustering methods [1]. Preclustering methods are mostly based on the statistical analysis of the color distribution of the images. Divisive preclustering methods start with a single cluster that contains all N' image colors. This initial cluster is recursively subdivided until C clusters are obtained. Well-known divisive methods include median-cut [11], octree [12], variance-based method [13], binary splitting method [14], and greedy orthogonal bipartitioning method [15]. On the other hand, agglomerative preclustering methods [16–18] start with N' singleton clusters each of which contains one image color. These clusters are repeatedly merged until C clusters remain. In contrast to preclustering methods that compute the palette only once, postclustering methods first determine an initial palette and then improve it iteratively. Essentially, any data clustering method can be used for this purpose. Since these methods involve iterative or stochastic optimization, they can obtain higher quality results when compared to preclustering methods at the expense of increased computational time. Clustering algorithms adapted to color quantization include hard c-means [19–22], competitive learning [23–27], fuzzy c-means [28–32], and self-organizing maps [33–35].

In this paper, we compare the performance of hard and fuzzy c-means algorithms within the context of color quantization. We implement several efficient variants of both algorithms, each one with a different initialization scheme, and then compare the resulting quantizers on a diverse set of images. The rest of the paper is organized as follows. Section 2 reviews the notions of hard and fuzzy partitions and gives an overview of the hard and fuzzy c-means algorithms. Section 3 describes the experimental setup and compares the hard and fuzzy c-means variants on the test images. Finally, Sect. 4 gives the conclusions.

2 Color quantization using c-means clustering algorithms

2.1 Hard versus fuzzy partitions

Given a data set X = {x₁, x₂, . . . , x _N } ∈ ℝ^D, a real matrix U = [u _ik ] _{C ×N} represents a hard C-partition of X if and only if its elements satisfy three conditions [36]:

$$\begin{array}{l}{u}_{ik}\in \left\{0,1\right\}1\le i\le C,1\le k\le N\\ \sum _{i=1}^C{u}_{ik}=11\le k\le N\\ 0<\sum _{k=1}^N{u}_{ik}<N1\le i\le C.\end{array}$$

(1)

Row i of U, say U _i = (u _{i 1}, u _{i 2}, . . . , u _iN ), exhibits the characteristic function of the i th partition (cluster) of X: u _ik is 1 if x _k is in the i th partition and 0 otherwise; ${\sum }_{i=1}^C{u}_{ik}=1\forall k$ means that each x _k is in exactly one of the C partitions; $0<{\sum }_{k=1}^N{u}_{ik}<N\forall i$ means that no partition is empty and no partition is all of X, i.e. 2 ≤ c ≤ N. For obvious reasons, U is often called a partition (membership) matrix.

The concept of hard C-partition can be generalized by relaxing the first condition in Equation 1 as u _ik ∈ 0[1] in which case the partition matrix U is said to represent a fuzzy C-partition of X [37]. In a fuzzy partition matrix U, the total membership of each x _k is still 1, but since 0 ≤ u _ik ≤ 1 ∀i, k, it is possible for each x _k to have an arbitrary distribution of membership among the C fuzzy partitions {U _i }.

2.2 Hard c-means (HCM) clustering algorithm

HCM is inarguably one of the most widely used methods for data clustering [38]. It attempts to generate optimal hard C-partitions of X by minimizing the following objective functional:

$$J\left(\mathbf{U},\mathbf{V}\right)=\sum _{k=1}^N\sum _{i=1}^C{u}_{ik}{\left(d_{ik}\right)}^2$$

(2)

where U is a hard partition matrix as defined in §2.1, V = {v₁, v₂, . . . , v _C } ∈ ℝ^D is a set of C cluster representatives (centers), e.g. v _i is the center of hard cluster U _i ∀i, and d _ik denotes the Euclidean $\left({ℒ}_2\right)$ distance between input vector x _k and cluster center v _i , i.e. d _ik = ||x _k - v _i ||₂.

Since u _ik = 1 ⇔ x _k ∈ U _i , and is zero otherwise, Equation 2 can also be written as:

$$J(U,V)=\sum _{i=1}^C\sum _{x_k\in U_i}{(d_{ik})}^2$$

This problem is known to be NP-hard even for C = 2 [39] or D = 2 [40], but a heuristic method developed by Lloyd [41] offers a simple solution. Lloyd's algorithm starts with C arbitrary centers, typically chosen uniformly at random from the data points. Each point is then assigned to the nearest center, and each center is recalculated as the mean of all points assigned to it. These two steps are repeated until a predefined termination criterion is met.

The complexity of HCM is $\mathcal{O}\left(NC\right)$ per iteration for a fixed D value. In color quantization applications, D often equals three since the clustering procedure is usually performed in a three-dimensional color space such as RGB or CIEL * a * b * [42].

From a clustering perspective, HCM has the following advantages:

◊ It is conceptually simple, versatile, and easy to implement.

◊ It has a time complexity that is linear in N and C.

◊ It is guaranteed to terminate [43] with a quadratic convergence rate [44].

Due to its gradient descent nature, HCM often converges to a local minimum of its objective functional [43] and its output is highly sensitive to the selection of the initial cluster centers. Adverse effects of improper initialization include empty clusters, slower convergence, and a higher chance of getting stuck in bad local minima. From a color quantization perspective, HCM has two additional drawbacks. First, despite its linear time complexity, the iterative nature of the algorithm renders the palette generation phase computationally expensive. Second, the pixel mapping phase is inefficient, since for each input pixel a full search of the palette is required to determine the nearest color. In contrast, preclustering methods often manipulate and store the palette in a special data structure (binary trees are commonly used), which allows for fast nearest neighbor search during the mapping phase. Note that these drawbacks are shared by the majority of postclustering methods, including the fuzzy c-means algorithm.

We have recently proposed a fast and exact HCM variant called Weighted Sort-Means (WSM) that utilizes data reduction and accelerated nearest neighbor search [21, 22]. When initialized with a suitable preclustering method, WSM has been shown to outperform a large number of classic and state-of-the-art quantization methods including median-cut [11], octree [12], variance-based method [13], binary splitting method [14], greedy orthogonal bipartitioning method [15], neu-quant [33], split and merge method [18], adaptive distributing units method [23, 26], finite-state HCM method [19], and stable-flags HCM method [20].

In this study, WSM is used in place of HCM since both algorithms give numerically identical results. However, in the remainder of this paper, WSM will be referred to as HCM for reasons of uniformity.

2.3 Fuzzy c-means (FCM) clustering algorithm

FCM is a generalization of HCM in which points can belong to more than one cluster [36]. It attempts to generate optimal fuzzy C-partitions of X by minimizing the following objective functional:

$$J_m\left(\mathbf{U},\mathbf{V}\right)=\sum _{k=1}^N\sum _{i=1}^C{\left(u_{ik}\right)}^m{\left(d_{ik}\right)}^2$$

(3)

where the parameter 1 ≤ m < ∞ controls the degree of membership sharing between fuzzy clusters in X.

As in the case of HCM, FCM is based on an alternating minimization procedure [45]. At each iteration, the fuzzy partition matrix U is updated by

$$u_{ik}={\left[{\sum _{j=1}^C\left(\frac{d_{ik}}{d_{jk}}\right)}^{2∕\left(m-1\right)}\right]}^{-1}.$$

(4)

which is followed by the update of the prototype matrix V by

$${\mathbf{v}}_i=\left(\sum _{k=1}^N{\left(u_{ik}\right)}^m{\mathbf{x}}_k\right)∕\left(\sum _{k=1}^N{\left(u_{ik}\right)}^m\right).$$

(5)

As $m\stackrel{+}{\to }1$, FCM converges to an HCM solution. Conversely, as m → ∞ it can be shown that u _ik → 1/C ∀i, k, so ${\mathbf{v}}_i\to \bar{\mathbf{X}}$, the centroid of X. In general, the larger m is, the fuzzier are the membership assignments; and conversely, as $m\stackrel{+}{\to }1$, FCM solutions become hard. In color quantization applications, in order to map each input color to the nearest (most similar) palette color, the membership values should be defuzzified upon convergence as follows:

$${\mathit{û}}_{ik}=\left\{\begin{array}{l}1u_{ik}=\underset{1\le j\le C}{max}{u}_{jk}\\ 0\mathsf{\text{otherwise}}\end{array}\right..$$

A näive implementation of FCM has a complexity of $\mathcal{O}\left(N{C}^2\right)$ per iteration, which is quadratic in the number of clusters. In this study, a linear complexity formulation, i.e. $\mathcal{O}\left(NC\right)$, described in [46] is used. In order to take advantage of the peculiarities of color image data (presence of duplicate samples, limited range, and sparsity), the same data reduction strategy used in WSM is incorporated into FCM.

3 Experimental results and discussion

3.1 Image set and performance criteria

Six publicly available, true-color images were used in the experiments. Five of these were natural images from the Kodak Lossless True Color Image Suite [47]: Hats (768 × 512; 34,871 unique colors), Motocross (768 × 512; 63,558 unique colors), Flowers and Sill (768 × 512; 37,552 unique colors), Cover Girl (768 × 512; 44,576 unique colors), and Parrots (768 × 512; 72,079 unique colors). The sixth image was synthetic, Poolballs (510 × 383; 13,604 unique colors) [48]. The images are shown in Figure 1.

Figure 1

Test images. a Hats, b Motocross, c Flowers and Sill, d Cover Girl, e Parrots, f Poolballs.

The effectiveness of a quantization method was quantified by the commonly used mean absolute error (MAE) and mean squared error (MSE) measures:

$$\begin{array}{l}\mathsf{\text{MAE}}\left(\mathbf{I},\widehat{\mathbf{I}}\right)=\frac{1}{HW}\sum _{h=1}^H\sum _{w=1}^W{∥\mathbf{I}\left(h,w\right)-\widehat{\mathbf{I}}\left(h,w\right)∥}_1\\ \mathsf{\text{MSE}}\left(\mathbf{I},\widehat{\mathbf{I}}\right)=\frac{1}{HW}\sum _{h=1}^H\sum _{w=1}^W{∥\mathbf{I}\left(h,w\right)-\widehat{\mathbf{I}}\left(h,w\right)∥}_2^2\end{array}$$

(6)

where I and $\widehat{\mathbf{I}}$ denote, respectively, the H × W original and quantized images in the RGB color space. MAE and MSE represent the average color distortion with respect to the ${ℒ}_1$ (City-block) and ${ℒ}_2^2$ (squared Euclidean) norms, respectively. Note that most of the other popular evaluation measures in the color quantization literature such as peak signal-to-noise ratio (PSNR), normalized MSE, root MSE, and average color distortion [24, 34] are variants of MAE or MSE.

The efficiency of a quantization method was measured by CPU time in milliseconds, which includes the time required for both the palette generation and the pixel mapping phases. The fast pixel mapping algorithm described in [49] was used in the experiments. All of the programs were implemented in the C language, compiled with the gcc v4.4.3 compiler, and executed on an Intel Xeon E5520 2.26 GHz machine. The time figures were averaged over 20 runs.

3.2 Comparison of HCM and FCM

The following well-known preclustering methods were used in the experiments:

• Median-cut (MC)[11]: This method starts by building a 32 × 32 × 32 color histogram that contains the original pixel values reduced to 5 bits per channel by uniform quantization (bit-cutting). This histogram volume is then recursively split into smaller boxes until C boxes are obtained. At each step, the box that contains the largest number of pixels is split along the longest axis at the median point, so that the resulting sub-boxes each contain approximately the same number of pixels. The centroids of the final C boxes are taken as the color palette.

• Octree (OCT)[12]: This two-phase method first builds an octree (a tree data structure in which each internal node has up to eight children) that represents the color distribution of the input image and then, starting from the bottom of the tree, prunes the tree by merging its nodes until C colors are obtained. In the experiments, the tree depth was limited to 6.

• Variance-based method (WAN)[13]: This method is similar to MC with the exception that at each step the box with the largest weighted variance (squared error) is split along the major (principal) axis at the point that minimizes the marginal squared error.

• Greedy orthogonal bipartitioning method (WU)[15]: This method is similar to WAN with the exception that at each step the box with the largest weighted variance is split along the axis that minimizes the sum of the variances on both sides.

Four variants of HCM/FCM, each one initialized with a different preclustering method, were tested. Each variant was executed until it converged. Convergence was determined by the following commonly used criterion [50]: (J_(i-1)- J_(i))/J_(i)≤ ε, where J_(i)denotes the value of the objective functional (Eqs. (2) and (3) for HCM and FCM, respectively) at the end of the i th iteration. The convergence threshold was set to ε = 0.001.

The weighting exponent (m) value recommended for color quantization applications ranges between 1.3 [30] and 2.0 [31]. In the experiments, four different m values were tested for each of the FCM variants: 1.25, 1.50, 1.75, and 2.00.

Tables 1 and 2 compare the effectiveness of the HCM and FCM variants on the test images. Similarly, Table 3 gives the efficiency comparison. For a given number of colors C (C ∈ {32, 64, 128, 256}), preclustering method P(P ∈ {MC, OCT, WAN, WU}), and input image I, the column labeled as 'Init' contains the MAE/MSE between I and $\widehat{\mathbf{I}}$ (the output image obtained by reducing the number of colors in I to C using P), whereas the one labeled as 'HCM' contains the MAE/MSE value obtained by HCM when initialized by P. The remaining four columns contain the MAE/MSE values obtained by the FCM variants. Note that HCM is equivalent to FCM with m = 1.00. The following observations are in order (note that each of these comparisons is made within the context of a particular C, P, and I combination):

Table 1 MAE comparison of the quantization methods

Table 2 MSE comparison of the quantization methods

Table 3 CPU time comparison of the quantization methods

⊳ The most effective initialization method is WU, whereas the least effective one is MC.

⊳ Both HCM and FCM reduces the quantization distortion regardless of the initialization method used. However, the percentage of MAE/MSE reduction is more significant for some initialization methods than others. In general, HCM/FCM is more likely to obtain a significant improvement in MAE/MSE when initialized by an ineffective preclustering algorithm such as MC or WAN. This is not surprising given that such ineffective methods generate outputs that are likely to be far from a local minimum, and hence HCM/FCM can significantly improve upon their results.

⊳ With respect to MAE, the HCM variant and the four FCM variants have virtually identical performance.

⊳ With respect to MSE, the performances of the HCM variant and the FCM variant with m = 1.25 are indistinguishable. Furthermore, the effectiveness of the FCM variants degrades with increasing m value.

⊳ On average, HCM is 92 times faster than FCM. This is because HCM uses hard memberships, which makes possible various computational optimizations that do not affect accuracy of the algorithm [51–55]. On the other hand, due to the intensive fuzzy membership calculations involved, accelerating FCM is significantly more difficult, which is why the majority of existing acceleration methods involve approximations [56–60]. Note that the fast HCM/FCM implementations used in this study give exactly the same results as the conventional HCM/FCM.

⊳ The FCM variant with m = 2.00 is the fastest since, among the m values tested in this study, only m = 2.00 leads to integer exponents in Equations 4 and 5.

Figure 2 shows sample quantization results for the Motocross image. Since WU is the most effective initialization method, only the outputs of HCM/FCM variants that use WU are shown. It can be seen that WU is unable to represent the color distribution of certain regions of the image (fenders of the leftmost and rightmost dirt bikes, helmet of the driver of the leftmost dirt bike, grass, etc.) In contrast, the HCM/FCM variants perform significantly better in allocating representative colors to these regions. Note that among the FCM variants, the one with m = 2.00 performs slightly worse in that the body color of the leftmost dirk bike and the color of the grass are mixed.

Figure 2

Sample quantization results for the Motocross image ( C = 32). a Original, b WU, c HCM-WU, d FCM-WU 1.25, e FCM-WU 1.50, f FCM-WU 1.75, g FCM-WU 2.00.

Figure 3 shows sample quantization for the Hats image. It can be seen that WU causes significant contouring in the sky region. It also adds a red tint to the pink hat. On the other hand, the HCM/FCM variants are significantly better in representing these regions. Once again, the less fuzzy FCM variants, i.e. those with smaller m values, are slightly better than the more fuzzy ones. For example, in the outputs of FCM 1.75 and 2.00, a brownish region can be discerned in the upper-right region where the white cloud and the blue sky merge.

Figure 3

Sample quantization results for the Hats image ( C = 64). a Original, b WU, c HCM-WU, d FCM-WU 1.25, e FCM-WU 1.50, f FCM-WU 1.75, g FCM-WU 2.00.

It could be argued that HCM's objective functional, Equation 2, is essentially equivalent to MSE, Equation 6, and therefore it is unreasonable to expect FCM to outperform HCM with respect to MSE unless m ≈ 1.00. However, neither HCM nor FCM minimizes MAE and yet their MAE performances are nearly identical. Hence, it can be safely concluded that FCM is not superior to HCM with respect to quantization effectiveness. Moreover, due to its simple formulation, HCM is amenable to various optimization techniques, whereas FCM's formulation permits only modest acceleration. Therefore, HCM should definitely be preferred over FCM when computationally efficiency is of prime importance.

4 Conclusions

In this paper, hard and fuzzy c-means clustering algorithms were compared within the context of color quantization. Fast and exact variants of both algorithms with several initialization schemes were compared on a diverse set of publicly available test images. The results indicate that fuzzy c-means does not seem to offer any advantage over hard c-means. Furthermore, due to the intensive membership calculations involved, fuzzy c-means is significantly slower than hard c-means, which makes it unsuitable for time-critical applications. In contrast, as was also demonstrated in a recent study [22], an efficient implementation of hard c-means with an appropriate initialization scheme can serve as a fast and effective color quantizer.