Fuzzy Clustering in the Kernel-Induced Feature Space Using Differential Evolution Algorithm

Part of the Studies in Computational Intelligence book series (SCI, volume 178)


This chapter introduces a scheme for clustering complex and linearly non-separable datasets, without any prior knowledge of the number of naturally occurring groups in the data. The proposed method is based on a modified version of the classical Differential Evolution (DE) algorithm, which uses the neighborhood-based mutation strategy. It also employs a kernel-induced similarity measure instead of the conventional sum-of-squares distance. Use of the kernel function makes it possible to cluster data that is linearly non-separable in the original input space into homogeneous groups in a transformed high-dimensional feature space. The performance of the proposed method has been extensively compared with a few state of the art clustering techniques over a test-suite of several artificial and real life datasets. Based on the computer simulations, some empirical guidelines have been provided for selecting the suitable parameters of the DE algorithm.


Particle Swarm Optimization Differential Evolution Fuzzy Cluster Synthetic Dataset Subspace Cluster 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Muller, K.R., Mika, S., Ratsch, G., Tsuda, K., Scholkopf, B.: An introduction to kernel-based learning algorithms. IEEE Transactions on Neural Networks 12(2), 181–202 (2001)CrossRefGoogle Scholar
  2. 2.
    Girolami, M.: Mercer kernel-based clustering in feature space. IEEE Transactions on Neural Networks 13(3), 780–784 (2002)CrossRefGoogle Scholar
  3. 3.
    Scholkopf, B., Smola, A.J.: Learning with Kernels. MIT Press, Cambridge (2002)Google Scholar
  4. 4.
    Vapnik, V.N.: Statistical Learning Theory. Wiley, New York (1998)zbMATHGoogle Scholar
  5. 5.
    Zhang, D.Q., Chen, S.C.: Clustering incomplete data using kernel-based fuzzy c-means algorithm. Neural Process Letters 18, 155–162 (2003)CrossRefGoogle Scholar
  6. 6.
    Zhang, R., Rudnick, A.I.: A large scale clustering scheme for kernel k-means. In: The Sixteenth International Conference on Pattern Recognition, pp. 289–292 (2002)Google Scholar
  7. 7.
    Das, S., Konar, A., Abraham, A., Chakraborty, U.K.: Differential Evolution Using a Neighborhood based Mutation Operator. In: IEEE Transactions on Evolutionary Computation. IEEE Press, USA (2008)Google Scholar
  8. 8.
    Parsopoulos, K.E., Vrahatis, M.N.: UPSO: a unified particle swarm optimization scheme. In: Proceedings of the Int. Conf. Comput. Meth. Sci. Eng (ICCMSE 2004). Lecture Series on Computer and Computational Sciences, vol. 1, pp. 868–873. VSP International Science Publishers, the Netherlands (2004)Google Scholar
  9. 9.
    Das, S., Sil, S., Chakraborty, U.K.: Kernel induced pixel clustering with differential evolution. In: IEEE Congress on Evolutionary Computation CEC 2008, IEEE World Congress on Computational Intelligence, WCCI 2008. IEEE Press, USA (2008)Google Scholar
  10. 10.
    Xie, X., Beni, G.: Validity measure for fuzzy clustering. IEEE Transactions on Pattern Analysis and Machine Learning 3, 841–846 (1991)CrossRefGoogle Scholar
  11. 11.
    Mezura-Montes, E., Velázquez-Reyes, J., Coello, C.A.C.: A comparative study of differential evolution variants for global optimization. In: Genetic and Evolutionary Computation Conference (GECCO 2006), pp. 485–492 (2006)Google Scholar
  12. 12.
    Storn, R., Price, K.V., Lampinen, J.: Differential Evolution - A Practical Approach to Global Optimization. Springer, Berlin (2005)zbMATHGoogle Scholar
  13. 13.
    Zielinski, K., Peters, D., Laur, R.: Run time analysis regarding stopping criteria for differential evolution and particle swarm optimization. In: Proc. of the 1st International Conference on Experiments/Process/System Modelling/Simulation/Optimization, Athens, Greece (2005)Google Scholar
  14. 14.
    Aho, A.V., Hopcroft, J.E., Ullman, J.D.: Data Structures and Algorithms. Addison-Wesley, Reading (1983)zbMATHGoogle Scholar
  15. 15.
    Cormen, T.H., Leiserson, C.E., Rivest, R.L.: Introduction to Algorithms, 1st edn. MIT Press and McGraw-Hill (1990)Google Scholar
  16. 16.
    Collet, P., Louchet, J., Lutton, E.: Issues on the optimization of evolutionary algorithms code. In: Proc. of the 2002 Congress on Evolutionary Computation (CEC 2002), Honolulu, HI, USA (2002)Google Scholar
  17. 17.
    Aronszajn, N.: Theory of reproducing kernels. Transactions of the American Mathematical Society 68(3), 337–404 (1950)zbMATHCrossRefMathSciNetGoogle Scholar
  18. 18.
    Saitoh, S.: Theory of Reproducing Kernels and its Applications. Longman Scientific & Technical, Harlow, England (1988)Google Scholar
  19. 19.
    Vapnik, V.N.: The Nature of Statistical Learning Theory. Springer, New York (1995)zbMATHGoogle Scholar
  20. 20.
    Pirooznia, M., Deng, Y.: SVM Classifier – a comprehensive java interface for support vector machine classification of microarray data. In: Proc. of Symposium of Computations in Bioinformatics and Bioscience (SCBB 2006), Hangzhou, China (2006)Google Scholar
  21. 21.
    Hertz, T., Bar, A., Daphna Weinshall, H.: Learning a Kernel Function for Classification with Small Training Samples. In: Proceedings of the 23rd International Conference on Machine Learning, Pittsburgh, PA (2006)Google Scholar
  22. 22.
    Wu, Z.D., Xie, W.X., Yu, J.P.: Fuzzy c-means clustering algorithm based on kernel method. In: ICCIMA (2003)Google Scholar
  23. 23.
    Zhang, D.Q., Chen, S.C.: Kernel based fuzzy and possibilistic c-means clustering. In: Proceedings of the International Conference Artificial Neural Network, pp. 122–125, Turkey (2003)Google Scholar
  24. 24.
    Zhang, D.Q., Chen, S.C.: A novel kernelized fuzzy c-means algorithm with application in medical image segmentation. Artificial Intelligence in Medicine 32(1), 37–50 (2004)CrossRefGoogle Scholar
  25. 25.
    Graepel, T., Obermayer, K.: Fuzzy topographic kernel clustering. In: Proceedings of the 5th GI Workshop Fuzzy Neuro Systems 1998, pp. 90–97 (1998)Google Scholar
  26. 26.
    Macdonald, D., Fyfe, C.: The kernel self-organising map. In: Fourth International Conference on Knowledge-Based Intelligent Engineering Systems and Allied Technologies, 2000, vol. 1, pp. 317–320 (2000)Google Scholar
  27. 27.
    Qinand, A.K., Suganthan, P.N.: Kernel neural gas algorithms with application to cluster analysis. In: ICPR, pp. 617–620 (2004)Google Scholar
  28. 28.
    Zhang, D.Q., Chen, S.C.: Fuzzy clustering using kernel method. In: The 2002 International Conference on Control and Automation, pp. 162–163 (2002)Google Scholar
  29. 29.
    Camastra, F., Verri, A.: A novel kernel method for clustering. IEEE Transactions on Pattern Analysis and Machine Intelligence 27(5), 801–804 (2005)CrossRefGoogle Scholar
  30. 30.
    Hur, A.B., Horn, D., Siegelmann, H.T., Vapnik, V.N.: Support vector clustering. Journal of Machine Learning Research 2, 125–137 (2001)Google Scholar
  31. 31.
    Filippone, M., Camastra, F., Masulli, F., Rovetta, S.: A survey of kernel and spectral methods for clustering. Pattern Recognition 41(1), 176–190 (2008)zbMATHCrossRefGoogle Scholar
  32. 32.
    Pal, N.R., Bezdek, J.C.: On cluster validity for the fuzzy c means model. IEEE Transactions on Fuzzy Systems 3(3), 370–379 (1995)CrossRefGoogle Scholar
  33. 33.
    Maulik, U., Bandyopadhyay, S.: Fuzzy Partitioning Using Real Coded Variable Length Genetic Algorithm for Pixel Classification. IEEE Transactions on Geosciences and Remote Sensing 41(5), 1075–1081 (2003)CrossRefGoogle Scholar
  34. 34.
    Omran, M.G., Engelbrecht, A.P., Salman, A.: Dynamic Clustering using Particle Swarm Optimization with Application in Unsupervised Image Classification. In: Proceedings of World Academy of Science, Engineering and Technology, vol. 9 (November 2005)Google Scholar
  35. 35.
    Das, S., Abraham, A., Konar, A.: Automatic Kernel Clustering with Multi-Elitist Particle Swarm Optimization Algorithm. Pattern Recognition Letters, Elsevier Science 29, 688–699 (2008)CrossRefGoogle Scholar
  36. 36.
    Dorigo, M., Maniezzo, V., Colorni, A.: Ant System: Optimization by a colony of cooperating agents. IEEE Transactions on Systems, Man, and Cybernetics-Part B 26(1), 29–41 (1996)CrossRefGoogle Scholar
  37. 37.
    Gu, Y., Hall, L.O.: Kernel Based Fuzzy Ant Clustering with Partition validity. In: IEEE International Conference on Fuzzy Systems. IEEE Press, Los Alamitos (2006)Google Scholar
  38. 38.
    Blake, C., Keough, E., Merz, C.J.: UCI repository of machine learning database (1998),
  39. 39.
    Huang, Z., Ng, M.K.: A fuzzy k-modes algorithm for clustering categorical data. IEEE Transactions on Fuzzy Systems 7(4), 446–452 (1999)CrossRefGoogle Scholar
  40. 40.
    Handl, J., Knowles, J.: Multiobjective clustering around medoids. In: Proceedings of the Congress on Evolutionary Computation (CEC 2005), vol. 1, pp. 632–639 (2005)Google Scholar
  41. 41.
    Zhang, T., Ramakrishnan, R., Livny, M.: BIRCH: An efficient data clustering method for very large databases. In: Proc. of the ACM SIGMOD Conference on Management of Data, Montreal, Canada (1996)Google Scholar
  42. 42.
    Handl, J., Knowles, J.: An evolutionary approach to multiobjective clustering. IEEE Transactions on Evolutionary Computation 11(1), 56–76 (2007)CrossRefGoogle Scholar
  43. 43.
    Agrawal, R., Gehrke, J., Gunopulos, D., Raghavan, P.: Automatic subspace clustering of high dimensional data for data mining applications. In: Proc. ACM-SIGMOD Int. Conf. Management of Data, Seattle, Washington (1998)Google Scholar
  44. 44.
    Evangelista, P.F., Embrecht, M.J., Szymanski, B.K.: Taming the Curse of Dimensionality in Kernels and Novelty Detection. In: Advances in Soft Computing: Applied Soft Computing Technologies: The Challenge of Complexity, pp. 425–438. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  45. 45.
    Hall, L.O., Özyurt, I.B., Bezdek, J.C.: Clustering with a genetically optimized approach. IEEE Transactions on Evolutionary Computing 3(2), 103–112 (1999)CrossRefGoogle Scholar
  46. 46.
    Gath, I., Geva, A.: Unsupervised optimal fuzzy clustering. IEEE Transactions on PAMI 11, 773–781 (1989)Google Scholar
  47. 47.
    Bensaid, A.M., Hall, L.O., Bezdek, J.C., Clarke, L.P.: Partially supervised clustering for image segmentation. Pattern Recognition 29, 859–871 (1996)CrossRefGoogle Scholar
  48. 48.
    Clark, M.C., Hall, L.O., Goldgof, D.B., Clarke, L.P., Velthuizen, R.P., Silbiger, M.S.: MRI segmentation using fuzzy clustering techniques. IEEE Eng. Med. Biol. 13, 730–742 (1994)CrossRefGoogle Scholar
  49. 49.
    Ahmed, M.N., Yamany, S.M., Mohamed, N., Farag, A.A., Moriarty, T.: A modified fuzzy c-means algorithm for bias field estimation and segmentation of MRI data. IEEE Transactions on Medical Imaging 21, 193–199 (2002)CrossRefGoogle Scholar
  50. 50.
    Wang, X., Wang, Y., Wang, L.: Improving fuzzy c-means clustering based on feature-weight learning. Pattern Recognition Letters 25, 1123–1132 (2004)CrossRefGoogle Scholar
  51. 51.
    Chen, S., Zhang, D.: Robust image segmentation using FCM with spatial constraints based on new kernel-induced distance metric. IEEE Transactions on System, Man and Cybernetics-Part B 34(4), 1907–1916 (2004)CrossRefGoogle Scholar
  52. 52.
    Gigandet, X., Cuadra, M.B., Pointet, A., Cammoun, L., Caloz, R., Thiran, J.: Region-based satellite image classification: method and validation ICIP (3), 832–835 (2005)Google Scholar
  53. 53.
  54. 54.
    Martin, D., Fowlkes, C., Tal, D., Malik, J.: A Database of Human Segmented Natural Images and its Application to Evaluating Segmentation Algorithms and Measuring Ecological Statistics. In: Proc. 8th International Conference on Computer Vision (2) 416—423 (2001)Google Scholar
  55. 55.
  56. 56.
    Congalton, R.G., Green, K.: Assessing the Accuracy of Remotely Sensed Data: Principles and Practices, Lewis, ch. 5, pp. 45–50 (1999)Google Scholar
  57. 57.
    Hubert, L., Arabie, P.: Comparing partitions. Journal of Classification, 193–218 (1985)Google Scholar
  58. 58.
    Rand, W.M.: Objective criteria for the evaluation of clustering methods. Journal of the American Statistical Association 66, 846–850 (1971)CrossRefGoogle Scholar
  59. 59.
    Dice, L.R.: Measures of the amount of ecologic association between species. Ecology. 26, 297–302 (1945)CrossRefGoogle Scholar
  60. 60.
    Unnikrishnan, R., Pantofaru, C., Hebert, M.: Toward Objective Evaluation of Image Segmentation Algorithms. IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI) 29(6), 929–944 (2007)CrossRefGoogle Scholar
  61. 61.
    Pal, N.R., Pal, S.K.: A Review on Image Segmentation Techniques. Pattern Recognition 26, 1277–1294 (1993)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

There are no affiliations available

Personalised recommendations