Advertisement

Dynamic clustering using particle swarm optimization with application in image segmentation

  • Mahamed G. H. Omran
  • Ayed Salman
  • Andries P. Engelbrecht
Theoretical Advances

Abstract

A new dynamic clustering approach (DCPSO), based on particle swarm optimization, is proposed. This approach is applied to image segmentation. The proposed approach automatically determines the “optimum” number of clusters and simultaneously clusters the data set with minimal user interference. The algorithm starts by partitioning the data set into a relatively large number of clusters to reduce the effects of initial conditions. Using binary particle swarm optimization the “best” number of clusters is selected. The centers of the chosen clusters is then refined via the K-means clustering algorithm. The proposed approach was applied on both synthetic and natural images. The experiments conducted show that the proposed approach generally found the “optimum” number of clusters on the tested images. A genetic algorithm and random search version of dynamic clustering is presented and compared to the particle swarm version.

Keywords

Unsupervised clustering Clustering validation Particle swarm optimization Image segmentation 

References

  1. 1.
    Jain AK, Murty MN, Flynn PJ (1999) Data clustering: a review. ACM Comput Surv 31(3):264–323CrossRefGoogle Scholar
  2. 2.
    Jain AK, Duin R, Mao J (2000) Statistical pattern recognition: a review. IEEE Trans Pattern Anal Mach Intell 22(1):4–37CrossRefGoogle Scholar
  3. 3.
    Judd D, Mckinley P, Jain AK (1998) Large-scale parallel data clustering. IEEE Trans Pattern Anal Mach Intell 20(8):871–876CrossRefGoogle Scholar
  4. 4.
    Abbas HM, Fahmy MM (1994) Neural networks for maximum likelihood clustering. Signal Process 36(1):111–126MATHCrossRefGoogle Scholar
  5. 5.
    Coleman GB, Andrews HC (1979) Image segmentation by clustering. Proc IEEE 67:773–785CrossRefGoogle Scholar
  6. 6.
    Jain AK, Dubes RC (1988) Algorithms for clustering data. Prentice Hall, New JerseyMATHGoogle Scholar
  7. 7.
    Ray S, Turi RH (1999) Determination of number of clusters in K-means clustering and application in colour image segmentation. In: Proceedings of the 4th international conference on advances in pattern recognition and digital techniques (ICAPRDT‘99), Calcutta, India, pp 137–143Google Scholar
  8. 8.
    Carpineto C, Romano G (1996) A lattice conceptual clustering system and its application to browsing retrieval. Mach Learn 24(2):95–122Google Scholar
  9. 9.
    Lee CY, Antonsson EK (2000) Dynamic partitional clustering using evolution strategies. In: The third Asia-Pacific conference on simulated evolution and learningGoogle Scholar
  10. 10.
    Hamerly G, Elkan C (2003) Learning the K in K-means. In: 7th annual conference on neural information processing systemsGoogle Scholar
  11. 11.
    Frigui H, Krishnapuram R (1999) A robust competitive clustering algorithm with applications in computer vision. IEEE Trans Pattern Anal Mach Intell 21(5):450–465CrossRefGoogle Scholar
  12. 12.
    Leung Y, Zhang J, Xu Z (2000) Clustering by space-space filtering. IEEE Trans Pattern Anal Mach Intell 22(12):1396–1410CrossRefGoogle Scholar
  13. 13.
    Halkidi M, Batistakis Y, Vazirgiannis M (2001) On clustering validation techniques. Intell Inform Syst J 17(2–3):107–145MATHCrossRefGoogle Scholar
  14. 14.
    Theodoridis S, Koutroubas K (1999) Pattern recognition. Academic, New YorkGoogle Scholar
  15. 15.
    Rosenberger C, Chehdi K (2000) Unsupervised clustering method with optimal estimation of the number of clusters: application to image segmentation. In: International conference on pattern recognition (ICPR’00) 1:1656–1659Google Scholar
  16. 16.
    Kuncheva L, Bezdek J (1998) Nearest prototype classification: clustering, genetic algorithms, or random search? IEEE Trans Syst Man Cybernet C: Appl Rev 28(1):160–164CrossRefGoogle Scholar
  17. 17.
    Forgy E (1965) Cluster analysis of multivariate data: efficiency versus interpretability of classification. Biometrics 21:768–769Google Scholar
  18. 18.
    Davies E (1997) Machine vision: theory, algorithms, practicalities, 2nd edn. Academic, New YorkGoogle Scholar
  19. 19.
    Bezdek J (1980) A convergence theorem for the fuzzy ISODATA clustering algorithms. IEEE Trans Pattern Anal Mach Intell 2:1–8MATHGoogle Scholar
  20. 20.
    Bezdek JC (1981) Pattern recognition with fuzzy objective function algorithms. Plenum, New YorkMATHGoogle Scholar
  21. 21.
    Bishop C (1995) Neural networks for pattern recognition. Clarendon, OxfordGoogle Scholar
  22. 22.
    McLachlan G, Krishnan T (1997) The EM algorithm and extensions. Wiley, New YorkMATHGoogle Scholar
  23. 23.
    Rendner R, Walker H (1984) Mixture densities, maximum likelihood and the EM algorithm. SIAM Rev 26(2)Google Scholar
  24. 24.
    Hamerly G (2003) Learning structure and concepts in data using data clustering, PhD thesis, University of California, San DiegoGoogle Scholar
  25. 25.
    Alldrin N, Smith A, Turnbull D (2003) Clustering with EM and K-means (November 15 2003), Unpublished Manuscript; http://louis.ucsd.edu/∼nalldrin/research/cse253_wi03.pdf.
  26. 26.
    Zhang Y, Brady M, Smith S (2001) Segmentation of brain MR images through a hidden Markov random field model and the expectation-maximization algorithm. IEEE Trans Med Imag 20(1):45–57CrossRefGoogle Scholar
  27. 27.
    Zhang B, Hsu M, Dayal U (1999) K-harmonic means–a data clustering algorithm. Technical report HPL-1999–124), Hewlett-Packard LabsGoogle Scholar
  28. 28.
    Zhang B (2000) Generalized K-harmonic means–boosting in unsupervised learning. Technical report HPL-2000–137), Hewlett-Packard LabsGoogle Scholar
  29. 29.
    Omran M, Engelbrecht A, Salman A (2005) Particle swarm optimization method for image clustering. Int J Pattern Recogn Artif Intell 19(3):297–322CrossRefGoogle Scholar
  30. 30.
    Frigui H, Krishnapuram R (1999) A robust competitive clustering algorithm with applications in computer vision. IEEE Trans Pattern Anal Mach Intell 21(5):450–465CrossRefGoogle Scholar
  31. 31.
    Ball G, Hall D (1967) A clustering technique for summarizing multivariate data. Behav Sci 12:153–155CrossRefGoogle Scholar
  32. 32.
    Huang K (2002) A synergistic automatic clustering technique (Syneract) for multispectral image analysis. Photogrammetric Eng Remote Sens 1(1):33–40Google Scholar
  33. 33.
    Pelleg D, Moore A (2000) X-means: extending K-means with efficient estimation of the number of clusters. In: Proceedings of the 17th international conference on machine learning, Morgan Kaufmann, San Francisco, CA, pp 727–734Google Scholar
  34. 34.
    Kass R, Wasserman L (1995) A reference Bayesian test for nested hypotheses and its relationship to the Schwarz criterion. J Am Stat Assoc 90(431):928–934MATHCrossRefMathSciNetGoogle Scholar
  35. 35.
    Hamerly G (2003) Learning structure and concepts in data using data clustering. PhD thesis, University of California, San DiegoGoogle Scholar
  36. 36.
    Wallace CS, Dowe DL (1994) Intrinsic classification by MML—the snob program. In: Proceedings 7th Australian joint conference on artificial intelligence, UNE, Armidale, NSW, Australia, pp 37–44Google Scholar
  37. 37.
    Wallace CS (1984) An improved program for classification. Technical report No. 47, Department of Computer Science, Monash University, AustraliaGoogle Scholar
  38. 38.
    Turi RH (2001) Clustering-based colour image segmentation. PhD Thesis, Monash University, AustraliaGoogle Scholar
  39. 39.
    Wallace CS, Boulton DM (1968) An information measure for classification. Comput J 11:185–194MATHGoogle Scholar
  40. 40.
    Oliver JJ, Hand D (1994) Introduction to minimum encoding inference. Technical report No. 94/205, Department of Computer Science, Monash University, AustraliaGoogle Scholar
  41. 41.
    Bischof H, Leonardis A, Selb A (1999) MDL principle for robust vector quantization. Pattern Anal Appl 2:59–72MATHCrossRefGoogle Scholar
  42. 42.
    Gath I, Geva A (1989) Unsupervised optimal fuzzy clustering. IEEE Trans Pattern Anal Mach Intell 11(7):773–781CrossRefGoogle Scholar
  43. 43.
    Lorette A, Descombes X, Zerubia J (2000) Fully unsupervised fuzzy clustering with entropy criterion. In: International conference on pattern recognition (ICPR’00) 3:3998-4001Google Scholar
  44. 44.
    Boujemaa N (2000) On competitive unsupervised clustering. In: International conference on pattern recognition (ICPR’00) 1:1631–1634Google Scholar
  45. 45.
    Frigui H, Krishnapuram R (1997) Clustering by competitive agglomeration. Pattern Recogn Lett 30(7):1109–1119Google Scholar
  46. 46.
    Kohonen T (1995) Self-organizing maps. Springer, Berlin Heidelberg New YorkGoogle Scholar
  47. 47.
    Mehrotra K, Mohan C, Rakka (1997) Elements of artificial neural networks. MIT, CambridgeGoogle Scholar
  48. 48.
    Pandya A, Macy R (1996) Pattern recognition with neural networks in C++. CRC, Boca RatonGoogle Scholar
  49. 49.
    Halkidi M, Vazirgiannis M (2001) Clustering validity assessment: finding the optimal partitioning of a data set. In: Proceedings of ICDM conference, CA, USAGoogle Scholar
  50. 50.
    Dunn JC (1974) Well separated clusters and optimal fuzzy partitions. J Cybern 4:95–104CrossRefMathSciNetGoogle Scholar
  51. 51.
    Davies, Bouldin (1979) A cluster separation measure. IEEE Trans Pattern Anal Mach Intell 1(2)Google Scholar
  52. 52.
    Halkidi M, Vazirgiannis M (2002) Clustering validity assessment using multi representative. In: Proceedings of SETN conference, Thessaloniki, GreeceGoogle Scholar
  53. 53.
    Kennedy J, Eberhart R (1995) Particle swarm optimization. In: Proceedings of IEEE international conference on neural networks, Perth, Australia 4:1942–1948Google Scholar
  54. 54.
    Kennedy J, Eberhart R (2001) Swarm intelligence. Morgan Kaufmann, San FranciscoGoogle Scholar
  55. 55.
    Engelbrecht A (2002) Computational intelligence: an introduction. Wiley, New YorkGoogle Scholar
  56. 56.
    Shi Y, Eberhart R (1998) Parameter selection in particle swarm optimization. Evolutionary Programming VII: Proceedings of EP 98:591–600Google Scholar
  57. 57.
    Suganthan P (1999) Particle Swarm Optimizer with Neighborhood Optimizer. In: Proceedings of the congress on evolutionary computation, pp 1958–1962Google Scholar
  58. 58.
    Shi Y, Eberhart R (1998) A modified particle swarm optimizer. In: Proceedings of the IEEE international conference on evolutionary computation, Piscataway, NJ, pp 69–73Google Scholar
  59. 59.
    Kennedy J, Small worlds and mega-minds: effects of neighborhood topology on particle swarm performance. In: Proceedings of the congress on evolutionary computation, pp 1931–1938Google Scholar
  60. 60.
    Kennedy J, Mendes R (2002) Population structure and particle performance. In: Proceedings of the IEEE congress on evolutionary computation, Honolulu, HawaiiGoogle Scholar
  61. 61.
    Van den Bergh F (2002) An analysis of particle swarm optimizers. PhD thesis, Department of Computer Science, University of PretoriaGoogle Scholar
  62. 62.
    van den Bergh F, Engelbrecht AP (2002) A new locally convergent particle swarm optimizer. In: Proceedings of the IEEE conference on systems, man, and cybernetics, Hammamet, TunisiaGoogle Scholar
  63. 63.
    Kennedy J, Eberhart R (1997) A discrete binary version of the particle swarm algorithm. In: Proceedings of the conference on systems, man, and cybernetics, pp 4104–4109Google Scholar
  64. 64.
    Pal NR, Pal SK (1993) A review on image segmentation techniques. Pattern Recogn 26:1277–1294CrossRefGoogle Scholar
  65. 65.
    Fu KS, Mui JK (1981) A survey on image segmentation. Pattern Recogn 13:3–16CrossRefMathSciNetGoogle Scholar
  66. 66.
    Goldberg D (1989) Genetic algorithms in search, optimization and machine learning. Addison-Wesley, ReadingMATHGoogle Scholar
  67. 67.
    Salman A, Omran M, Engelbrecht A (2005) SIGT: synthetic image generation tool for clustering algorithms. ICGST Int J Graph Vision Image Process (GVIP) 2:33–44Google Scholar

Copyright information

© Springer-Verlag London Limited 2005

Authors and Affiliations

  • Mahamed G. H. Omran
    • 1
  • Ayed Salman
    • 2
  • Andries P. Engelbrecht
    • 1
  1. 1.Department of Computer Science, School of Information TechnologyUniversity of PretoriaPretoriaSouth Africa
  2. 2.Department of Computer EngineeringKuwait UniversityKuwaitKuwait

Personalised recommendations