Skip to main content

Advertisement

SpringerLink
Log in

Dynamic clustering using particle swarm optimization with application in image segmentation

  • Theoretical Advances
  • Published:
Pattern Analysis and Applications Aims and scope Submit manuscript

Abstract

A new dynamic clustering approach (DCPSO), based on particle swarm optimization, is proposed. This approach is applied to image segmentation. The proposed approach automatically determines the “optimum” number of clusters and simultaneously clusters the data set with minimal user interference. The algorithm starts by partitioning the data set into a relatively large number of clusters to reduce the effects of initial conditions. Using binary particle swarm optimization the “best” number of clusters is selected. The centers of the chosen clusters is then refined via the K-means clustering algorithm. The proposed approach was applied on both synthetic and natural images. The experiments conducted show that the proposed approach generally found the “optimum” number of clusters on the tested images. A genetic algorithm and random search version of dynamic clustering is presented and compared to the particle swarm version.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Jain AK, Murty MN, Flynn PJ (1999) Data clustering: a review. ACM Comput Surv 31(3):264–323

    Article  Google Scholar 

  2. Jain AK, Duin R, Mao J (2000) Statistical pattern recognition: a review. IEEE Trans Pattern Anal Mach Intell 22(1):4–37

    Article  Google Scholar 

  3. Judd D, Mckinley P, Jain AK (1998) Large-scale parallel data clustering. IEEE Trans Pattern Anal Mach Intell 20(8):871–876

    Article  Google Scholar 

  4. Abbas HM, Fahmy MM (1994) Neural networks for maximum likelihood clustering. Signal Process 36(1):111–126

    Article  MATH  Google Scholar 

  5. Coleman GB, Andrews HC (1979) Image segmentation by clustering. Proc IEEE 67:773–785

    Article  Google Scholar 

  6. Jain AK, Dubes RC (1988) Algorithms for clustering data. Prentice Hall, New Jersey

    MATH  Google Scholar 

  7. Ray S, Turi RH (1999) Determination of number of clusters in K-means clustering and application in colour image segmentation. In: Proceedings of the 4th international conference on advances in pattern recognition and digital techniques (ICAPRDT‘99), Calcutta, India, pp 137–143

  8. Carpineto C, Romano G (1996) A lattice conceptual clustering system and its application to browsing retrieval. Mach Learn 24(2):95–122

    Google Scholar 

  9. Lee CY, Antonsson EK (2000) Dynamic partitional clustering using evolution strategies. In: The third Asia-Pacific conference on simulated evolution and learning

  10. Hamerly G, Elkan C (2003) Learning the K in K-means. In: 7th annual conference on neural information processing systems

  11. Frigui H, Krishnapuram R (1999) A robust competitive clustering algorithm with applications in computer vision. IEEE Trans Pattern Anal Mach Intell 21(5):450–465

    Article  Google Scholar 

  12. Leung Y, Zhang J, Xu Z (2000) Clustering by space-space filtering. IEEE Trans Pattern Anal Mach Intell 22(12):1396–1410

    Article  Google Scholar 

  13. Halkidi M, Batistakis Y, Vazirgiannis M (2001) On clustering validation techniques. Intell Inform Syst J 17(2–3):107–145

    Article  MATH  Google Scholar 

  14. Theodoridis S, Koutroubas K (1999) Pattern recognition. Academic, New York

    Google Scholar 

  15. Rosenberger C, Chehdi K (2000) Unsupervised clustering method with optimal estimation of the number of clusters: application to image segmentation. In: International conference on pattern recognition (ICPR’00) 1:1656–1659

  16. Kuncheva L, Bezdek J (1998) Nearest prototype classification: clustering, genetic algorithms, or random search? IEEE Trans Syst Man Cybernet C: Appl Rev 28(1):160–164

    Article  Google Scholar 

  17. Forgy E (1965) Cluster analysis of multivariate data: efficiency versus interpretability of classification. Biometrics 21:768–769

    Google Scholar 

  18. Davies E (1997) Machine vision: theory, algorithms, practicalities, 2nd edn. Academic, New York

    Google Scholar 

  19. Bezdek J (1980) A convergence theorem for the fuzzy ISODATA clustering algorithms. IEEE Trans Pattern Anal Mach Intell 2:1–8

    MATH  Google Scholar 

  20. Bezdek JC (1981) Pattern recognition with fuzzy objective function algorithms. Plenum, New York

    MATH  Google Scholar 

  21. Bishop C (1995) Neural networks for pattern recognition. Clarendon, Oxford

    Google Scholar 

  22. McLachlan G, Krishnan T (1997) The EM algorithm and extensions. Wiley, New York

    MATH  Google Scholar 

  23. Rendner R, Walker H (1984) Mixture densities, maximum likelihood and the EM algorithm. SIAM Rev 26(2)

  24. Hamerly G (2003) Learning structure and concepts in data using data clustering, PhD thesis, University of California, San Diego

  25. Alldrin N, Smith A, Turnbull D (2003) Clustering with EM and K-means (November 15 2003), Unpublished Manuscript; http://louis.ucsd.edu/∼nalldrin/research/cse253_wi03.pdf.

  26. Zhang Y, Brady M, Smith S (2001) Segmentation of brain MR images through a hidden Markov random field model and the expectation-maximization algorithm. IEEE Trans Med Imag 20(1):45–57

    Article  Google Scholar 

  27. Zhang B, Hsu M, Dayal U (1999) K-harmonic means–a data clustering algorithm. Technical report HPL-1999–124), Hewlett-Packard Labs

  28. Zhang B (2000) Generalized K-harmonic means–boosting in unsupervised learning. Technical report HPL-2000–137), Hewlett-Packard Labs

  29. Omran M, Engelbrecht A, Salman A (2005) Particle swarm optimization method for image clustering. Int J Pattern Recogn Artif Intell 19(3):297–322

    Article  Google Scholar 

  30. Frigui H, Krishnapuram R (1999) A robust competitive clustering algorithm with applications in computer vision. IEEE Trans Pattern Anal Mach Intell 21(5):450–465

    Article  Google Scholar 

  31. Ball G, Hall D (1967) A clustering technique for summarizing multivariate data. Behav Sci 12:153–155

    Article  Google Scholar 

  32. Huang K (2002) A synergistic automatic clustering technique (Syneract) for multispectral image analysis. Photogrammetric Eng Remote Sens 1(1):33–40

    Google Scholar 

  33. Pelleg D, Moore A (2000) X-means: extending K-means with efficient estimation of the number of clusters. In: Proceedings of the 17th international conference on machine learning, Morgan Kaufmann, San Francisco, CA, pp 727–734

  34. Kass R, Wasserman L (1995) A reference Bayesian test for nested hypotheses and its relationship to the Schwarz criterion. J Am Stat Assoc 90(431):928–934

    Article  MATH  MathSciNet  Google Scholar 

  35. Hamerly G (2003) Learning structure and concepts in data using data clustering. PhD thesis, University of California, San Diego

  36. Wallace CS, Dowe DL (1994) Intrinsic classification by MML—the snob program. In: Proceedings 7th Australian joint conference on artificial intelligence, UNE, Armidale, NSW, Australia, pp 37–44

  37. Wallace CS (1984) An improved program for classification. Technical report No. 47, Department of Computer Science, Monash University, Australia

  38. Turi RH (2001) Clustering-based colour image segmentation. PhD Thesis, Monash University, Australia

  39. Wallace CS, Boulton DM (1968) An information measure for classification. Comput J 11:185–194

    MATH  Google Scholar 

  40. Oliver JJ, Hand D (1994) Introduction to minimum encoding inference. Technical report No. 94/205, Department of Computer Science, Monash University, Australia

  41. Bischof H, Leonardis A, Selb A (1999) MDL principle for robust vector quantization. Pattern Anal Appl 2:59–72

    Article  MATH  Google Scholar 

  42. Gath I, Geva A (1989) Unsupervised optimal fuzzy clustering. IEEE Trans Pattern Anal Mach Intell 11(7):773–781

    Article  Google Scholar 

  43. Lorette A, Descombes X, Zerubia J (2000) Fully unsupervised fuzzy clustering with entropy criterion. In: International conference on pattern recognition (ICPR’00) 3:3998-4001

  44. Boujemaa N (2000) On competitive unsupervised clustering. In: International conference on pattern recognition (ICPR’00) 1:1631–1634

  45. Frigui H, Krishnapuram R (1997) Clustering by competitive agglomeration. Pattern Recogn Lett 30(7):1109–1119

    Google Scholar 

  46. Kohonen T (1995) Self-organizing maps. Springer, Berlin Heidelberg New York

    Google Scholar 

  47. Mehrotra K, Mohan C, Rakka (1997) Elements of artificial neural networks. MIT, Cambridge

    Google Scholar 

  48. Pandya A, Macy R (1996) Pattern recognition with neural networks in C++. CRC, Boca Raton

    Google Scholar 

  49. Halkidi M, Vazirgiannis M (2001) Clustering validity assessment: finding the optimal partitioning of a data set. In: Proceedings of ICDM conference, CA, USA

  50. Dunn JC (1974) Well separated clusters and optimal fuzzy partitions. J Cybern 4:95–104

    Article  MathSciNet  Google Scholar 

  51. Davies, Bouldin (1979) A cluster separation measure. IEEE Trans Pattern Anal Mach Intell 1(2)

  52. Halkidi M, Vazirgiannis M (2002) Clustering validity assessment using multi representative. In: Proceedings of SETN conference, Thessaloniki, Greece

  53. Kennedy J, Eberhart R (1995) Particle swarm optimization. In: Proceedings of IEEE international conference on neural networks, Perth, Australia 4:1942–1948

  54. Kennedy J, Eberhart R (2001) Swarm intelligence. Morgan Kaufmann, San Francisco

    Google Scholar 

  55. Engelbrecht A (2002) Computational intelligence: an introduction. Wiley, New York

    Google Scholar 

  56. Shi Y, Eberhart R (1998) Parameter selection in particle swarm optimization. Evolutionary Programming VII: Proceedings of EP 98:591–600

    Google Scholar 

  57. Suganthan P (1999) Particle Swarm Optimizer with Neighborhood Optimizer. In: Proceedings of the congress on evolutionary computation, pp 1958–1962

  58. Shi Y, Eberhart R (1998) A modified particle swarm optimizer. In: Proceedings of the IEEE international conference on evolutionary computation, Piscataway, NJ, pp 69–73

  59. Kennedy J, Small worlds and mega-minds: effects of neighborhood topology on particle swarm performance. In: Proceedings of the congress on evolutionary computation, pp 1931–1938

  60. Kennedy J, Mendes R (2002) Population structure and particle performance. In: Proceedings of the IEEE congress on evolutionary computation, Honolulu, Hawaii

  61. Van den Bergh F (2002) An analysis of particle swarm optimizers. PhD thesis, Department of Computer Science, University of Pretoria

  62. van den Bergh F, Engelbrecht AP (2002) A new locally convergent particle swarm optimizer. In: Proceedings of the IEEE conference on systems, man, and cybernetics, Hammamet, Tunisia

  63. Kennedy J, Eberhart R (1997) A discrete binary version of the particle swarm algorithm. In: Proceedings of the conference on systems, man, and cybernetics, pp 4104–4109

  64. Pal NR, Pal SK (1993) A review on image segmentation techniques. Pattern Recogn 26:1277–1294

    Article  Google Scholar 

  65. Fu KS, Mui JK (1981) A survey on image segmentation. Pattern Recogn 13:3–16

    Article  MathSciNet  Google Scholar 

  66. Goldberg D (1989) Genetic algorithms in search, optimization and machine learning. Addison-Wesley, Reading

    MATH  Google Scholar 

  67. Salman A, Omran M, Engelbrecht A (2005) SIGT: synthetic image generation tool for clustering algorithms. ICGST Int J Graph Vision Image Process (GVIP) 2:33–44

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, School of Information Technology, University of Pretoria, Pretoria, 0002, South Africa

    Mahamed G. H. Omran & Andries P. Engelbrecht

  2. Department of Computer Engineering, Kuwait University, Kuwait, Kuwait

    Ayed Salman

Authors
  1. Mahamed G. H. Omran
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ayed Salman
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Andries P. Engelbrecht
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Andries P. Engelbrecht.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Omran, M.G.H., Salman, A. & Engelbrecht, A.P. Dynamic clustering using particle swarm optimization with application in image segmentation. Pattern Anal Applic 8, 332–344 (2006). https://doi.org/10.1007/s10044-005-0015-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10044-005-0015-5

Keywords

Search

Navigation