Skip to main content
SpringerLink
Log in

Niching particle swarm optimization based on Euclidean distance and hierarchical clustering for multimodal optimization

  • Original paper
  • Published:
Nonlinear Dynamics Aims and scope Submit manuscript

Abstract

Multimodal optimization is still one of the most challenging tasks in the evolutionary computation field, when multiple global and local optima need to be effectively and efficiently located. In this paper, a niching particle swarm optimization (PSO)-based Euclidean distance and hierarchical clustering (EDHC) for multimodal optimization is proposed. This technique first uses the Euclidean distance-based PSO algorithm to perform preliminarily search. In this phase, the particles are rapidly clustered around peaks. Secondly, hierarchical clustering is applied to identify and concentrate the particles distributed around each peak to finely search as a whole. Finally, a small-world network topology is adopted in each niche to improve the exploitation ability of the algorithm. At the end of this paper, the proposed EDHC-PSO algorithm is applied to the traveling salesman problems (TSP) after being discretized. The experiments demonstrate that the proposed method outperforms existing niching techniques on benchmark problems and is effective for TSP.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Eberhart, R.C., Kennedy, J.: A new optimizer using particle swarm theory. In: Proceedings of the 6th International Symposium on Micro-machine and Human Science. Nagoya, Japan (1995)

  2. Kennedy, J., Eberhart, R.C.: Particle swarm optimization. In: Proceedings of IEEE International Conference Neural Networks. Perth, Australia (1995)

  3. Li, X.: Niching without niching parameters: particle swarm optimization using a ring topology. IEEE Trans. Evol. Comput. 14(1), 150–169 (2010)

    Google Scholar 

  4. Kennedy, J., Mendes, R.: Population structure and particle swarm performance. (2002)

  5. Mendes, R., Kennedy, J., Neves, J.: The fully informed particle swarm: simpler, maybe better. IEEE Trans. Evol. Comput. 8(3), 204–210 (2004)

    Google Scholar 

  6. Bassett, D.S., Bullmore, E.: Small-world brain networks. The Neuroscientist 12(6), 512–523 (2006)

    Google Scholar 

  7. Kennedy, J., Mendes, R.: Neighborhood topologies in fully informed and best-of-neighborhood particle swarms. IEEE Trans. Syst. Man Cybern. Part C Appl. Rev. 36(4), 515 (2006)

    Google Scholar 

  8. Liu, Q., van Wyk, B.J., Sun, Y.: Small world network based dynamic topology for particle swarm optimization. In: 11th International Conference on Natural Computation (ICNC), 2015. IEEE (2015)

  9. Mahfoud, S.W.: Crowding and preselection revisited. Urbana 51, 61801 (1992)

    Google Scholar 

  10. Thomsen, R.: Multimodal optimization using crowding-based differential evolution. In: Congress on Evolutionary Computation, 2004. CEC2004. IEEE (2004)

  11. Pétrowski, A.: A clearing procedure as a niching method for genetic algorithms. In: Proceedings of IEEE International Conference on Evolutionary Computation. IEEE (1996)

  12. Goldberg, D.E., Richardson, J.: Genetic algorithms with sharing for multimodal function optimization. In: Genetic algorithms and their applications: Proceedings of the Second International Conference on Genetic Algorithms. Hillsdale, NJ: Lawrence Erlbaum (1987)

  13. Beasley, D., Bull, D.R., Martin, R.R.: A sequential niche technique for multimodal function optimization. Evol. Comput. 1(2), 101–125 (1993)

    Google Scholar 

  14. Yin, X., Germay, N.: A fast genetic algorithm with sharing scheme using cluster analysis methods in multimodal function optimization. In: Artificial Neural Nets and Genetic Algorithms. Springer (1993)

  15. Li, J.-P., et al.: A species conserving genetic algorithm for multimodal function optimization. Evol. Comput. 10(3), 207–234 (2002)

    Google Scholar 

  16. Bessaou, M., Pétrowski, A., Siarry, P.: Island model cooperating with speciation for multimodal optimization. In: International Conference on Parallel Problem Solving from Nature. Paris, France: Springer (2000)

  17. Harik, G.R.: Finding Multimodal Solutions Using Restricted Tournament Selection. In: ICGA. (1995)

  18. Brits, R., Engelbrecht, A.P., Van den Bergh, F.: A niching particle swarm optimizer. In: Proceedings of the 4th Asia-Pacific Conference on Simulated Evolution and Learning. Singapore: Orchid Country Club (2002)

  19. Veeramachaneni, K., et al.: Optimization using particle swarms with near neighbor interactions. In: Genetic and Evolutionary Computation Conference. Springer (2003)

  20. Li, X.: A multimodal particle swarm optimizer based on fitness Euclidean-distance ratio. In: Proceedings of the 9th Annual Conference on Genetic and Evolutionary Computation. ACM (2007)

  21. Li, X.: Efficient differential evolution using speciation for multimodal function optimization. In: Proceedings of the 7th Annual Conference on Genetic and Evolutionary Computation. ACM (2005)

  22. Li, X.: Adaptively choosing neighbourhood bests using species in a particle swarm optimizer for multimodal function optimization. In: Genetic and Evolutionary Computation–GECCO 2004. Springer (2004)

  23. Parrott, D., Li, X.: Locating and tracking multiple dynamic optima by a particle swarm model using speciation. IEEE Trans. Evol. Comput. 10(4), 440–458 (2006)

    Google Scholar 

  24. Qu, B., Suganthan, P.N., Liang, J.-J.: Differential evolution with neighborhood mutation for multimodal optimization. IEEE Trans. Evol. Comput. 16(5), 601–614 (2012)

    Google Scholar 

  25. Qu, B., Suganthan, P.N., Das, S.: A distance-based locally informed particle swarm model for multimodal optimization. IEEE Trans. Evol. Comput. 17(3), 387–402 (2013)

    Google Scholar 

  26. Gao, W., Yen, G.G., Liu, S.: A cluster-based differential evolution with self-adaptive strategy for multimodal optimization. IEEE Trans. Cybern. 44(8), 1314–1327 (2014)

    Google Scholar 

  27. Hui, S., Suganthan, P.N.: Ensemble and arithmetic recombination-based speciation differential evolution for multimodal optimization. IEEE Trans. Cybern. 46(1), 64–74 (2016)

    Google Scholar 

  28. Mohamed, A.W., Suganthan, P.N.: Real-parameter unconstrained optimization based on enhanced fitness-adaptive differential evolution algorithm with novel mutation. Soft Computing 1–21 (2017)

  29. Bošković, B., Brest, J.: Clustering and differential evolution for multimodal optimization. In: IEEE Congress on Evolutionary Computation (CEC), 2017. IEEE (2017)

  30. Sengupta, S., et al.: Particle Swarm Optimization: A survey of historical and recent developments with hybridization perspectives. arXiv preprint arXiv:1804.05319, (2018)

  31. Wang, F., et al.: A hybrid particle swarm optimization algorithm using adaptive learning strategy. Inf. Sci. 436, 162–177 (2018)

    Google Scholar 

  32. Xia, X., Gui, L., Zhan, Z.-H.: A multi-swarm particle swarm optimization algorithm based on dynamical topology and purposeful detecting. Appl. Soft Comput. 67, 126–140 (2018)

    Google Scholar 

  33. Rana, P.B., Patel, J.L., Lalwani, D.: Parametric optimization of turning process using evolutionary optimization techniques—a review (2000–2016), In: Soft Computing for Problem Solving. Springer. pp. 165-180 (2019)

  34. Liu, Q., et al.: Dynamic Small World Network Topology for Particle Swarm Optimization. Int. J. Pattern Recognit Artif Intell. 30(09), 1660009 (2016)

    Google Scholar 

  35. Kennedy, J.: Small worlds and mega-minds: effects of neighborhood topology on particle swarm performance. In: Proceedings of the 1999 Congress on Evolutionary Computation, 1999. CEC 99. IEEE (1999)

  36. Shi, Y., Eberhart, R.: A modified particle swarm optimizer. In: Proceedings of the 1999 Congress on Evolutionary Computation Proceedings, 1998. IEEE World Congress on Computational Intelligence, 1998. IEEE (1998)

  37. Shi, Y., Eberhart, R.C.: Parameter selection in particle swarm optimization. In: International Conference on Evolutionary Programming. Springer (1998)

  38. Shi, Y., Eberhart, R.C.: Fuzzy adaptive particle swarm optimization. In: Proceedings of the 2001 Congress on Evolutionary Computation, 2001. IEEE (2001)

  39. Ratnaweera, A., Halgamuge, S.K., Watson, H.C.: Self-organizing hierarchical particle swarm optimizer with time-varying acceleration coefficients. IEEE Trans. Evol. Comput. 8(3), 240–255 (2004)

    Google Scholar 

  40. Shi, Y., Eberhart, R.C.: Empirical study of particle swarm optimization. In: Proceedings of the 1999 Congress on Evolutionary Computation, 1999. CEC 99. IEEE (1999)

  41. Zhang, L., et al.: A new particle swarm optimization algorithm with adaptive inertia weight based on Bayesian techniques. Appl. Soft Comput. 28, 138–149 (2015)

    Google Scholar 

  42. Suganthan, P.N.: Particle swarm optimiser with neighbourhood operator. In: Proceedings of the 1999 Congress on Evolutionary Computation, 1999. CEC 99. IEEE (1999)

  43. Vora, M., Mirnalinee, T.: Small world particle swarm optimizer for global optimization problems. In: Pattern Recognition and Machine Intelligence, Springer. pp. 575–580 (2013)

  44. Zhan, Z.-H., et al.: Multiple populations for multiple objectives: a coevolutionary technique for solving multiobjective optimization problems. IEEE Trans. Cybern. 43(2), 445–463 (2013)

    Google Scholar 

  45. Lynn, N., Suganthan, P.N.: Heterogeneous comprehensive learning particle swarm optimization with enhanced exploration and exploitation. Swarm Evol. Comput. 24, 11–24 (2015)

    Google Scholar 

  46. Mo, S., Zeng, J., Xu, W.: Attractive and repulsive fully informed particle swarm optimization based on the modified fitness model. Soft. Comput. 20(3), 863–884 (2016)

    Google Scholar 

  47. Pornsing, C., Sodhi, M.S., Lamond, B.F.: Novel self-adaptive particle swarm optimization methods. Soft. Comput. 20(9), 3579–3593 (2016)

    Google Scholar 

  48. Liu, J., et al.: Ecosystem particle swarm optimization. Soft. Comput. 21(7), 1667–1691 (2017)

    Google Scholar 

  49. Parsopoulos, K., Vrahatis, M.: Modification of the particle swarm optimizer for locating all the global minima. In: Artificial Neural Nets and Genetic Algorithms. Springer (2001)

  50. Sibson, R.: SLINK: an optimally efficient algorithm for the single-link cluster method. Comput. J. 16(1), 30–34 (1973)

    Google Scholar 

  51. Defays, D.: An efficient algorithm for a complete link method. Comput. J. 20(4), 364–366 (1977)

    Google Scholar 

  52. Matsushita, H., Nishio, Y.: Network-structured particle swarm optimizer with small-world topology. In: Proceedings of Int. Symposium on Nonlinear Theory and its Applications. (2009)

  53. Matsushita, H., Nishio, Y.: Network-structured particle swarm optimizer considering neighborhood relationships. In: International Joint Conference on Neural Networks, 2009. IJCNN 2009. IEEE (2009)

  54. Matsushita, H., Nishio, Y., Chi, K.T.: Network-structured particle swarm optimizer that considers neighborhood distances and behaviors. J. Signal Process. 18(6), 291–302 (2014)

    Google Scholar 

  55. Vora, M., Mirnalinee, T.: Small-World Particle Swarm Optimizer for Real-World Optimization Problems, In: Artificial Intelligence and Evolutionary Algorithms in Engineering Systems. Springer. pp. 465–472 (2015)

  56. Wei, J., et al.: Optimal Randomness in Swarm-Based Search. arXiv:1905.02776. (2019)

  57. Watts, D.J., Strogatz, S.H.: Collective dynamics of ‘small-world’ networks. Nature 393(6684), 440–442 (1998)

    Google Scholar 

  58. Watts, D.J.: Small Worlds: The Dynamics of Networks Between Order and Randomness. Princeton University Press, Princeton (1999)

    Google Scholar 

  59. Clerc, M., Kennedy, J.: The particle swarm-explosion, stability, and convergence in a multidimensional complex space. IEEE Trans. Evol. Comput. 6(1), 58–73 (2002)

    Google Scholar 

  60. Qu, B.-Y., Liang, J.J., Suganthan, P.N.: Niching particle swarm optimization with local search for multi-modal optimization. Inf. Sci. 197, 131–143 (2012)

    Google Scholar 

  61. Stoean, C., et al.: Multimodal optimization by means of a topological species conservation algorithm. IEEE Trans. Evol. Comput. 14(6), 842–864 (2010)

    Google Scholar 

  62. Ackley, D.H.: An empirical study of bit vector function optimization. Genetic algorithms and simulated annealing 1, 170–204 (1987)

    Google Scholar 

  63. Deb, K.: Genetic algorithms in multimodal function optimization. Clearinghouse for Genetic Algorithms, Department of Engineering Mechanics, University of Alabama (1989)

  64. Michalewicz, Z.: Genetic Algorithms + Data Structures = Evolution Programs. Springer, New York (1996)

    Google Scholar 

  65. DeJong, K.: An analysis of the behavior of a class of genetic adaptive systems. Ph. D. Thesis, University of Michigan, (1975)

  66. Ursem, R.K.: Multinational evolutionary algorithms. In: Proceedings of the 1999 Congress on Evolutionary Computation, 1999. CEC 99. IEEE (1999)

  67. Croes, G.A.: A method for solving traveling-salesman problems. Oper. Res. 6(6), 791–812 (1958)

    Google Scholar 

  68. Davis, L.: Applying adaptive algorithms to epistatic domains. In: IJCAI. (1985)

  69. Tao, G., Michalewicz, Z.: Evolutionary algorithms for the TSP. Parallel Probl. Solv. Nat. 1498, 803–812 (1998)

    Google Scholar 

  70. Wang, K., et al.: Particle swarm optimization for traveling salesman problem. In: International Conference on Machine Learning and Cybernetics, 2003. IEEE (2003)

  71. Clerc, M.: Discrete particle swarm optimization, illustrated by the traveling salesman problem, In: New optimization Techniques in Engineering, Springer. pp. 219–239 (2004)

  72. Zhi, X.-H., et al.: A discrete PSO method for generalized TSP problem. In: Proceedings of 2004 International Conference on Machine Learning and Cybernetics, 2004. IEEE (2004)

  73. Shi, X.H., et al.: Particle swarm optimization-based algorithms for TSP and generalized TSP. Inf. Process. Lett. 103(5), 169–176 (2007)

    Google Scholar 

  74. Mahi, M., Baykan, Ö.K., Kodaz, H.: A new hybrid method based on particle swarm optimization, ant colony optimization and 3-opt algorithms for traveling salesman problem. Appl. Soft Comput. 30, 484–490 (2015)

    Google Scholar 

  75. Zhong, Y., Lin, J., Wang, L., Zhang, H.: Discrete comprehensive learning particle swarm optimization algorithm with Metropolis acceptance criterion for traveling salesman problem. Swarm Evol, Comput 42, 77–88 (2018)

    Google Scholar 

  76. Michalewicz, Z., Fogel, D.B.: How to Solve It: Modern Heuristics. Springer Science & Business Media, Berlin (2013)

    Google Scholar 

  77. Paul, P.V., et al.: Performance analyses over population seeding techniques of the permutation-coded genetic algorithm: an empirical study based on traveling salesman problems. Appl. Soft Comput. 32, 383–402 (2015)

    Google Scholar 

  78. Wang, J., et al.: Multi-offspring genetic algorithm and its application to the traveling salesman problem. Appl. Soft Comput. 43, 415–423 (2016)

    Google Scholar 

  79. Chen, W.-N., et al.: A novel set-based particle swarm optimization method for discrete optimization problems. IEEE Trans. Evol. Comput. 14(2), 278–300 (2010)

    Google Scholar 

  80. Escario, J.B., Jimenez, J.F., Giron-Sierra, J.M.: Ant colony extended: experiments on the travelling salesman problem. Expert Syst. Appl. 42(1), 390–410 (2015)

    Google Scholar 

  81. Ismkhan, H.: Effective heuristics for ant colony optimization to handle large-scale problems. Swarm Evol. Comput. 32, 140–149 (2017)

    Google Scholar 

  82. Xu, Z., et al.: Immune algorithm combined with estimation of distribution for traveling salesman problem. IEEJ Trans. Electric. Electron. Eng. 11(S1), S142 (2016)

    Google Scholar 

  83. Wang, H., Zhang, N., Créput, J.-C.: A massively parallel neural network approach to large-scale Euclidean traveling salesman problems. Neurocomputing 240, 137–151 (2017)

    Google Scholar 

  84. Reinelt, G.: TSPLIB–A traveling salesman problem library. ORSA J. Comput. 3(4), 376–384 (1991)

    Google Scholar 

  85. Davis, L.: Genetic Algorithms and Simulated Annealing. Morgan Kaufmann Publishers Inc., San Francisco, CA (1987)

    Google Scholar 

Download references

Acknowledgements

This work is based on the research supported in part by the National Research Foundation of South Africa (Grant Numbers 93539).

Author information

Authors and Affiliations

  1. School of Mechanical and Electrical Engineering, Kunming University, Kunming, 650214, China

    Qingxue Liu

  2. College of Electrical Engineering and Automation, Shandong University of Science and Technology, Qingdao, 266590, China

    Qingxue Liu

  3. Department of Electrical Engineering, Tshwane University of Technology, Pretoria, 0001, South Africa

    Qingxue Liu, Shengzhi Du & Barend Jacobus van Wyk

  4. Department of Electrical and Electronic Engineering Science, University of Johannesburg, Johannesburg, 2006, South Africa

    Yanxia Sun

Authors
  1. Qingxue Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shengzhi Du
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Barend Jacobus van Wyk
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yanxia Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shengzhi Du.

Ethics declarations

Conflict of interest

The authors certify that they have no conflict of interest regarding copyright and finance.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, Q., Du, S., van Wyk, B.J. et al. Niching particle swarm optimization based on Euclidean distance and hierarchical clustering for multimodal optimization. Nonlinear Dyn 99, 2459–2477 (2020). https://doi.org/10.1007/s11071-019-05414-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11071-019-05414-7

Keywords

Search

Navigation