# Improved particle swarm optimization algorithm using design of experiment and data mining techniques

## Abstract

Particle swarm optimization (PSO) is a relatively new global optimization algorithm. Benefitting from its simple concept, fast convergence speed and strong ability of optimization, it has gained much attention in recent years. However, PSO suffers from premature convergence problem because of the quick loss of diversity in solution search. In order to improve the optimization capability of PSO, design of experiment method, which spreads the initial particles across a design domain, and data mining technique, which is used to identify the promising optimization regions, are studied in this research to initialize the particle swarm. From the test results, the modified PSO algorithm initialized by OLHD (Optimal Latin Hypercube Design) technique successfully enhances the efficiency of the basic version but has no obvious advantage compared with other modified PSO algorithms. An extension algorithm, namely OLCPSO (Optimal Latin hypercube design and Classification and Regression tree techniques for improving basic PSO), is developed by consciously distributing more particles into potential optimal regions. The proposed method is tested and validated by benchmark functions in contrast with the basic PSO algorithm and five PSO variants. It is found from the test studies that the OLCPSO algorithm successfully enhances the efficiency of the basic PSO and possesses competitive optimization ability and algorithm stability in contrast to the existing initialization PSO methods.

## Keywords

Particle swarm optimization Design of experiment Data mining Optimization search Global optimization Algorithm stability## Nomenclature

- PSO
Particle Swarm Optimization

- OLHD
Optimal Latin Hypercube Design

- OLCPSO
Optimal Latin hypercube design and Classification and regression tree techniques for improving basic PSO

- DOE
Design of experiments

- CART
Classification and Regression Tree

- ESE
Enhanced Stochastic Evolutionary

- GPSO
Particle Swarm Optimization initialized by Gaussian distribution

- EPSO
Particle Swarm Optimization initialized by Exponential distribution

- LNPSO
Particle Swarm Optimization initialized by Lognormal distribution

- VC-PSO
Particle Swarm Optimization initialized by Vander Corput sequence

- SO-PSO
Particle Swarm Optimization initialized by Sobol sequence

- D
Dimension

**X**The position vector of a particle

**V**The velocity vector of a particle

**P**_{i}The best previously visited position vector

**P**_{g}The global best position vector of the swarm

*t*Generation number

*c*_{1}Cognitive scaling parameter

*c*_{2}Social scaling parameter

*w*Inertia weight

*r*_{1}Random numbers

*r*_{2}Random numbers

**y**Vector of classes

*M*Number of response

*N*Numbers of design variable

**xc**Matrix of design variable

*xc*_{j}^{R}Best splitting value of variable

*xc*_{ j }*ϕ*_{p}Optimal criterion for design of experiment

*d*_{i}Euclidean distance

*J*_{i}Number of pairs separated by

*d*_{ i }in the design*s*Number of distinct distance values

*p*A positive integer

*T*_{h}Threshold of the cooling and warming schedule

**X**_{E}Current design of DOE

**X**_{try}Best design based on ESE

*S*A uniform random number between 0 and 1

- P
Particle number

*s*_{f}Sample ratio factor

*t*_{c}Supplement coefficient

*L*Length of variable’s range

*P*_{s}Particle number in sub-region

## Notes

### Acknowledgments

The authors acknowledge the support from Ford University Research Program (URP) and the Adjunct Professor position provided by the Shanghai Jiao Tong University to Dr. Wei Chen.

## References

- Bochenek B, Forys P (2006) Structural optimization for post-buckling behavior using particle swarms. Struct Multidiscip Optim 32:521–531CrossRefGoogle Scholar
- Breiman L, Friedman JH, Olshen RA, Stone CG (1984) Classification and Regression Trees. Wadsworth International Group, USAzbMATHGoogle Scholar
- Brits R (2002) Niching Strategies for particle Swarm Optimization. Masters Thesis, Department of Computer Science, University of PretoriaGoogle Scholar
- Cheng YM, Li L, Sun YJ, Au SK (2012) A coupled particle swarm and harmony search optimization algorithm for difficult geotechnical problems. Struct Multidiscip Optim 45:489–501CrossRefGoogle Scholar
- Clark LA, Pregibon D (1992) Tree-based models. In: Statistical models in S. Wadsworth and Brooks. pp 377–420Google Scholar
- Cui H, Turan O (2010) Application of a new multi-agent hybrid co-evolution based particle swarm optimization methodology in ship design. Comput Aided Des 42:1013–1027zbMATHCrossRefGoogle Scholar
- Eberhart RC, Shi Y (1998) A modified particle swarm optimizer. In: Proceedings of the 1998 International Conference on Neural Networks and Brain. pp 5–13Google Scholar
- Eberhart RC, Shi Y (2001) Particle Swarm Optimization: developments, Applications and Resources. In: IEEE Int. Conference on Evolutionary Computation. pp 81–86Google Scholar
- Elegbede C (2005) Structural reliability assessment based on particles swarm optimization. Struct Saf 27(2):171–186CrossRefGoogle Scholar
- Fourie PC, Groenwold AA (2002) The particle swarm optimization algorithm in size and shape optimization. Struct Multidiscip Optim 23:259–267CrossRefGoogle Scholar
- Grosan C, Abrahamand A, Nicoara M (2005) Search optimization using hybrid particle sub-swarms and evolutionary algorithms. Int J Simul Syst Sci Technol 6:60–79Google Scholar
- Hart CG, Vlahopoulos N (2010) An integrated multidisciplinary particle swarm optimization approach to conceptual ship design. Struct Multidiscip Optim 41:481–489CrossRefGoogle Scholar
- Iman RL, Conover WJ (1980) Small sample sensitivity analysis techniques for computer models with an application to risk assessment. Commun Stat A Theory Methods 17:1749–1842MathSciNetCrossRefGoogle Scholar
- Jin R, Chen W, Sudjianto A (2005) An efficient algorithm for constructing optimal design of computer experiments. J Stat Plan Inf 134:268–287zbMATHMathSciNetCrossRefGoogle Scholar
- Kennedy J (1997) The Particle Swarm: Social Adaptation of Knowledge. In: IEEE International Conference on Evolutionary Computation (Indianapolis, Indiana). pp 303–308Google Scholar
- Kennedy J, Eberhart R (1995) Particle swarm optimization. In: Proceedings of the IEEE International Conference on Neural Networks. pp 1942–1948Google Scholar
- Khare A, Rangnekar S (2013) A review of particle swarm optimization and its application in solar photovoltaic system. Appl Soft Comput 13:2997–3006CrossRefGoogle Scholar
- Kou XY, Parks GT, Tan ST (2012) Optimal design of functionally graded materials using a procedural model and particle swarm optimization. Comput Aided Des 44:300–310CrossRefGoogle Scholar
- Krohling R A (2004) Gaussian Swarm: A Novel Particle Swarm Optimization Algorithm. In: In Proc. of the 2004 I.E. Conference on Cybernetics and Intelligent Systems. pp 372–376Google Scholar
- Krohling RA, Coelho LS (2006) PSO-E: Particle Swarm with Exponential Distribution. In: Proc. of IEEE Congress on Evolutionary Computation. pp 1428–1433Google Scholar
- Lakshmi K, Rama Mohan Rao A (2013) Optimal design of laminate composite isogrid with dynamically reconfigurable quantum PSO. Struct Multidiscip Optim 48:1001–1021CrossRefGoogle Scholar
- Li H, Chandrashekhara K (2012) Structural optimization of laminated composite blade using particle swarm optimization. In: Proceedings of the ASME 2012 International Mechanical Engineering Congress &Exposition. pp 281–287Google Scholar
- Ma M, Zhang LB (2007) Particle swarm optimization algorithm design for fuzzy neural network. Adv Soft Comput 40:309–314CrossRefGoogle Scholar
- McKay MD, Beckman RJ, Conover WJ (1979) A comparison of three methods for selecting values of input variables from a computer code. Technometrics 21:239–245zbMATHMathSciNetGoogle Scholar
- Montgomery DC (2004) Design and analysis of experiments. Wiley, USAGoogle Scholar
- Morris MD, Mitchell TJ (1995) Exploratory designs for computational experiments. J Stat Plan Inf 43:381–402zbMATHCrossRefGoogle Scholar
- Pant M, Thangaraj R, Singh VP (2007) Particle Swarm Optimization: Experimenting the Distributions of Random Numbers. In: 3rd Indian Int. Conf. on Artificial Intelligence. pp 412 – 420Google Scholar
- Pant M, Thangaraj R, Grosan C, Abraham A (2008) Improved Particle Swarm Optimization with Low-discrepancy Sequences. In: IEEE Congress on Evolutionary Computation (CEC’08). pp 3016–3023Google Scholar
- Pant M, Thangaraj R, Abraham A (2009) Particle swarm optimization: performance tuning and empirical analysis. Found Comput Intell 3(203):101–128Google Scholar
- Parsopoulos KE, Varahatis MN (2002) Initializing Particle Swarm Optimizer using non linear simplex Method. In: Advances in Intelligent systems, Fuzzy systems, Evolutionary Computation. pp 216–221Google Scholar
- Ripley BD (1996) Pattern recognition and neural networks. Cambridge University Press, CambridgezbMATHCrossRefGoogle Scholar
- Shi YH, Eberhart RC (1998) A Modified Particle Swarm Optimizer. In: IEEE International Conference on Evolutionary Computation. pp 69–73Google Scholar
- Simpson TW, Lin DKJ, Chen W (2001) Sampling strategies for computer experiments: design and analysis. Int J Reliab Appl 2(3):209–240Google Scholar
- Uy NQ, Hoai NX, Mckay RI, Tuan PM (2007) Initialising PSO with Randomized Low-Discrepancy Sequences: The Comparative Results. In: Proc. of IEEE Congress Evolutionary Computation. pp 1985–1992Google Scholar
- Wang L, Singh C (2007) Compromise between cost and reliability in optimum design of an autonomous hybrid power system using mixed-integer PSO algorithm. In: IEEE. pp 682–689Google Scholar
- Wang J, Yin Z (2008) A ranking selection-based particle swarm optimizer for engineering design optimization problems. Struct Multidiscip Optim 37:131–147CrossRefGoogle Scholar