Skip to main content
SpringerLink
Log in

A two-layer surrogate-assisted particle swarm optimization algorithm

  • Focus
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

Like most evolutionary algorithms, particle swarm optimization (PSO) usually requires a large number of fitness evaluations to obtain a sufficiently good solution. This poses an obstacle for applying PSO to computationally expensive problems. This paper proposes a two-layer surrogate-assisted PSO (TLSAPSO) algorithm, in which a global and a number of local surrogate models are employed for fitness approximation. The global surrogate model aims to smooth out the local optima of the original multimodal fitness function and guide the swarm to fly quickly to an optimum or the global optimum. In the meantime, a local surrogate model constructed using the data samples near each particle is built to achieve a fitness estimation as accurate as possible. The contribution of each surrogate in the search is empirically verified by experiments on uni- and multi-modal problems. The performance of the proposed TLSAPSO algorithm is examined on ten widely used benchmark problems, and the experimental results show that the proposed algorithm is effective and highly competitive with the state-of-the-art, especially for multimodal optimization problems.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17

Similar content being viewed by others

References

  • Abou El-Ela A, Fetouh T, Bishr M, Saleh R (2008) Power systems operation using particle swarm optimization technique. Electr Power Syst Res 78(11):1906–1913

  • Acar E, Rais-Rohani M (2009) Ensemble of metamodels with optimized weight factors. Struct Multidiscip Optim 37(3):279–294

    Article  Google Scholar 

  • Bird S, Li X (2010) Improving local convergence in particle swarms by fitness approximation using regression. In: Computational intelligence in expensive optimization problems. Adaptation learning and optimization, vol 2. Springer, Berlin, Heidelberg, New York, pp 265–293

  • Buche D, Schraudolph NN, Koumoutsakos P (2005) Accelerating evolutionary algorithms with gaussian process fitness function models. IEEE Trans Syst Man Cybern Part C Appl Rev 35(2):183–194

    Article  Google Scholar 

  • Clerc M, Kennedy J (2002) The particle swarm-explosion, stability, and convergence in a multidimensional complex space. IEEE Trans Evol Comput 6(1):58–73

    Article  Google Scholar 

  • Eberhart R, Kennedy J (1995) A new optimizer using particle swarm theory. In: Proceedings of the sixth international symposium on micro machine and human science, pp 39–43

  • Eberhart RC, Shi Y (2000) Comparing inertia weights and constriction factors in particle swarm optimization. In: Proceedings of the 2000 congress on evolutionary computation, vol 1, pp 84–88

  • Farina M (2002) A neural network based generalized response surface multiobjective. In: Proceedings of the 2002 congress on evolutionary computation, vol 1, pp 956–961

  • Fonseca LG, Lemonge AC, Barbosa HJ (2012) A study on fitness inheritance for enhanced efficiency in real-coded genetic algorithms. In: Proceedings of the 2012 IEEE congress on evolutionary computation (CEC), pp 1–8

  • Goel T, Haftka RT, Shyy W, Queipo NV (2007) Ensemble of surrogates. Struct Multidiscip Optim 33(3):199–216

  • He S, Prempain E, Wu Q (2004) An improved particle swarm optimizer for mechanical design optimization problems. Eng Optim 36(5):585–605

    Article  MathSciNet  Google Scholar 

  • Hendtlass T (2007) Fitness estimation and the particle swarm optimisation algorithm. In: Proceedings of the IEEE congress on evolutionary computation, pp 4266–4272

  • Jin Y, Sendhoff B (2004) Reducing fitness evaluations using clustering techniques and neural network ensembles. In: Proceedings of the genetic and evolutionary computation (GECCO 2004). Lecture notes in computer science, vol 3102. Springer, New York, pp 688– 699

  • Jin Y, Olhofer M, Sendhoff B (2002) A framework for evolutionary optimization with approximate fitness. IEEE Trans Evol Comput 6(5):481–494

    Article  Google Scholar 

  • Jin Y (2005) A comprehensive survey of fitness approximation in evolutionary computation. Soft Comput 9(1):3–12

    Article  Google Scholar 

  • Joseph VR, Hung Y, Sudjianto A (2008) Blind kriging: a new method for developing metamodels. J Mech Des 130(3):031102.1– 031102.8

  • Kattan A, Galvan E (2012) Evolving radial basis function networks via gp for estimating fitness values using surrogate models. In: Proceedings of the 2012 IEEE congress on evolutionary computation (CEC), pp 1–7

  • Lian Y, Liou M-S (2005) Multiobjective optimization using coupled response surface model. AIAA J 43(6):1316–1325

    Article  Google Scholar 

  • Lim D, Jin Y, Ong Y-S, Sendhoff B (2010) Generalizing surrogate-assisted evolutionary computation. IEEE Trans Evol Comput 14(3):329–355

    Article  Google Scholar 

  • Liu B, Zhang Q, Gielen G (2014) A Gaussian process surrogate model assisted evolutionary algorithm for medium scale expensive optimization problems. IEEE Trans Evol Comput 18(2):180–192

  • Lu J, Li B, Jin Y (2013) An evolution strategy assisted by an ensemble of local gaussian process models. In: Proceedings of the fifteenth annual conference on genetic and evolutionary computation conference, ACM, pp 447–454

  • Lu X, Tang K, Yao X (2011) Classification-assisted differential evolution for computationally expensive problems. In: Proceedings of the 2011 IEEE congress on evolutionary computation (CEC), pp 1986–1993

  • Ong YS, Nair PB, Keane AJ, Wong KW (2004) Surrogate-assisted evolutionary optimization frameworks for high-fidelity engineering design problems. In: Jin Y (ed) Knowledge incorporation in evolutionary computation. Studies in fuzziness and soft computing series. Springer, pp 307–331

  • Ong YS, Nair PB, Keane AJ (2003) Evolutionary optimization of computationally expensive problems via surrogate modeling. AIAA J 41(4):687–696

    Article  Google Scholar 

  • Ong Y-S, Nair PB, Lum KY (2006) Max-min surrogate-assisted evolutionary algorithm for robust design. IEEE Trans Evol Comput 10(4):392–404

    Article  Google Scholar 

  • Parno M, Hemker T, Fowler K (2012) Applicability of surrogates to improve efficiency of particle swarm optimization for simulation-based problems. Eng Optim 44(5):521–535

    Article  Google Scholar 

  • Praveen C, Duvigneau R (2009) Low cost pso using metamodels and inexact preevaluation: application to aerodynamic shape design. Comput Methods Appl Mech Eng 198(9):1087–1096

    Article  MATH  Google Scholar 

  • Ratle A (2001) Kriging as a surrogate fitness landscape in evolutionary optimization. AI EDAM 15(01):37–49

    Google Scholar 

  • Regis RG (2014) Particle swarm with radial basis function surrogates for expensive blackbox optimization. J Comput Sci 5(1):12–23

    Article  MathSciNet  Google Scholar 

  • Reyes-Sierra M, Coello CAC (2005) A study of fitness inheritance and approximation techniques for multi-objective particle swarm optimization. In: Proceedings of the 2005 IEEE congress on evolutionary computation, vol 1, pp 65–72

  • Sha D, Hsu C-Y (2008) A new particle swarm optimization for the open shop scheduling problem. Comput Oper Res 35(10):3243–3261

    Article  MATH  Google Scholar 

  • Shi Y, Eberhart R (1998) A modified particle swarm optimizer. In: Proceedings of the 1998 IEEE international conference on evolutionary computation, IEEE world congress on computational intelligence, pp 69–73

  • Smith RE, Dike BA, Stegmann SA (1995) Fitness inheritance in genetic algorithms. In: Proceedings of the 1995 ACM symposium on applied computing, pp 345–350

  • Storn R (1996) On the usage of differential evolution for function optimization. In: Proceedings of the 1996 biennial conference of the North American Fuzzy Information Processing Society, NAFIPS, pp 519–523

  • Suganthan PN, Hansen N, Liang JJ, Deb K, Chen Y, Auger A, Tiwari S (2005) Problem definitions and evaluation criteria for the cec 2005 special session on realparameter optimization. Technical Report, Nanyang Technological University, Singapore and KanGAL Report #2005005, IIT Kanpur, India

  • Sun C, Zeng J, Pan J, Jin Y (2013) Similarity-based evolution control for fitness estimation in particle swarm optimization. In: Proceedings of the 2013 IEEE symposium on computational intelligence in dynamic and uncertain environments (CIDUE), pp 1–8

  • Sun C, Zeng J, Pan J, Xue S, Jin Y (2012) A new fitness estimation strategy for particle swarm optimization. Inf Sci 221:355–370

  • Sun X, Gong D, Jin Y, Chen S (2013) A new surrogate-assisted interactive genetic algorithm with weighted semisupervised learning. IEEE Trans Cybern 43(2):685–698

    Article  Google Scholar 

  • Tang Y, Chen J, Wei J (2013) A surrogate-based particle swarm optimization algorithm for solving optimization problems with expensive black box functions. Eng Optim 45(5):557–576

    Article  MathSciNet  Google Scholar 

  • Tenne Y, Armfield SW (2009) A framework for memetic optimization using variable global and local surrogate models. Soft Comput 13(8–9):781–793

    Article  Google Scholar 

  • Ulmer H, Streichert F, Zell A (2003) Evolution strategies assisted by Gaussian processes with improved preselection criterion. In: Proceedings of the 2003 congress on evolutionary computation (CEC’03), vol 1, pp 692–699

  • Zhou Z, Ong YS, Nguyen MH, Lim D (2005) A study on polynomial regression and gaussian process global surrogate model in hierarchical surrogate-assisted evolutionary algorithm. In: Proceedings of the 2005 IEEE congress on evolutionary computation, vol 3, pp 2832–2839

  • Zhou Z, Ong YS, Nair PB, Keane AJ, Lum KY (2007) Combining global and local surrogate models to accelerate evolutionary optimization. IEEE Trans Syst Man Cybern Part C Appl Rev 37(1):66–76

    Article  Google Scholar 

Download references

Acknowledgments

This work was supported in part by Youth Foundation of Shanxi Province of China under Grant No. 2011021019-3, the Doctoral Foundation of Taiyuan University of Science and Technology under Grant No. 20122010, and the State Key Laboratory of Software Engineering, Nanjing University, China, Project No. KFKT2013A05.

Author information

Authors and Affiliations

  1. Complex System and Computational Intelligence Laboratory, Taiyuan University of Science and Technology, Taiyuan, 030024, Shanxi, China

    Chaoli Sun & Jianchao Zeng

  2. Department of Computing, University of Surrey, Guildford, GU2 7XH, UK

    Yaochu Jin

  3. National Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, 200093, China

    Yang Yu

Authors
  1. Chaoli Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yaochu Jin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jianchao Zeng
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yang Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chaoli Sun.

Additional information

Communicated by Y. Jin.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sun, C., Jin, Y., Zeng, J. et al. A two-layer surrogate-assisted particle swarm optimization algorithm. Soft Comput 19, 1461–1475 (2015). https://doi.org/10.1007/s00500-014-1283-z

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-014-1283-z

Keywords

Search

Navigation