Skip to main content

An adaptive surrogate-assisted particle swarm optimization for expensive problems

Abstract

To solve engineering problems with evolutionary algorithms, many expensive function evaluations (FEs) are required. To alleviate this difficulty, surrogate-assisted evolutionary algorithms (SAEAs) have attracted increasingly more attention in both academia and industry. Most existing SAEAs either waste computational resources due to the lack of accuracy of the surrogate model or easily fall into the local optimum as the dimension increases. To address these problems, this paper proposes an adaptive surrogate-assisted particle swarm optimization algorithm. In the proposed algorithm, a surrogate model is adaptively selected from a single model and an ensemble model by comparing the best existing solution and the latest obtained solution. Additionally, a model output criterion based on the standard deviation is suggested to improve the stability and generalization ability of the ensemble model. To verify the performance of the proposed algorithm, 10 benchmark functions with different modalities from 10 to 50 dimensions are tested, and the results are compared with those of five state-of-the-art SAEAs. The experimental results indicate that the proposed algorithm performs well for most benchmark functions within a limited number of FEs. Moreover, the performance of the proposed algorithm in solving engineering problems is verified by applying the algorithm to the PX oxidation process.

This is a preview of subscription content, access via your institution.

We’re sorry, something doesn't seem to be working properly.

Please try refreshing the page. If that doesn't work, please contact support so we can address the problem.

Fig. 1
Fig. 2
Fig. 3
Fig. 4

References

  1. Alizadeh R, Allen JK, Mistree F (2020) Managing computational complexity using surrogate models: a critical review. Res Eng Des 31(3):275–298. https://doi.org/10.1007/s00163-020-00336-7

    Article  Google Scholar 

  2. Awad NH, Ali MZ, Mallipeddi R, Suganthan PN (2018) An improved differential evolution algorithm using efficient adapted surrogate model for numerical optimization. Inf Sci 451:326–347. https://doi.org/10.1016/j.ins.2018.04.024

    MathSciNet  Article  Google Scholar 

  3. Díaz-Manríquez A, Toscano G, Coello Coello CA (2017) Comparison of metamodeling techniques in evolutionary algorithms. Soft Comput 21(19):5647–5663. https://doi.org/10.1007/s00500-016-2140-z

    Article  Google Scholar 

  4. Emmerich MTM, Giannakoglou KC, Naujoks B (2006) Single- and multi-objective evolutionary optimization assisted by gaussian random field metamodels. IEEE Trans Evol Comput 10(4):421–439. https://doi.org/10.1109/TEVC.2005.859463

    Article  Google Scholar 

  5. Fan CD, Hou B, Xiao LY, Yi LZ (2020) A surrogate-assisted particle swarm optimization using ensemble learning for expensive problems with small sample datasets. Appl Soft Comput. https://doi.org/10.1016/j.asoc.2020.106242

    Article  Google Scholar 

  6. Fan QQ, Yan XF (2015) Differential evolution algorithm with self-adaptive strategy and control parameters for p-xylene oxidation process optimization. Soft Comput 19(5):1363–1391. https://doi.org/10.1007/s00500-014-1349-y

    Article  Google Scholar 

  7. Fu CB, Wang P, Zhao L, Wang XJ (2020) A distance correlation-based Kriging modeling method for high-dimensional problems. Knowl-Based Syst. https://doi.org/10.1016/j.knosys.2020.106356

    Article  Google Scholar 

  8. Gao KF, Mei G, Cuomo S, Piccialli F, Xu NX (2020) ARBF: adaptive radial basis function interpolation algorithm for irregularly scattered point sets. Soft Comput. https://doi.org/10.1007/s00500-020-05211-0

    Article  Google Scholar 

  9. Goel T, Haftka RT, Shyy W, Queipo NV (2007) Ensemble of surrogates. Struct Multidiscip Optim 33(3):199–216. https://doi.org/10.1007/s00158-006-0051-9

    Article  Google Scholar 

  10. Huang CW, Radi B, Hami AE, Bai H (2018) CMA evolution strategy assisted by Kriging model and approximate ranking. Appl Intell 48(11):4288–4304. https://doi.org/10.1007/s10489-018-1193-3

    Article  Google Scholar 

  11. Hüllen G, Zhai JY, Kim SH et al (2020) Managing uncertainty in data-driven simulation-based optimization. Comput Chem Eng. https://doi.org/10.1016/j.compchemeng.2019.106519

    Article  Google Scholar 

  12. Jia LY, Alizadeh R, Jia H et al (2020) A rule-based method for automated surrogate model selection. Adv Eng Inf. https://doi.org/10.1016/j.aei.2020.101123

    Article  Google Scholar 

  13. Jin YC (2011) Surrogate-assisted evolutionary computation: recent advances and future challenges. Swarm Evol Comput 1(2):61–70. https://doi.org/10.1016/j.swevo.2011.05.001

    Article  Google Scholar 

  14. Liang S, Rasheed K (2008) ASAGA: an adaptive surrogate-assisted genetic algorithm. In: Proceeding of the 2008 conference on genetic and evolutionary computation conference(GECCO). https://doi.org/10.1145/1389095.1389289

  15. Li F, Cai XW, Gao L (2019) Ensemble of surrogates assisted particle swarm optimization of medium scale expensive problems. Appl Soft Comput 74:291–305. https://doi.org/10.1016/j.asoc.2018.10.037

    Article  Google Scholar 

  16. Li F, Shen WM, Cai XW (2020) A fast surrogate-assisted particle swarm optimization algorithm for computationally expensive problems. Appl Soft Comput 92:106303. https://doi.org/10.1016/j.asoc.2020.106303

    Article  Google Scholar 

  17. Li YH (2020) A Kriging-based multi-point sequential sampling optimization method for complex black-box problem. Evol Intell. https://doi.org/10.1007/s12065-020-00352-5

    Article  Google Scholar 

  18. Liu B, Grout V, Nikolaeva A (2018) Efficient global optimization of actuator based on a surrogate model assisted hybrid algorithm. IEEE Trans Ind Electron 65(7):5712–5721. https://doi.org/10.1109/TIE.2017.2782203

    Article  Google Scholar 

  19. Liu B, Zhang QF, Gielen GGE (2014) A Gaussian process surrogate model assisted evolutionary algorithm for medium scale expensive optimization problems. IEEE Trans Evol Comput 18(2):180–192. https://doi.org/10.1109/TEVC.2013.2248012

    Article  Google Scholar 

  20. Liu HT, Meng JG, Xu SL, Yang SH, Wang XF (2016) Optimal weighted pointwise ensemble of radial basis functions with different basis functions. AIAA J 54(10):3117–3133. https://doi.org/10.2514/1.J054664

    Article  Google Scholar 

  21. Liu HT, Cai JF, Ong Y (2017) An adaptive sampling approach for Kriging metamodeling by maximizing expected prediction error. Comput Chem Eng 106:171–182. https://doi.org/10.1016/j.compchemeng.2017.05.025

    Article  Google Scholar 

  22. Liu RR, Li ZM (2014) Soft sensor for COx content in tail gas of PX oxidation side reactions based on particle filters and EM algorithm. In: 2013 International Conference on Future Software Engineering and Multimedia Engineering 6: 63–71. https://doi.org/10.1016/j.ieri.2014.03.011

  23. Loshchilov I, Schoenauer M, Sebag M (2012) Self-adaptive surrogate-assisted covariance matrix adaptation evolution strategy. In: Proceedings of the 2012 international conference on genetic and evolutionary computation. https://doi.org/10.1145/2330163.2330210

  24. Masato K, Hidetoshi M (2020) The limiting distribution of combining the t and Wilcoxon rank sum tests. Stat 54(4):871–884. https://doi.org/10.1080/02331888.2020.1809662

    MathSciNet  Article  MATH  Google Scholar 

  25. Mohamed WA (2017) A novel differential evolution algorithm for solving constrained engineering optimization problems. J Intell Manuf 29(3):659–692. https://doi.org/10.1007/s10845-017-1294-6

    Article  Google Scholar 

  26. Nguyen DD, Long N (2021) An adaptive control for surrogate assisted multi-objective evolutionary algorithms. pp: 123–132. https://doi.org/10.1007/978-981-15-8289-9_12

  27. Pan JS, Liu NX, Chu SC, Lai TT (2021) An efficient surrogate-assisted hybrid optimization algorithm for expensive optimization problems. Inf Sci 561:304–325. https://doi.org/10.1016/j.ins.2020.11.056

    Article  Google Scholar 

  28. Pan LQ, He C, Tian Y, Wang HD, Zhang XY, Jin YC (2019) A classification-based surrogate-assisted evolutionary algorithm for expensive many-objective optimization. IEEE Trans Evol Comput 23(1):74–88. https://doi.org/10.1109/TEVC.2018.2802784

    Article  Google Scholar 

  29. Regis RG (2014) Particle swarm with radial basis function surrogates for expensive black-box optimization. J Comput Sci 5(1):12–23. https://doi.org/10.1016/j.jocs.2013.07.004

    MathSciNet  Article  Google Scholar 

  30. Si T, Jana ND, Sil J (2011) Constrained function optimization using PSO with polynomial mutation. In: International conference on swarm 7076: 209-216

  31. Sun CL, Jin YC, Cheng R, Ding JL, Zeng JC (2017) Surrogate-assisted cooperative swarm optimization of high-dimensional expensive problems. IEEE Trans Evol Comput 21(4):644–660. https://doi.org/10.1109/TEVC.2017.2675628

    Article  Google Scholar 

  32. Sun CL, Jin YC, Zeng JC, Yu Y (2015) A two-layer surrogate-assisted particle swarm optimization algorithm. Soft Comput 19(6):1461–1475. https://doi.org/10.1007/s00500-014-1283-z

    Article  Google Scholar 

  33. Urquhart M, Ljungskog E, Sebben S (2020) Surrogate-based optimization using adaptively scaled radial basis functions. Appl Soft Comput 88:106050. https://doi.org/10.1016/j.asoc.2019.106050

    Article  Google Scholar 

  34. Wang HD, Jin YC, Sun CL, Doherty J (2019) Offline data-driven evolutionary optimization using selective surrogate ensembles. IEEE Trans Evol Comput 23(2):203–216. https://doi.org/10.1109/TEVC.2018.2834881

    Article  Google Scholar 

  35. Wang HD, Jin YC, Doherty J (2017) Committee-based active learning for surrogate-assisted particle swarm optimization of expensive problems. IEEE Trans Cybern 47(9):2664–2677. https://doi.org/10.1109/TCYB.2017.2710978

    Article  Google Scholar 

  36. Wu JL, Luo Z, Zhang N, Gao W (2018) A new sequential sampling method for constructing the high-order polynomial surrogate models. Eng Comput 35(2):529–564. https://doi.org/10.1108/EC-05-2016-0160

    Article  Google Scholar 

  37. Ye PC, Pan G, Dong ZM (2018) Ensemble of surrogate based global optimization methods using hierarchical design space reduction. Struct Multidiscip Optim 58(2):537–554. https://doi.org/10.1007/s00158-018-1906-6

    MathSciNet  Article  Google Scholar 

  38. Yondo R, Andrés E, Valero E (2018) A review on design of experiments and surrogate models in aircraft real-time and many-query aerodynamic analyses. Prog Aeosp Sci 96:23–61. https://doi.org/10.1016/j.paerosci.2017.11.003

    Article  Google Scholar 

  39. Zhai JY, Boukouvala F (2019) Nonlinear variable selection algorithms for surrogate modeling. AIChE J 65(8):e16601. https://doi.org/10.1002/aic.16601

    Article  Google Scholar 

  40. Zhang XY, Tian Y, Cheng R, Jin YC (2015) An efficient approach to non-dominated sorting for evolutionary multiobjective optimization. IEEE Trans Evol Comput 19(2):201–213. https://doi.org/10.1109/TEVC.2014.2308305

    Article  Google Scholar 

  41. Zhu H, Hu YM, Zhu WD (2019) A dynamic adaptive particle swarm optimization and genetic algorithm for different constrained engineering design optimization problems. Adv Mech Eng 11(3):1–27. https://doi.org/10.1177/1687814018824930

    Article  Google Scholar 

Download references

Acknowledgments

The authors of this paper appreciate the support from the National Natural Science Foundation of China (Project No. 21676086).

Author information

Affiliations

Authors

Contributions

Xuemei Li contributed to conceptualization, methodology, software, writing-reviewing and editing. Shaojun Li contributed to conceptualization, writing-reviewing, resources, supervision, project administration and funding acquisition.

Corresponding author

Correspondence to Shaojun Li.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Li, X., Li, S. An adaptive surrogate-assisted particle swarm optimization for expensive problems. Soft Comput 25, 15051–15065 (2021). https://doi.org/10.1007/s00500-021-06348-2

Download citation

Keywords

  • Surrogate-assisted evolutionary algorithm
  • Ensemble model
  • Radial basis functions
  • Particle swarm optimization