Abstract
The optimization problems and algorithms are the basics subfield in artificial intelligence, which is booming in the almost any industrial field. However, the computational cost is always the issue which hinders its applicability. This paper proposes a novel hybrid optimization algorithm for solving expensive optimizing problems, which is based on particle swarm optimization (PSO) combined with Gaussian process (GP). In this algorithm, the GP is used as an inexpensive fitness function surrogate and a powerful tool to predict the global optimum solution for accelerating the local search of PSO. In order to improve the predictive capacity of GP, the training datasets are dynamically updated through sorting and replacing the worst fitness function solution with the better solution during the iterative process. A numerical study is carried out using twelve different benchmark functions with 10, 20 and 30 dimensions, respectively. Regarding solving of the ill-conditioned computationally expensive optimization problems, results show that the proposed algorithm is much more efficient and suitable than the standard PSO alone.
Article PDF
Avoid common mistakes on your manuscript.
References
A.R.M. Rao, K. Sivasubramanian, Multi-objective optimal design of fuzzy logic controller using a self configurable swarm intelligence algorithm, Comput. Struct. 86 (2008), 2141–2154
A.H. Gandomi, A.R. Kashani, D.A. Roke, M. Mousavi, Optimization of retaining wall design using recent swarm intelligence techniques, Eng. Struct. 103 (2015), 72–84
E. Bonabeau, D.D.R.D.F. Marco, M. Dorigo, G. Theraulaz, Swarm Intelligence: From Natural to Artificial Systems, 1999.
J. Liao, L. Tang, G. Shao, Q. Qiu, C. Wang, S. Zheng, X. Su, A neighbor decay cellular automata approach for simulating urban expansion based on particle swarm intelligence, Int. J. Geogr. Inf. Sci. 28 (2014), 720–738
Z. Cui, B. Sun, G. Wang, Y. Xue, J. Chen, A novel oriented cuckoo search algorithm to improve DV-Hop performance for cyber-physical systems, J. Parallel Distr. Com. 103 (2017), 42–52
M. Zhang, H. Wang, Z. Cui, J. Chen, Hybrid multi-objective cuckoo search with dynamical local search, Memet. Comput. 10 (2018), 199–208
X. Cai, X. Gao, Y. Xue, Improved bat algorithm with optimal forage strategy and random disturbance strategy, Int. J. Bio-Inspir. Com. 8 (2016), 205–214
X. Cai, H. Wang, Z. Cui, J. Cai, Y. Xue, L. Wang, Bat algorithm with triangle-flipping strategy for numerical optimization, Int. J. Mach. Learn. Cyb. 9 (2018), 199–215
M. Celik, F. Koylu, D. Karaboga, CoABCMiner: an algorithm for cooperative rule classification system based on artificial bee colony, Int. J. Artif. Intell. T. 25 (2016), 1550028.
H.M. Jiang, K. Xie, Y.F. Wang, Optimization of pump parameters for gain flattened Raman fiber amplifiers based on artificial fish school algorithm, Opt. Commun. 284 (2011), 5480–5483
M. Dorigo, V. Maniezzo, A. Colorni, Ant system: optimization by a colony of cooperating agents, IEEE Trans. Syst. Man. Cybern. Part B-Cybern. 26 (1996), 29–41
R. Eberhart, J. Kennedy, New optimizer using particle swarm theory, in MHS’95. Proceedings of the Sixth International Symposium on Micro Machine and Human Science, Nagoya, Japan, 1995, pp. 39–43.
M.M. Ali, P. Kaelo, Improved particle swarm algorithms for global optimization, Appl. Math. Comput. 196 (2008), 578–593
M.M. Noel, A new gradient based particle swarm optimization algorithm for accurate computation of global minimum, Appl. Soft Comput. 12 (2012), 353–359
M. Sopa, N. Angkawisittpan, An application of cuckoo search algorithm for series system with cost and multiple choices constraints, Procedia Comput. Sci. 86 (2016), 453–456
T. Gaber, S. Abdelwahab, M. Elhoseny, A.E. Hassanien, Trust-based secure clustering in WSN-based intelligent transportation systems. Comput. Netw. 146 (2018), 151–158
J. Kennedy, R. Eberhart, Particle swarm optimization (PSO), in Proceedings of IEEE International Conference on Neural Networks, Piscataway, NJ, 1995, pp. 1942–1948.
D.E. Goldberg, GA in Search, Optimization, and Machine Learning, Addison-Wesley, Boston, MA, 1989.
R. Hassan, B. Cohanim, O. De Weck, G. Venter, A comparison of particle swarm optimization and the genetic algorithm, in 46th AIAA/ASME/ASCE/AHS/ASC structures, structural dynamics and materials conference, Austin, TX, 2005, pp. 1897.
D. Wang, Z. Wu, Y. Fei, W. Zhang, Structural design employing a sequential approximation optimization approach, Comput. Struct. 134 (2014), 75–87
N.H. Awad, M.Z. Ali, R. Mallipeddi, P.N. Suganthan, An improved differential evolution algorithm using efficient adapted surrogate model for numerical optimization, Inf. Sci. 451 (2018), 326–347
G. Su, Q. Jiang, A cooperative optimization algorithm based on gaussian process and particle swarm optimization for optimizing expensive problems, in 2009 International Joint Conference on Computational Sciences and Optimization, IEEE, Hainan, Sanya, China, 2009, Vol. 2, pp. 929–933.
K. Mistry, L. Zhang, S.C. Neoh, C.P. Lim, B. Fielding, A micro-GA embedded PSO feature selection approach to intelligent facial emotion recognition, IEEE Trans. Cybern. 47 (2016), 1496–1509
H. Garg, A hybrid GSA-GA algorithm for constrained optimization problems, Inf. Sci. 478 (2019), 499–523
L.Y. Chuang, H.W. Chang, C.J. Tu, C.H. Yang, Improved binary PSO for feature selection using gene expression data, Comput. Biol. Chem. 32, (2008), 29–38.
C. Praveen, R. Duvigneau, Low cost PSO using metamodels and inexact pre-evaluation: application to aerodynamic shape design, Comput. Methods Appl. Mech. Eng. 198 (2009), 1087–1096
S. Selleri, M. Mussetta, P. Pirinoli, R.E. Zichm, L. Matekovits, Differentiated meta-PSO methods for array optimization, IEEE Trans. Antennas Propag. 56 (2008), 67–75
M.V.J.J. Suresh, K.S. Reddy, A.K. Kolar, ANN-GA based optimization of a high ash coal-fired supercritical power plant, Appl. Energ. 88 (2011), 4867–4873
A. Ratle, Accelerating the convergence of evolutionary algorithms by fitness landscape approximation, in International Conference on Parallel Problem Solving from Nature, Springer, Berlin, Germany, 1998, pp. 87–96.
K.C. Giannakoglou, Optimization and inverse design in aeronautics: how to couple genetic algorithms with radial basis function networks, in: J. Periaux, P. Joly, O. Pironneau, E. Onate (Eds.), Innovative Tools for Scientific Computation in Aeronautical Engineering, CIMNE, Barcelona, Spain, 2001.
Y. Jin, A comprehensive survey of fitness approximation in evolutionary computation, Soft Comput. 9 (2005), 3–12
Y. Jin, Surrogate-assisted evolutionary computation: recent advances and future challenges, Swarm Evol. Comput. 1 (2011), 61–70
J. Kacprzyk, Advances in Soft Computing, Heidelberg, 2001.
Y. Zhang, Y. Jun, G. Wei, L. Wu, Find multi-objective paths in stochastic networks via chaotic immune PSO, Expert Syst. Appl. 37 (2010), 1911–1919
M.T.M. Emmerich, K.C. Giannakoglou, B. Naujoks, Single-and multiobjective evolutionary optimization assisted by Gaussian random field metamodels, IEEE Trans. Evol. Comput. 10 (2006), 421–439
J. Knowles, ParEGO: a hybrid algorithm with on-line landscape approximation for expensive multiobjective optimization problems, IEEE Trans. Evolut. Comput. 10 (2006), 50–66
Q. Zhang, W. Liu, E. Tsang, B. Virginas, Expensive multiobjective optimization by MOEA/D with Gaussian process model, IEEE Trans. Evol. Comput. 14 (2009), 456–474
N. Higashi, H. Iba, Particle swarm optimization with Gaussian mutation, in Proceedings of the 2003 IEEE Swarm Intelligence Symposium. SIS’03 (Cat. No.03EX706), Indianapolis, IN, 2003, pp. 72–79.
R.A. Krohling, Gaussian Particle Swarm with jumps, in 2005 IEEE Congress on Evolutionary Computation, Edinburgh, Scotland, UK, 2005, pp. 1226–1231.
Y. Zhang, B. Liu, Y. Hou, Z. Zeng, An intelligent back analysis optimization method of constitutive parameters for surrounding rock, Unsaturated soil mechanics - from theory to practice, in Proceedings of the 6th Asia Pacific Conference on Unsaturated Soils, Guilin, China, 2015, pp. 601–605.
G. Wang, Z. Ma, Hybrid particle swarm optimization for first-order reliability method, Comput. Geotech. 81 (2017), 49–58
J. Kennedy, The behavior of particles, in: V.W. Porto, N. Saravanan, D. Waagen, A.E. Eiben (Eds.), Evolutionary Programming VII, Springer, Berlin, Heidelberg, 1998, pp. 581–589.
D.J.C. MacKay, Introduction to Gaussian processes, in: C.M. Bishop (Ed.), Neural Networks and Machine Learning, NATO ASI Series, Springer, Berlin, Germany, 1998, pp. 133–166.
G. Su, L. Yan, Y. Song, Gaussian process for non-linear displacement time series prediction of landslide, J. China Univ. Geosci. 18 (2007), 219–221
J. Hensman, R. Mills, S.G. Pierce, K. Worden, M. Eaton, Locating acoustic emission sources in complex structures using Gaussian processes, Mech. Syst. Signal Proc. 24 (2010), 211–223
M. Pal, S. Deswal, Modelling pile capacity using Gaussian process regression, Comput. Geotech. 37 (2010), 942–947
C.K.I. Williams, C.E.Rasmussen, Gaussian Processes for Machine Learning, Cambridge, MA, 2006.
G. Su, Gaussian process assisted differential evolution algorithm for computationally expensive optimization problems, in 2008 IEEE Pacific-Asia Workshop on Computational Intelligence and Industrial Application, Wuhan, China, 2008, pp. 272–276.
Y. Shi, H. Liu, L. Gao, G. Zhang, Cellular particle swarm optimization, Inf. Sci. 181 (2011), 4460–4493
Y. Wang, B. Li, T. Weise, J. Wang, B. Yuan, Q. Tian, Self-adaptive learning based particle swarm optimization, Inf. Sci. 181 (2011), 4515–4538
W. Zhang, Y. Liu, Reactive power optimization based on PSO in a practical power system, in IEEE Power Engineering Society General Meeting, Denver, CO, 2004, pp. 239–243.
Y. Shi, R.C. Eberhart, A modified particle swarm optimizer, in Proceedings IEEE Conference on Evolutionary Computation, Anchorage, AK, 1998.
J. Kennedy, R. Mendes, Population structure and particle swarm performance, in Proceedings of the 2002 Congress on Evolutionary Computation. CEC’02 (Cat. No.02TH8600), Honolulu, HI, 2002, pp.1671–1676.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
This is an open access article distributed under the CC BY-NC 4.0 license (http://creativecommons.org/licenses/by-nc/4.0/).
About this article
Cite this article
Zhang, Y., Li, H., Bao, E. et al. A Hybrid Global Optimization Algorithm Based on Particle Swarm Optimization and Gaussian Process. Int J Comput Intell Syst 12, 1270–1281 (2019). https://doi.org/10.2991/ijcis.d.191101.004
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.2991/ijcis.d.191101.004