Optimization of Gaussian Process Models with Evolutionary Algorithms
Gaussian process (GP) models are non-parametric, black-box models that represent a new method for system identification. The optimization of GP models, due to their probabilistic nature, is based on maximization of the probability of the model. This probability can be calculated by the marginal likelihood. Commonly used approaches for maximizing the marginal likelihood of GP models are the deterministic optimization methods. However, their success critically depends on the initial values. In addition, the marginal likelihood function often has a lot of local minima in which the deterministic method can be trapped. Therefore, stochastic optimization methods can be considered as an alternative approach. In this paper we test their applicability in GP model optimization. We performed a comparative study of three stochastic algorithms: the genetic algorithm, differential evolution, and particle swarm optimization. Empirical tests were carried out on a benchmark problem of modeling the concentration of CO2 in the atmosphere. The results indicate that with proper tuning differential evolution and particle swarm optimization significantly outperform the conjugate gradient method.
KeywordsGaussian process models hyperparameters optimization evolutionary algorithms
Unable to display preview. Download preview PDF.
- 1.Birge, B.: Matlab Central: Particle swarm optimization toolbox, http://www.mathworks.com/matlabcentral/fileexchange/7506-particle-swarm-optimization-toolbox
- 2.Carbon Dioxide Information Analysis Center. Atmospheric CO 2 values collected at Mauna Loa, Hawaii, USA, http://cdiac.esd.ornl.gov/ftp/trends/co2/maunaloa.co2
- 5.Kennedy, J., Eberhart, R., Shi, Y.: Swarm Intelligence. Morgan Kaufmann, San Francisco (2001)Google Scholar
- 6.Pohlheim, H.: Geatbx – The Genetic and Evolutionary Algorithm Toolbox for Matlab, http://www.geatbx.com/
- 8.Rassmusen, C.E., Williams, C.K.I.: Gaussian Processes for Machine Learning. The MIT Press, Cambridge (2006)Google Scholar
- 9.Storn, R., Price, K.: Differential evolution – A Simple and Efficient Heuristic for Global Optimization over Continuous Spaces. Journal of Global Optimization (11), 341–359 (1997)Google Scholar