Abstract
In this chapter we give an overview of the consensus-based global optimization algorithm and its recent variants. We recall the formulation and analytical results of the original model, and then we discuss variants using component-wise independent or common noise. In combination with mini-batch approaches those variants were tailored for machine learning applications. Moreover, it turns out that the analytical estimates are dimension independent, which is useful for high-dimensional problems. We discuss the relationship of consensus-based optimization with particle swarm optimization, a method widely used in the engineering community. Then we survey a variant of consensus-based optimization that is proposed for global optimization problems constrained to hyper-surfaces. We conclude the chapter with remarks on applications, preprints and open problems.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Mohan, B. C. and Baskaran, R.: A survey: Ant colony optimization-based recent research and implementation on several engineering domain. Expert Syst. Appl. 39, 4618–4627 (2012)
Karaboga, D., Gorkemli, B., Ozturk, C., and Karaboga, N.: A comprehensive survey: Artificial bee colony (ABC) algorithm and applications. Artif. Intell. Rev. 42, 21–57 (2014)
Yang, X.-S.: Firefly Algorithms for Multimodal Optimization. In: Stochastic Algorithms: Foundations and Applications, SAGA 2009, Lecture Notes in Computer Sciences, Vol. 5792, pp. 169–178 (2009)
Jamil, M. and Yang, X.-S.: A literature survey of benchmark functions for global optimisation problems. Int. J. Math. Model. Numer. Optim. 4, 150–194 (2013)
Bayraktar, Z., Komurcu, M. Bossard, J.A. and Werner, D.H.: The Wind Driven Optimization Technique and its Application in Electromagnetics. In: IEEE Transactions on Antennas and Propagation 61, 2745–2757 (2013)
Henderson, D., Jacobson, S.H. and Johnson, A.W.: The theory and practice of simulated annealing. In: Handbook of Metaheuristics, International Series in Operations Research & Management Science, 57 287–319, Springer, Boston (2003)
Monmarché, P.: Hypocoercivity in metastable settings and kinetic simulated annealing, Probab. Theory Relat. Fields 172,1215–1248 (2018)
Chak, M., Kantas, N. and Pavliotis, G.A.: On the Generalised Langevin Equation for Simulated Annealing, arXiv:2003.06448v3 (2021)
Kennedy, J. and Eberhart, R.C.: Particle Swarm Optimization. Proc. IEEE Int. Conf. Neu. Net. 4, 1942–1948 (1995)
Hegselmann, R. and Krause, U.: Opinion dynamics and bounded confidence models, analysis, and simulation. J. Artif. Soc. Social Simulat. 5, 1–33 (2002)
Pinnau, R., Totzeck, C., Tse, O. and Martin, S.: A consensus-based model for global optimization and its mean-field limit. Math. Meth. Mod. Appl. Sci. 27,183–204 (2017)
Carrillo, J.A., Choi, Y.-P., Totzeck, C. and Tse, O.: An analytical framework for a consensus-based global optimization method. Math. Mod. Meth. Appl. Sci. 28, 1037–1066 (2018)
Carrillo, J.A., Jin, S., Li, L. and Zhu, Y.: A consensus-based global optimization method for high dimensional machine learning problems. ESAIM: COCV 27, S5 (2021)
Ha, S.-Y. and Jin, S. and Kim, D.: Convergence of a first-order consensus-based global optimization algorithm. Math. Mod. Meth. Appl. Sci. 30, 2417–2444 (2020)
Totzeck, C and Wolfram, M.-T.: Consensus-Based Global Optimization with Personal Best. Math. Biosci. Eng. 17, 6026–6044 (2020)
Fornasier, M. and Huang, H. and Pareschi, L. and Sünnen, P.: Consensus-based optimization on hypersurfaces: well-posedness and mean-field limit. Math. Mod. Meth. Appl. Sci. 30, 2725–2751 (2020)
Dembo, A. and Zeitouni, O.: Large Deviations Techniques and Applications, Applications of Mathematics Vol. 38, Springer Science and Business Media (2009)
Higham, D.J.: An Algorithmic Introduction to Numerical Simulation of Stochastic Differential Equations. SIAM Rev. 43, 525–546 (2001)
Robbins, H. and Monro, S.: A stochastic approximation method. Ann. Math. Stat. 22, 400–407 (1951)
Jin, S. and Li, L., Liu, J.-G.: Random batch methods (RBM) for interacting particle systems. J. Comput. Phys. 400, 108877 (2020)
Ha, S.-Y. and Jin, S. and Kim, D.: Convergence and error estimates for time-discrete consensus-based optimization algorithms. Numer. Math. 147, 255–282 (2021)
Poli, R. and Kennedy, J. and Blackwell, T.: Particle swarm optimization - An overview. Swarm Intell. 1, 33–57 (2007)
Grassi, S. and Pareschi, L.: From particle swarm optimization to consensus based optimization: stochastic modeling and mean-field limit. arXiv:2012.05613 (2020)
Fornasier, M. and Huang, H. and Pareschi, L. and Sünnen, P.: Consensus-based optimization on hypersurfaces: well-posedness and mean-field limit. arXiv:2001.11988 (2020)
Sznitman, A.-S.: Topics in propagation of chaos. In: Ecole d’été de probabilités de Saint-Flour XIX – 2089, 165–251. Springer (1991)
Totzeck, C., Pinnau, R., Blauth, S. and Schotthöfer, S.: A Numerical Comparison of Consensus-Based Global Optimization to other Particle-based Global Optimization Schemes. PAMM 18, e201800291 (2018)
Golse, F.: On the Dynamics of Large Particle Systems in the Mean Field Limit. In: Macroscopic and Large Scale Phenomena: Coarse Graining, Mean Field Limits and Ergodicity, pp. 1–144. Springer (2016)
Chen, J., Jin, S. and Lyu, L.: A Consensus-based global optimization method with adaptive momentum estimation, arXiv:2012.04827 (2020)
Huang, H. and Qiu, J.: On the mean-field limit for the consensus-based optimization. arXiv:2105.12919v1 (2021)
Fornasier, M., Klock, T. and Riedl, K.: Consensus-based optimization methods converge globally in mean-field law, arXiv:2103.15130v3 (2021)
Carrillo, J.A., Hoffmann, F., Stuart, A.M. and Vaes, U.: Consensus Based Sampling. arXiv:2106.02519v1 (2021)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this chapter
Cite this chapter
Totzeck, C. (2022). Trends in Consensus-Based Optimization. In: Bellomo, N., Carrillo, J.A., Tadmor, E. (eds) Active Particles, Volume 3. Modeling and Simulation in Science, Engineering and Technology. Birkhäuser, Cham. https://doi.org/10.1007/978-3-030-93302-9_6
Download citation
DOI: https://doi.org/10.1007/978-3-030-93302-9_6
Published:
Publisher Name: Birkhäuser, Cham
Print ISBN: 978-3-030-93301-2
Online ISBN: 978-3-030-93302-9
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)