Abstract
This paper presents the multi-recombinative constraint active matrix adaptation evolution strategy (Constraint Active-MA-ES). It extends the MA-ES recently introduced by Beyer and Sendhoff in order to handle constrained black-box optimization problems. The active covariance matrix adaptation approach for constraint handling similar to the method proposed by Arnold and Hansen for the \((1+1)\) covariance matrix adaptation evolution strategy is used. As a first step toward constraint handling, active covariance matrix adaptation is incorporated into the MA-ES and evaluated on unconstrained problems (Active-MA-ES). As the second step, constraint handling based on active covariance matrix adaptation for the MA-ES is proposed (Constraint Active-MA-ES). The algorithm has been tested on different test functions and it has been compared to other methods. The experiments show that for cases where directional sampling of the offspring mutations is beneficial, the Active-MA-ES can reach the target faster than the MA-ES. In particular, the Active-MA-ES reaches the final target precision on average by a factor of 1.4 generations (Ellipsoid), a factor of 1.6 generations (Different Powers), and a factor of 2.0 generations (Tablet) faster than the MA-ES. The experiments for the Constraint Active-MA-ES reveal that it achieves 80% of the considered targets with about \(N \times 10^5\) function and constraint evaluations. With this result, it is the best method among the compared approaches.
Similar content being viewed by others
Notes
We have also empirically investigated the effect of using negative weights for the next generation’s parental candidate solution update (Line 16 in Algorithm 1) as well as for the \(\mathbf {s}\)-update (Line 17 in Algorithm 1). To this end, we have first used an approximation for the optimal weights computed by Arnold (2006) for the infinite-dimensional sphere model and second the weights proposed by Hansen (2016). It turned out that for the dimensions considered, negative weights in the centroid computation and the \(\mathbf {s}\)-update are not beneficial.
Note that by distributing M over the terms in the brackets, the matrix–matrix multiplication is not necessary. It can thus be implemented in a running time of \(O(\lambda N^2)\).
https://www.lri.fr/~hansen/purecmaes.m (with last change on April, 29, 2014). The pure CMA-ES implementation was used instead of the production CMA-ES implementation because the latter code contains ad hoc solutions for improving the performance of the CMA-ES.
Technically, Matlab’s implementation in the function ranksum has been used.
We provide additional experimental results in “Appendix” (Sect. 6.2). There, the results are presented in more detail. In particular, the results are shown for every function separately.
In the BBOB COCO framework, one call to the constraint evaluation function yields the values of all the constraints at the given query point.
Because the bbob-constrained suite has still been under development during creation of this manuscript, we provide a fork in https://github.com/patsp/coco. This makes our results reproducible. Even though it is not the final BBOB COCO constrained framework, it gives a good indication of the algorithm’s performance in comparison with other methods. The changes we made are in our fork in the new branch development-sppa-2. This branch is based on the development branch of https://github.com/numbbo/coco with changes up to and including Dec 10, 2017.
V. Klee and G. J. Minty, “How good is the simplex algorithm?” in Inequalities, O. Shisha, Ed. New York: Academic Press, 1972, vol. III, pp. 159–175.
References
Arnold DV (2006) Weighted multirecombination evolution strategies. Theor Comput Sci 361(1):18–37
Arnold DV (2016) An active-set evolution strategy for optimization with known constraints. In: International conference on parallel problem solving from nature. Springer, pp 192–202
Arnold DV (2017) Reconsidering constraint release for active-set evolution strategies. In: Proceedings of the genetic and evolutionary computation conference, GECCO ’17. ACM, New York, NY, USA, pp 665–672. https://doi.org/10.1145/3071178.3071294
Arnold DV, Hansen N (2012) A (1 + 1)-CMA-ES for constrained optimisation. In: Proceedings of the 14th annual conference on genetic and evolutionary computation. ACM, pp 297–304
Arnold DV, Porter J (2015) Towards an augmented Lagrangian constraint handling approach for the (1 + 1)-ES. In: Proceedings of the 2015 annual conference on genetic and evolutionary computation, GECCO ’15. ACM, New York, NY, USA, pp 249–256. https://doi.org/10.1145/2739480.2754813
Atamna A, Auger A, Hansen N (2016) Augmented Lagrangian constraint handling for CMA-ES—case of a single linear constraint. In: International conference on parallel problem solving from nature. Springer, pp 181–191
Atamna A, Auger A, Hansen N (2017) Linearly convergent evolution strategies via augmented Lagrangian constraint handling. In: Proceedings of the 14th ACM/SIGEVO conference on foundations of genetic algorithms. ACM, pp 149–161
Banks A, Vincent J, Anyakoha C (2007) A review of particle swarm optimization. Part I: background and development. Nat Comput 6(4):467–484
Banks A, Vincent J, Anyakoha C (2008) A review of particle swarm optimization. Part II: hybridisation, combinatorial, multicriteria and constrained optimization, and indicative applications. Nat Comput 7(1):109–124
Beyer HG (1989) Ein Evolutionsverfahren zur mathematischen Modellierung stationärer Zustände in dynamischen Systemen. Hochschule für Architektur und Bauwesen, Weimar
Beyer HG, Finck S (2012) On the design of constraint covariance matrix self-adaptation evolution strategies including a cardinality constraint. IEEE Trans Evol Comput 16(4):578–596
Beyer HG, Sendhoff B (2017) Simplify your covariance matrix adaptation evolution strategy. IEEE Trans Evol Comput 21(5):746–759
Brockhoff D, Hansen N, Mersmann O, Ros R, Tusar D, Tusar T (2017) COCO documentation repository. http://github.com/numbbo/coco-doc
Chocat R, Brevault L, Balesdent M, Defoort S (2015) Modified covariance matrix adaptation—evolution strategy algorithm for constrained optimization under uncertainty, application to rocket design. Int J Simul Multidiscip Des Optim 6:A1
Conn AR, Scheinberg K, Vicente LN (2009) Introduction to derivative-free optimization. SIAM, Philadelphia
Eaton JW, Bateman D, Hauberg S, Wehbring R (2016) GNU Octave version 4.2.0 manual: a high-level interactive language for numerical computations. http://www.gnu.org/software/octave/doc/interpreter
Hansen N (2016) The CMA evolution strategy: a tutorial. CoRR arXiv:1604.00772
Hansen N, Auger A, Brockhoff D, Tusar D, Tusar T (2016) COCO: performance assessment. ArXiv e-prints arXiv:1605.03560
Huang VL, Qin AK, Suganthan PN (2006) Self-adaptive differential evolution algorithm for constrained real-parameter optimization. In: IEEE congress on evolutionary computation (CEC). IEEE, pp 17–24
Jamal MB, Ming F, Zhengang J (2018) Solving constrained optimization problems by using covariance matrix adaptation evolutionary strategy with constraint handling methods. In: Proceedings of the 2nd international conference on innovation in artificial intelligence, ICIAI ’18. ACM, New York, NY, USA, pp 6–15. https://doi.org/10.1145/3194206.3194215
Jastrebski GA, Arnold DV (2006) Improving evolution strategies through active covariance matrix adaptation. In: 2006 IEEE international conference on evolutionary computation, pp 2814–2821
Kolda TG, Lewis RM, Torczon V (2003) Optimization by direct search: new perspectives on some classical and modern methods. SIAM Rev 45(3):385–482
Kramer O (2010) A review of constraint-handling techniques for evolution strategies. Appl Comput Intell Soft Comput
Kusakci AO, Can M (2013) An adaptive penalty based covariance matrix adaptation-evolution strategy. Comput Oper Res 40(10):2398–2417. https://doi.org/10.1016/j.cor.2013.03.013. http://www.sciencedirect.com/science/article/pii/S03050548HrB13000890HrB
Loshchilov I, Glasmachers T, Beyer HG (2018) Large scale black-box optimization by limited-memory matrix adaptation. IEEE Trans Evol Comput 1. https://doi.org/10.1109/TEVC.2018.2855049
Mallipeddi R, Suganthan PN (2010) Differential evolution with ensemble of constraint handling techniques for solving CEC 2010 benchmark problems. In: IEEE congress on evolutionary computation (CEC). IEEE, pp 1–8
Mezura-Montes E, Coello CAC (2005) A simple multimembered evolution strategy to solve constrained optimization problems. IEEE Trans Evol Comput 9(1):1–17. https://doi.org/10.1109/TEVC.2004.836819
Mezura-Montes E, Coello CAC (2011) Constraint-handling in nature-inspired numerical optimization: past, present and future. Swarm Evol Comput 1(4):173–194
Oyman AI, Deb K, Beyer HG (1999) An alternative constraint handling method for evolution strategies. In: Angeline P (ed) Proceedings of the CEC’99 conference. IEEE, Piscataway, NJ, pp 612–619
Poláková R (2017) L-SHADE with competing strategies applied to constrained optimization. In: 2017 IEEE congress on evolutionary computation (CEC), pp 1683–1689. https://doi.org/10.1109/CEC.2017.7969504
Rechenberg I (1973) Evolutionsstrategie Optimierung technischer Systeme nach Prinzipien der biologishen Evolution. Frommann-Holzboog, Stuttgart
Runarsson TP, Yao X (2000) Stochastic ranking for constrained evolutionary optimization. IEEE Trans Evol Comput 4(3):284–294. https://doi.org/10.1109/4235.873238
Sakamoto N, Akimoto Y (2017) Modified box constraint handling for the covariance matrix adaptation evolution strategy. In: Proceedings of the genetic and evolutionary computation conference companion, GECCO ’17. ACM, New York, NY, USA, pp 183–184. https://doi.org/10.1145/3067695.3075986
Schwefel HP (1981) Numerical optimization of computer models. Wiley, Hoboken
Takahama T, Sakai S (2010) Constrained optimization by the \(\varepsilon \) constrained differential evolution with an archive and gradient-based mutation. In: IEEE congress on evolutionary computation (CEC). IEEE, pp 1–9
Trosset MW (1997) I know it when I see it: toward a definition of direct search methods. SIAG/OPT Views News Forum SIAM Act Group Optim 9:7–10
Wright MH (1996) Direct search methods: once scorned, now respectable. In: Griffiths DF, Watson GA (eds) Numerical analysis 1995 (proceedings of the 1995 Dundee Biennial conference in numerical analysis), Pitman research notes in mathematics, vol 344. CRC Press, Boca Raton, pp 191–208
Acknowledgements
This work was supported by the Austrian Science Fund FWF under Grant P29651-N32.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Ethical approval
This article does not contain any studies with human participants or animals performed by any of the authors.
Additional information
Communicated by A. Di Nola.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
6 Appendix
6 Appendix
This appendix contains further material accompanying the main paper. It is organized as follows: Section 6.1 shows the derivation of the \(\mathbf {M}\)-update for the infeasible mutations. Section 6.2 presents additional experimentation results, and Sect. 6.3 summarizes the changes we performed in the BBOB COCO framework.
1.1 6.1 Derivation of \(\mathbf {M} \leftarrow \mathbf {M}\left( \mathbf {I} - \beta \mathbf {v}_j\mathbf {v}_j^\mathrm{T}\right) \)
The covariance matrix \(\mathbf {C}\) is factored into
With this, Lines 12 and 13 of Algorithm 2 correspond to sampling the mutation from \(\mathcal {N}(\mathbf {0}, \mathbf {C})\). Hence, for taking into account the mutations of infeasible offspring in the active covariance adaptation, we aim for
where
That is, the vectors \(\mathbf {w}_j\) are the fading records of the violation vectors corresponding to the mutations that are denoted \(\tilde{\mathbf {d}}\) (cf. Line 13 of Algorithm 2). Notice that the sum in Eq. (9) can also be written as a for loop
Now, to obtain Eq. (5), the derivations are very similar to those in Beyer and Sendhoff (2017), Eqs. (25)–(30). In the following, let j be the index of a violated constraint. The starting point of the derivation is
Taking into account Eqs. (8) and (10),
follows. Pulling out \(\mathbf {M}^{(g)}\) and \(\left( {\mathbf {M}^{(g)}}\right) ^\mathrm{T}\) in Eq. (13) results in
Expansion of Eq. (14) into a matrix power series and cutoff after the linear term analogously to Beyer and Sendhoff (2017), Eqs. (27)–(30) yields
which corresponds to Eq. (5) with neglecting the higher-order terms.
1.2 6.2 Constraint active-\((\mu /\mu _w,\lambda )\)-MA-ES experimental evaluation: presentation of additional results
Figures 5 and 6 show the ECDF plots of the Constraint Active-\((\mu /\mu _w, \lambda )\)-MA-ES for all the functions and dimensions in the BBOB COCO constrained suite. Figures 7 and 8 show the aRT plots.
Tables 4 show the performance in tabular form of the Constraint Active-\((\mu /\mu _w, \lambda )\)-MA-ES in comparison with the conSaDE, the \(\varepsilon \)DEag, and the \((1+1)\)-CMA-ES for constrained optimization for dimension 40.
1.3 6.3 BBOB COCO adaptations
Work has already been done for a constrained test suite in the BBOB COCO framework. But some changes of the BBOB COCO framework were necessary to perform the experiments. The GitHub repository of the original BBOB COCO framework is here: https://github.com/numbbo/coco. A fork was created for this work in https://github.com/patsp/coco. The adaptations are in a new branch called development-sppa-2. The main changes are summarized here. For all the details, it is referred to the GitHub repository https://github.com/patsp/coco. The commit logs show all the changes in detail.Footnote 8
-
Log of the relative instead of the absolute objective function distance to the optimum (in code-experiments/src/logger_bbob.c): this is more informative because of the easier comparison of large objective function values with small ones. In the original code, the value \(f - f_{\mathrm{opt}}\) is logged. This change is done to log the value \(|f - f_{\mathrm{opt}}| / |f_{\mathrm{opt}}|\). The division is only done if \(|f_{\mathrm{opt}}|\) is not too small; currently, \(|f_{\mathrm{opt}}| > 10^{-5}\) must hold for the division to be done. Absolute values are used to avoid negative values.
-
Possibility to disable the asymmetric and oscillation nonlinear transformations for the bbob-constrained test suite (in code-experiments/src/transform_vars_oscillate.c and code-experiments/src/transform_vars_asymmetric.c). Currently, this is done with compile-time definitions ENABLE_NON_LINEAR_TRANSFORMATIONS_ON_CONSTRAINTS and ENABLE_NON_LINEAR_TRANSFORMATIONS_ON_OBJECTIVEFUNC that enable the oscillation and asymmetry transformations for the constraint functions and objective function, respectively. This means that by default the transformations are disabled in the current changed version.
-
Addition of a new test suite called custom. It contains at the time of writing this document only the Klee-Minty problemFootnote 9 (new file code-experiments/src/suite_custom.c and changes in code-experiments/src/coco_suite.c).
-
Python post-processing module: Support for the new custom test suite.
-
Python post-processing module: Negative values on the x-axis of the ECDF-plots are clipped to 0. It is possible that negative values occur in the x-axis of the ECDF-plots. This is due to the division by the dimension. It occurs if the value of function and constraint evaluations is less than the dimension because then the value after division is less than 1. The log is taken, and thus, we have a negative value.
-
Python post-processing module: Smaller font size in the plots.
About this article
Cite this article
Spettel, P., Beyer, HG. A multi-recombinative active matrix adaptation evolution strategy for constrained optimization. Soft Comput 23, 6847–6869 (2019). https://doi.org/10.1007/s00500-018-03736-z
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00500-018-03736-z