Runtime Analysis of \((1+1)\) Evolutionary Algorithm Controlled with Q-learning Using Greedy Exploration Strategy on OneMax+ZeroMax Problem
There exist optimization problems with the target objective, which is to be optimized, and several extra objectives. The extra objectives may or may not be helpful in optimization process in terms of the number of objective evaluations necessary to reach an optimum of the target objective.
OneMax+ZeroMax is a previously proposed benchmark optimization problem where the target objective is OneMax and a single extra objective is ZeroMax, which is equal to the number of zero bits in the bit vector. This is an example of a problem where extra objectives are not good, and objective selection methods should ignore the extra objectives. The EA+RL method is a method which selects objectives to be optimized by evolutionary algorithms (EA) using reinforcement learning (RL). Previously it was shown that it runs in \(\varTheta (N \log N)\) on OneMax+ZeroMax when configured to use the randomized local search algorithm and the Q-learning algorithm with the greedy exploration strategy.
We present the runtime analysis for the case when the \((1+1)\)-EA algorithm is used. It is shown that the expected running time is at most \(3.12 e N \log N\).
- 2.Buzdalov, M., Buzdalova, A., Shalyto, A.: A first step towards the runtime analysis of evolutionary algorithm adjusted with reinforcement learning. In: Proceedings of the International Conference on Machine Learning and Applications, vol. 1, pp. 203–208. IEEE Computer Society (2013)Google Scholar
- 3.Buzdalova, A., Buzdalov, M.: Increasing efficiency of evolutionary algorithms by choosing between auxiliary fitness functions with reinforcement learning. In: Proceedings of the International Conference on Machine Learning and Applications, vol. 1, pp. 150–155 (2012)Google Scholar
- 10.Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (1998)Google Scholar
- 11.Witt, C.: Optimizing linear functions with randomized search heuristics - the robustness of mutation. In: Proceedings of the 29th Annual Symposium on Theoretical Aspects of Computer Science, pp. 420–431 (2012)Google Scholar