Abstract
A connection has recently been drawn between dynamic optimization and reinforcement learning problems as subsets of a broader class of sequential decision-making problems. We present a unified approach that enables the cross-pollination of ideas between established communities, and could help to develop rigorous methods for algorithm comparison and selection for real-world resource-constrained problems.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Notation has been adjusted to aid comparison to RLP (2).
- 2.
Maximization problems are considered without a loss of generality.
References
Oliveto, P.S., He, J., Yao, X.: Time complexity of evolutionary algorithms for combinatorial optimization: a decade of results. Int. J. Autom. Comput. 4(3), 281–293 (2007)
Nguyen, T., Yang, S., Branke, J.: Evolutionary dynamic optimization: a survey of the state of the art. Swarm Evol. Comput. 6, 1–24 (2012)
Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction, vol. 1. MIT press, Cambridge (1998)
Fu, H., Lewis, P.R., Sendhoff, B., Tang, K., Yao, X.: What are dynamic optimization problems? In: IEEE Congress on Evolutionary Computing (CEC), pp. 1550–1557 (2014)
Fu, H., Lewis, P.R., Yao, X.: A Q-learning based evolutionary algorithm for sequential decision making problems. In: Parallel Problem Solving from Nature (PPSN). VUB AI Lab (2014)
Wiering, M., van Otterlo, M.: Reinforcement Learning: State-of-the-Art, vol. 12. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-27645-3
Myers, P.L., Spencer, D.B.: Application of a multi-objective evolutionary algorithm to the spacecraft stationkeeping problem. Acta Astronautica 127, 76–86 (2016)
Tan, K.C., Cheong, C.Y., Goh, C.K.: Solving multiobjective vehicle routing problem with stochastic demand via evolutionary computation. Eur. J. Oper. Res. 177(2), 813–839 (2007)
Münst, W., Dannheim, C., Gay, N., Malnar, B., Al-mamun, M., Icking, C., Hagen, F.: Managing intersections in the cloud, pp. 329–334 (2015)
Fergus, R., Fei-Fei, L., Perona, P., Zisserman, A.: Learning object categories from Google’s image search. In: Tenth IEEE International Conference on Computer Vision, ICCV 2005, vol. 2, pp. 1816–1823. IEEE (2005)
Laumanns, M., Thiele, L., Deb, K., Zitzler, E.: Combining convergence and diversity in evolutionary multiobjective optimization. Evol. Comput. 10(3), 263–282 (2002)
Silver, D., Huang, A., Maddison, C.J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., Dieleman, S., Grewe, D., Nham, J., Kalchbrenner, N., Sutskever, I., Lillicrap, T., Leach, M., Kavukcuoglu, K., Graepel, T., Hassabis, D.: Mastering the game of Go with deep neural networks and tree search. Nature 529(7587), 484–489 (2016)
Jin, Y.J.Y., Branke, J.: Evolutionary optimization in uncertain environments-a survey. IEEE Trans. Evol. Comput. 9(3), 303–317 (2005)
Drugan, M.M.: Synergies between evolutionary algorithms and reinforcement learning. In: Proceedings of the Companion Publication of the 2015 Annual Conference on Genetic and Evolutionary Computation, GECCO Companion 2015, pp. 723–740. ACM (2015)
Eiben, A.E., Schoenauer, M.: Evolutionary computing. Inf. Process. Lett. 82(1), 1–6 (2002)
Soni, A., Lewis, P.R., Ekárt, A.: Offline and online time in sequential decision-making problems. In: IEEE CIDUE. IEEE Press (2016)
Uzor, C.J., Gongora, M., Coupland, S., Passow, B.N.: Real-world dynamic optimization using an adaptive-mutation compact genetic algorithm. In: 2014 IEEE Symposium on Computational Intelligence in Dynamic and Uncertain Environments (CIDUE), pp. 17–23. IEEE (2014)
Cruz, C., González, J.R., Pelta, D.A.: Optimization in dynamic environments: a survey on problems, methods and measures. Soft Comput. 15(7), 1427–1448 (2011)
Dearden, R., Friedman, N., Andre, D.: Model based Bayesian exploration. In: Proceedings of the Fifteenth Conference on Uncertainty in Artificial Intelligence, pp. 150–159. Morgan Kaufmann Publishers Inc. (1999)
Piatkowski, N., Lee, S., Morik, K.: Integer undirected graphical models for resource-constrained systems. Neurocomputing 173, 9–23 (2016)
Graves, A.: Adaptive computation time for recurrent neural networks. arXiv preprint arXiv:1603.08983 (2016)
Hutter, F., Xu, L., Hoos, H.H., Leyton-Brown, K.: Algorithm runtime prediction: methods & evaluation. In: International Joint Conference on Artificial Intelligence (IJCAI), pp. 4197–4201, January 2015
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG, part of Springer Nature
About this paper
Cite this paper
Soni, A., Lewis, P.R., Ekárt, A. (2018). Synergies Between Reinforcement Learning and Evolutionary Dynamic Optimisation. In: Lewis, P., Headleand, C., Battle, S., Ritsos, P. (eds) Artificial Life and Intelligent Agents. ALIA 2016. Communications in Computer and Information Science, vol 732. Springer, Cham. https://doi.org/10.1007/978-3-319-90418-4_7
Download citation
DOI: https://doi.org/10.1007/978-3-319-90418-4_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-90417-7
Online ISBN: 978-3-319-90418-4
eBook Packages: Computer ScienceComputer Science (R0)