Abstract
In this article, I will consider Markov Decision Processes with two criteria, each defined as the expected value of an infinite horizon cumulative return. The second criterion is either itself subject to an inequality constraint, or there is maximum allowable probability that the single returns violate the constraint. I describe and discuss three new reinforcement learning approaches for solving such control problems.
Chapter PDF
References
Altman, E.: Constrained Markov Decision Processes. Chapman and Hall/CRC, Boca Raton (1999)
Bertsekas, D.P.: Dynamic Programming and Optimal Control. vol. 1, 2. Athena Scientific, Belmont (1995)
Dolgov, D.A., Durfee, E.H.: Constructing optimal policies for agents with constrained architectures. In: AAMAS, pp. 974–975 (2003)
Dolgov, D.A., Durfee, E.H.: Approximating optimal policies for agents with limited execution resources. In: Proceedings of the Eighteenth International Joint Conference on Artificial Intelligence, pp. 1107–1112. AAAI Press, Menlo Park (2004)
Feinberg, E.A., Shwartz, A.: Constrained markov decision models with weighted discounted rewards. Math. of Operations Research 20(4), 302–320 (1995)
Gabor, Z., Kalmar, Z., Szepesvari, C.: Multi-criteria reinforcement learning. In: Proc. 15th International Conf. on Machine Learning, pp. 197–205. Morgan Kaufmann, San Francisco (1998)
Geibel, P., Wysotzki, F.: Risk-sensitive reinforcement learning applied to chance constrained control. JAIR 24, 81–108 (2005)
Sutton, R.S., Barto, A.G.: Reinforcement Learning – An Introduction. MIT Press, Cambridge (1998)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Geibel, P. (2006). Reinforcement Learning for MDPs with Constraints. In: Fürnkranz, J., Scheffer, T., Spiliopoulou, M. (eds) Machine Learning: ECML 2006. ECML 2006. Lecture Notes in Computer Science(), vol 4212. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11871842_63
Download citation
DOI: https://doi.org/10.1007/11871842_63
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-45375-8
Online ISBN: 978-3-540-46056-5
eBook Packages: Computer ScienceComputer Science (R0)