Abstract
This chapter introduces the very general formalism of Markov decision processes (MDPs) that allows representation of various sequential decision making problems. Thus a Markov decision process can be used to model stochastic path problems, stopping problems as well as problems in reinforcement learning, experiment design, and control.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Thus, the result is weakly polynomial complexity, due to the dependence on the input size description.
References
Chernoff, H.: Sequential design of experiments. Ann. Math. Stat. 30(3), 755–770 (1959)
Chernoff, H.: Sequential models for clinical trials. In: Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability, vol. 4, pp. 805–812. University of California Press (1966)
Puterman, M.L.: Markov Decision Processes Discrete Stochastic Dynamic Programming. Wiley, New Jersey, US (1994)
Tseng, P.: Solving H-horizon, stationary Markov decision problems in time proportional to log(H). Oper. Res. Lett. 9(5), 287–297 (1990)
Ye, Y.: The simplex and policy-iteration methods are strongly polynomial for the Markov decision problem with a fixed discount rate. Math. Oper. Res. 36(4), 593–603 (2011)
DeGroot, M.H.: Optimal Statistical Decisions. Wiley (1970)
O’Gordon Duff, M.: Optimal learning computational procedures for bayes-adaptive markov decision processes. Ph.D. thesis, University of Massachusetts at Amherst (2002)
Gittins, J.C.: Multi-armed Bandit Allocation Indices. Wiley, New Jersey, US (1989)
Bertsekas, D.P., Tsitsiklis, J.N.: Neuro-Dynamic Programming. Athena Scientific (1996)
Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press (1998)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this chapter
Cite this chapter
Dimitrakakis, C., Ortner, R. (2022). Experiment Design and Markov Decision Processes. In: Decision Making Under Uncertainty and Reinforcement Learning. Intelligent Systems Reference Library, vol 223. Springer, Cham. https://doi.org/10.1007/978-3-031-07614-5_6
Download citation
DOI: https://doi.org/10.1007/978-3-031-07614-5_6
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-07612-1
Online ISBN: 978-3-031-07614-5
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)