Abstract
The objective of this chapter is to introduce the stochastic control processes we are interested in; these are the so-called (discrete-time) controlled Markov processes (CMP’s), also known as Markov decision processes or Markov dynamic programs. The main part is Section 1.2. It contains some basic definitions and the statement of the optimal and the adaptive control problems studied in this book. In Section 1.3 we present several examples; the idea is to illustrate the main concepts and provide sources for possible applications. Also in Section 1.3 we discuss (briefly) more general control systems, such as non-stationary CMP’s and semi-Markov control models. The chapter is concluded in Section 1.4 with some comments on related references.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Author information
Authors and Affiliations
Rights and permissions
Copyright information
© 1989 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Hernández-Lerma, O. (1989). Controlled Markov Processes. In: Adaptive Markov Control Processes. Applied Mathematical Sciences, vol 79. Springer, New York, NY. https://doi.org/10.1007/978-1-4419-8714-3_1
Download citation
DOI: https://doi.org/10.1007/978-1-4419-8714-3_1
Publisher Name: Springer, New York, NY
Print ISBN: 978-1-4612-6454-5
Online ISBN: 978-1-4419-8714-3
eBook Packages: Springer Book Archive