Dynamic consistency for stochastic optimal control problems
- 415 Downloads
For a sequence of dynamic optimization problems, we aim at discussing a notion of consistency over time. This notion can be informally introduced as follows. At the very first time step t 0, the decision maker formulates an optimization problem that yields optimal decision rules for all the forthcoming time steps t 0,t 1,…,T; at the next time step t 1, he is able to formulate a new optimization problem starting at time t 1 that yields a new sequence of optimal decision rules. This process can be continued until the final time T is reached. A family of optimization problems formulated in this way is said to be dynamically consistent if the optimal strategies obtained when solving the original problem remain optimal for all subsequent problems. The notion of dynamic consistency, well-known in the field of economics, has been recently introduced in the context of risk measures, notably by Artzner et al. (Ann. Oper. Res. 152(1):5–22, 2007) and studied in the stochastic programming framework by Shapiro (Oper. Res. Lett. 37(3):143–147, 2009) and for Markov Decision Processes (MDP) by Ruszczynski (Math. Program. 125(2):235–261, 2010). We here link this notion with the concept of “state variable” in MDP, and show that a significant class of dynamic optimization problems are dynamically consistent, provided that an adequate state variable is chosen.
KeywordsStochastic optimal control Dynamic consistency Dynamic programming Risk measures
This study was made within the Systems and Optimization Working Group (SOWG), which is composed of Laetitia Andrieu, Kengy Barty, Pierre Carpentier, Jean-Philippe Chancelier, Guy Cohen, Anes Dallagi, Michel De Lara and Pierre Girardeau, and based at Université Paris-Est, CERMICS, Champs sur Marne, 77455 Marne la Vallée Cedex 2, France.
- Bellman, R. (1957). Dynamic programming. Princeton: Princeton University Press. Google Scholar
- Bertsekas, D. (2000). Dynamic programming and optimal control (2nd ed.). Nashua: Athena Scientific. Google Scholar
- Cheridito, P., Delbaen, F., & Kupper, M. (2006). Dynamic monetary risk measures for bounded discrete-time processes. Electronic Journal of Probability, 11(3), 57–106. Google Scholar
- Ekeland, I., & Lazrak, A. (2006). Being serious about non-commitment: subgame perfect equilibrium in continuous time. arXiv:math.OC/0604264.
- Prékopa, A. (1995). Stochastic programming. Dordrecht: Kluwer Academic. Google Scholar
- Ruszczynski, A., & Shapiro, A. (Eds.) (2003). Handbooks in operations research and management science: Vol. 10. Stochastic programming. Amsterdam: Elsevier. Google Scholar
- Whittle, P. (1982). Optimization over time. New York: Wiley. Google Scholar