Abstract
In Chap. 2, we presented static stochastic optimization problems with open-loop control solutions. In Chap. 3, we introduced various tools to handle information. Now, we examine dynamic stochastic decision issues characterized by the sequence: information \(\rightarrow \) decision \(\rightarrow \) information \(\rightarrow \) decision \(\rightarrow \) etc. This chapter focuses on the interplay between information and decision. First, we provide a “guided tour” of stochastic dynamic optimization issues by examining a simple one-dimensional, two-period linear dynamical system with a quadratic criterion. We examine the celebrated Witsenhausen counterexample, then describe how different information patterns deeply modify the optimal solutions. Second, we present the classical state control dynamical model. Within this formalism, when an optimal solution is searched for among functions of the state, optimization problems with time-additive criterion can be solved by Dynamic Programming (DP), by means of the well-known Bellman equation. This equation connects the value functions between two successive times by means of a static optimization problem over the control set and parameterized by the state. This provides an optimal feedback. We conclude this chapter with more advanced material. We present a more general form of optimal stochastic control problems relative to the state model. Following Witsenhausen, we recall that a Dynamic Programming equation also holds in such a context, due to sequentiality. This equation also connects the value functions between two successive times by means of a static optimization problem. However, the optimization is over a set of feedbacks, and it is parameterized by an information state, the dimension of which is much larger than that of the original state.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
In [155], the random vector is supposed to be Gaussian under the probability \(\mathbb {P}\), but we do not need this assumption in what follows.
- 2.
In fact, although time-additive criteria are widely used, other criteria are also adapted to Dynamic Programming [20, 51, 152, 153].
- 3.
See footnote 3 in Sect. 1.2.1.
- 4.
To take the expectation in (4.57b), we need that the expression be measurable. For that, it suffices that the value function \(V_{t+1}\) be measurable. However, the proof that induction (4.57) preserves measurability is quite delicate, and requires proper technical assumptions. We refer the reader to [21, 63] for an in-depth analysis.
- 5.
We do not treat the case with decision constraints in this section, but the extension is rather straightforward (for instance, decision constraints can be incorporated in the criterion with characteristic functions).
- 6.
See footnote 3 in Sect. 1.2.1.
- 7.
- 8.
We use the terminology “support function” despite, traditionally, the support function of a subset is a supremum of linear functions over this subset, whereas we consider here an infimum. Indeed, our optimization problem (4.96a) is one of minimization, and not of maximization.
- 9.
To avoid confusion, we temporarily adopt the notation \( \overline{\eta }_t \) for a generic feedback in \({\varGamma }_t^{\overline{\mathcal {I}}_t} \), instead of \( \overline{\gamma }_t\).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this chapter
Cite this chapter
Carpentier, P., Chancelier, JP., Cohen, G., De Lara, M. (2015). Information and Stochastic Optimization Problems. In: Stochastic Multi-Stage Optimization. Probability Theory and Stochastic Modelling, vol 75. Springer, Cham. https://doi.org/10.1007/978-3-319-18138-7_4
Download citation
DOI: https://doi.org/10.1007/978-3-319-18138-7_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-18137-0
Online ISBN: 978-3-319-18138-7
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)