Let \((\varOmega ,{\mathcal {F}},{\mathbb {P}})\) be a probability space. Throughout, we identify two sets in \({\mathcal {F}}\) whenever their symmetric difference is a null set, and identify two functions on \(\varOmega \) if they coincide a.s. (almost surely). Let \({\mathcal {G}}\) be a sub-\(\sigma \)-algebra of \({\mathcal {F}}\). Denote by \(\varPi _{\mathcal {G}}\) the set of partitions \((A_k)\) of \(\varOmega \) where \(A_k\in {\mathcal {G}}\) for all k. Let \(L^0_{\mathcal {G}}, L^0_{\mathcal {G}}({\mathbb {N}}), L^0_{{\mathcal {G}},+}, L^0_{{\mathcal {G}},++}, {{\underline{\hbox {L}}}}^0_{\mathcal {G}}\), and \({\bar{L}}^0_{\mathcal {G}}\) denote the spaces of \({\mathcal {G}}\)-measurable random variables with values in \({\mathbb {R}}, {\mathbb {N}}, [0,+\infty [, ]0,+\infty [, {\mathbb {R}}\cup \{-\infty \}\), and \({\mathbb {R}}\cup \{\pm \infty \}\), respectively. Recall that \(L^0_{\mathcal {G}}\) with the pointwise a.s. order is a Dedekind complete lattice-ordered ring. The essential supremum and the essential infimum are denoted by \(\sup \) and \(\inf \), respectively. Inequalities between random variables with values in an ordered set are always understood in the pointwise a.s. sense.
Definition 2.1
A \({\mathcal {G}}\)-conditional metric on a non-empty set X is a function \(d\,{:}\,X\times X\rightarrow L^0_{{\mathcal {G}},+}\), such that the following conditions hold:
-
(i)
\(d(x,y)=0\) if and only if \(x=y\),
-
(ii)
\(d(x,y)=d(y,x),\)
-
(iii)
\(d(x,z)\le d(x,y)+d(y,z)\),
-
(iv)
for every sequence \((x_k)\) in X and \((A_k)\in \varPi _{\mathcal {G}}\), there exists exactly one element \(x\in X\) such that \(1_{A_k}d(x,x_k)=0\) for all \(k\in {\mathbb {N}}\).
The pair (X, d) is called a \({\mathcal {G}}\)-conditional metric space.
In the following, we call the unique element in (iv) the concatenation of the sequence \((x_k)\) along the partition \((A_k)\), and denote it by \(\sum \nolimits _k 1_{A_k}x_k\). For a sequence \((x_n)\) in a conditional metric space (X, d), we write \(x_n\rightarrow x\) whenever \(d(x,x_n)\rightarrow 0\) a.s.. Further, a conditional subsequence\((x_{n_k})\) of \((x_n)\) is of the form \(x_{n_k}:=\sum \nolimits _{j\in {\mathbb {N}}} 1_{\{n_k=j\}}x_j\), where \((n_k)\) is a sequence in \(L^0_{{\mathcal {G}}}({\mathbb {N}})\) such that \(n_k<n_{k+1}\) for all \(k\in {\mathbb {N}}\).
Definition 2.2
Let \((X,d_X)\) and \((Z,d_Z)\) be \({\mathcal {G}}\)-conditional metric spaces, and H and G subsets of X and Z, respectively.
The set H is called \({\mathcal {G}}\)-stable if \(H \ne \emptyset \) and \(\sum \nolimits _k 1_{A_k} x_k\in H\) for all \((A_k)\in \varPi _{\mathcal {G}}\) and every sequence \((x_k)\) in H, and sequentially closed if H contains every \(x\in X\), such that there is a sequence \((x_k)\) in H with \(x_k\rightarrow x\).
A function \(f:H\rightarrow G\) is said to be \({\mathcal {G}}\)-stable if \(f\left( \sum _k 1_{A_k} x_k\right) =\sum \nolimits _k 1_{A_k} f(x_k)\) for all \((A_k)\in \varPi _{\mathcal {G}}\) and every sequence \((x_k)\) in H, where H and G are assumed to be \({\mathcal {G}}\)-stable.
Remark 2.1
-
1.
If (X, d) is a \({\mathcal {G}}\)-conditional metric space, then the metric d is \({\mathcal {G}}\)-stable, i.e., \(d\left( \sum \nolimits _k 1_{A_k}x_k, \sum \nolimits _k 1_{A_k}y_k\right) =\sum \nolimits _k 1_{A_k}d(x_k,y_k)\) for all sequences \((x_k)\) and \((y_k)\) in X and \((A_k)\in \varPi _{\mathcal {G}}\). Indeed, denoting by \(x=\sum \nolimits _k 1_{A_k}x_k\) and \(y=\sum \nolimits _k 1_{A_k}y_k\) the respective concatenations, it follows from the triangular inequality that
$$\begin{aligned} 1_{A_k}d(x,y)&\le 1_{A_k}d(x,x_k)+1_{A_k}d(x_k,y_k)+1_{A_k}d(y_k,y)=1_{A_k}d(x_k,y_k)\\&\le 1_{A_k}d(x_k,x)+1_{A_k}d(x,y)+1_{A_k}d(y_k,y)=1_{A_k}d(x,y), \end{aligned}$$
which shows that \(1_{A_k}d(x,y)=1_{A_k}d(\sum _k 1_{A_k}x_k,\sum \nolimits _k 1_{A_k}y_k)=1_{A_k}d(x_k,y_k)\) for all \(k\in {\mathbb {N}}\). Summing up over all k yields the desired \({\mathcal {G}}\)-stability.
-
2.
Let \((X,d_X)\) and \((Y,d_Y)\) be two \({\mathcal {G}}\)-conditional metric spaces. Then, its product \(X\times Y\) endowed with the \({\mathcal {G}}\)-conditional metric
$$\begin{aligned}d_{X\times Y}\left( (x,y),(x^\prime ,y^\prime )\right) :=\max \{d_{X}(x,x^\prime ),d_{Y}(y,y^\prime )\}\end{aligned}$$
is a \({\mathcal {G}}\)-conditional metric space.
-
3.
Let \((X,d_X)\) be a \({\mathcal {G}}\)-conditional metric space. Then, the set \({\mathbf {X}}\) of all pairs \((x,A)\in X\times {\mathcal {G}}\), where (x, A) and (y, B) are identified if \(A=B\) and \(1_A d(x,y)=0\), is a conditional set as introduced in
[2]; in
[2, Section 4] conditional metric spaces are defined.
We next introduce the parameter-dependent stochastic optimal control problem for conditional metric spaces. For a fixed finite time horizon \(T\in {\mathbb {N}}\), we consider a filtration \({\mathcal {F}}_0\subset {\mathcal {F}}_1\subset \cdots \subset {\mathcal {F}}_T={\mathcal {F}}\). For simplicity, we often abbreviate the index \({\mathcal {F}}_t\) by t, and write for instance \(L^0_{t}\) for \(L^0_{{\mathcal {F}}_t}\). For each \(t=0,\ldots ,T\), let \((X_t,d_{X_t})\) and \((Z_t,d_{Y_t})\) be \({\mathcal {F}}_t\)-conditional metric spaces. Our aim is to study control problems, for which the control set \(\varTheta _t\) depends on \({\mathcal {F}}_t\), but also on a state parameter \(x\in X_t\). For every \(t=0,\ldots ,T-1\), we assume that the state-dependent control set\(\varTheta _t\) satisfies
-
(c1)
\(\emptyset \ne \varTheta _t(x)\subset Z_t\) for all \(x\in X_t\),
-
(c2)
\(\varTheta _t\) is \({\mathcal {F}}_t\)-stable, i.e.,
$$\begin{aligned} \varTheta _t\left( \sum _k 1_{A_k}x_k\right) =\sum _k 1_{A_k} \varTheta _t(x_k):=\left\{ \sum _k 1_{A_k} z_k:z_k\in \varTheta _t(x_k) \text { for all }k\right\} \end{aligned}$$
for all \((A_k)\in \varPi _t\) and every sequence \((x_k)\) in \(X_t\),
-
(c3)
for every \(x\in X_t\), the set \(\varTheta _t(x)\) is conditionally sequentially compact, i.e., for every sequence \((z_n)\) in \(\varTheta _t(x)\), there exists a conditional subsequence \(n_1<n_2<\cdots \) with \(n_k\in L^0_t({\mathbb {N}})\) such that \(z_{n_k}\rightarrow z\in \varTheta _t(x)\),
-
(c4)
for every sequence \((x_n)\) in \(X_t\) such that \(x_n\rightarrow x\in X_t\) and every sequence \((z_n)\) in \(\varTheta _t(x_n)\), there exists a conditional subsequence \(n_1<n_2<\cdots \) with \(n_k\in L^0_t({\mathbb {N}})\) and a sequence \((z_k^\prime )\) in \(\varTheta _t(x)\) such that \(d_{Z_t}(z_{n_k},z_{k}^\prime )\rightarrow 0\) a.s..
Note that \({\mathcal {F}}_t\)-stability of \(\varTheta _t\) implies that \(\varTheta _t(x)\) is \({\mathcal {F}}_t\)-stable for all \(x\in X_t\).
We consider forward generators
$$\begin{aligned} v_t:X_t\times Z_t\rightarrow X_{t+1},\quad t=0,\ldots ,T-1, \end{aligned}$$
which are
-
(v1)
\({\mathcal {F}}_t\)-stable, i.e., \(v_t\left( \sum _k 1_{A_k}x_k, \sum \nolimits _k 1_{A_k}z_k\right) =\sum \nolimits _k 1_{A_k}v_t(x_k,z_k)\) for every partition \((A_k)\in \varPi _t\), and all sequences \((x_k)\) in \(X_t\) and \((z_k)\) in \(Z_t\),
-
(v2)
sequentially continuous, i.e., \(v_t(x_n,z_n)\rightarrow v_t(x,z)\) whenever \(x_n\rightarrow x\) in \(X_t\) and \(z_n\rightarrow z\) in \(Z_t\).
For every \(x_t\in X_t\), we consider the set
$$\begin{aligned} C_t(x_t)&:=\left\{ ((x_s)_{s=t+1}^\mathrm{T},(z_s)_{s=t}^{T-1}) :x_{s+1}=v_s(x_s,z_s),z_s\in \varTheta _s(x_s)\right. \\&\left. \qquad \text { for all }s=t,\ldots ,T-1\right\} \end{aligned}$$
of all parameter processes \((x_s)_{s=t}^\mathrm{T}\), which can be realized by the state-dependent controls \(z_s\in \varTheta _t(x_s)\) for \(s=t,\ldots ,T-1\).
As for the objective function, we consider backward generators
$$\begin{aligned} u_t:X_t\times {{\underline{\hbox {L}}}}^0_{t+1}\times Z_t \rightarrow {{\underline{\hbox {L}}}}^0_t,\quad t=0,1,\ldots ,T-1, \end{aligned}$$
which are
-
(u1)
\({\mathcal {F}}_t\)-stable, i.e., \(u_t\left( \sum _k 1_{A_k}x_k,\sum \nolimits _k 1_{A_k} y_k,\sum \nolimits _k 1_{A_k} z_k\right) = \sum \nolimits _k 1_{A_k} u_t(x_k,y_k,z_k)\) for all \((A_k)\in \varPi _t\), and sequences \((x_k)\) in \(X_t, (y_k)\) in \(Y_t\), and \((z_k)\) in \(Z_t\),
-
(u2)
increasing in the second component, i.e., \(u_t(x,y,z)\le u_t(x,y^\prime ,z)\) whenever \(y\le y^\prime \),
-
(u3)
sequentially upper semi-continuous, i.e.,
$$\begin{aligned}\limsup _{n\rightarrow \infty } u_t(x_n,y_n,z_n)\le u_t(x,y,z),\end{aligned}$$
whenever \(x_n\rightarrow x\) in \(X_t, y_n\rightarrow y\) in \({\underline{\hbox {L}}}^0_{t+1}\), and \(z_n\rightarrow z\) in \(Z_t\).
We assume that \(u_T\,{:}\,X_T\rightarrow {\underline{\hbox {L}}}^0_T\) is \({\mathcal {F}}_{T}\)-stable and sequentially upper semi-continuous.Footnote 2
Given such a family \((u_t)_{t=0}^\mathrm{T}\) of backward generators, our goal is to maximize
$$\begin{aligned} y_t(x_t):=\sup _{((x_s)_{s=t+1}^\mathrm{T},(z_s)_{s=t}^{T-1})\in C_t(x_t) }u_t(x_t,\cdot ,z_t)\circ \cdots \circ u_{T-1}(x_{T-1},\cdot ,z_{T-1})\circ u_T (x_T),\nonumber \\ \end{aligned}$$
(1)
over all realizable state processes initialized at \(x_t\in X_t\). In (1) we consider the composition of the functions \(u_T, u_{T-1}(x_{T-1},\cdot ,z_{T-1}),\ldots ,u_{t}(x_{t},\cdot ,z_{t})\), where \(u_s(x_s,\cdot ,z_s)\) denotes the function \({\underline{\hbox {L}}}^0_{s+1}\rightarrow {\underline{\hbox {L}}}^0_{s}, y\mapsto u_s(x_s,y,z_s)\).
Remark 2.2
The objective function in the stochastic control problem (1) is recursively defined. Its generators are functions between conditional metric spaces which are not necessarily (conditional) expected utilities. In case of (conditional) expected utility, the generators are closely related with dynamic and conditional risk measures; see
[9,10,11,12,13,14]. The preferences which underly conditional expected utility functionals were studied in
[15].
In decision theory, there is an extensive literature on recursive utilities starting with the seminal work
[16, 17]. The preferences therein are defined on sets of temporal lotteries (probability trees), and follow a kind of Bellman recursive structure, which is similar (on a formal level) to the construction above; see
[16, Theorem 1]. This was later extended in
[18], where non-expected utilities were incorporated as well, and established under the name of Epstein–Zin utilities. See also
[19] for a survey on non-expected utility theory. With the techniques of conditional analysis and based on results in BSDE theory,
[20] solves a utility maximization problem in continuous time for Epstein–Zin utilities.
The following result shows that the global supremum in (1) is attained and can be reduced to local optimization problems by the following Bellman’s principle.
Theorem 2.1
Suppose that \((\hbox {c}1)\)–\((\hbox {c}4)\), \((\hbox {v}1)\)–\((\hbox {v}2)\), and \((\hbox {u}1)\)–\((\hbox {u}3)\) are fulfilled. Then, the functions \(y_t:X_{t}\rightarrow {\underline{\hbox {L}}}^0_t\) are \({\mathcal {F}}_t\)-stable and sequentially upper semi-continuous for all \(t=0,\ldots , T\), and can be computed by backward recursion
$$\begin{aligned} y_T(x_T)&=u_T(x_T)\\ y_t(x_t)&=\max _{z_t\in \varTheta _t(x_t)} u_t(x_t,y_{t+1}(v_t(x_t,z_t)),z_t), \quad t=0,\ldots ,T-1. \end{aligned}$$
Moreover, for every \(x_t\in X_t\) the process \(((x^*_s)_{s=t}^\mathrm{T},(z^*_s)_{s=t}^{T-1})\), given by \(x_t^*=x_t\), and the forward recursion \(x^*_{s+1}=v_s(x^*_s,z^*_s)\), where
$$\begin{aligned} z_s^*\in \mathop {\hbox {argmax}}\limits _{z_s\in \varTheta _s(x_s^*)} u_s\left( x_s^*,y_{s+1}(v_t(x^*_s,z_s)),z_s\right) ,\quad s=t,\ldots T-1, \end{aligned}$$
(2)
satisfies \(((x^*_s)_{s=t+1}^\mathrm{T},(z^*_s)_{s=t}^{T-1})\in C_t(x_t)\) and
$$\begin{aligned} y_t(x_t)= u_t(x_t,\cdot ,z^*_t)\circ \cdots \circ u_{T-1}(x^*_{T-1},\cdot ,z^*_{T-1})\circ u_T (x^*_T). \end{aligned}$$
Proof
The proof is by backward induction. For \(t=T\), it follows from (1) that \(y_T=u_T\), which by assumption is an \({\mathcal {F}}_t\)-stable and sequentially upper semi-continuous function from \(X_T\) to \({\underline{\hbox {L}}}^0_T\).
As for the induction step, assume that \(y_{t+1}:X_{t+1}\rightarrow {\underline{\hbox {L}}}^0_{t+1}\) is \({\mathcal {F}}_{t+1}\)-stable and sequentially upper semi-continuous, and that for each \(x_{t+1}\in X_{t+1}\) there exists \(((x^*_s)_{s=t+2}^\mathrm{T},(z^*_s)_{s=t+1}^{T-1})\in C_{t+1}(x_{t+1})\) such that
$$\begin{aligned} y_{t+1}(x_{t+1})=u_{t+1}(x_{t+1},\cdot ,z^*_{t+1})\circ \cdots \circ u_T (x^*_T). \end{aligned}$$
By (u1) and (v1), the function
$$\begin{aligned} X_t\times Z_t\ni (x,z)\mapsto u_t\left( x,y_{t+1}(v_t(x,z)),z\right) \end{aligned}$$
is \({\mathcal {F}}_t\)-stable. Moreover, it is sequentially upper semi-continuous. Indeed, let \((x_k,z_k)\) be a sequence in \(X_t\times Z_t\) such that \(x_k\rightarrow x\in X_t\) and \(z_k\rightarrow z\in Z_t\). Since \(v(x_k,z_k)\rightarrow v(x,z)\) by (v2), it follows from the induction hypothesis that
$$\begin{aligned} \limsup _{k\rightarrow \infty } y_{t+1}(v_t(x_k,z_k))\le y_{t+1}(v(x,z))<+\infty . \end{aligned}$$
Since
$$\begin{aligned} \left\{ \sup _{k\ge 1} y_{t+1}(v_t(x_{k},z_{k}))=+\infty \right\}&=\bigcap _{k\ge 1}\left\{ \sup _{k^\prime \ge k} y_{t+1}(v_t(x_{k^\prime },z_{k^\prime }))=+\infty \right\} \\&=\left\{ \limsup _{k\rightarrow \infty } y_{t+1}(v_t(x_k,z_k))=+\infty \right\} , \end{aligned}$$
we have \(\sup _{k\ge 1} y_{t+1}(v_t(x_{k},z_{k}))\in {\underline{\hbox {L}}}^0_{t+1}\). Hence, by (u2), (u3) and (v2), we get
$$\begin{aligned} \limsup _{k\rightarrow \infty } u_t\left( x_k,y_{t+1}(v_t(x_k,z_k)),z_k\right)&\le \limsup _{k\rightarrow \infty } u_t\left( x_k,\sup _{k^\prime \ge k} y_{t+1}(v_t(x_{k^\prime },z_{k^\prime })),z_k\right) \nonumber \\&\le u_t\left( x,\limsup _{k\rightarrow \infty } y_{t+1}(v_t(x_{k},z_{k})),z\right) \nonumber \\&\le u_t\left( x, y_{t+1}(v_t(x,z)),z\right) , \end{aligned}$$
(3)
which shows the desired sequential upper semi-continuity. As a consequence, the supremum in
$$\begin{aligned} f_t(x_t):=\sup _{z\in \varTheta _t(x_t)} u_t\left( x_t,y_{t+1}(v_t(x_t,z)),z\right) \end{aligned}$$
(4)
is attained for each \(x_t\in X_t\). Indeed, since \(z\mapsto u_t\left( x,y_{t+1}(v_t(x,z)),z\right) \) and \(\varTheta _t(x_t)\) are \({\mathcal {F}}_t\)-stable, it follows from standard properties of the essential supremum that there exists a sequence \(z_n\in \varTheta _t(x_t)\) such that
$$\begin{aligned} u_t\left( x_t,y_{t+1}(v_t(x_t,z_n)),z_n\right) \rightarrow f_t(x_t). \end{aligned}$$
By (c3), there is a conditional subsequence \(n_1<n_2<\cdots \) with \(n_k\in L^0_t({\mathbb {N}})\) such that \(z_{n_k}\rightarrow z\in \varTheta _t(x_t)\) a.s.. Since \(z\mapsto u_t\left( x,y_{t+1}(v_t(x,z)),z\right) \) is sequentially upper semi-continuous and \({\mathcal {F}}_t\)-stable, it follows that
$$\begin{aligned} u_t\left( x_t,y_{t+1}(v_t(x_t,z)),z\right) \ge \limsup _{k\rightarrow \infty } u_t\left( x_t,y_{t+1}(v_t(x_t,z_{n_k})),z_{n_k}\right) = f_t(x_t), \end{aligned}$$
which shows that the supremum in (4) is attained.
We next show that \(f_t\,{:}\,X_t\rightarrow {\underline{\hbox {L}}}^0_{t+1}\) is sequentially upper semi-continuous. By contradiction, suppose that \((x_k)\) is a sequence in \(X_t\) such that \(x_k\rightarrow x\in X_t\) and \(f_t(x)<\limsup _{k\rightarrow \infty } f_t(x_k)\) on some \(A\in {\mathcal {F}}\) with \({\mathbb {P}}(A)>0\). Note that \(f_t\) is \({\mathcal {F}}_t\)-stable. Thus, by possibly passing to a conditional subsequence , we can suppose that there exists \(r\in L^0_{t,++}\) such that
$$\begin{aligned} f_t(x)+r< f_t(x_k)\text { on }A,\quad \text { for all }k\in {\mathbb {N}}. \end{aligned}$$
(5)
Denote by \(z_k\in \varTheta _t(x_k)\) a respective maximizer of \(f_t(x_k)\). By (c4), there exists \(z_k^\prime \in \varTheta _t(x)\) such that \(d_{Z_t}(z_k,z^\prime _k)\rightarrow 0\) a.s. by possibly passing to a conditional subsequence . By (c3), there exists a conditional subsequence \(k_1<k_2<\cdots \) with \(k_l\in L^0_t({\mathbb {N}})\) such that \(z^\prime _{k_l}\rightarrow z^\prime \in \varTheta _t(x)\). Since \(d_{Z_t}(z_{k_l},z^\prime _{k_l})\rightarrow 0\) a.s., by \({\mathcal {F}}_t\)-stability of the conditional metric \(d_{Z_t}\), it follows from the triangular inequality that \(z_{k_l}\rightarrow z^\prime \in \varTheta _t(x)\). By the \({\mathcal {F}}_t\)-stability of \(f_t\) and (c2), it follows that \(z_{k_l}\) is in \(\varTheta _t(x_{k_l})\) and maximizes \(f_t(x_{k_l})\). Hence, it follows from (3) that
$$\begin{aligned} \limsup _{l\rightarrow \infty } f_t(x_{k_l})&=\limsup _{l\rightarrow \infty } u_t\left( x_{k_l},y_{t+1}(v_t(x_{k_l},z_{k_l})),z_{k_l}\right) \\&\le u_t\left( x,y_{t+1}(v_t(x,z^\prime )),z^\prime \right) \\&\le \sup _{z\in \varTheta _t(x)} u_t\left( x,y_{t+1}(v_t(x,z),z\right) =f_t(x). \end{aligned}$$
Notice that, due to the \({\mathcal {F}}_t\)-stability of \(f_t\), (5) is satisfied for any conditional subsequence of \((x_k)\). Thus, we have that \(f_t(x)+r\le \limsup _{l\rightarrow \infty } f_t(x_{k_l})\le f_t(x)\) on A, which is a contradiction. We conclude that \(f_t\) is sequentially upper semi-continuous.
Finally, we show that \(y_t=f_t\). By induction hypothesis, for every \(x_t\in X_t\) and \(z_t\in Z_t\), there exists \(((x^*_s)_{s=t+2}^\mathrm{T},(z^*_s)_{s=t+1}^{T-1} )\in C_{t+1}(v_t(x_t,z_t))\) such that
$$\begin{aligned} y_{t+1}(v_t(x_t,z_t))=u_{t+1}(v_t(x_t,z_t),\cdot ,z^*_{t+1})\circ \cdots \circ u_{T-1}(x^*_{T-1},\cdot ,z^*_{T-1})\circ u_T(z^*_T). \end{aligned}$$
In particular, for \(x_t\in X_t\) and \(z_t^*\in Z_t\) being a maximizer in (4), it holds
$$\begin{aligned} f_t(x_t)&=\sup _{z\in \varTheta _t(x_t)} u_t\left( x_t,y_{t+1}(v_t(x_t,z)),z\right) \\&= u_t\left( x_t,y_{t+1}(v_t(x_t,z_t^*)),z_t^*\right) \\&= u_t\left( x_t,\cdot ,z_t^*\right) \circ u_{t+1}(v_t(x_t,z^*_t),\cdot ,z^*_{t+1})\circ \cdots \circ u_{T-1}(x^*_{T-1},\cdot ,z^*_{T-1})\circ u_T(z^*_T) \\&= \sup _{(x,z)\in C_{t+1}(v(x_t,z^*_t)) } u_t\left( x_t,\cdot ,z_t^*\right) \circ u_{t+1}(v_t(x_t,z_t),\cdot ,z_{t+1})\circ \cdots \circ \circ u_T(z_T) \\&= \sup _{z_t\in \varTheta _t(x_t)} \sup _{(x,z)\in C_{t+1}(v(x_t,z_t)) } u_t\left( x_t,\cdot ,z_t\right) \circ u_{t+1}(v_t(x_t,z_t),\cdot ,z_{t+1})\circ \cdots \circ u_T(z_T) \\&= \sup _{(x,z)\in C_{t}(x_t) } u_t\left( x_t,\cdot ,z_t\right) \circ u_{t+1}(v_t(x_t,z_t),\cdot ,z_{t+1})\circ \cdots \circ u_T(z_T) \\&=y_t(x_t). \end{aligned}$$
This shows that \(((x^*_s)_{s=t+1}^\mathrm{T},(z^*_s)_{s=t}^\mathrm{T})\in C_{t}(x_t)\) is an optimizer of (1) whenever it satisfies the local optimality criterion
$$\begin{aligned} z^*_s\in {\mathop {\hbox {argmax}}\nolimits }_{z\in \varTheta _t(x^*_t)} u_s\left( x_s^*,y_{s+1}(v_t(x^*_s,z_s)),z_s\right) \quad \text{ and }\quad x^*_{s+1}=v_s(x^*_s,z^*_s) \end{aligned}$$
for all \(s=t,\ldots ,T\), where \(x_t^*=x_t\). In particular, every process which satisfies the forward recursion (2) is an optimizer for (1). \(\square \)
Example 2.1
As for the illustration, we provide examples of \({\mathcal {F}}_t\)-conditional metric spaces, which are of interest for the control and parameter spaces.
-
1.
Given a nonempty metric space (X, d), denote by \(L^0_t(X)\) the set of all strongly \({\mathcal {F}}_t\)-measurable functions \(x:\varOmega \rightarrow X\). The metric d extends from X to \(L^0_t(X)\) by defining
$$\begin{aligned} d_{L^0_t(X)}(x,{{\bar{x}}})(\omega ):=d(x(\omega ),{{\bar{x}}} (\omega ))\quad \text{ for } \text{ a.a. } \omega \in \varOmega \text{ and } \text{ all } x,{{\bar{x}}}\in L^0_t(X). \end{aligned}$$
Then, \((L^0_t(X),d_{L^0_t(X)})\) is a \({\mathcal {F}}_t\)-conditional metric space.
-
2.
The conditional Euclidean space with dimension \(n=\sum _k 1_{A_k} n_k\in L^0_t({\mathbb {N}})\) is defined as
$$\begin{aligned} L^0_t({\mathbb {R}})^n=\sum _k 1_{A_k} L^0_t({\mathbb {R}}^{n_k}):=\left\{ \sum _k 1_{A_k} x_k :x_k\in L^0_t({\mathbb {R}}^{n_k}) \text { for all }k\right\} . \end{aligned}$$
The \({\mathcal {F}}_t\)-conditional metric on \(L^0_t({\mathbb {R}})^n\) is defined by
$$\begin{aligned} d_{L^0_t({\mathbb {R}})^n}(x,{{\bar{x}}}):=\sum _k 1_{A_k} d_{ L^0_t({\mathbb {R}}^{n_k}) }(x_k,{{\bar{x}}}_k), \end{aligned}$$
where \(x=\sum _k 1_{A_k} x_k\) and \({{\bar{x}}}=\sum _k 1_{A_k} {{\bar{x}}}_k\). Here, \(d_{L^0_t({\mathbb {R}}^{n_k})}\) denotes the \({\mathcal {F}}_t\)-conditional metric on \(L^0_t({\mathbb {R}}^{n_k})\). Straightforward verification shows that \((L^0_t({\mathbb {R}})^n, d_{L^0_t({\mathbb {R}})^n})\) is a \({\mathcal {F}}_t\)-conditional metric space.
-
3.
For \(1\le p<\infty \), we define the conditional \(L^p\)-space
$$\begin{aligned} L^p_t:=\{x\in L^0_T :{\mathbb {E}}[|x|^p|{\mathcal {F}}_t]<+\infty \} \end{aligned}$$
with \({\mathcal {F}}_t\)-conditional metric \(d_{L^p_t}(x,\bar{x}):={\mathbb {E}}[|x-{{\bar{x}}}|^p|{\mathcal {F}}_t]^{1/p}\). Then, \((L^p_t,d_{L^p_t})\) is a \({\mathcal {F}}_t\)-conditional metric space.