Appendix
Proof of Theorem 2. From the definition of the sets \({{\mathcal{I}}}_{k}\) and \({{\mathcal{O}}}_{k}\) and the dynamic programming relations (3)–(5) it follows that
$${{\mathcal{I}}}_{k}=\left\{x\in {{\mathbb{R}}}^{n}:\mathop{\rm{max}}\limits_{{u}_{k}\in {U}_{k}}{{\bf{M}}}_{k}\left[{{\bf{I}}}_{{{\mathcal{F}}}_{k}}\left({x}_{k}\right){{\rm{B}}}_{k+1}\left({f}_{k}\left({x}_{k},{u}_{k},{\xi }_{k}\right)\right)| {x}_{k}=x\right]=1\right\},$$
(A.1)
$${{\mathcal{O}}}_{k}=\left\{x\in {{\mathbb{R}}}^{n}:\mathop{\rm{max}}\limits_{{u}_{k}\in {U}_{k}}{{\bf{M}}}_{k}\left[{{\bf{I}}}_{{{\mathcal{F}}}_{k}}\left({x}_{k}\right){{\rm{B}}}_{k+1}\left({f}_{k}\left({x}_{k},{u}_{k},{\xi }_{k}\right)\right)| {x}_{k}=x\right]=0\right\}.$$
(A.2)
Consider some step k of the dynamic programming algorithm (3)–(5). Due to the basic properties of the sets \({{\mathcal{I}}}_{k}\), \({{\mathcal{B}}}_{k}\), and \({{\mathcal{O}}}_{k}\),
$$\begin{array}{cc}{{\rm{B}}}_{k+1}\left(x\right)={{\bf{I}}}_{{{\mathcal{I}}}_{k}}\left(x\right){{\rm{B}}}_{k+1}\left(x\right)+{{\bf{I}}}_{{{\mathcal{B}}}_{k}}\left(x\right){{\rm{B}}}_{k+1}\left(x\right)+{{\bf{I}}}_{{{\mathcal{O}}}_{k}}\left(x\right){{\rm{B}}}_{k+1}\left(x\right)\\ ={{\bf{I}}}_{{{\mathcal{I}}}_{k}}\left(x\right){{\rm{B}}}_{k+1}\left(x\right)+{{\bf{I}}}_{{{\mathcal{B}}}_{k}}\left(x\right){{\rm{B}}}_{k+1}\left(x\right).\end{array}$$
Introduce a system of hypotheses forming a complete group of incompatible events:
$$\left\{{f}_{k}\left({x}_{k},{u}_{k},{\xi }_{k}\right)\in {{\mathcal{I}}}_{k+1}\right\},\quad \left\{{f}_{k}\left({x}_{k},{u}_{k},{\xi }_{k}\right)\notin {{\mathcal{I}}}_{k+1}\right\}.$$
Rewrite the expression (4) using the total mathematical expectation formula:
$$\begin{array}{ccccc}{{\rm{B}}}_{k}\left(x\right)=\mathop{\rm{max}}\limits_{{u}_{k}\in {U}_{k}}{{\bf{M}}}_{k}\left[{{\bf{I}}}_{{{\mathcal{F}}}_{k}}\left({x}_{k}\right){{\rm{B}}}_{k+1}\left({f}_{k}\left({x}_{k},{u}_{k},{\xi }_{k}\right)\right)| {x}_{k}=x\right]\\ =\mathop{\rm{max}}\limits_{{u}_{k}\in {U}_{k}}{{\bf{M}}}_{k}\left[{{\bf{I}}}_{{{\mathcal{F}}}_{k}}\left(x\right){{\rm{B}}}_{k+1}\left({f}_{k}\left(x,{u}_{k},{\xi }_{k}\right)\right)\right]\\ =\mathop{\rm{max}}\limits_{{u}_{k}\in {U}_{k}}{{\bf{I}}}_{{{\mathcal{F}}}_{k}}\left(x\right){{\bf{M}}}_{k}\left[{{\rm{B}}}_{k+1}\left({f}_{k}\left(x,{u}_{k},{\xi }_{k}\right)\right)\right]\\ =\mathop{\rm{max}}\limits_{{u}_{k}\in {U}_{k}}{{\bf{I}}}_{{{\mathcal{F}}}_{k}}\left(x\right)\left({\bf{P}}\left({f}_{k}\left(x,{u}_{k},{\xi }_{k}\right)\in {{\mathcal{I}}}_{k+1}\right){{\bf{M}}}_{k}\left[\left.{{\rm{B}}}_{k+1}\left({f}_{k}\left(x,{u}_{k},{\xi }_{k}\right)\right)\right|{f}_{k}\left(x,{u}_{k},{\xi }_{k}\right)\in {{\mathcal{I}}}_{k+1}\right]\right.\\ \left.+{\bf{P}}\left({f}_{k}\left(x,{u}_{k},{\xi }_{k}\right)\notin {{\mathcal{I}}}_{k+1}\right){{\bf{M}}}_{k}\left[\left.{{\rm{B}}}_{k+1}\left({f}_{k}\left(x,{u}_{k},{\xi }_{k}\right)\right)\right|{f}_{k}\left(x,{u}_{k},{\xi }_{k}\right)\notin {{\mathcal{I}}}_{k+1}\right]\right).\end{array}$$
(A.3)
In view of the equality
$${{\bf{M}}}_{k}\left[\left.{{\rm{B}}}_{k+1}\left({f}_{k}\left(x,{u}_{k},{\xi }_{k}\right)\right)\right|{f}_{k}\left(x,{u}_{k},{\xi }_{k}\right)\in {{\mathcal{I}}}_{k+1}\right]=1,$$
the expression (A.3) takes the form
$$\begin{array}{cccc}{{\rm{B}}}_{k}\left(x\right)=\mathop{\rm{max}}\limits_{{u}_{k}\in {U}_{k}}{{\bf{I}}}_{{{\mathcal{F}}}_{k}}\left(x\right)\left({\bf{P}}\left({f}_{k}\left(x,{u}_{k},{\xi }_{k}\right)\in {{\mathcal{I}}}_{k+1}\right)\right.\\ \left.+{\bf{P}}({f}_{k}(x,{u}_{k},{\xi }_{k})\notin {{\mathcal{I}}}_{k+1}){{\bf{M}}}_{k}\left[\left.{{\rm{B}}}_{k+1}({f}_{k}(x,{u}_{k},{\xi }_{k}))\right|{f}_{k}(x,{u}_{k},{\xi }_{k})\notin {{\mathcal{I}}}_{k+1}\right]\right)\\ =\mathop{\rm{max}}\limits_{{u}_{k}\in {U}_{k}}{{\bf{I}}}_{{{\mathcal{F}}}_{k}}(x)\left({\bf{P}}({f}_{k}(x,{u}_{k},{\xi }_{k})\in {{\mathcal{I}}}_{k+1})+(1-{\bf{P}}({f}_{k}(x,{u}_{k},{\xi }_{k})\in {{\mathcal{I}}}_{k+1}))\right.\\ \left.\times {{\bf{M}}}_{k}\left[\left.{{\rm{B}}}_{k+1}\left({f}_{k}\left(x,{u}_{k},{\xi }_{k}\right)\right)\right|{f}_{k}\left(x,{u}_{k},{\xi }_{k}\right)\notin {{\mathcal{I}}}_{k+1}\right]\right).\end{array}$$
(A.4)
Since
$${{\bf{M}}}_{k}\left[\left.{{\rm{B}}}_{k+1}\left({f}_{k}\left(x,{u}_{k},{\xi }_{k}\right)\right)\right|{f}_{k}\left(x,{u}_{k},{\xi }_{k}\right)\notin {{\mathcal{I}}}_{k+1}\right]\in \left[0,1\right),$$
the right-hand side of (A.4) is equal to 1 if and only if
$$\mathop{\rm{max}}\limits_{{u}_{k}\in {U}_{k}}{{\bf{I}}}_{{{\mathcal{F}}}_{k}}\left(x\right){\bf{P}}\left({f}_{k}\left(x,{u}_{k},{\xi }_{k}\right)\in {{\mathcal{I}}}_{k+1}\right)=1.$$
Hence, (A.1) reduces to
$$\begin{array}{cccc}{{\mathcal{I}}}_{k}=\left\{x\in {{\mathbb{R}}}^{n}:\mathop{\rm{max}}\limits_{{u}_{k}\in {U}_{k}}{{\bf{M}}}_{k}\left[{{\bf{I}}}_{{{\mathcal{F}}}_{k}}\left({x}_{k}\right){{\rm{B}}}_{k+1}\left({f}_{k}\left({x}_{k},{u}_{k},{\xi }_{k}\right)\right)| {x}_{k}=x\right]=1\right\}\\ =\left\{x\in {{\mathbb{R}}}^{n}:\quad \mathop{\rm{max}}\limits_{{u}_{k}\in {U}_{k}}{{\bf{I}}}_{{{\mathcal{F}}}_{k}}\left(x\right){\bf{P}}\left({f}_{k}\left(x,{u}_{k},{\xi }_{k}\right)\in {{\mathcal{I}}}_{k+1}\right)=1\right\}\\ ={{\mathcal{F}}}_{k}\cap \left\{x\in {{\mathbb{R}}}^{n}:\quad \mathop{\rm{max}}\limits_{{u}_{k}\in {U}_{k}}{\bf{P}}\left({f}_{k}\left(x,{u}_{k},{\xi }_{k}\right)\in {{\mathcal{I}}}_{k+1}\right)=1\right\}\\ ={{\mathcal{F}}}_{k}\cap \left\{x\in {{\mathbb{R}}}^{n}:\quad \exists {u}_{k}\in {U}_{k}:\quad {\bf{P}}\left({f}_{k}\left(x,{u}_{k},{\xi }_{k}\right)\in {{\mathcal{I}}}_{k+1}\right)=1\right\}.\end{array}$$
(A.5)
From (A.5) it follows that for \({x}_{k}\in {{\mathcal{I}}}_{k}\), the optimal control is any element from the set
$${U}_{k}^{{\mathcal{I}}}\left({x}_{k}\right)=\left\{u\in {U}_{k}:\,{\bf{P}}\left({f}_{k}\left({x}_{k},u,{\xi }_{k}\right)\in {{\mathcal{I}}}_{k+1}\right)=1\right\}.$$
(A.6)
Items 1, 3, and 5 of Theorem 2 are established.
Now introduce another system of hypotheses forming a complete group of incompatible events:
$$\left\{{f}_{k}\left({x}_{k},{u}_{k},{\xi }_{k}\right)\in {{\mathcal{I}}}_{k+1}\right\},\quad \left\{{f}_{k}\left({x}_{k},{u}_{k},{\xi }_{k}\right)\in {{\mathcal{B}}}_{k+1}\right\},\quad \left\{{f}_{k}\left({x}_{k},{u}_{k},{\xi }_{k}\right)\in {{\mathcal{O}}}_{k+1}\right\}.$$
Rewrite the expression (4) using the total mathematical expectation formula:
$$\begin{array}{ccccccc}{{\rm{B}}}_{k}\left(x\right)=\mathop{\rm{max}}\limits_{{u}_{k}\in {U}_{k}}{{\bf{M}}}_{k}\left[{{\bf{I}}}_{{{\mathcal{F}}}_{k}}\left({x}_{k}\right){{\rm{B}}}_{k+1}\left({f}_{k}\left({x}_{k},{u}_{k},{\xi }_{k}\right)\right)| {x}_{k}=x\right]\\ =\mathop{\rm{max}}\limits_{{u}_{k}\in {U}_{k}}{{\bf{M}}}_{k}\left[{{\bf{I}}}_{{{\mathcal{F}}}_{k}}\left(x\right){{\rm{B}}}_{k+1}\left({f}_{k}\left(x,{u}_{k},{\xi }_{k}\right)\right)\right]\\ =\mathop{\rm{max}}\limits_{{u}_{k}\in {U}_{k}}{{\bf{I}}}_{{{\mathcal{F}}}_{k}}\left(x\right){{\bf{M}}}_{k}\left[{{\rm{B}}}_{k+1}\left({f}_{k}\left(x,{u}_{k},{\xi }_{k}\right)\right)\right]\\ =\mathop{\rm{max}}\limits_{{u}_{k}\in {U}_{k}}{{\bf{I}}}_{{{\mathcal{F}}}_{k}}\left(x\right)\left({\bf{P}}\left({f}_{k}\left(x,{u}_{k},{\xi }_{k}\right)\in {{\mathcal{I}}}_{k+1}\right)\right.\\ \times {{\bf{M}}}_{k}\left[\left.{{\rm{B}}}_{k+1}\left({f}_{k}\left(x,{u}_{k},{\xi }_{k}\right)\right)\right|{f}_{k}\left(x,{u}_{k},{\xi }_{k}\right)\in {{\mathcal{I}}}_{k+1}\right]\\ +{\bf{P}}\left({f}_{k}\left(x,{u}_{k},{\xi }_{k}\right)\in {{\mathcal{B}}}_{k+1}\right){{\bf{M}}}_{k}\left[\left.{{\rm{B}}}_{k+1}\left({f}_{k}\left(x,{u}_{k},{\xi }_{k}\right)\right)\right|{f}_{k}\left(x,{u}_{k},{\xi }_{k}\right)\in {{\mathcal{B}}}_{k+1}\right]\\ \left.+{\bf{P}}\left({f}_{k}\left(x,{u}_{k},{\xi }_{k}\right)\in {{\mathcal{O}}}_{k+1}\right){{\bf{M}}}_{k}\left[\left.{{\rm{B}}}_{k+1}\left({f}_{k}\left(x,{u}_{k},{\xi }_{k}\right)\right)\right|{f}_{k}\left(x,{u}_{k},{\xi }_{k}\right)\in {{\mathcal{O}}}_{k+1}\right]\right).\end{array}$$
(A.7)
Due to the equalities
$$\begin{array}{l}{{\bf{M}}}_{k}\left[\left.{{\rm{B}}}_{k+1}\left({f}_{k}\left(x,{u}_{k},{\xi }_{k}\right)\right)\right|{f}_{k}\left(x,{u}_{k},{\xi }_{k}\right)\in {{\mathcal{I}}}_{k+1}\right]=1,\\ {{\bf{M}}}_{k}\left[\left.{{\rm{B}}}_{k+1}\left({f}_{k}\left(x,{u}_{k},{\xi }_{k}\right)\right)\right|{f}_{k}\left(x,{u}_{k},{\xi }_{k}\right)\in {{\mathcal{O}}}_{k+1}\right]=0,\end{array}$$
the expression (A.1) takes the form
$$\begin{array}{lcl}{{\rm{B}}}_{k}\left(x\right)=\mathop{\rm{max}}\limits_{{u}_{k}\in {U}_{k}}{{\bf{I}}}_{{{\mathcal{F}}}_{k}}\left(x\right)\left({\bf{P}}\left({f}_{k}\left(x,{u}_{k},{\xi }_{k}\right)\in {{\mathcal{I}}}_{k+1}\right)\right.\\ \left.+{\bf{P}}\left({f}_{k}\left(x,{u}_{k},{\xi }_{k}\right)\in {{\mathcal{B}}}_{k+1}\right){{\bf{M}}}_{k}\left[\left.{{\rm{B}}}_{k+1}\left({f}_{k}\left(x,{u}_{k},{\xi }_{k}\right)\right)\right|{f}_{k}\left(x,{u}_{k},{\xi }_{k}\right)\in {{\mathcal{B}}}_{k+1}\right]\right).\end{array}$$
(A.8)
Since
$${{\bf{M}}}_{k}\left[\left.{{\rm{B}}}_{k+1}\left({f}_{k}\left(x,{u}_{k},{\xi }_{k}\right)\right)\right|{f}_{k}\left(x,{u}_{k},{\xi }_{k}\right)\in {{\mathcal{B}}}_{k+1}\right]\in \left(0,1\right),$$
the right-hand side of the expression (A.8) is equal to 0 if
$$\left\{\begin{array}{l}\mathop{\rm{max}}\limits_{{u}_{k}\in {U}_{k}}{{\bf{I}}}_{{{\mathcal{F}}}_{k}}\left(x\right){\bf{P}}\left({f}_{k}\left(x,{u}_{k},{\xi }_{k}\right)\in {{\mathcal{I}}}_{k+1}\right)=0\\ \mathop{\rm{max}}\limits_{{u}_{k}\in {U}_{k}}{{\bf{I}}}_{{{\mathcal{F}}}_{k}}\left(x\right){\bf{P}}\left({f}_{k}\left(x,{u}_{k},{\xi }_{k}\right)\in {{\mathcal{B}}}_{k+1}\right)=0,\end{array}\right.$$
which is equivalent to
$$\forall {u}_{k}\in {U}_{k}:\left\{\begin{array}{l}{{\bf{I}}}_{{{\mathcal{F}}}_{k}}(x){\bf{P}}({f}_{k}(x,{u}_{k},{\xi }_{k})\in {{\mathcal{I}}}_{k+1})=0\\ {{\bf{I}}}_{{{\mathcal{F}}}_{k}}(x)(1-{\bf{P}}({f}_{k}(x,{u}_{k},{\xi }_{k})\in {{\mathcal{I}}}_{k+1})-{\bf{P}}({f}_{k}(x,{u}_{k},{\xi }_{k})\ \in \ {{\mathcal{O}}}_{k+1}))=0.\end{array}\right.$$
In view of (A.2), then it follows that
$$\begin{array}{cccc}{{\mathcal{O}}}_{k}=\left\{x\in {{\mathbb{R}}}^{n}:\mathop{\rm{max}}\limits_{{u}_{k}\in {U}_{k}}{{\bf{M}}}_{k}\left[{{\bf{I}}}_{{{\mathcal{F}}}_{k}}\left({x}_{k}\right){{\rm{B}}}_{k+1}\left({f}_{k}\left({x}_{k},{u}_{k},{\xi }_{k}\right)\right)| {x}_{k}=x\right]=0\right\}\\ =\left\{x\in {{\mathbb{R}}}^{n}:\quad \mathop{\rm{max}}\limits_{{u}_{k}\in {U}_{k}}{{\bf{I}}}_{{{\mathcal{F}}}_{k}}\left(x\right){\bf{P}}\left({f}_{k}\left(x,{u}_{k},{\xi }_{k}\right)\in {{\mathcal{O}}}_{k+1}\right)=1\right\}\\ ={\overline{{\mathcal{F}}}}_{k}\cup \left\{x\in {{\mathbb{R}}}^{n}:\quad \mathop{\rm{max}}\limits_{{u}_{k}\in {U}_{k}}\left(x\right){\bf{P}}\left({f}_{k}\left(x,{u}_{k},{\xi }_{k}\right)\in {{\mathcal{O}}}_{k+1}\right)=1\right\}\\ ={\overline{{\mathcal{F}}}}_{k}\cup \left\{x\in {{\mathbb{R}}}^{n}:\quad \forall {u}_{k}\in {U}_{k}:\quad \left(x\right){\bf{P}}\left({f}_{k}\left(x,{u}_{k},{\xi }_{k}\right)\in {{\mathcal{O}}}_{k+1}\right)=1\right\}.\end{array}$$
(A.9)
As can be observed from (A.9), for \({x}_{k}\in {{\mathcal{O}}}_{k}\) any admissible control uk ∈ Uk is optimal.
Items 2 and 4 of Theorem 2 are established.
Consider the Bellman equation (A.8). Let \(x\in {{\mathcal{B}}}_{k};\) in this case,
$$\begin{array}{ccccccc}{{\rm{B}}}_{k}\left(x\right)=\mathop{\rm{max}}\limits_{{u}_{k}\in {U}_{k}}\left\{{\bf{P}}\left({f}_{k}\left(x,{u}_{k},{\xi }_{k}\right)\in {{\mathcal{I}}}_{k+1}\right)+{\bf{P}}\left({f}_{k}\left(x,{u}_{k},{\xi }_{k}\right)\in {{\mathcal{B}}}_{k+1}\right)\right.\\ \left.\times {{\bf{M}}}_{k}\left[\left.{{\rm{B}}}_{k+1}\left({f}_{k}\left(x,{u}_{k},{\xi }_{k}\right)\right)\right|{f}_{k}\left(x,{u}_{k},{\xi }_{k}\right)\in {{\mathcal{B}}}_{k+1}\right]\right\}\\ =\mathop{\rm{max}}\limits_{{u}_{k}\in {U}_{k}}\left\{{\bf{P}}\left({f}_{k}\left(x,{u}_{k},{\xi }_{k}\right)\in {{\mathcal{I}}}_{k+1}\right)\right.\\ +\left(1-{\bf{P}}\left({f}_{k}\left(x,{u}_{k},{\xi }_{k}\right)\in {{\mathcal{I}}}_{k+1}\right)-{\bf{P}}\left({f}_{k}\left(x,{u}_{k},{\xi }_{k}\right)\in {{\mathcal{O}}}_{k+1}\right)\right)\\ \left.\times {{\bf{M}}}_{k}\left[\left.{{\rm{B}}}_{k+1}\left({f}_{k}\left(x,{u}_{k},{\xi }_{k}\right)\right)\right|{f}_{k}\left(x,{u}_{k},{\xi }_{k}\right)\in {{\mathcal{B}}}_{k+1}\right]\right\}\\ =\mathop{\rm{max}}\limits_{{u}_{k}\in {U}_{k}}\left\{{\bf{P}}\left({f}_{k}\left(x,{u}_{k},{\xi }_{k}\right)\in {{\mathcal{I}}}_{k+1}\right)\left(1-{{\bf{M}}}_{k}\left[\left.{{\rm{B}}}_{k+1}\left({f}_{k}\left(x,{u}_{k},{\xi }_{k}\right)\right)\right|{f}_{k}\left(x,{u}_{k},{\xi }_{k}\right)\in {{\mathcal{B}}}_{k+1}\right]\right)\right.\\ \left.+\left(1-{\bf{P}}\left({f}_{k}\left(x,{u}_{k},{\xi }_{k}\right)\in {{\mathcal{O}}}_{k+1}\right)\right){{\bf{M}}}_{k}\left[\left.{{\rm{B}}}_{k+1}\left({f}_{k}\left(x,{u}_{k},{\xi }_{k}\right)\right)\right|{f}_{k}\left(x,{u}_{k},{\xi }_{k}\right)\in {{\mathcal{B}}}_{k+1}\right]\right\}.\end{array}$$
(A.10)
Take advantage of the well-known inequality
$$\mathop{\rm{min}}\limits_{i = \overline{1,n}}{z}_{i}\leqslant \mathop{\sum }\limits_{i = 1}^{n}{a}_{i}{z}_{i}\leqslant \mathop{\rm{max}}\limits_{i = \overline{1,n}}{z}_{i},\quad {a}_{1},\ldots ,{a}_{n}\geqslant 0,\quad \mathop{\sum }\limits_{i = 1}^{n}{a}_{i}=1.$$
Formally letting
$$\begin{array}{ccc}{a}_{1}={{\bf{M}}}_{k}\left[\left.{{\rm{B}}}_{k+1}\left({f}_{k}\left(x,{u}_{k},{\xi }_{k}\right)\right)\right|{f}_{k}\left(x,{u}_{k},{\xi }_{k}\right)\in {{\mathcal{B}}}_{k+1}\right],\\ {a}_{2}=1-{{\bf{M}}}_{k}\left[\left.{{\rm{B}}}_{k+1}\left({f}_{k}\left(x,{u}_{k},{\xi }_{k}\right)\right)\right|{f}_{k}\left(x,{u}_{k},{\xi }_{k}\right)\in {{\mathcal{B}}}_{k+1}\right],\\ {z}_{1}={\bf{P}}\left({f}_{k}\left(x,{u}_{k},{\xi }_{k}\right)\in {{\mathcal{I}}}_{k+1}\right),\quad {z}_{2}=1-{\bf{P}}\left({f}_{k}\left(x,{u}_{k},{\xi }_{k}\right)\in {{\mathcal{O}}}_{k+1}\right),\end{array}$$
gives the following two-sided estimate for the right-hand side of the Bellman equation:
$$\mathop{\rm{min}}\limits_{i = \overline{1,2}}{z}_{i}\leqslant {{\bf{M}}}_{k}\left[{{\rm{B}}}_{k+1}\left({f}_{k}\left(x,{u}_{k},{\xi }_{k}\right)\right)\right]\leqslant \mathop{\rm{max}}\limits_{i = \overline{1,2}}{z}_{i},x\in {{\mathcal{B}}}_{k}.$$
Due to the equality
$${\bf{P}}({f}_{k}(x,{u}_{k},{\xi }_{k})\in {{\mathcal{I}}}_{k+1})+{\bf{P}}({f}_{k}(x,{u}_{k},{\xi }_{k})\in {{\mathcal{B}}}_{k+1})+{\bf{P}}({f}_{k}(x,{u}_{k},{\xi }_{k})\in {{\mathcal{O}}}_{k+1})=1,$$
z2 can be also represented as
$${z}_{2}={\bf{P}}\left({f}_{k}\left(x,{u}_{k},{\xi }_{k}\right)\in {{\mathcal{I}}}_{k+1}\right)+{\bf{P}}\left({f}_{k}\left(x,{u}_{k},{\xi }_{k}\right)\in {{\mathcal{B}}}_{k+1}\right).$$
Hence, z2 ⩾ z1, and
$${z}_{1}\leqslant {{\bf{M}}}_{k}\left[{{\rm{B}}}_{k+1}\left({f}_{k}\left(x,{u}_{k},{\xi }_{k}\right)\right)\right]\leqslant {z}_{2}.$$
In accordance with items 1 and 2 of Theorem 2, the inclusions \({{\mathcal{I}}}_{k}\subseteq {{\mathcal{F}}}_{k}\) and \({\overline{{\mathcal{F}}}}_{k}\subseteq {{\mathcal{O}}}_{k}\) hold for all \(k=\overline{0,N}\). Consequently, \({{\mathcal{I}}}_{k}\cup {{\mathcal{B}}}_{k}\subseteq {{\mathcal{F}}}_{k}\), and then
$${\bf{P}}\left({f}_{k}\left(x,{u}_{k},{\xi }_{k}\right)\in {{\mathcal{I}}}_{k+1}\right)+{\bf{P}}\left({f}_{k}\left(x,{u}_{k},{\xi }_{k}\right)\in {{\mathcal{B}}}_{k+1}\right)\leqslant {\bf{P}}\left({f}_{k}\left(x,{u}_{k},{\xi }_{k}\right)\in {{\mathcal{F}}}_{k+1}\right).$$
As a result,
$${{\bf{M}}}_{k}\left[{{\rm{B}}}_{k+1}\left({f}_{k}\left(x,{u}_{k},{\xi }_{k}\right)\right)\right]\leqslant {\bf{P}}\left({f}_{k}\left(x,{u}_{k},{\xi }_{k}\right)\in {{\mathcal{F}}}_{k+1}\right).$$
Thus, the system of inequalities (8) is valid. Taking the supremum over uk ∈ Uk in all sides of the inequalities finally yields (10). Items 6 and 7 of Theorem 2 are established.
The proof of Theorem 2 is complete.
Proof of Theorem 3. Introduce a system of hypotheses forming a complete group of incompatible events:
$$\left\{\mathop{\bigcap }\limits_{k = 0}^{N}\left\{{\underline{x}}_{k}\in {{\mathcal{I}}}_{k}\right\}\right\},\quad \left\{\mathop{\bigcup }\limits_{k = 0}^{N}\left\{{\underline{x}}_{k}\notin {{\mathcal{I}}}_{k}\right\}\right\}.$$
Then the total probability formula gives the chain of equalities
$$\begin{array}{ccc}{P}_{\varphi }\left(\underline{u}\left(\cdot \right)\right)={\bf{P}}\left(\mathop{\bigcap }\limits_{k = 0}^{N}\left\{{\underline{x}}_{k+1}\in {{\mathcal{F}}}_{k+1}\right\}\right)\\ ={\bf{P}}\left(\mathop{\bigcap }\limits_{k = 0}^{N}\left\{{\underline{x}}_{k}\in {{\mathcal{I}}}_{k}\right\}\right){\bf{P}}\left(\left.\mathop{\bigcap }\limits_{k = 0}^{N}\left\{{\underline{x}}_{k+1}\in {{\mathcal{F}}}_{k+1}\right\}\right|\mathop{\bigcap }\limits_{k=0}^{N}\left\{{\underline{x}}_{k}\in {{\mathcal{I}}}_{k}\right\}\right)\\ +{\bf{P}}\left(\mathop{\bigcup }\limits_{k = 0}^{N}\left\{{\underline{x}}_{k}\notin {{\mathcal{I}}}_{k}\right\}\right){\bf{P}}\left(\left.\mathop{\bigcap }\limits_{k = 0}^{N}\left\{{\underline{x}}_{k+1}\in {{\mathcal{F}}}_{k+1}\right\}\right|\mathop{\bigcup }\limits_{k = 0}^{N}\left\{{\underline{x}}_{k}\notin {{\mathcal{I}}}_{k}\right\}\right).\end{array}$$
(A.11)
From the definition of the control \(\underline{u}\left(\cdot \right)\) and the properties of the set \({{\mathcal{I}}}_{k}\) (item 1 of Theorem 2) it follows that
$${\bf{P}}\left({\underline{x}}_{k+1}\in {{\mathcal{I}}}_{k+1}\right)=1\quad \,\text{for}\,\quad {\underline{x}}_{k}\in {{\mathcal{I}}}_{k},\quad k=\overline{0,N}.$$
(A.12)
Then, due to the inclusion \({{\mathcal{I}}}_{k}\subseteq {{\mathcal{F}}}_{k}\) holding for all \(k=\overline{1,N+1}\),
$${\bf{P}}\left({\underline{x}}_{k+1}\in {{\mathcal{F}}}_{k+1}\right)=1\quad \,\text{for}\,\quad {\underline{x}}_{k}\in {{\mathcal{I}}}_{k},\quad k=\overline{0,N},$$
and consequently
$${\bf{P}}\left(\mathop{\bigcap }\limits_{k = 0}^{N}\left\{\left.{\underline{x}}_{k+1}\in {{\mathcal{F}}}_{k+1}\right\}\right|\mathop{\bigcap }\limits_{k = 0}^{N}\left\{{\underline{x}}_{k}\in {{\mathcal{I}}}_{k}\right\}\right)=1.$$
In view of this equality, the expression (A.11) can be written as
$$\begin{array}{cc}{P}_{\varphi }\left(\underline{u}\left(\cdot \right)\right)={\bf{P}}\left(\mathop{\bigcap }\limits_{k = 0}^{N}\left\{{\underline{x}}_{k}\in {{\mathcal{I}}}_{k}\right\}\right)\\ +{\bf{P}}\left(\mathop{\bigcup }\limits_{k = 0}^{N}\left\{{\underline{x}}_{k}\notin {{\mathcal{I}}}_{k}\right\}\right){\bf{P}}\left(\left.\mathop{\bigcap }\limits_{k = 0}^{N}\left\{{\underline{x}}_{k+1}\in {{\mathcal{F}}}_{k+1}\right\}\right|\mathop{\bigcup }\limits_{k = 0}^{N}\left\{{\underline{x}}_{k}\notin {{\mathcal{I}}}_{k}\right\}\right).\end{array}$$
(A.13)
Now introduce another system of hypotheses forming a complete group of incompatible events:
$$\left\{\mathop{\bigcap }\limits_{k = 0}^{N-1}\left\{{\underline{x}}_{k}\in {{\mathcal{I}}}_{k}\right\}\right\},\quad \left\{\mathop{\bigcup }\limits_{k = 0}^{N-1}\left\{{\underline{x}}_{k}\notin {{\mathcal{I}}}_{k}\right\}\right\}.$$
Apply the total probability formula for the first term in the right-hand side of (A.13):
$$\begin{array}{cc}{\bf{P}}\left(\mathop{\bigcap }\limits_{k = 0}^{N}\left\{{\underline{x}}_{k}\in {{\mathcal{I}}}_{k}\right\}\right)={\bf{P}}\left(\mathop{\bigcap }\limits_{k = 0}^{N-1}\left\{{\underline{x}}_{k}\in {{\mathcal{I}}}_{k}\right\}\right){\bf{P}}\left(\mathop{\bigcap }\limits_{k = 0}^{N-1}\left\{\left.{\underline{x}}_{k+1}\in {{\mathcal{I}}}_{k+1}\right\}\right|\mathop{\bigcap }\limits_{k = 0}^{N-1}\left\{{\underline{x}}_{k}\in {{\mathcal{I}}}_{k}\right\}\right)\\ +{\bf{P}}\left(\mathop{\bigcup }\limits_{k = 0}^{N-1}\left\{{\underline{x}}_{k}\notin {{\mathcal{I}}}_{k}\right\}\right){\bf{P}}\left(\mathop{\bigcap }\limits_{k = 0}^{N-1}\left\{\left.{\underline{x}}_{k+1}\in {{\mathcal{I}}}_{k+1}\right\}\right|\mathop{\bigcup }\limits_{k = 0}^{N-1}\left\{{\underline{x}}_{k}\notin {{\mathcal{I}}}_{k}\right\}\right).\end{array}$$
(A.14)
Taking (A.12) into account, transform the right-hand side of (A.14) to obtain
$$\begin{array}{cc}{\bf{P}}\left(\mathop{\bigcap }\limits_{k = 0}^{N}\left\{{\underline{x}}_{k}\in {{\mathcal{I}}}_{k}\right\}\right)={\bf{P}}\left(\mathop{\bigcap }\limits_{k = 0}^{N-1}\left\{{\underline{x}}_{k}\in {{\mathcal{I}}}_{k}\right\}\right)\\ +{\bf{P}}\left(\mathop{\bigcup }\limits_{k = 0}^{N-1}\left\{{\underline{x}}_{k}\notin {{\mathcal{I}}}_{k}\right\}\right){\bf{P}}\left(\left.\mathop{\bigcap }\limits_{k = 0}^{N-1}\left\{{\underline{x}}_{k+1}\in {{\mathcal{I}}}_{k+1}\right\}\right|\mathop{\bigcup }\limits_{k = 0}^{N-1}\left\{{\underline{x}}_{k}\notin {{\mathcal{I}}}_{k}\right\}\right).\end{array}$$
(A.15)
Substituting (A.15) into (A.13) yields
$$\begin{array}{ccc}{P}_{\varphi }\left(\underline{u}\left(\cdot \right)\right)={\bf{P}}\left(\mathop{\bigcap }\limits_{k = 0}^{N-1}\left\{{\underline{x}}_{k}\in {{\mathcal{I}}}_{k}\right\}\right)\\ +{\bf{P}}\left(\mathop{\bigcup }\limits_{k = 0}^{N-1}\left\{{\underline{x}}_{k}\notin {{\mathcal{I}}}_{k}\right\}\right){\bf{P}}\left(\left.\mathop{\bigcap }\limits_{k = 0}^{N-1}\left\{{\underline{x}}_{k+1}\in {{\mathcal{I}}}_{k+1}\right\}\right|\mathop{\bigcup }\limits_{k = 0}^{N-1}\left\{{\underline{x}}_{k}\notin {{\mathcal{I}}}_{k}\right\}\right)\\ +{\bf{P}}\left(\mathop{\bigcup }\limits_{k = 0}^{N}\left\{{\underline{x}}_{k}\notin {{\mathcal{I}}}_{k}\right\}\right){\bf{P}}\left(\left.\mathop{\bigcap }\limits_{k = 0}^{N}\left\{{\underline{x}}_{k+1}\in {{\mathcal{F}}}_{k+1}\right\}\right|\mathop{\bigcup }\limits_{k = 0}^{N}\left\{{\underline{x}}_{k}\notin {{\mathcal{I}}}_{k}\right\}\right).\end{array}$$
(A.16)
Perform the analogous transformations for the first term in (A.16), introducing the systems of hypotheses
$$\left\{\mathop{\bigcap }\limits_{k = 0}^{l}\left\{{\underline{x}}_{k}\in {{\mathcal{I}}}_{k}\right\}\right\},\quad \left\{\mathop{\bigcup }\limits_{k = 0}^{l}\left\{{\underline{x}}_{k}\notin {{\mathcal{I}}}_{k}\right\}\right\},\quad l=\overline{1,\ldots ,N-2},$$
to obtain the following expression for the value of the probabilistic criterion under the control \(\underline{u}\left(\cdot \right)\):
$$\begin{array}{ccc}{P}_{\varphi }\left(\underline{u}\left(\cdot \right)\right)={\bf{P}}\left({\underline{x}}_{1}\in {{\mathcal{I}}}_{1}\right)\\ +\mathop{\sum }\limits_{l=1}^{N-1}{\bf{P}}\left(\mathop{\bigcup }\limits_{k = 0}^{l}\left\{{\underline{x}}_{k}\notin {{\mathcal{I}}}_{k}\right\}\right){\bf{P}}\left(\left.\mathop{\bigcap }\limits_{k = 0}^{l}\left\{{\underline{x}}_{k+1}\in {{\mathcal{I}}}_{k+1}\right\}\right|\mathop{\bigcup }\limits_{k = 0}^{l}\left\{{\underline{x}}_{k}\notin {{\mathcal{I}}}_{k}\right\}\right)\\ +{\bf{P}}\left(\mathop{\bigcup }\limits_{k = 0}^{N}\left\{{\underline{x}}_{k}\notin {{\mathcal{I}}}_{k}\right\}\right){\bf{P}}\left(\left.\mathop{\bigcap }\limits_{k = 0}^{N}\left\{{\underline{x}}_{k+1}\in {{\mathcal{F}}}_{k+1}\right\}\right|\mathop{\bigcup }\limits_{k = 0}^{N}\left\{{\underline{x}}_{k}\notin {{\mathcal{I}}}_{k}\right\}\right).\end{array}$$
(A.17)
Note that the first term in (A.17) satisfies the chain of equalities
$${\bf{P}}\left({\underline{x}}_{1}\in {{\mathcal{I}}}_{1}\right)={\bf{P}}\left({f}_{0}\left(X,{\underline{u}}_{0},{\xi }_{0}\right)\in {{\mathcal{I}}}_{1}\right)={\underline{{\rm{B}}}}_{0}\left(X\right),$$
which implies the expression (14).
Item 1 of Theorem 3 is established.
For establishing item 3 of Theorem 3, it suffices to observe that for \({x}_{k}\in {{\mathcal{I}}}_{k}\), \({u}_{k}^{* }={\underline{u}}_{k}\) for all \(k=\overline{0,N}\). Following the same procedure as above, this equality can be used for deriving the expression (14) for the optimal-value function of the probabilistic criterion on the trajectories \({\left\{{x}_{k}^{* }\right\}}_{k = 1}^{N+1}\) of the closed loop system with the optimal control \({u}^{* }\left(\cdot \right)\).
Item 2 of Theorem 3 is established.
Item 3 of Theorem 3 directly follows from item 7 of Theorem 2 and item 1 of Theorem 3.
The proof of Theorem 3 is complete.
Proof of Lemma 1. Using item 1 of Theorem 2, write the isobell of level 1 of the Bellman function for k = N:
$${{\mathcal{I}}}_{N}={{\mathcal{F}}}_{N}\cap \left\{x\in {{\mathbb{R}}}^{2}:\exists u\in {U}_{N}:{\bf{P}}\left({\left\Vert \Omega \left({A}_{N}x+\left({A}_{N}^{\xi }x+{B}_{N}u\right){\xi }_{N}\right)\right\Vert }_{\infty }\leqslant \varphi \right)=1\right\}.$$
Since the random variable ξN has an unbounded support, the optimal control for k = N and \({x}_{N-1}\in {{\mathcal{I}}}_{N}\) is unique and given by \({u}_{N}^{* }=-{B}_{N}^{{\mathtt{T}}}{A}_{N}^{\xi }{x}_{N-1}\); moreover, the isobell of level 1 of the Bellman function has the form
$${{\mathcal{I}}}_{N}={{\mathcal{F}}}_{N}\cap \left\{x\in {{\mathbb{R}}}^{2}:{\left\Vert \Omega {A}_{N}x\right\Vert }_{\infty }\leqslant \varphi \right\}=\left\{x\in {{\mathbb{R}}}^{2}:{\rm{max}}\left\{{\left\Vert \Omega x\right\Vert }_{\infty },\left|{\beta }_{N}^{{\mathtt{T}}}x\right|\right\}\leqslant \varphi \right\}.$$
In accordance with item 2 of Theorem 2, for k = N the isobell of level 0 of the Bellman function is given by
$$\begin{array}{cc}{{\mathcal{O}}}_{N}={\overline{{\mathcal{F}}}}_{N}\cup \left\{x\in {{\mathbb{R}}}^{2}:\forall u\in {U}_{N}:{\bf{P}}\left({\left\Vert \Omega \left({A}_{N}x+\left({A}_{N}^{\xi }x+{B}_{N}u\right){\xi }_{N}\right)\right\Vert }_{\infty }>\varphi \right)=1\right\}\\ ={\overline{{\mathcal{F}}}}_{N}\cup \varnothing =\left\{x\in {{\mathbb{R}}}^{2}:{\left\Vert \Omega x\right\Vert }_{\infty }>\varphi \right\}.\end{array}$$
By analogy with step k = N, for k = N − 1 the isobell of level 1 can be written as
$$\begin{array}{lccc}{{\mathcal{I}}}_{N-1}={{\mathcal{F}}}_{N-1}\cap \left\{x\in {{\mathbb{R}}}^{2}:\ \exists u\in {U}_{N-1}:\right.\\ {\bf{P}}\left({\rm{max}}\left\{{\left\Vert \Omega \left({A}_{N-1}x+\left({A}_{N-1}^{\xi }x+{B}_{N-1}u\right){\xi }_{N-1}\right)\right\Vert }_{\infty },\right.\right.\\ \left.\left.\left.{\left\Vert \Omega {A}_{N}\left({A}_{N-1}x+\left({A}_{N-1}^{\xi }x+{B}_{N-1}u\right){\xi }_{N-1}\right)\right\Vert }_{\infty }\right\}\leqslant \varphi \right)=1\right\}\\ =\left\{x\in {{\mathbb{R}}}^{2}:max\left\{{\left\Vert \Omega x\right\Vert }_{\infty },\mathop{\rm{max}}\limits_{j = \overline{N-1,N}}\left|{\beta }_{j}^{{\mathtt{T}}}x\right|\right\}\leqslant \varphi \right\}.\end{array}$$
Note that, like in the case k = N, if \({x}_{N-1}\in {{\mathcal{I}}}_{N-1}\), then
$${u}_{N-1}^{* }=-{B}_{N-1}^{{\mathtt{T}}}{A}_{N-1}^{\xi }{x}_{N-1}$$
and the isobell of level 0 of the Bellman function has the form
$$\begin{array}{l}{{\mathcal{O}}}_{N-1}={\overline{{\mathcal{F}}}}_{N-1}\cup \left\{x\in {{\mathbb{R}}}^{2}:\right.\\ \left.\forall u\in {U}_{N-1}:{\bf{P}}\left({\left\Vert \Omega \left({A}_{N-1}x+\left({A}_{N-1}^{\xi }x+{B}_{N-1}u\right){\xi }_{N-1}\right)\right\Vert }_{\infty }>\varphi \right)=1\right\}\\ ={\overline{{\mathcal{F}}}}_{N-1}\cup \varnothing =\left\{x\in {{\mathbb{R}}}^{2}:{\left\Vert \Omega x\right\Vert }_{\infty }>\varphi \right\}.\end{array}$$
These considerations are continued further by induction, and the conclusion follows.
The proof of Lemma 1 is complete.
Proof of Lemma 2. Perform a series of transformations for the right-hand side of the expression (20) as follows:
$$\begin{array}{l}{\underline{{\rm{B}}}}_{k}\left(x\right)=\mathop{\rm{max}}\limits_{u}{\bf{P}}\left({\rm{max}}\left\{\dot{\omega }\left|{e}_{2}^{{\mathtt{T}}}\left({A}_{k}x+\left({A}_{k}^{\xi }x+{B}_{k}{u}_{k}\right){\xi }_{k}\right)\right|,\right.\right.\\ \left.\left.\mathop{\rm{max}}\limits_{l = \overline{k,N}}\omega \left|{\beta }_{l+1}^{{\mathtt{T}}}\left({A}_{k}x+\left({A}_{k}^{\xi }x+{B}_{k}{u}_{k}\right){\xi }_{k}\right)\right|\right\}\leqslant \varphi \right)\\ ={\bf{P}}\left(\mathop{\bigcap }\limits_{l = k}^{N}\left\{\omega \left|{\beta }_{l+1}^{{\mathtt{T}}}\left({A}_{k}x+\left({A}_{k}^{\xi }x+{B}_{k}{u}_{k}\right){\xi }_{k}\right)\right|\leqslant \varphi \right\}\right.\\ \left.\cap \left\{\dot{\omega }\left|{e}_{2}^{{\mathtt{T}}}\left({A}_{k}x+\left({A}_{k}^{\xi }x+{B}_{k}{u}_{k}\right){\xi }_{k}\right)\right|\leqslant \varphi \right\}\right)\\ ={\bf{P}}\left(\mathop{\bigcap }\limits_{l = k}^{N}\left\{\frac{-\varphi {\omega }^{-1}-{\beta }_{l+1}^{{\mathtt{T}}}{A}_{k}x}{{e}_{2}^{{\mathtt{T}}}{\beta }_{l+1}}\leqslant {e}_{2}^{{\mathtt{T}}}\left({A}_{k}^{\xi }x+{B}_{k}{u}_{k}\right){\xi }_{k}\leqslant \frac{-\varphi {\omega }^{-1}-{\beta }_{l+1}^{{\mathtt{T}}}{A}_{k}x}{{e}_{2}^{{\mathtt{T}}}{\beta }_{l+1}}\right\}\right.\\ \left.\cap \left\{-\varphi {\dot{\omega }}^{-1}-{e}_{2}^{{\mathtt{T}}}{A}_{k}x\leqslant {e}_{2}^{{\mathtt{T}}}\left({A}_{k}^{\xi }x+{B}_{k}{u}_{k}\right){\xi }_{k}\leqslant \varphi {\dot{\omega }}^{-1}-{e}_{2}^{{\mathtt{T}}}{A}_{k}x\right\}\right).\end{array}$$
Introduce the change of variables for the control \({\tilde{u}}_{k}={e}_{2}^{{\mathtt{T}}}\left({A}_{k}^{\xi }x+{B}_{k}{u}_{k}\right)\), and denote by \({\mathop \xi \limits^\circ _k} = {\xi _k} - {m_\xi }\) the centered random variable. Continue transformations for the right-hand side of the expression (20):
$$\begin{array}{cccccccccccc}{\bf{P}}\left(\mathop{\bigcap }\limits_{l = k}^{N}\left\{\frac{-\varphi {\omega }^{-1}-{\beta }_{l+1}^{{\mathtt{T}}}{A}_{k}x}{{e}_{2}^{{\mathtt{T}}}{\beta }_{l+1}}\leqslant {\tilde{u}}_{k}{\xi }_{k}\leqslant \frac{-\varphi {\omega }^{-1}-{\beta }_{l+1}^{{\mathtt{T}}}{A}_{k}x}{{e}_{2}^{{\mathtt{T}}}{\beta }_{l+1}}\right\}\right.\\ \left.\cap \left\{-\varphi {\dot{\omega }}^{-1}-{e}_{2}^{{\mathtt{T}}}{A}_{k}x\leqslant {\tilde{u}}_{k}{\xi }_{k}\leqslant \varphi {\dot{\omega }}^{-1}-{e}_{2}^{{\mathtt{T}}}{A}_{k}x\right\}\right)\\ ={\bf{P}}\left(\mathop{\bigcap }\limits_{l = k}^{N}\left\{\frac{-\varphi {\omega }^{-1}-{\beta }_{l+1}^{{\mathtt{T}}}{A}_{k}x}{{e}_{2}^{{\mathtt{T}}}{\beta }_{l+1}}\leqslant {\tilde{u}}_{k}\left({\xi }_{k}^{\circ }+{m}_{\xi }\right)\leqslant \frac{-\varphi {\omega }^{-1}-{\beta }_{l+1}^{{\mathtt{T}}}{A}_{k}x}{{e}_{2}^{{\mathtt{T}}}{\beta }_{l+1}}\right\}\right.\\ \left.\cap \left\{-\varphi {\dot{\omega }}^{-1}-{e}_{2}^{{\mathtt{T}}}{A}_{k}x\leqslant {\tilde{u}}_{k}\left({\xi }_{k}^{\circ }+{m}_{\xi }\right)\leqslant \varphi {\dot{\omega }}^{-1}-{e}_{2}^{{\mathtt{T}}}{A}_{k}x\right\}\right)\\ ={\bf{P}}\left(\mathop{\bigcap }\limits_{l = k}^{N}\left\{\frac{-\varphi {\omega }^{-1}-{\beta }_{l+1}^{{\mathtt{T}}}{A}_{k}x}{{m}_{\xi }{e}_{2}^{{\mathtt{T}}}{\beta }_{l+1}}\leqslant {\tilde{u}}_{k}\left({m}_{\xi }^{-1}{\xi }_{k}^{\circ }+1\right)\leqslant \frac{-\varphi {\omega }^{-1}-{\beta }_{l+1}^{{\mathtt{T}}}{A}_{k}x}{{m}_{\xi }{e}_{2}^{{\mathtt{T}}}{\beta }_{l+1}}\right\}\right.\\ \left.\cap \left\{\frac{-\varphi {\dot{\omega }}^{-1}-{e}_{2}^{{\mathtt{T}}}{A}_{k}x}{{m}_{\xi }}\leqslant {\tilde{u}}_{k}\left({m}_{\xi }^{-1}{\xi }_{k}^{\circ }+1\right)\leqslant \frac{\varphi {\dot{\omega }}^{-1}-{e}_{2}^{{\mathtt{T}}}{A}_{k}x}{{m}_{\xi }}\right\}\right)\\ ={\bf{P}}\left({\rm{max}}\left\{\frac{-\varphi {\dot{\omega }}^{-1}-{e}_{2}^{{\mathtt{T}}}{A}_{k}x}{{m}_{\xi }},\mathop{\rm{max}}\limits_{l = \overline{k,N}}\frac{-\varphi {\omega }^{-1}-{\beta }_{l+1}^{{\mathtt{T}}}{A}_{k}x}{{m}_{\xi }{e}_{2}^{{\mathtt{T}}}{\beta }_{l+1}}\right\}\leqslant {\tilde{u}}_{k}\left({m}_{\xi }^{-1}{\xi }_{k}^{\circ }+1\right)\right.\\ \left.\leqslant {\rm{min}}\left\{\frac{\varphi {\dot{\omega }}^{-1}-{e}_{2}^{{\mathtt{T}}}{A}_{k}x}{{m}_{\xi }},\mathop{\rm{min}}\limits_{l = \overline{k,N}}\frac{\varphi {\omega }^{-1}-{\beta }_{l+1}^{{\mathtt{T}}}{A}_{k}x}{{m}_{\xi }{e}_{2}^{{\mathtt{T}}}{\beta }_{l+1}}\right\}\right)\\ ={\bf{P}}\left({\underline{\varphi }}_{k}\left(x\right)\leqslant {\tilde{u}}_{k}\left({m}_{\xi }^{-1}{\xi }_{k}^{\circ }+1\right)\leqslant {\overline{\varphi }}_{k}\left(x\right)\right)\\ ={\bf{P}}\left(-\frac{{\overline{\varphi }}_{k}\left(x\right)-{\underline{\varphi }}_{k}\left(x\right)}{2}+\frac{{\overline{\varphi }}_{k}\left(x\right)+{\overline{\varphi }}_{k}\left(x\right)}{2}\leqslant {\tilde{u}}_{k}\left({m}_{\xi }^{-1}{\xi }_{k}^{\circ }+1\right)\right.\\ \left.\leqslant \frac{{\overline{\varphi }}_{k}\left(x\right)-{\underline{\varphi }}_{k}\left(x\right)}{2}+\frac{{\overline{\varphi }}_{k}\left(x\right)+{\overline{\varphi }}_{k}\left(x\right)}{2}\right)\\ ={\bf{P}}\left(-{r}_{k}\left(x\right)-{c}_{k}\left(x\right)\leqslant {\tilde{u}}_{k}\left({m}_{\xi }^{-1}{\xi }_{k}^{\circ }+1\right)\leqslant {r}_{k}\left(x\right)-{c}_{k}\left(x\right)\right).\end{array}$$
For the sake of convenience, introduce the notations \({\eta _k} = m_\xi ^{ - 1}{\mathop \xi \limits^\circ _k}\), \({\eta }_{k} \sim {\mathcal{N}}\left(0,{\sigma }_{\eta }^{2}\right)\), and \({\sigma }_{\eta }^{2}={\sigma }_{\xi }^{2}{m}_{\xi }^{-2}\). In view of the transformations and notations presented above, the expression (20) can be written as
$${\underline{{\mathcal{B}}}}_{k}\left(x\right)=\mathop{\rm{max}}\limits_{{\tilde{u}}_{k}}{\bf{P}}\left(\left|{c}_{k}\left(x\right)+{\tilde{u}}_{k}\left(1+{\eta }_{k}\right)\right|\leqslant {r}_{k}\left(x\right)\right).$$
(A.18)
The solution of the stochastic programming problem in the right-hand side of (A.18) is known up to the parameters \({c}_{k}\left(x\right)\) and \({r}_{k}\left(x\right);\) see [2, 15]. Under the condition \(x\in {{\mathcal{B}}}_{k}\) this solution has the form
$${\tilde{u}}_{k}^{* }=-2{c}_{k}\left(x\right){\left(1+\sqrt{1+2{\sigma }_{\eta }^{2}\mathrm{ln}\,\left(\frac{\left|{c}_{k}\left(x\right)\right|+{r}_{k}\left(x\right)}{\left|{c}_{k}\left(x\right)\right|-{r}_{k}\left(x\right)}\right)}\right)}^{-1}.$$
(A.19)
With the change of variables for \(\underline{u}\), this finally gives (22).
Items 1 and 2 of Lemma 2 are established.
For establishing items 3 and 4 of Lemma 2, it suffices to observe that, by analogy with the transformations used for proving items 1 and 2 of this lemma, the Bellman function (21) can be estimated from above as
$$\begin{array}{ccc}{\overline{{\rm{B}}}}_{k}\left(x\right)=\mathop{\rm{max}}\limits_{{u}_{k}}{\bf{P}}\left(\dot{\omega }\left|{e}_{2}^{{\mathtt{T}}}\left({A}_{k}x+\left({A}_{k}^{\xi }x+{B}_{k}{u}_{k}\right){\xi }_{k}\right)\right|\leqslant \varphi \right)\\ =\mathop{\rm{max}}\limits_{{\tilde{u}}_{k}}{\bf{P}}\left(\left|{e}_{2}^{{\mathtt{T}}}{A}_{k}{m}_{\xi }^{-1}x+{\tilde{u}}_{k}\left({m}_{\xi }^{-1}{\xi }_{k}^{\circ }+1\right)\right|\leqslant \varphi {\dot{\omega }}^{-1}{m}_{\xi }^{-1}\right)\\ =\mathop{\rm{max}}\limits_{{\tilde{u}}_{k}}{\bf{P}}\left(\left|{e}_{2}^{{\mathtt{T}}}{A}_{k}{m}_{\xi }^{-1}x+{\tilde{u}}_{k}\left(1+{\eta }_{k}\right)\right|\leqslant \varphi {\dot{\omega }}^{-1}{m}_{\xi }^{-1}\right).\end{array}$$
(A.20)
The stochastic programming problem in the right-hand side of (A.20) coincides, up to the parameters, with the stochastic programming problem in the right-hand side of (A.18). The same considerations as in the proof of items 1 and 2 of Lemma 2 finally yield (24) and (25).
The proof of Lemma 2 is complete.
