Lax-Oleinik-Type Formulas and Efficient Algorithms for Certain High-Dimensional Optimal Control Problems

Abstract

Two of the main challenges in optimal control are solving problems with state-dependent running costs and developing efficient numerical solvers that are computationally tractable in high dimensions. In this paper, we provide analytical solutions to certain optimal control problems whose running cost depends on the state variable and with constraints on the control. We also provide Lax-Oleinik-type representation formulas for the corresponding Hamilton-Jacobi partial differential equations with state-dependent Hamiltonians. Additionally, we present an efficient, grid-free numerical solver based on our representation formulas, which is shown to scale linearly with the state dimension, and thus, to overcome the curse of dimensionality. Using existing optimization methods and the min-plus technique, we extend our numerical solvers to address more general classes of convex and nonconvex initial costs. We demonstrate the capabilities of our numerical solvers using implementations on a central processing unit (CPU) and a field-programmable gate array (FPGA). In several cases, our FPGA implementation obtains over a 10 times speedup compared to the CPU, which demonstrates the promising performance boosts FPGAs can achieve. Our numerical results show that our solvers have the potential to serve as a building block for solving broader classes of high-dimensional optimal control problems in real-time.

Appendices

Appendix A Some Technical Lemmas for the Analytical Solutions

Lemma A1

Let a, b, t be positive scalars and $x,u$ be real numbers satisfying $u-bt\leqslant x\leqslant u+at$. Let $V$ be the function defined in (9) and  (15) and $[0,t]\ni s\mapsto \gamma (s;x,t,u,a,b)\in \mathbb {R}$ be the trajectory defined in (12),  (13), (14), and  (16) for different cases. Then, there holds

$$\begin{aligned} \int _0^t \dfrac{1}{2} \left( \gamma (s;x,t,u,a,b)\right) ^2 \textrm{d}s = V(x,t; u,a,b). \end{aligned}$$

(A1)

Proof

If $(x,t,u)\in \varOmega _1$ holds, we have

$$\begin{aligned}&\int _0^t \dfrac{1}{2} \left( \gamma (s;x,t,u,a,b)\right) ^2 \textrm{d}s\\ =&\int _0^{\frac{-x+u+ at}{a+b}} \dfrac{1}{2}(u- bs)^2 \textrm{d}s + \int _{\frac{-x+u+ at}{a+b}}^t \dfrac{1}{2}(as - at+x)^2 \textrm{d}s\\ =&-\left. \dfrac{1}{6b}(u-bs)^3\right| _0^{\frac{-x+u+ at}{a+b}} + \left. \dfrac{1}{6a}(as-at+x)^3\right| _{\frac{-x+u+ at}{a+b}}^t\\ =&-\left( \dfrac{1}{6b}+\dfrac{1}{6a}\right) \left( \dfrac{au+bx-abt}{a+b}\right) ^3 + \dfrac{u^3}{6b} + \dfrac{x^3}{6a}\\ =&V(x,t;u,a,b). \end{aligned}$$

If $(x,t,u)\in \varOmega _2$ holds, we have

$$\begin{aligned} \begin{aligned} \int _0^t \dfrac{1}{2} \left( \gamma (s;x,t,u,a,b)\right) ^2 \textrm{d}s&= \int _0^{\frac{u}{b}} \dfrac{1}{2}(u- bs)^2 \textrm{d}s + \int _{t-\frac{x}{a}}^t \dfrac{1}{2}(as - at+x)^2 \textrm{d}s\\&= -\left. \dfrac{1}{6b}(u-bs)^3\right| _0^{\frac{u}{b}} + \left. \dfrac{1}{6a}(as-at+x)^3\right| _{t-\frac{x}{a}}^t\\&= \dfrac{u^3}{6b} + \dfrac{x^3}{6a}\\&= V(x,t;u,a,b). \end{aligned} \end{aligned}$$

If $(x,t,u)\in \varOmega _3$ holds, we have

$$\begin{aligned} \begin{aligned} \int _0^t \dfrac{1}{2} \left( \gamma (s;x,t,u,a,b)\right) ^2 \textrm{d}s&= \int _0^{\frac{u}{b}} \dfrac{1}{2}(u- bs)^2 \textrm{d}s + \int _{t-\frac{-x}{b}}^t \dfrac{1}{2}(-bs +bt+x)^2 \textrm{d}s\\&= -\left. \frac{1}{6b}(u-bs)^3\right| _0^{\frac{u}{b}} - \left. \frac{1}{6b}(-bs+bt+x)^3\right| _{t-\frac{-x}{b}}^t\\&= \frac{u^3}{6b} - \frac{x^3}{6b}\\&= V(x,t;u,a,b). \end{aligned} \end{aligned}$$

If $u<0$, we have

$$\begin{aligned} \begin{aligned} \int _0^t \dfrac{1}{2} \left( \gamma (s;x,t,u,a,b)\right) ^2 \textrm{d}s&= \int _0^t \dfrac{1}{2} \left( \gamma (s;-x,t,-u,b,a)\right) ^2 \textrm{d}s\\&= V(-x,t;-u,b,a) = V(x,t;u,a,b). \end{aligned} \end{aligned}$$

Therefore, (A1) holds for any $(x,t,u)\in \mathbb {R}\times (0,+\infty )\times \mathbb {R}$ satisfying $u-bt\leqslant x\leqslant u+at$.

Lemma A2

Let a, b be positive scalars. Let $V$ be the function defined in (9) and  (15). Then, for any $x\in \mathbb {R}$, $t>0$, the function $\mathbb {R}\ni u\mapsto V(x,t;u,a,b)\in \mathbb {R}\cup \{+\infty \}$ is strictly convex and twice continuously differentiable in its domain.

Proof

In this proof, we regard the function $V(x,t;u,a,b)$ as a function of $u$ from its domain $[x-at,x+bt]$ to $\mathbb {R}$, and we use its derivative to mean the derivative of $V$ with respect to $u$, by default. To prove the statement, we need to prove that $V$ is twice continuously differentiable and that the second-order derivative is positive almost everywhere in the domain. We consider the following cases.

First, assume $x\geqslant at$ holds. After some computation, the function $u\mapsto V(x,t;u,a,b)$ can be written as follows:

$$\begin{aligned} V(x,t;u,a,b) = \dfrac{u^3}{6b} + \dfrac{x^3}{6a} - \left( \dfrac{1}{6a} + \dfrac{1}{6b}\right) \left( \frac{au+ bx - abt}{a+b}\right) ^3, \quad \forall u\in [x-at, x+bt], \end{aligned}$$

which is twice continuously differentiable. The second-order derivative is given by

$$\begin{aligned} \begin{aligned} \dfrac{\partial ^2 V(x,t;u,a,b)}{\partial u^2}&= \dfrac{u}{b} - \dfrac{a}{b(a+b)^2} (au+ bx-abt) = \dfrac{(b^2+2ab)u- ab(x -at)}{b(a+b)^2}\\&\geqslant \dfrac{(b^2+ab)u}{b(a+b)^2}\geqslant 0, \end{aligned} \end{aligned}$$

(A2)

where the first and second inequalities hold since we have $u\geqslant x-at\geqslant 0$. Moreover, the second inequality becomes equality if and only if $u$ is zero. In other words, the second-order derivative in (A2) is positive almost everywhere, and hence, the conclusion holds in this case.

Next, assume that x is a point in [0, at). In this case, the function $u\mapsto V(x,t;u,a,b)$ can be written as follows:

$$\begin{aligned} V(x,t;u,a,b) = {\left\{ \begin{array}{ll} -\dfrac{u^3}{6a} + \dfrac{x^3}{6a}, &{} x-at\leqslant u<0,\\ \dfrac{u^3}{6b} + \dfrac{x^3}{6a}, &{} 0\leqslant u<bt-\dfrac{bx}{a},\\ \dfrac{u^3}{6b} + \dfrac{x^3}{6a} - \left( \dfrac{1}{6a} + \dfrac{1}{6b}\right) \left( \dfrac{au+ bx - abt}{a+b}\right) ^3, &{} bt-\dfrac{bx}{a}\leqslant u\leqslant x+bt. \end{array}\right. } \end{aligned}$$

It is straightforward to check that this function is twice continuously differentiable and that the second-order derivative reads

$$\begin{aligned} \dfrac{\partial ^2 V(x,t;u,a,b)}{\partial u^2} = {\left\{ \begin{array}{ll} -\dfrac{u}{a}, &{} x-at< u< 0,\\ \dfrac{u}{b}, &{} 0\leqslant u< bt-\dfrac{bx}{a},\\ \dfrac{(b^2+2ab)u- ab(x -at)}{b(a+b)^2}, &{} bt-\dfrac{bx}{a}\leqslant u< x+bt, \end{array}\right. } \end{aligned}$$

(A3)

where the first line is positive since $u<0$ holds in the first line, the second line is positive almost everywhere since $u>0$ holds almost everywhere in the second line, and the third line is positive since the inequalities in (A2) also hold according to the condition on u (there holds $u \geqslant bt-\frac{bx}{a}>0>x-at$). Therefore, the conclusion follows in this case.

Finally, we consider the case when $x<0$. By definition, we have that $V(x,t;u,a,b) = V(-x,t;-u,b,a)$, where the right-hand side is twice continuously differentiable and whose second-order derivative with respect to $-u$ is positive almost everywhere by the same argument above. Therefore, the function $V(x,t;u,a,b)$ is also strictly convex and twice continuously differentiable with respect to $u$, and the conclusion holds.

Appendix B Some Computations for the Numerical Implementation

1.1 Appendix B.1 A Numerical Method for Computing the Proximal Point of $u\mapsto \frac{1}{\lambda }V(x,t;u,a,b)$

Here, we discuss how to compute the proximal point of the function $\mathbb {R}\ni u\mapsto \frac{1}{\lambda }V(x,t;u,a,b)\in \mathbb {R}\cup \{+\infty \}$, i.e., how to solve the following convex optimization problem:

$$\begin{aligned} u^*{} & {} = \mathop {\mathrm {arg\,min}}\limits _{u\in {\mathbb {R}}} \left\{ V(x,t; u,a,b) + \dfrac{\lambda }{2}(u - y)^2\right\} \nonumber \\{} & {} = \mathop {\mathrm {arg\,min}}\limits _{u\in [x-at,x+bt]} \left\{ V(x,t; u,a,b) + \dfrac{\lambda }{2}(u - y)^2\right\} \end{aligned}$$

(B1)

for any $\lambda ,t,a,b>0$, and $x,y\in \mathbb {R}$. We consider the following two cases for the variable u.

If $u \leqslant 0$, after some computation, the objective function in (B1) can be written as

$$\begin{aligned} \begin{aligned} F(u;x,t,a,b):=&V(x,t; u,a,b) + \frac{\lambda }{2}(u - y)^2 \\ =&{\left\{ \begin{array}{ll} \frac{u^3}{6b} + \frac{x^3}{6a} - \left( \frac{1}{6a} + \frac{1}{6b}\right) \left( \frac{au + bx - abt}{a+b}\right) ^3 + \frac{\lambda }{2}(u - y)^2, &{} u\in \varOmega _1(x,t,a,b),\\ \frac{u^3}{6b} + \frac{x^3}{6a} + \frac{\lambda }{2}(u - y)^2, &{} u\in \varOmega _2(x,t,a,b),\\ \frac{u^3}{6b} - \frac{x^3}{6b} + \frac{\lambda }{2}(u - y)^2, &{} u\in \varOmega _3(x,t,a,b),\\ +\infty , &{} \text {otherwise}, \end{array}\right. } \end{aligned} \end{aligned}$$

where the three regions $\varOmega _1(x,t,a,b), \varOmega _2(x,t,a,b), and~\varOmega _3(x,t,a,b)\subset [0,+\infty )$ are defined by

$$\begin{aligned} \begin{aligned} \varOmega _1(x,t,a,b)&:= \left\{ u\in (bt,+\infty ):x-at\leqslant u\leqslant x+bt\right\} \\&\qquad \bigcup \left\{ u\in [0,bt]:u\geqslant x-at,\,\, u\geqslant bt-\frac{bx}{a}\right\} ,\\ \varOmega _2(x,t,a,b)&:= {\left\{ \begin{array}{ll} \left[ 0,bt-\frac{bx}{a}\right) , &{} x\geqslant 0, \\ \varnothing , &{} x<0, \end{array}\right. }\\ \varOmega _3(x,t,a,b),&:= {\left\{ \begin{array}{ll} [0,x+bt], &{} x< 0,\\ \varnothing , &{} x\geqslant 0. \end{array}\right. } \end{aligned} \end{aligned}$$

In this case, the derivative of F with respect to u is given by

$$\begin{aligned} \begin{aligned}&\qquad \dfrac{\partial }{\partial u} F(u;x,t,a,b) \\&\quad = {\left\{ \begin{array}{ll} \dfrac{(2a+b)u^2 - 2a(x-at)u - b(x-at)^2}{2(a+b)^2} &{}\\ \quad + \lambda (u - y), &{} u\in \varOmega _1(x,t,a,b),\\ \dfrac{u^2}{2b} + \lambda (u - y), &{} u\in \varOmega _2(x,t,a,b) \cup \varOmega _3(x,t,a,b), \end{array}\right. } \end{aligned} \end{aligned}$$

(B2)

and the second derivative of F with respect to u can be easily computed using (A2) and (A3), for different cases. To get possible candidates for the minimizer $u^*$ of F in this case, we compute the roots of the functions in the two lines of (B2) and select the roots where the second derivative of F is non-negative. After some calculations, the candidates are given by $u_1$ and $u_2$, which are defined as follows:

$$\begin{aligned} \begin{aligned} {\left\{ \begin{array}{ll} u_1 &{}:= -\frac{\lambda (a+b)^2 - a(x-at)}{2a+b} \\ &{}\qquad + \sqrt{\left( \frac{\lambda (a+b)^2-a(x-at)}{2a+b}\right) ^2 + \frac{b(x-at)^2}{2a+b} + \frac{2\lambda (a+b)^2y}{2a+b}},\\ u_2 &{}:= -\lambda b + \sqrt{(\lambda b)^2 + 2 \lambda b y} \end{array}\right. }. \end{aligned} \end{aligned}$$

(B3)

Note that $u_1$ and $u_2$ may be not well-defined if the term under the square root is negative, in which case the corresponding function does not provide a possible candidate for $u^*$. Therefore, we assign $u_i$ ($i=1,2$) to be an arbitrary point in $[x-at,x+bt]$ if it is not well-defined.

If $u < 0$, after some computation, the objective function in (B1) can be written as

$$\begin{aligned} F(u;x,t,a,b)&:= V(x,t; u,a,b) + \dfrac{\lambda }{2}(u - y)^2 \\&= V(-x,t; -u,b,a) + \dfrac{\lambda }{2}(u - y)^2 \\&= {\left\{ \begin{array}{ll} -\dfrac{u^3}{6a} - \dfrac{x^3}{6b} + \left( \dfrac{1}{6a} + \dfrac{1}{6b}\right) \left( \dfrac{bu + ax + abt}{a+b}\right) ^3 + \dfrac{\lambda }{2}(u - y)^2, &{} -u\in \varOmega _1(-x,t,b,a),\\ -\dfrac{u^3}{6a} - \dfrac{x^3}{6b} + \dfrac{\lambda }{2}(u - y)^2, &{} -u\in \varOmega _2(-x,t,b,a), \\ -\dfrac{u^3}{6a} + \dfrac{x^3}{6a} + \dfrac{\lambda }{2}(u - y)^2, &{} -u\in \varOmega _3(-x,t,b,a),\\ +\infty , &{} \text {otherwise}. \end{array}\right. } \end{aligned}$$

Thus, for $u < 0$, the derivative of F with respect to u is given by

$$\begin{aligned} \begin{aligned}&\qquad \frac{\partial }{\partial u} F(u;x,t,a,b) \\&\quad = {\left\{ \begin{array}{ll} \dfrac{-(a+2b)u^2 + 2b(x+bt)u + a(x+bt)^2}{2(a+b)^2} + \lambda (u - y), &{} -u\in \varOmega _1(-x,t,b,a),\\ -\dfrac{u^2}{2a} + \lambda (u - y), &{} -u\in \varOmega _2(-x,t,b,a) \cup \varOmega _3(-x,t,b,a). \end{array}\right. } \end{aligned} \end{aligned}$$

(B4)

Similarly as in the first case, we take the roots of the two functions in (B4), such that the second order derivative of F is non-negative. These roots provide possible candidates for $u^*$. We denote these candidates by $u_1'$ and $u_2'$, which are defined by

$$\begin{aligned} \begin{aligned} {\left\{ \begin{array}{ll} u_1' &{}:= \frac{\lambda (a+b)^2 + b(x+bt)}{a+2b} \\ &{}\, -\sqrt{\left( \frac{\lambda (a+b)^2 + b(x+bt)}{a+2b}\right) ^2 + \frac{a(x+bt)^2}{a+2b} - \frac{2\lambda (a+b)^2y}{a+2b}}, \\ u_2' &{}:= \lambda a - \sqrt{(\lambda a)^2 - 2\lambda a y}. \end{array}\right. } \end{aligned} \end{aligned}$$

(B5)

Similarly, if $u_1'$ or $u_2'$ is not well-defined, we set it to be any point in $[x-at,x+bt]$.

Note that the objective function F is strictly convex and twice continuously differentiable with respect to u by Lemma  A2. Then, by the first and second derivative tests, the minimizer $u^*$ in (B1) is selected among the possible candidates $u_1,u_2,u'_1, u_2'$ defined in (B3) and (B5), as well as the boundary points $x-at$ and $x+bt$. In other words, the minimizer $u^*$ satisfies

$$\begin{aligned} u^* = \mathop {\mathrm {arg\,min}}\limits _{u\in \{u_1,u_2,u'_1,u'_2,x - at, x+bt\}} F(u;x,t,a,b). \end{aligned}$$

(B6)

Numerically, we solve the optimization problem  (B1) by computing the six candidates $u_1,u_2,u_1',u_2',x-at,x+bt$ and comparing the objective function values at those points. Then, the minimizer $u^*$ is selected using (B6). Therefore, the complexity of solving (B1) is $\varTheta (1)$.

1.2 Appendix B.2 An Equivalent Expression for $V(x,t;u,a,b)$ and $\gamma (s;x,t,u,a,b)$

Let $V$ be the function defined by (9) and (15), and let $\gamma$ be the function defined by (12),  (13), (14), and (16) for different cases. Now, we present an equivalent expression for $V$ and $\gamma$, which is used in our numerical implementation.

By straightforward calculation, the function $V$ can equivalently be expressed as

$$\begin{aligned} \begin{aligned}&\qquad V(x,t;u,a,b) \\&\quad ={\left\{ \begin{array}{ll} \max \{V_3(x,t;u,a,b), \min \{V_1(x,t;u,a,b), V_2(x,t;u,a,b)\}\} &{} \text {if } u\in [0, x+bt],\\ \max \{V_3(-x,t;-u,b,a), \min \{V_1(-x,t;-u,b,a), V_2(-x,t;-u,b,a)\}\} &{} \text {if } u\in [x-at, 0),\\ +\infty &{}\text {otherwise}, \end{array}\right. } \end{aligned} \end{aligned}$$

where $V_1,V_2$, and $V_3$ are the functions in the first, second, and third lines of (9), respectively.

Similarly, assuming $u\in [x-at,x+bt]$ holds, the function $\gamma$ can be expressed as

$$\begin{aligned} \gamma (s;x,t,u,a,b) = {\left\{ \begin{array}{ll} \max \{u-bs, a(s-t)+x,0\} &{} \text {if } x\geqslant 0, 0\leqslant u\leqslant x+bt,\\ \max \{u-bs,0\} + \min \{-b(s-t)+x,0\} &{} \text {if } x< 0, 0\leqslant u\leqslant x+bt,\\ \min \{u+as, -b(s-t)+x,0\} &{} \text {if } x< 0, x-at\leqslant u<0,\\ \min \{u+as,0\} + \max \{a(s-t)+x,0\} &{} \text {if } x\geqslant 0, x-at\leqslant u<0. \end{array}\right. } \end{aligned}$$

Compared to the definitions (9),  (15), (12),  (13), (14), and  (16), these equivalent formulas involve less conditional branching, and hence, they are more favorable for the performance of our numerical implementation.

Appendix C Proofs of Convergence Results in Sect. 3

In this Appendix, we provide the proof of Proposition 7 in Appendix C.1 and the proof of Proposition 8 in Appendix C.2.

1.1 Appendix C.1 Proof of Proposition 7

Let $\varvec{v}^N$, $\varvec{d}^N$, and $\varvec{u}^N$ be the corresponding vectors in the algorithm at the N-th iteration. Let $\varvec{u}^*$ be the minimizer of the minimization problem in (24), which is unique since $\varPhi$ is convex and each $u_i\mapsto V(x_i,t;u_i,a,b)$ is strictly convex by Lemma A2. According to [28, Theorem 2.2] whose assumptions are proved using [28, Remark 2.2], both $\varvec{v}^N$ and $\varvec{d}^N$ converge to the point $\varvec{u}^*$ as N approaches infinity, and hence, $\varvec{u}^N$ in Algorithm 1 also converges to $\varvec{u}^*$. Since $\varPhi$ is a real-valued convex function, it is continuous in $\mathbb {R}^n$ and we have

$$\begin{aligned} \lim _{N\rightarrow \infty } \varPhi (\varvec{u}^N) = \varPhi (\varvec{u}^*). \end{aligned}$$

(C1)

Note that the domain of the function $\varvec{u}\mapsto \sum _{i=1}^nV(x_i,t;u_i, a_i,b_i)$ equals

$$\begin{aligned} \prod _{i=1}^n [x_i-a_it,x_i+b_it], \end{aligned}$$

(C2)

and the point $\varvec{u}^N = \varvec{d}^N$ is in the set in (C2) by definition of $\varvec{d}^N$. Thus, the point $\varvec{u}^N$ is in the domain of the function $\varvec{u}\mapsto \sum _{i=1}^nV(x_i,t;u_i, a_i,b_i)$. By Lemma A2, the function $\varvec{u}\mapsto \sum _{i=1}^nV(x_i,t;u_i, a_i,b_i)$ is continuous in its domain. It is also straightforward to check that the function $\varvec{u}\mapsto \sum _{i=1}^nV(x_i,t;u_i, a_i,b_i)$ is Lipschitz in its domain, and we denote its Lipschitz constant by $L_i$. Thus, we have that

$$\begin{aligned} \left| \sum _{i=1}^nV(x_i,t;u^N_i, a_i,b_i) -\sum _{i=1}^nV(x_i,t;u^*_i, a_i,b_i)\right| \leqslant \left( \sum _{i=1}^n L_i\right) \Vert \varvec{u}^N - \varvec{u}^*\Vert . \end{aligned}$$

(C3)

Then, the convergence of $\hat{V}^N(\varvec{x},t)$ to $V(\varvec{x},t)$ follows from (C1) and (C3).

Now, it remains to prove the second formula in (44). We have proved that $\varvec{u}^N$ and $\varvec{u}^*$ are both in the set in (C2). Let $\{\varvec{u}^{N_j}\}_j$ be a subsequence of $\{\varvec{u}^{N}\}_N$ (i.e., we assume $N_1<N_2<\cdots$ and $\lim _{j\rightarrow \infty }N_j=+\infty$), such that for each $i\in \{1,\cdots , n\}$, the i-th component $\{u_i^{N_j}\}_j$ of the subsequence satisfies one of the following assumptions:

• (i) there exists an index $r_i$ in $\{1,2,3\}$, such that there hold

  $$\begin{aligned} u_i^{N_j}\geqslant 0 \quad \text { and }\quad (x_i,t,u_i^{N_j})\in \bar{\varOmega }_{r_i}(a_i,b_i),\quad \forall \, j\in \mathbb {N}; \end{aligned}$$

• (ii) there exists an index $r_i$ in $\{1,2,3\}$, such that there hold

  $$\begin{aligned} u_i^{N_j}< 0 \quad \text { and }\quad (-x_i,t,-u_i^{N_j})\in \bar{\varOmega }_{r_i}(b_i,a_i),\quad \forall \, j\in \mathbb {N}. \end{aligned}$$

Here, to emphasize the dependence on $a_i$ and $b_i$, we use $\varOmega _{r_i}(a_i,b_i)$ and $\bar{\varOmega }_{r_i}(a_i,b_i)$ to respectively denote the set defined in (10) with constants $a=a_i$ and $b=b_i$ and its closure. Note that the situations considered in cases (i) and (ii) give a partition of the set in (C2) (where some sets in the partition may be empty and the sets may overlap on the boundary, but neither of these possibilities affect the result). Hence, if the statement is proved for any such subsequence $\{\varvec{u}^{N_j}\}_j$, then the statement also holds for the whole sequence $\{\varvec{u}^{N}\}_N$. Thus, it suffices to prove the statement for the subsequence $\{\varvec{u}^{N_j}\}_j$.

Let $i\in \{1,\cdots ,n\}$ be any index. Assume case (i) holds for $\{u_i^{N_j}\}_j$ with the index $r_i$. In other words, we assume $(x_i,t,u_i^{N_j})\in \bar{\varOmega }_{r_i}(a_i,b_i)$ holds for any $j\in \mathbb {N}$. Since the set $\bar{\varOmega }_{r_i}(a_i,b_i)$ is closed and the subsequence $\{u_i^{N_j}\}_j$ converges to $u_i^*$, we conclude that $(x_i,t,u_i^*)\in \bar{\varOmega }_{r_i}(a_i,b_i)$ also holds. Then, by definition of $\gamma$ in $\varOmega _{r_i}(a_i,b_i)$, it is straightforward to check that

$$\begin{aligned} \sup _{s\in [0,t]} \left| \gamma (s;x_i,t, u_i^{N_j},a_i,b_i) - \gamma (s;x_i,t, u_i^*,a_i,b_i)\right| \leqslant \left| u_i^{N_j}-u_i^*\right| . \end{aligned}$$

(C4)

The proof for case (ii) is similar, so we omit it. Note that (C4) holds for any arbitrary index $i\in \{1,\cdots ,n\}$. Hence, we have

$$\begin{aligned} \begin{aligned} \sup _{s\in [0,t]}\left\| \hat{\varvec{\gamma }}^{N_j}(s;\varvec{x},t) - \varvec{\gamma }(s;\varvec{x},t)\right\| ^2&= \sup _{s\in [0,t]}\sum _{i=1}^n\left| \gamma (s;x_i,t, u_i^{N_j},a_i,b_i) - \gamma (s;x_i,t, u_i^*,a_i,b_i)\right| ^2\\&\leqslant \sum _{i=1}^n \sup _{s\in [0,t]}\left| \gamma (s;x_i,t, u_i^{N_j},a_i,b_i) - \gamma (s;x_i,t, u_i^*,a_i,b_i)\right| ^2\\&\leqslant \sum _{i=1}^n\left| u_i^{N_j}-u_i^*\right| ^2\\&=\left\| \varvec{u}^{N_j}-\varvec{u}^*\right\| ^2, \end{aligned} \end{aligned}$$

where the second inequality holds by (C4). Thus, the second formula in (44) holds for the subsequence by the convergence of $\varvec{u}^{N_j}$ to $\varvec{u}^*$. Moreover, the argument holds for any such subsequence, and hence, the statement holds for the whole sequence.

1.2 Appendix C.2 Proof of Proposition 8

Let r be the index defined in (51), and let the index set ${\mathcal {J}}\subseteq \{1,\cdots ,m\}$ be defined by

$$\begin{aligned} {\mathcal {J}}:= \mathop {\mathrm {arg\,min}}\limits _{i\in \{1,\cdots ,m\}} V_i(\varvec{x},t). \end{aligned}$$

Then, we have

$$\begin{aligned} V(\varvec{x},t) - \hat{V}(\varvec{x},t) = V(\varvec{x},t) - \hat{V}_r(\varvec{x},t) \leqslant V_r(\varvec{x},t) - \hat{V}_r(\varvec{x},t) \leqslant \epsilon , \end{aligned}$$

where the first equality holds by definition of r and the first inequality holds since $V$ satisfies  (47). Similarly, for any $j\in {\mathcal {J}}$, we have

$$\begin{aligned} V(\varvec{x},t) - \hat{V}(\varvec{x},t) = V_j(\varvec{x},t) - \hat{V}(\varvec{x},t) \geqslant V_j(\varvec{x},t) - \hat{V}_j(\varvec{x},t) \geqslant -\epsilon , \end{aligned}$$

where the first equality holds by definition of ${\mathcal {J}}$ and the first inequality holds since $\hat{V}$ satisfies (52). Therefore, (54) holds.

Now, assume $V_j(\varvec{x},t) > V(\varvec{x},t) + 2\epsilon$ holds for each index j satisfying $V_j(\varvec{x},t) \ne V(\varvec{x},t)$. We prove $r\in {\mathcal {J}}$ by contradiction. Assume r is not in ${\mathcal {J}}$. Then, we have $V_r(\varvec{x},t) \ne V(\varvec{x},t)$. However, from straightforward calculation, we also have

$$\begin{aligned} \begin{aligned} V_r(\varvec{x},t) - V(\varvec{x},t)&\leqslant (V_r(\varvec{x},t) - \hat{V}_r(\varvec{x},t)) + (\hat{V}_r(\varvec{x},t) - \hat{V}(\varvec{x},t)) + (\hat{V}(\varvec{x},t)- V(\varvec{x},t))\\&\leqslant \epsilon + 0 + \epsilon = 2\epsilon , \end{aligned} \end{aligned}$$

which leads to a contradiction with our assumption. Therefore, we have $r\in {\mathcal {J}}$, and hence (55) holds by definition of r and ${\mathcal {J}}$.