Error Estimates for Numerical Approximation of Hamilton–Jacobi Equations Related to Hybrid Control Systems

Abstract

Hybrid control systems are dynamical systems that can be controlled by a combination of both continuous and discrete actions. In this paper we study the approximation of optimal control problems associated to this kind of systems, and in particular of the quasi-variational inequality which characterizes the value function. Our main result features the error estimates between the value function of the problem and its approximation. We also focus on the hypotheses describing the mathematical model and the properties defining the class of numerical scheme for which the result holds true.

Appendix

1.1 A.1 The Upper Bounds of Lipschitz Constants

Proof of Lemma 5.2

For $q \in \mathbb I$ and $\epsilon >0$, set

$$\begin{aligned} m_{\epsilon }:=\sup _{x,y \in {\mathbb R}^d}\varphi (x,y):=\sup _{x,y \in {\mathbb R}^d}\left\{ w(x,q)-w(y,q)-\frac{\delta }{2}|x-y|^2-\frac{\epsilon }{2}(|x|^2+|y|^2)\right\} . \end{aligned}$$

Let $x_0,y_0\in {\mathbb R}^d$ such that $m_{\epsilon } = \varphi (x_0,y_0)$. Taking into account the HJB equation satisfied by w and applying the notion of viscosity solution, we get:

$$\begin{aligned} \begin{aligned} 0\le \max \big \{&\lambda w(y_0,q)+H(y_0,q,p_y) - \lambda w(x_0,q) - H(x_0,q,p_x),\\&w(y_0,q)-\Phi (y_0,q)-w(x_0,q)+\Phi (x_0,q)\big \}. \end{aligned} \end{aligned}$$

Two cases have to be considered.

• 1. The max is attained by its first argument.

  In this case, we get

  $$\begin{aligned} \lambda w(y_0,q)+H(y_0,q,p_y) - \lambda w(x_0,q) - H(x_0,q,p_x)\ge 0 \end{aligned}$$

  where

  $$\begin{aligned} \begin{aligned} p_x&=\delta (x_0-y_0)+\epsilon x_0\\ p_y&= \delta (x_0-y_0)-\epsilon y_0. \end{aligned} \end{aligned}$$

  This is a standard case (see [1]), and we have that

  $$\begin{aligned} w(x,q)-w(y,q) \le \frac{L_\ell }{\lambda -L_f}|x-y|. \end{aligned}$$

• 2. The max is attained by its second argument.

  In this case,

  $$\begin{aligned} w(y_0,q)-\Phi (y_0,q)-w(x_0,q)+\Phi (x_0,q)\ge 0, \end{aligned}$$

  so that we get $w(x_0,q)-w(y_0,q) \le L_{\Phi }|x_0-y_0|$, and we can infer that

  $$\begin{aligned} m_{\epsilon } \le L_{\Phi }|x_0-y_0|-\frac{\delta }{2}|x_0-y_0|^2. \end{aligned}$$

  (8.1)

  Setting $r:=|x_0-y_0|$, we have $\max _{r\ge 0} \left\{ L_{\Phi }r-\frac{\delta }{2} r^2\right\} =L_{\Phi }^2/2 \delta$, and hence we obtain

  $$\begin{aligned} m_{\epsilon }\le \frac{L_{\Phi }^2}{2 \delta }. \end{aligned}$$

  (8.2)

  Define now $m:=\lim _{\epsilon \rightarrow 0} m_{\epsilon }$. Applying a simple calculus argument (see [14, Lemma 2.3]), for fixed $\delta$, we have:

  $$\begin{aligned} m=\sup _{x,y \in {\mathbb R}^d}\big \{w(x,q)-w(y,q)-\delta |x-y|^2\big \} \le \frac{L_{\Phi }^2}{2 \delta }, \end{aligned}$$

  where the inequality follows from (8.2). Therefore, by definition of m, we have that:

  $$\begin{aligned} w(x,q)-w(y,q) \le \frac{L_{\Phi }^2}{2 \delta }+\frac{\delta }{2}|x-y|^2. \end{aligned}$$

  Observing now that

  $$\begin{aligned} \min _{\delta \ge 0} \left\{ \frac{L_{\Phi }^2}{2 \delta }+\frac{\delta }{2}|x-y|^2\right\} =L_{\Phi }|x-y|, \end{aligned}$$

  we finally obtain:

  $$\begin{aligned} w(x,q)-w(y,q) \le L_{\Phi }|x-y|. \end{aligned}$$

In conclusion, for both cases, we have

$$\begin{aligned} L_{w}= \max \left\{ L_{\Phi }, \frac{L_\ell }{\lambda -L_f}\right\} , \end{aligned}$$

and, using similar arguments, we can also bound $L_{w^{\epsilon }}$ as:

$$\begin{aligned} L_{w^{\epsilon }}= \max \left\{ L_{\Phi }, \frac{L_\ell }{\lambda -L_f}\right\} . \end{aligned}$$

$\square$

1.2 A.2 Lipschitz Stability for the SL Scheme

In this section, we prove that the numerical approximation $w_h$ for the solution w of the obstacle problem is Lipschitz continuous. We consider schemes approximating (5.1) in the fixed point form:

$$\begin{aligned} W_h(x,q)= \min \big \{\Sigma ^h(x,q,W_h), \Phi (x,q)\big \}\quad (x,q)\in {\mathbb R}^d\times \mathbb I, \end{aligned}$$

(8.3)

where, in the case of Semi-Lagrangian schemes, the operator $\Sigma ^h$ reads

$$\begin{aligned} \Sigma ^h(x,q,W_h):= \Pi _{\Delta x} \circ \min _{u\in U}\Big \{h \ell (x,q,u) + e^{-\lambda h} W_h \big (x+h f(x,q,u), q\big )\Big \}, \end{aligned}$$

(8.4)

where $\Pi _{\Delta x}$ is an interpolation operator based on a space grid of step $\Delta x$. It is well-known that $\Sigma ^h$ is non-expansive in the $\infty$-norm, provided the interpolation operator $\Pi _{\Delta x}$ is monotone (see [11]).

Theorem A.1

Under assumptions (A1)–(A6), (S1)–(S4), the solution $W_h$ of problem (6.1) obtained with the Semi-Lagrangian scheme (8.3)–(8.4) is Lipschitz continuous with

$$\begin{aligned} | W_h |_1 \le L_{W_h}=\big (1+(\lambda -L_f)h\big )\max \left\{ L_\Phi ,\frac{L_\ell }{\lambda -L_f}\right\} . \end{aligned}$$

Proof

For $(x,q) \in {\mathbb R}^d\times \mathbb I$, consider the iterative solution of the fixed point equation (8.3):

$$\begin{aligned} W^{(k+1)}_h(x,q)=\min \Big \{\Sigma ^h\big (x,q,W_h^{(k)}\big ), \Phi (x,q)\Big \} \end{aligned}$$

where $W^{(k)}_h$ is the approximation of $W_h$ at the k–th iteration. For any $x_1,x_2\in {\mathbb R}^d$, we have

$$\begin{aligned} \begin{aligned}&\big |W^{(k+1)}_h(x_1,q)-W^{(k+1)}_h(x_2,q)\big |\le \max \Big \{hL_\ell +e^{-\lambda h}(1+\lambda L_f)L_{W^{(k)}_h}, L_\Phi \Big \}\le \\&\qquad \qquad \le \max \Big \{ hL_\ell +e^{-(\lambda -L_f)h}L_{W^{(k)}_h}, L_\Phi \Big \}\le \\&\qquad \qquad \le \max \left\{ \frac{L_\ell }{\lambda -L_f}, L_\Phi \right\} \max \left\{ (\lambda -L_f)h+\frac{e^{-(\lambda -L_f)h}L_{W^{(k)}_h}}{\max \left\{ \frac{L_\ell }{\lambda -L_f}, L_\Phi \right\} }, 1\right\} . \end{aligned} \end{aligned}$$

By setting

$$\begin{aligned} \begin{aligned} m&:=(\lambda -L_f)h > 0,\\ M_k&:=\frac{L_{W^{(k)}_h}}{\max \left\{ \frac{L_\ell }{\lambda -L_f}, L_\Phi \right\} }, \end{aligned} \end{aligned}$$

we have

$$\begin{aligned} M_{k+1}\le \max \{m+e^{-m}M_k,1\}. \end{aligned}$$

Note that, if $M_{k}\le 1+m$, then $e^{-m}M_k\le e^{-m}(1+m)\le 1$. Hence,

$$\begin{aligned} M_{k+1}\le \max \{m+e^{-m}M_k,1\}\le \max \{1+m,1\} = 1+m. \end{aligned}$$

It suffices therefore to initialize the fixed point iterations with $W^{(0)}_h$ such that $M_0=0$ to guarantee $M_k\le 1+m$ for every $k\ge 0$, and, by the definitions of m and $M_k$, we obtain

$$\begin{aligned} \frac{L_{W^{(k)}_h}}{\max \left\{ \frac{L_\ell }{\lambda -L_f}, L_\Phi \right\} }\le 1+(\lambda -L_f)h \end{aligned}$$

which implies

$$\begin{aligned} L_{W^{(k)}_h}\le \big (1+(\lambda -L_f)h\big )\max \left\{ \frac{L_\ell }{\lambda -L_f}, L_\Phi \right\} . \end{aligned}$$

Now, since $W^{(k)}_h$ converges towards the solution $W_h$ of the scheme (8.3)–(8.4) as $k \rightarrow +\infty$, we conclude

$$\begin{aligned} L_{W_h}\le \big (1+(\lambda -L_f)h\big )\max \left\{ \frac{L_\ell }{\lambda -L_f}, L_\Phi \right\} . \end{aligned}$$

$\square$

1.3 A.3 Estimate on the perturbed value function of the stopping problem

The cost function corresponding to the stopping problem can be defined as :

$$\begin{aligned} J(x, q; \theta ) := \int _0^\xi \ell \big (X(t, q, u),q,u(t)\big ) e^{-\lambda t} dt +e^{-\lambda \xi }\Phi \big (X(\xi , q, u),q\big ) \end{aligned}$$

where $\Phi : {\mathbb R}\rightarrow {\mathbb R}$ is Lipschitz continuous. Then, the value function given by:

$$\begin{aligned} w(x,q):=\inf _{\theta \in \Theta ^0}J(x,q;\theta ), \end{aligned}$$

satisfies (5.1). Now, for a given $\varepsilon$, we replace the dynamics (2.1) with (6.8), and define the value function $w^\varepsilon$:

$$\begin{aligned} w^\varepsilon (x,q):=\inf _{\theta ^\varepsilon \in \Theta ^\varepsilon }J^\varepsilon (x,q;\theta ^\varepsilon ) \end{aligned}$$

with $\Theta ^\varepsilon :=\mathcal {U}\times \mathbb {R}_+ \times \mathcal {F}^\varepsilon$ and

$$\begin{aligned} J^\varepsilon (x, q; \theta ^\varepsilon ) := \int _0^{\xi } \ell \big (X^\varepsilon (t, q, u)+e(t),q,u(t)\big ) e^{-\lambda t} dt +e^{-\lambda {\xi }}\Phi \big (X^\varepsilon ({\xi }, q, u),q\big ). \end{aligned}$$

With this definition, the value function $w^\varepsilon$ is the unique solution of the equation:

$$\begin{aligned} \max \Big (\lambda w^{\varepsilon }(x,q) + \max _{|e|\le \varepsilon } H\big (x+e,q,D_x w^{\varepsilon }(x,q)\big ),w^\varepsilon (x,q)-\Phi (x,q)\Big )= 0 \quad \text{ in } {\mathbb R}^d\times \mathbb {I}, \nonumber \\ \end{aligned}$$

(8.5)

here, we have used the notation $\cdot ^\varepsilon$ to distinguish between the quantities related to the perturbed problems and the ones related to the unperturbed problem. Note that the equations in the system (8.5) [reps. (8.5)] are not connected. So in the sequel, we will drop the dependency with respect to q.

The rest of this section is dedicated to deriving an estimate for the difference between $w^\varepsilon$ and w. Since every control $\theta \in \Theta ^0$ can be considered as an admissible control in $\Theta ^\varepsilon$ with a perturbation function $e\equiv 0$. Hence, we have:

$$\begin{aligned} w^\varepsilon (x) \le w(x)\quad x\in {\mathbb R}^d. \end{aligned}$$

(8.6)

Let $\theta =(u,\xi )\in \Theta ^0$ and let $e\in {\mathcal F}^\varepsilon$ for some $\varepsilon >0$. Let X be a trajectory solution of (2.1) associated to $\theta$ and let $X^\varepsilon$ solution of (6.8) associated to $\theta ^\varepsilon =(u,\xi ,e)$. For every $t>0$, we have:

$$\begin{aligned} \begin{aligned} |X^\varepsilon _x(t) - X_x(t)|&\le \int _0^t|f\big (X^\varepsilon _x(s) + e(s),u(s)\big ) - f\big (X_x(s),u(s)\big )|\,ds \\&\le \int _0^t(\varepsilon L_f + L_f \big |X^\varepsilon _x(s) - X_x(s)\big |)\,ds. \end{aligned} \end{aligned}$$

Then applying Grönwall’s inequality we obtain

$$\begin{aligned} \big |X^\varepsilon _x(t) - X_x(t)\big | \le \varepsilon \big (e^{L_f t}-1\big ) \quad \text{ for } \text{ every } t\ge 0. \end{aligned}$$

(8.7)

We can now derive the estimate error between the solutions of problems (6.6) and (5.1).

Theorem A.2

Assume (A1)–(A6) hold. Then, for every $x \in \mathbb {R}^d$ and $\varepsilon >0$ we have:

$$\begin{aligned} \big |w(x,q)-w^\varepsilon (x,q)\big | \le \varepsilon \left( \frac{L_\ell }{\lambda - L_f} + L_\Phi \right) . \end{aligned}$$

Proof

From the definition of the value function $w^\varepsilon$ we know that for each $\delta > 0$ there exists $\theta _\delta ^\varepsilon =(u_\delta ,\xi _\delta ,e_\delta )\in \Theta ^\varepsilon$ such that

$$\begin{aligned} \begin{aligned} J^\varepsilon (x,q;\theta _\delta ^\varepsilon )&\le w^\varepsilon (x,q)+\delta . \end{aligned} \end{aligned}$$

Set $\theta _\delta =(u_\delta ,\xi _\delta )$ and denote by $X^{\varepsilon }_{\delta ,x}$ [resp. $X_{\delta ,x}$] the solution of (6.8) (resp. of (2.1)) associated to $u_\delta$. It follows that

$$\begin{aligned} \begin{aligned} 0\le w(x,q)-w^\varepsilon (x,q)&\le J(x,q;\theta _\delta ) - J^\varepsilon (x,q;\theta ^\varepsilon _\delta ) + \delta \\&\le \int _0^{\xi _\delta } e^{-\lambda s}|\ell (X_{\delta ,x}(s)+e(s),u_\delta (s))-\ell (X^\varepsilon _ {\delta ,x}(s),u_\delta (s))|\,ds \\& +e^{-\lambda \xi _\delta }|\Phi (X^\varepsilon _ {\delta ,x}(T))-\Phi (X_ {\delta ,x}(T))| +\delta \\&\le L_\ell \int _0^{\xi _\delta } \!\!e^{-\lambda s} \bigg [|X^\varepsilon _ {\delta ,x}(s)-X_ {\delta ,x}(s)|+|e(s)|\bigg ]\,ds \\& + e^{-\lambda \xi _\delta } L_\Phi |X^\varepsilon _ {\delta ,x} (\xi _\delta )- X_ {\delta ,x}(\xi _\delta )|+ \delta \\&\le \varepsilon L_\ell \int _0^{\xi _\delta } e^{-(\lambda -L_f) s}\,ds + \varepsilon L_\Phi e^{-\lambda \xi _\delta } (e^{L_f\xi _\delta }-1) +\delta \\&\le \varepsilon \left( L_\ell \frac{1-e^{-(\lambda -L_f)\xi _\delta }}{\lambda -L_f} + L_\Phi e^{-(\lambda -L_f)\xi _\delta } \right) +\delta \\&\le \varepsilon \left( \frac{L_\ell }{\lambda -L_f} + L_\Phi \right) +\delta . \end{aligned} \end{aligned}$$

The above estimate being valid for any $\delta >0$, we can conclude that the statement of the theorem is proved. $\square$

1.4 A.4 Estimate on the Perturbed Numerical Approximation

We want to examine here the difference between the numerical approximations of respectively the HJB equation with a given (known) obstacle and its perturbed version in the case of a Semi-Lagrangian scheme.

We recall that the unperturbed system is

$$\begin{aligned} {} \max \Big \{\lambda w(x,q) + H\big (x,q,D_x w(x,q)\big ),w(x,q)-\Phi (x,q)\Big \} \le 0 \qquad (x,q)\in {\mathbb R}^d\times \mathbb I. \nonumber \\ \end{aligned}$$

(8.8)

It can be approximated with the scheme

$$\begin{aligned} W_h(x,q) = \mathcal {T}^h(x,q,W_h) := \min \big \{\Sigma ^h(x,q,W_h), \Phi (x,q)\big \} (x,q)\in ({\mathbb R}^d\times \mathbb I). \nonumber \\ \end{aligned}$$

(8.9)

The perturbed SL scheme is obtained by replacing $\Sigma ^h$ in (8.9) with the mapping

$$\begin{aligned}&\Sigma ^{\varepsilon ,h}(x,q,W_h^\varepsilon )= \Pi _{\Delta x} \circ \min _{u\in U,|e|\le \varepsilon }\Big \{ h\ell (x+e,q,u)\nonumber \\&\quad +\, (1-\lambda h)W_h^\varepsilon \big (x+hf(x+e,q,u),q\big ) \Big \}. \end{aligned}$$

(8.10)

We start by giving the following general result:

Theorem A.3

Let (A1)–(A6) and (S1)–(S9) hold, and let $W_h$ and $W_h^\varepsilon$ be respectively solution of (8.9) and its perturbed version (8.10) with $\Phi$ finite or infinite. Then, the perturbed SL scheme has a unique bounded and uniformly Lipschitz continuous solution $W_h^\varepsilon$.

Proof

It suffices to note that, with the addition of the term e, the problem still satisfies the basic assumptions, and all the relevant constants of the problem remain unchanged. Then, the result follows from Theorem A.1, implying

$$\begin{aligned} |W_h^\varepsilon |_1\le \big (1+(\lambda -L_f)h\big )\max \left\{ L_\Phi ,\frac{L_\ell }{\lambda -L_f}\right\} . \end{aligned}$$

$\square$

Let now $W^\varepsilon _h$ denote the numerical solution for the perturbed SL scheme. We prove the following.

Theorem A.4

Let (A1)–(A6) and (S1)–(S9) hold, and let $W_h$ and $W_h^\varepsilon$ be respectively solution of (8.9) and its perturbed version (8.10) with $\Phi$ finite or infinite. Then, for $\varepsilon$ and h small enough, we have

$$\begin{aligned} |W_h-W^\varepsilon _h|_0 \le \varepsilon K_{W_h,h} \end{aligned}$$

(8.11)

with

$$\begin{aligned} K_{W_h,h}:=\max \left\{ (L_\ell + L_{W_h} L_f)h ,\frac{L_\ell + L_{W_h} L_f}{\lambda }\right\} . \end{aligned}$$

Proof

We recall that both the exact and the approximate solutions for either the original or the perturbed problem are Lipschitz continuous.

Using a scheme in fixed point SL form, the unperturbed QVI is approximated by (8.9), whereas its perturbed version is given by

$$\begin{aligned} W^{\varepsilon }_h(x,q) = \mathcal {T}^{\varepsilon ,h}(x,q,W^{\varepsilon }_h) := \min \big \{\Sigma ^{\varepsilon ,h}(x,q,W^{\varepsilon }_h), \Phi (x,q)\big \} \qquad (x,q)\in ({\mathbb R}^d\times \mathbb I).\nonumber \\ \end{aligned}$$

(8.12)

The plan is to apply the two schemes to Lipschitz continuous numerical solutions $W_h$ and $W^\varepsilon _h$ and estimate, for the various operators, differences of the form

$$\begin{aligned} \begin{aligned} \big | T^h(\>\cdot \>,\>\cdot \>,W_h) - T^{\varepsilon ,h}(\>\cdot \>,\>\cdot \>,W^\varepsilon _h) \big |_0&\le \big | T^h(\>\cdot \>, \>\cdot \>,W_h) - T^h(\>\cdot \>, \>\cdot \>,W^\varepsilon _h) \big |_0 + \\&\quad + \big | T^h(\>\cdot \>, \>\cdot \>,W^\varepsilon _h) - T^{\varepsilon ,h}(\>\cdot \>, \>\cdot \>,W^\varepsilon _h) \big |_0 \end{aligned} \end{aligned}$$

(8.13)

Using now, for $T=\mathcal {T}^h,\mathcal {T}^{\varepsilon ,h},\Sigma ^h,\Sigma ^{\varepsilon ,h}$ and $U=W_h,W^{\varepsilon }_h$, the shorthand notation

$$\begin{aligned} T(U) := T(\>\cdot \>,\cdot \>,U) \end{aligned}$$

we can single out three cases:

• (a) $\mathcal {T}^h(x,q,W_h)=\Sigma ^h(x,q,W_h)$ and $\mathcal {T}^{\varepsilon ,h}(x,q,W_h^\varepsilon )= \Sigma ^{\varepsilon ,h}(x,q,W_h^\varepsilon )$.

  In this case, we can bound the first term in the right-hand side of (8.13) as

  $$\begin{aligned} \big | \mathcal {T}^h(x,q,W_h) - \mathcal {T}^h(x,q,W^\varepsilon _h) \big |_0= & {} \big | \Sigma ^h(x,q,W_h) - \Sigma ^h(x,q,W^\varepsilon _h)\big |_0 \nonumber \\\le & {} (1-\lambda h)\big |W_h - W^\varepsilon _h\big |_0, \end{aligned}$$

  (8.14)

  which is a known property of the SL scheme. For the second, considering the Lipschitz continuity of $\ell$ and f and the bound on |e|, we have

  $$\begin{aligned} \begin{aligned} \big |\ell (x,q,u) - \ell (x+e,q,u)\big |&\le L_\ell \varepsilon , \\ \big |f(x,q,u) - f(x+e,q,u)\big |&\le L_f \varepsilon \end{aligned} \end{aligned}$$

  so that, taking into account the Lipschitz continuity of $W_h$, by a standard argument we obtain

  $$\begin{aligned} \big |\Sigma ^h(x,q,W_h) - \Sigma ^{\varepsilon ,h}(x,q,W_h) \big |_0 \le (L_\ell + L_{W_h} L_f)h \varepsilon . \end{aligned}$$

  (8.15)

• (b) $\mathcal {T}^h(x,q,W_h)=\Phi (x,q) =\mathcal {T}^{\varepsilon ,h}(x,q,W_h^\varepsilon )$. In this case there is nothing else to prove.

• (c) The $\min$ is achieved by different operators, e.g., let $\mathcal {T}^h(x,q,W_h)=\Sigma ^h(x,q,W_h)$ and $\mathcal {T}^h(x,q,W^\varepsilon _h)=\Phi (x,q)$. Working in terms of unilateral estimates, we have

  $$\begin{aligned} \mathcal {T}^h(x,q,W_h) - \mathcal {T}^h(x,q,W^\varepsilon _h)= & {} \Sigma ^h(x,q,W_h) - \Phi (x,q) \\\le & {} \Phi (x,q) - \Phi (x,q) = 0 \end{aligned}$$

  in which we get the inequality by replacing the argmin in $\Theta ^h(x,q,W_h)$ with the other choice. In a parallel form, we obtain the reverse inequality, as

  $$\begin{aligned} \begin{aligned} \mathcal {T}^h(x,q,W^\varepsilon _h) - \mathcal {T}^h(x,q,W_h)&= \Phi (x,q) - \Sigma ^h(x,q,W_h) \le \\&\le \Sigma ^h(x,q,W^\varepsilon _h) - \Sigma ^h(x,q,W_h)\le \\&\le (1-\lambda h)|W_h - W^\varepsilon _h|_0 \end{aligned} \end{aligned}$$

  The same arguments can then be applied to the case in which the choice of the operators is reversed, so that we finally obtain (8.14).

  We obtain therefore, by iterating the estimate (8.13) in (8.9) and (8.12) from the same initial guess $W_{h}^{(0)}=W^{\varepsilon (0)}_h$:

  $$\begin{aligned} | W_h - W^{\varepsilon }_h|_0 \le (L_\ell + L_{W_h}L_f)\varepsilon h \sum _{k\ge 0} (1-\lambda h)^k = \frac{L_\ell + L_{W_h}L_f}{\lambda } \varepsilon \end{aligned}$$

We can therefore conclude by collecting all the cases above in the bound

$$\begin{aligned} | W_h - W^{\varepsilon }_h |_0 \le \max \left\{ (L_\ell + L_{W_h} L_f)h ,\frac{L_\ell + L_{W_h} L_f}{\lambda }\right\} \varepsilon . \end{aligned}$$

$\square$