Phase transitions arising in stochastic ergodic control associated with viscous Hamilton–Jacobi equations with bounded inward drift

Abstract

This paper is concerned with certain phase transition phenomena arising in a family of stochastic ergodic control problems having real parameter $\beta$. We show that the large time behavior of the optimal diffusion changes drastically in the vicinity of some critical value $\beta ={\beta _{c}}$. Specifically, the optimal diffusion is recurrent for $\beta <{\beta _{c}}$, while it is transient for $\beta >{\beta _{c}}$. We also investigate the large time behavior of the optimal diffusion for $\beta ={\beta _{c}}$ which turns out to be different from the previous two cases and more subtle. Our proof is based on the Lyapunov method giving analytical criteria for recurrence and transience of diffusions. The key lies in the analysis of solutions to the associated viscous Hamilton–Jacobi equation with bounded inward drift. In particular, a refined version of the gradient estimate for solutions to viscous Hamilton–Jacobi equations plays a substantial role.

Appendix

In this appendix, we prove Theorem A.1 below, which is a slight refinement of [12, Théorème IV.1]. This kind of gradient estimates have been used in [3, 7,8,9,10] to obtain a bound of $|D\phi |$. Here we establish an estimate which enables one to obtain the decay rate of $|D\phi (x)|$ as $|x|\rightarrow \infty$.

Let $U\subset {{\mathbf {R}}^{d}}$ be a domain. We consider the elliptic equation

$$\begin{aligned} -\Delta \phi +g\cdot D\phi +G(x,D\phi ) -f=0\quad \text {in }\ U, \end{aligned}$$

(A.1)

where $g\in C^1(U;{{\mathbf {R}}^{d}})$, $G\in C^1(U\times {{\mathbf {R}}^{d}})$, and $f\in C^1(U)$. We assume that G satisfies the following condition:

(G) There exist some $m>1$, $\kappa >0$, and ${M_1},{M_2},{N_1},{N_2}:U\rightarrow [0,\infty )$ such that, for all $x\in U$ and $p\in {{\mathbf {R}}^{d}}$,

$$\begin{aligned} G(x,p)\ge \kappa {|p|^{m}}, \quad |D_xG(x,p)|\le M_1(x)+M_2(x)|p|^{m},\quad |D_pG(x,p)|&\le N_1(x)+N_2(x)|p|^{m-1}. \end{aligned}$$

Theorem A.1

Let (G) hold. Then, there exists some $K>0$ depending only on d, m, and $\kappa$ such that, for any solution $\phi \in C^3(U)$ of (A.1) and $x_0\in U$, $r>0$ with $B_r(x_0)\subset U$,

$$\begin{aligned} \begin{aligned} \sup _{B_{r/2}(x_0)}|D\phi |&\le K\sup _{B_r(x_0)} \left( f_+^{\frac{1}{m}}+|Df|^{\frac{1}{2m-1}}+|g|^{\frac{1}{m-1}}+|Dg|^{\frac{1}{2m-2}} +M_1^{\frac{1}{2m-1}}+M_2^{\frac{1}{m-1}}\right) \\&\quad + K \left\{ (\sup _{B_r(x_0)}|g|^{\frac{1}{2m-2}}+N_1^{\frac{1}{2m-2}})r^{-\frac{1}{2m-2}}+(1+N_2^{\frac{1}{m-1}})r^{-\frac{1}{m-1}}\right\} . \end{aligned} \end{aligned}$$

where $z_\pm :=\max \{\pm z,0\}$ for $z\in {\mathbf {R}}$.

Proof

We set $w:=(1/2)|D\phi |^2$. Then, $Dw=(D^2\phi )( D\phi )$ and

$$\begin{aligned} \Delta w={\text {tr}}((D^2\phi )^2)+D(\Delta \phi )\cdot D\phi , \end{aligned}$$

(A.2)

where ${\text {tr}}(A)$ denotes the trace of a square matrix A. Since first and second terms of the right-hand side of (A.2) can be estimated as

$$\begin{aligned} {\text {tr}}( (D^2\phi )^2)&\ge \frac{1}{d}({\text {tr}}(D^2\phi ))^2=\frac{1}{d} (\Delta \phi )^2 =\frac{1}{d} (g\cdot D\phi +G+ f_--f_+)^2\\&\ge \frac{1}{3d}G^2-\frac{3}{d} |g|^2|D\phi |^2 -\frac{3}{d} {f_{+}^2} \end{aligned}$$

and

$$\begin{aligned} D(&\Delta \phi )\cdot D\phi \\&= D(g\cdot D\phi +G-f )\cdot D\phi \\&= (Dg)(D\phi )\cdot (D\phi )+ g\cdot (D^2\phi )(D\phi ) +D_xG\cdot D\phi + D_pG\cdot (D^2\phi )(D\phi )-Df\cdot D\phi \\&\ge -|Dg||D\phi |^2- (|g|+|D_pG|)|Dw|-(|D_xG| +|Df|)|D\phi |\\&\ge -|Dg||D\phi |^2-(|g|+N_1+N_2|D\phi |^{m-1})|Dw| -(M_1+M_2|D\phi |^{m}+|Df|)| D\phi |, \end{aligned}$$

respectively, we see by plugging them into (A.2) that

$$\begin{aligned} \begin{aligned} \Delta w&\ge \frac{1}{3d}G^2-\Big (\frac{3}{d}|g|^2+|Dg|\Big )|D\phi |^2-\frac{3}{d} {f_{+}^2}\\&\qquad -(M_1+|Df|)|D\phi |-M_2|D\phi |^{m+1}-(|g|+N_1+N_2|D\phi |^{m-1})| Dw|. \end{aligned} \end{aligned}$$

(A.3)

Now, let $\zeta \in C^2(U)$ be any cut-off function such that $\zeta \equiv 1$ in $B_{1/2}$, ${\text {supp}}\zeta \subset B_1$, and $0\le \zeta \le 1$ in $B_1$. We set

$$\begin{aligned} \xi (x):=\Big \{\zeta \Big (\frac{x-x_0}{r}\Big )\Big \}^{\gamma },\quad u(x):=\xi (x)w(x) \end{aligned}$$

for $x\in U$, where $\gamma >1$ is some constant which will be determined later. Since $u\equiv 0$ outside $B_r(x_0)$, we observe that u attains its positive maximum at a point ${\hat{x}}\in B_r(x_0)$. In particular, we have $u({\hat{x}})=\xi ({\hat{x}}) w({\hat{x}})>0$, $Du({\hat{x}})=\xi ({\hat{x}}) Dw({\hat{x}})+w({\hat{x}})D\xi ({\hat{x}})=0$, and $\Delta u({\hat{x}})\le 0$, which also implies that

$$\begin{aligned} Dw({\hat{x}})=-w({\hat{x}})\frac{D\xi ({\hat{x}})}{\xi ({\hat{x}})}=-\frac{1}{2}|D\phi ({\hat{x}})|^2\frac{D\xi ({\hat{x}})}{\xi ({\hat{x}})}. \end{aligned}$$

We now set $Z:=\xi ({\hat{x}})|D\phi ({\hat{x}})|^{2m}$ and find an upper bound of Z. To this end, we first evaluate $\Delta u$ at $x={\hat{x}}$. In what follows, we drop ${\hat{x}}$ for the simplicity of notation. Then,

$$\begin{aligned} 0\ge \Delta u&=\xi \Delta w+2D\xi \cdot Dw +w\Delta \xi =\xi \Delta w-|D\phi |^2\frac{|D\xi |^2}{\xi }+\frac{1}{2} |D\phi |^2\Delta \xi \\&=\xi \Delta w-(\xi |D\phi |^{2m})^{\frac{1}{m}}\,(\xi ^{-\frac{m+1}{2m}}|D\xi |)^2+\frac{1}{2}(\xi |D\phi |^{2m})^{\frac{1}{m}}\,(\xi ^{-\frac{1}{m}}\Delta \xi )\\&=\xi \Delta w-(\xi ^{-\frac{m+1}{2m}}|D\xi |)^2Z^{\frac{1}{m}}-\frac{1}{2}(\xi ^{-\frac{1}{m}}|\Delta \xi |)Z^{\frac{1}{m}}. \end{aligned}$$

We next estimate the value of $\xi \Delta w$ at $x={\hat{x}}$. Since $0\le \xi \le \xi ^{\delta }\le 1$ for any $0\le \delta \le 1$ and $2\xi |Dw|=|D\phi |^2|D\xi |$ at $x={\hat{x}}$, we see in view of (A.3) that

$$\begin{aligned} \xi \Delta w&\ge \frac{\kappa ^2}{3d}(\xi |D\phi |^{2m})-\Big (\frac{3}{d} |g|^2+|Dg|\Big )(\xi |D\phi |^{2m})^{\frac{1}{m}}-\frac{3}{d} \xi {f_{+}^2}\\&\qquad -(M_1+|Df|)(\xi |D\phi |^{2m})^{\frac{1}{2m}}-M_2(\xi |D\phi |^{2m})^{\frac{m+1}{2m}}\\&\qquad -\frac{1}{2}(|g|+N_1)(\xi ^{-\frac{1}{m}}|D\xi |)(\xi |D\phi |^{2m})^{\frac{1}{m}}-\frac{1}{2} N_2(\xi ^{-\frac{m+1}{2m}}|D\xi |)(\xi |D\phi |^{2m})^{\frac{m+1}{2m}}\\&=\frac{\kappa ^2}{3d}Z-\Big (\frac{3}{d} |g|^2+|Dg|\Big )Z^{\frac{1}{m}}-\frac{3}{d} \xi {f_{+}^2}-(M_1+|Df|)Z^{\frac{1}{2m}}-M_2Z^{\frac{m+1}{2m}}\\&\qquad -\frac{1}{2}(|g|+N_1)(\xi ^{-\frac{1}{m}}|D\xi |)Z^{\frac{1}{m}} -\frac{1}{2} N_2(\xi ^{-\frac{m+1}{2m}}|D\xi |)Z^{\frac{m+1}{2m}}. \end{aligned}$$

Plugging this into the previous inequality, we obtain

$$\begin{aligned} Z&\le K\{ ( |g|^2+|Dg|)Z^{\frac{1}{m}} + {f_{+}^2} +(M_1+|Df|)Z^{\frac{1}{2m}} +M_2Z^{\frac{m+1}{2m}}\\&\quad +(|g|+N_1)(\xi ^{-\frac{1}{m}}|D\xi |)Z^{\frac{1}{m}} +N_2(\xi ^{-\frac{m+1}{2m}}|D\xi |)Z^{\frac{m+1}{2m}}\\&\quad +(\xi ^{-\frac{m+1}{2m}}|D\xi |)^2Z^{\frac{1}{m}} +(\xi ^{-\frac{1}{m}}|\Delta \xi |)Z^{\frac{1}{m}}\}, \end{aligned}$$

where $K>0$ is some constant depending only on d and $\kappa$. In the following, we denote by K various constants depending only on d, $\kappa$, and m.

Now, we recall Young’s inequality to see that, for any $L>0$ and $\varepsilon >0$,

$$\begin{aligned} LZ^{\frac{1}{2m}}\le C_\varepsilon L^{\frac{2m}{2m-1}}+\varepsilon Z,\qquad LZ^{\frac{1}{m}}\le C_\varepsilon L^{\frac{m}{m-1}}+\varepsilon Z,\qquad LZ^{\frac{m+1}{2m}}\le C_\varepsilon L^{\frac{2m}{m-1}}+\varepsilon Z, \end{aligned}$$

where $C_\varepsilon >0$ is a constant depending only on $\varepsilon$ and m. Then, we have

$$\begin{aligned} Z&\le K\{{f_{+}^2}+(|g|^2+|Dg|)^{\frac{m}{m-1}}+(M_1+|Df|)^{\frac{2m}{2m-1}}+M_2^{\frac{2m}{m-1}}\\&\quad +(|g|+N_1)^{\frac{m}{m-1}}(\xi ^{-\frac{1}{m}}|D\xi |)^{\frac{m}{m-1}} +N_2^{\frac{2m}{m-1}}(\xi ^{-\frac{m+1}{2m}}|D\xi |)^{\frac{2m}{m-1}}\\&\quad +(\xi ^{-\frac{m+1}{2m}}|D\xi |)^{\frac{2m}{m-1}}+(\xi ^{-\frac{1}{m}}|\Delta \xi |)^{\frac{m}{m-1}}\}. \end{aligned}$$

We finally find upper bounds for $\xi ^{-\frac{1}{m}}|D\xi |$, $\xi ^{-\frac{m+1}{2m}}|D\xi |$, and $\xi ^{-\frac{1}{m}}|\Delta \xi |$. Since $\xi ^{-\frac{1}{m}}<\xi ^{-\frac{m+1}{2m}}$, it suffices to consider $\xi ^{-\theta }|D\xi |$ and $\xi ^{-\theta }|\Delta \xi |$ for $\theta :=(m+1)/(2m)$. By choosing $\gamma :=4m/(m-1)$, which is equivalent to $\gamma -2-\gamma \theta =0$, we see by direct computations that

$$\begin{aligned} \xi ^{-\theta }|D\xi |&=\frac{\gamma }{r}\zeta ^{\gamma -1-\gamma \theta }|D\zeta |=\frac{\gamma }{r}\zeta |D\zeta |,\\ \xi ^{-\theta }|\Delta \xi |&\le \frac{\gamma }{r^2}\{\zeta ^{\gamma -1-\gamma \theta }|\Delta \zeta |+(\gamma -1)\zeta ^{\gamma -2-\gamma \theta }|D\zeta |^2\}=\frac{\gamma }{r^2}\{\zeta |\Delta \zeta |+(\gamma -1)|D\zeta |^2\}. \end{aligned}$$

Hence, there exists some $C>0$ depending only on m, $\sup _{B_1}|D\zeta |$, and $\sup _{B_1}|\Delta \zeta |$ such that, at ${\hat{x}}$,

$$\begin{aligned} Z=\xi |D\phi |^{2m}&\le K({f_{+}^2}+|Df|^{\frac{2m}{2m-1}}+|g|^{\frac{2m}{m-1}}+|Dg|^{\frac{m}{m-1}} +M_1^{\frac{2m}{2m-1}}+M_2^{\frac{2m}{m-1}})\\&\quad + KC\{(|g|^{\frac{m}{m-1}}+N_1^{\frac{m}{m-1}})r^{-\frac{m}{m-1}}+(1+N_2^{\frac{2m}{m-1}})r^{-\frac{2m}{m-1}}\}. \end{aligned}$$

Since $\xi \equiv 1$ in $B_{r/2}(x_0)$, we see that

$$\begin{aligned} \sup _{x\in B_{r/2}(x_0)}|D\phi (x)|^2&=2\sup _{x\in B_{r/2}(x_0)}\xi (x)w(x) =2\sup _{x\in B_{r/2}(x_0)}u(x)\\&\le 2\sup _{x\in B_r(x_0)}u(x) =2u({\hat{x}}) =2\xi ({\hat{x}})w({\hat{x}})\\&=\xi ({\hat{x}})^{\frac{m-1}{m}}(\xi ({\hat{x}})|D\phi ({\hat{x}})|^{2m})^{\frac{1}{m}} \le Z^{\frac{1}{m}}, \end{aligned}$$

which implies the desired estimate. Hence, we have completed the proof. $\square$