Fast Convergence of Inertial Gradient Dynamics with Multiscale Aspects

Abstract

In this paper, the asymptotic properties as $t\rightarrow +\infty$ of the following second-order differential equation in a Hilbert space ${\mathcal {H}}$ are studied,

$$\begin{aligned} \ddot{x}(t)+\gamma (t){\dot{x}}(t)+\beta (t)\Big (\nabla \Phi (x(t))+\epsilon (t)\nabla U(x(t))\Big )=0, \end{aligned}$$

where $\Phi ,U:{\mathcal {H}}\rightarrow {\mathbb {R}}$ are convex differentiable, $\gamma (t)$ is a positive damping coefficient, $\beta (t)$ is a time scale coefficient and $\epsilon (t)$ is a positive nonincreasing function, $\gamma (t)$, $\beta (t)$ and $\epsilon (t)$ are all continuously differentiable. This system has applications in the fields of mechanics and optimization. Based on the proper tuning of $\gamma (t)$ and $\beta (t)$, we obtain the convergence rates for the values, and the conclusion is that, under the different conditions, the trajectories either converge to one minimizer of $\Phi$ weakly, or converge to one common minimizer of $\Phi$ and U weakly. When $\epsilon (t)$ tends to 0 as t goes to infinity, under the condition that $\Phi$ or U is convex, the trajectories converge to the unique minimizer of $\Phi$, or the unique minimizer of U, respectively. Finally, some particular cases are examined, and some numerical experiments are conducted to illustrate our main results.

Appendix

In what follows, we prove the existence and the uniqueness of a global solution to the Cauchy problem associated with the evolution system (1). We will use the following lemma, which have been established in [15, Prop. 6.2.1].

Lemma A.1

Let $F:I\times {\mathcal {X}}\rightarrow {\mathcal {X}}$ where $I=[t_0,+\infty )$ and ${\mathcal {X}}$ is a Banach space. Assume that

• (i) for every $x\in {\mathcal {X}}$, $F(\cdot ,x)\in L^1_{loc}(I,{\mathcal {X}})$;

• (ii) for a.e. $t\in I$, for every $x,y\in {\mathcal {X}}$,

  $$\begin{aligned} \left\| F(t,x)-F(t,y)\right\| \leqslant K(t,\left\| x\right\| +\left\| y\right\| )\left\| x-y\right\| \end{aligned}$$

  where $K(\cdot ,r)\in L^1_{loc}(I), \forall r\in {\mathbb {R}}_+$;

• (iii) for a.e. $t\in I$, for every $x\in {\mathcal {X}}$,

  $$\begin{aligned} \left\| F(t,x)\right\| \leqslant P(t)(1+\left\| x\right\| ), \end{aligned}$$

  where $P\in L^1_{loc}(I)$.

Then, for every $s\in I, x\in {\mathcal {X}}$, there exists a unique solution $u_{s,x}\in W_{loc}^{1,1}(I,{\mathcal {X}})$ of the Cauchy problem:

$$\begin{aligned} {\dot{u}}_{s,x}(t)=F(t,u_{s,x}(t)) \text { for a.e. } t\in I , \text { and } u_{s,x}(s)=x . \end{aligned}$$

The following theorem gives the existence and uniqueness of system (1).

Theorem A.1

Suppose that $\Phi ,U:{\mathcal {H}}\rightarrow {\mathbb {R}}$ are convex, ${\mathcal {C}}^1$, with Lipschitz continuous gradient $\nabla \Phi$ and $\nabla U$. Assume that $\beta ,\gamma ,\epsilon :[t_0,+\infty )\rightarrow {\mathbb {R}}_+^*$ are locally integrable. Then, the evolution system (1), with initial condition $(x(t_0),{\dot{x}}(t_0))=(x_0,{\dot{x}}_0)\in {\mathcal {H}}\times {\mathcal {H}}$, admits a unique global solution $x:[t_0\rightarrow +\infty ) \rightarrow {\mathcal {H}}$.

Proof

To prove the existence and uniqueness for (1) with initial condition $(x(t_0),{\dot{x}}(t_0))=(x_0,{\dot{x}}_0)$, we formulate it in the phase space. Set $I=[t_0,+\infty )$, and define $F:I\times {\mathcal {H}}\times {\mathcal {H}}\rightarrow {\mathcal {H}}$ by

$$\begin{aligned} F(t,x,y)=(y,-\gamma (t)y-\beta (t)(\nabla \Phi +\epsilon (t)\nabla U(x))) \end{aligned}$$

We set $u(t)=(x(t),y(t))$. The Cauchy problem for (1) can be equivalently formulated as:

$$\begin{aligned} \begin{aligned} {\dot{u}}(t)&=F(t,u(t))\quad \text {for a.e. } t\in I ,\\ u(t_0)&=(x_0,{\dot{x}}_0). \end{aligned} \end{aligned}$$

We are going to verify the three conditions of Lemma A.1.

(i) For each $(x,y)\in {\mathcal {H}} \times {\mathcal {H}}$, $F(\cdot ,x,y)\in L^1_{loc}(I,{\mathcal {H}})$, since the functions $\beta ,\gamma ,\epsilon$ are so.

(ii) Denote by $L_1$ and $L_2$ the Lipschitz constant of $\nabla \Phi$ and $\nabla U$. For every $u=(x,y), u'=(x',y')\in {\mathcal {H}}\times {\mathcal {H}}$ and a.e. $t\in I$,

$$\begin{aligned} \begin{aligned}&\left\| F(t,u)-F(t,u')\right\| \\&\quad ={}\left\| y-y'\right\| +\left\| \gamma (t)(y-y')+\beta (t)(\nabla \Phi (x)-\nabla \Phi (x')+\epsilon (t)(\nabla U(x)-\nabla U(x')))\right\| \\&\quad \leqslant {}(1+\gamma (t)+L_1\beta (t)+L_2\beta (t)\epsilon (t))(\left\| x-x'\right\| +\left\| y-y'\right\| )\\&\quad ={}(1+\gamma (t)+L_1\beta (t)+L_2\beta (t)\epsilon (t))\left\| (x,y)-(x',y')\right\| \end{aligned} \end{aligned}$$

Hence, the second condition is verified, since the real function $t\mapsto 1+\gamma (t)+L_1\beta (t)+L_2\beta (t)\epsilon (t)$ belongs to $L_{loc}^1(I,{\mathbb {R}})$.

(iii) For every $u=(x,y)\in {\mathcal {H}}\times {\mathcal {H}}$ and a.e. $t\in I$,

$$\begin{aligned} \begin{aligned}&\left\| F(t,u)\right\| \\&\quad ={}\left\| y\right\| +\left\| \gamma (t)y+\beta (t)(\nabla \Phi (x)-\nabla \Phi (x_0)+\nabla \Phi (x_0) \right. \\&\qquad \left. +\epsilon (t)(\nabla U(x)-\nabla U(x_0)+\nabla U(x_0)))\right\| \\&\quad \leqslant {}(1+\gamma (t))\left\| y\right\| +(L_1\beta (t)+L_2\beta (t)\epsilon (t))\left\| x-x_0\right\| \\&\qquad +\beta (t)\left\| \nabla \Phi (x_0)\right\| +\beta (t)\epsilon (t)\left\| \nabla U(x_0)\right\| \\&\quad \leqslant {} v(t)(1+\left\| x\right\| +\left\| y\right\| ), \end{aligned} \end{aligned}$$

where

$$\begin{aligned} v(t)= & {} \max (1+\gamma (t),L_1\beta (t)+L_2\beta (t)\epsilon (t),\beta (t)((L_1+L_2\epsilon (t))\left\| x_0\right\| \\&+\nabla \Phi (x)+\epsilon (t)\nabla U(x))). \end{aligned}$$

Since $v(\cdot )\in L_{loc}^1(I,{\mathbb {R}})$, we conclude that all the conditions of Lemma A.1 are satisfied. Therefore, there exists a unique global solution of system (1) satisfying the initial condition $(x(t_0),{\dot{x}}(t_0))=(x_0,{\dot{x}}_0)$. $\square$

The following lemma can be found in [26, Lemma 3.2].

Lemma A.2

Let $\delta >0$, and let $w:[\delta ,+\infty )\rightarrow {\mathbb {R}}$ be a continuously differentiable function which is bounded from below. If the positive part $[{\dot{w}}]_+$ of ${\dot{w}}$ belongs to $L^1(\delta ,+\infty )$, then $\lim _{t\rightarrow +\infty }w(t)$ exists.

Proof

Since $[{\dot{w}}]_+$ belongs to $L^1(\delta ,+\infty )$, we have $\int _{\delta }^{+\infty }{[{\dot{w}}(t)]_+\mathrm {d}t}<+\infty$. Let $\theta (t)=w(t)-\int _{\delta }^{t}{[{\dot{w}}(t)]_+\mathrm {d}s}$, hence we obtain ${\dot{\theta }}(t)=\dot{w }(t)-[{\dot{w}}(t)]_+\leqslant 0$, which means that $\theta (t)$ is nonincreasing. Since w(t) is bounded from below, we immediately deduce that $\theta (t)$ is also bounded from below, it implies that $\lim _{t\rightarrow +\infty }\theta (t)$ exists. Therefore,

$$\begin{aligned} \lim _{t\rightarrow +\infty }w(t)=\lim _{t\rightarrow +\infty }\theta (t)+\int _{\delta }^{+\infty }{[{\dot{w}}(t)]_+\mathrm {d}t} \end{aligned}$$

exists. $\square$

To establish the weak convergence of the solutions of (1), we shall use Opial’s lemma in [24], which has the continuous form. This argument was first used in [13] to obtain the convergence of nonlinear contraction semigroups.

Lemma A.3

Let S be a nonempty subset of ${\mathcal {H}}$ and let $x:[0,+\infty )\rightarrow {\mathcal {H}}$. Assume that

• (i) for every $z\in S$, $\lim _{t\rightarrow +\infty }\left\| x(t)-z\right\|$ exists;

• (ii) every weak sequential limit point of x(t) , as $t\rightarrow +\infty$, belongs to S.

Then, x(t) converges weakly as $t\rightarrow +\infty$ to a point in S.

Introducing three lemmas about $\gamma (t)$ and $\Gamma (t)$, which has been established in [2].

Lemma A.4

Let $\Gamma$ defined by (3). If there exists $m>0$ such that $\gamma (t)\leqslant m$ for every $t\geqslant t_2$. Then, we have $\Gamma (t)\geqslant \frac{1}{m}$ for every $t\geqslant t_2$.

Lemma A.5

Let $\Gamma$ defined by (3). If there exists $t_1 \geqslant t_0$ and $a\in [0,1)$ such that $\frac{{\dot{\gamma }}(t)}{\gamma (t)^2}\geqslant -a$ for every $t\geqslant t_1$. Then, we have $\int _{t_0}^{+\infty }{\mathrm {e}^{-\int _{t_0}^{t}{\gamma (u)\mathrm {d}u}}}<+\infty$ and $\Gamma (t)\leqslant \frac{1}{(1-a)\gamma (t)}$ for every $t\geqslant t_1$.

Lemma A.6

Let $\Gamma$ defined by (3). If there exists $c\in [0,1[$ such that $\lim _{t\rightarrow +\infty }\frac{{\dot{\gamma }}(t)}{\gamma (t)^2}=-c$. Then we have $\Gamma (t)\sim \frac{1}{(1-c)\gamma (t)}$ as $t\rightarrow +\infty$.