A notion of stability in probability of stochastic nonlinear systems

Abstract

The concepts of stability in probability of nontrivial solutions for stochastic nonlinear systems are analyzed in terms of a control Lyapunov function which is smooth except possibly at the origin. We show under certain hypothesis that the neighborhood of the origin is stable in probability. An illustrating example is provided.

MSC:60H10, 93C10, 93D05, 93D15, 93D21, 93E15.

1 Introduction

In recent years, the stability analysis of stochastic nonlinear systems (SNS) has been one of the most active and important areas in control theory. Although there are many research papers (see, for instance, [1–13]) on the stability of trivial solutions of SNS, but there are very few results on the stability of nontrivial solutions of stochastic systems. It is known that there are many types of stochastic systems all of whose solutions tend to each other although they do not have the trivial solution. These systems are worth being interested in since the solutions with different initial value has similar large-time properties. In this paper, we study the stability in probability of a nontrivial solution of the SNS.

The main results of the present paper is to explore further the stabilization in probability problem for a wider class of SNS than that described in [14]. This paper is also intended to fill in the gap of the previous works by establishing necessary and sufficient conditions for stability in probability of nontrivial solution for the SNS. Our main result, Theorem 3.1, which is an extension of Theorem 3.1 established in [2], asserts that the SNS is stable in probability if and only if it admits a control Lyapunov function (CLF). The main tool used in this paper is a stochastic version of the converse Lyapunov theorem established in Kushner [9].

The paper is organized as follows. In Section 2, we introduce a class of stochastic systems, some basic definitions and properties of stochastic CLF which play an important role in the stability theory. In Section 3, we state and prove the main results of the paper on the stability in probability of nontrivial solution of SNS. Finally, in Section 4, we provide a numerical example to validate our results.

2 Stochastic stability

In this section we first introduce a class of stochastic systems and define the stability in probability of nontrivial solutions of this system. We also focus on the properties of CLF which play an important role to the stability in Section 3.

Let $(\mathrm{\Omega },F,P)$ be a complete probability space, and denote by ${(w_t)}_{t\ge 0}$ a standard ${\mathbb{R}}^m$-valued Wiener process defined on this space.

Consider the following SNS

$$dx=f(t,x)dt+g(t,x)d{w}_t,$$

(1)

where $x\in {\mathbb{R}}^n$ is the system state vector, $f:{\mathbb{R}}^{+}\times {\mathbb{R}}^n\to {\mathbb{R}}^n$ and $g:{\mathbb{R}}^{+}\times {\mathbb{R}}^n\to {\mathbb{R}}^{n\times m}$ are continuous with $f(t,0)\ne 0$, $g(t,0)\ne 0$, and there exist constants $C_1,C_2\ge 0$ such that the following conditions hold:

$$\begin{array}{c}|f(t,x)-f(t,y)|+|g(t,x)-g(t,y)|\le C_1(|x-y|),\hfill \\ |f(t,x)|+|g(t,x)|\le C_2(1+|x|).\hfill \end{array}$$

We note that SNS (1) satisfies a sufficient condition, under which there is some unique global solution $x(t,x_0)$ to SNS (1) for any given initial value $x(0)=x_0\in {\mathbb{R}}^n$ (see, for instance, Khasminskii [8]).

Throughout this paper we adopt the following notations:

• For any $x_0\in {\mathbb{R}}^n$, denote by $x(t)=x(t,x_0)$ the solution of SNS (1).

• Denote by $|\cdot |$ the usual Euclidean norm in ${\mathbb{R}}^n$.

• We use ${\mathrm{\Phi }}_0=\mathrm{\Phi }(0,x_0)$ to denote the initial value of a Lyapunov function.

• $\mathcal{N}$ denotes the class of nonnegative $C^0$ functions $\mu :{\mathbb{R}}^{+}\to {\mathbb{R}}^{+}$, for which ${\int }_0^{+\mathrm{\infty }}\mu (t)dt<+\mathrm{\infty }$ and ${lim}_{t\to +\mathrm{\infty }}\mu (t)=0$ hold.

• $S_r=\{x\in {\mathbb{R}}^n;|x|\le r,r>0\}$ is a ball.

• We define $\tau \wedge t=min\{\tau ,t\}$.

Definition 2.1 A function $\gamma :{\mathbb{R}}^{+}\to {\mathbb{R}}^{+}$ is a

• (i) K-function if it is continuous, strictly increasing and $\gamma (0)=0$, and

• (ii) $K_{\mathrm{\infty }}$-function if it is a K-function and also $\gamma (r)\to \mathrm{\infty }$ as $r\to \mathrm{\infty }$.

We now give the definition of stability in probability for SNS (1) when the origin is not a trivial solution. In this case we study the stability in probability of solutions with respect to the small neighborhood of the origin.

Definition 2.2 We say that $S_r$, $r>0$ is stable in probability if for any $c>r$ and any $ϵ\in (0,1)$, there exists $\delta =\delta (ϵ,c)$ such that for all $t\ge t_0\ge 0$ and $|x_0|<\delta$, the following condition holds:

$$P\{|x(t)|<c\}\ge 1-ϵ.$$

(2)

In the following we consider the SNS

$$dx=(f(t,x)+h(t,x)u)dt+g(t,x)d{w}_t,$$

(3)

where $x\in {\mathbb{R}}^n$, $u\in {\mathbb{R}}^p$, the dynamics $f(t,x)$, $g(t,x)$ and $h:{\mathbb{R}}^{+}\times {\mathbb{R}}^n\to {\mathbb{R}}^{n\times p}$, are continuous and Lipschitz functions with $f(t,0)\ne 0$ and $g(t,0)\ne 0$.

Theorem 3.1 of this paper is an extension of Theorem 3.1 provided in [2] that guarantees the existence of a $C^{\mathrm{\infty }}$ control law $u=\varphi (t,x)$ in such a way that $S_r$ satisfy the stability in probability property with respect to the closed-loop system

$$dx=(f(t,x)+h(t,x)\varphi (t,x))dt+g(t,x)d{w}_t.$$

(4)

Denote by D the infinitesimal generator of the stochastic process solution of the uncontrolled part of SNS (3), that is, D is the second-order differential operator defined for any function Φ in $C^{2,1}({\mathbb{R}}^{+}\times {\mathbb{R}}^n,\mathbb{R})$ by

$$\mathbf{D}\mathrm{\Phi }(t,x)=\frac{\partial \mathrm{\Phi }(t,x)}{\partial t}+\sum _{i=1}^n{f}^i\frac{\partial \mathrm{\Phi }(t,x)}{\partial x_i}+\frac{1}{2}\sum _{i,j=1}^n\sum _{k=1}^m{g}_k^i{g}_k^j\frac{{\partial }^2\mathrm{\Phi }(t,x)}{\partial x_i\partial x_j}.$$

For any $z\in (1,\dots ,p)$, denote by ${\mathbf{D}}_z$ the first-order differential operator defined for any function Φ in $C^{1,1}({\mathbb{R}}^{+}\times {\mathbb{R}}^n,\mathbb{R})$ by

$${\mathbf{D}}_z\mathrm{\Phi }(t,x)=\sum _{i=1}^n{h}_z^i(t,x)\frac{\partial \mathrm{\Phi }(t,x)}{\partial x_i}.$$

Define Y as the infinitesimal generator for the stochastic process solution of the closed-loop system (4), that is, Y is the differential operator defined for any function Φ in $C^{2,1}({\mathbb{R}}^{+}\times {\mathbb{R}}^n,\mathbb{R})$ by

$$\mathbf{Y}\mathrm{\Phi }(t,x)=\mathbf{D}\mathrm{\Phi }(t,x)+\sum _{z=1}^p{\mathbf{D}}_z\mathrm{\Phi }(t,x)u_z.$$

In the following we extend the concept of stochastic Lyapunov condition introduced in Definition 2.4 provided in [2] used for stability in probability of SNS (3) at the neighborhood of the origin.

Definition 2.3 SNS (3) admits a CLF if there exists a $C^{2,1}$ function $\mathrm{\Phi }:{\mathbb{R}}^{+}\times {\mathbb{R}}^n\to {\mathbb{R}}^{+}$, class $K_{\mathrm{\infty }}$ functions $a_1$, $a_2$, a $C^0$ positive definite function $\rho :{\mathbb{R}}^{+}\to {\mathbb{R}}^{+}$, and a nonnegative $C^0$ function μ of class $\mathcal{N}$ such that for all $(t,x)\in {\mathbb{R}}^{+}\times {\mathbb{R}}^n$, the following conditions hold:

$$a_1(|x|)\le \mathrm{\Phi }(t,x)\le a_2(|x|),$$

(5)

$${\mathbf{D}}_z\mathrm{\Phi }(t,x)=0\Rightarrow \mathbf{D}\mathrm{\Phi }(t,x)\le -\rho (\mathrm{\Phi }(t,x))+\mu (t).$$

(6)

Remark 2.4 We note that condition (6) obtained in Definition 2.3 can be replaced with the following condition

$$\mathbf{D}\mathrm{\Phi }(t,x)+\sum _{z=1}^p{\mathbf{D}}_z\mathrm{\Phi }(t,x)u_z\le -\rho (\mathrm{\Phi }(t,x))+\mu (t).$$

In the next section we will turn the attention to the class of SNS (3) and provide some results related to the stability in probability of this system.

3 Main results

In this section, where the dynamics $f(t,0)$ and $g(t,0)$ may be non-vanishing at the origin, we study the stability in probability of nontrivial solutions of SNS (3). In the following theorem that is an extension of Theorem 3.1 established in [2], we derive the necessary and sufficient conditions for stability in probability of nontrivial solutions of SNS (3).

Theorem 3.1 Consider SNS (3). Then the following three statements are equivalent:

• (i) There exists a $C^{\mathrm{\infty }}$ control law $u=\varphi :{\mathbb{R}}^{+}\times {\mathbb{R}}^n\to {\mathbb{R}}^m$ with $\varphi (t,0)=0$ for all $t\ge 0$ such that $S_r$ satisfy the stability in probability property with respect to closed-loop system (4).

• (ii) There exists a $C^0$ control law $u=\varphi :{\mathbb{R}}^{+}\times {\mathbb{R}}^n\to {\mathbb{R}}^m$ with $\varphi (t,0)=0$ for all $t\ge 0$, being locally Lipschitz in x, in such a way that $S_r$ satisfy the stability in probability property with respect to closed-loop system (4).

• (iii) There exists a $C^{2,1}$ function $\mathrm{\Phi }:{\mathbb{R}}^{+}\times {\mathbb{R}}^n\to {\mathbb{R}}^{+}$, class $K_{\mathrm{\infty }}$ functions $a_1$, $a_2$, a $C^0$ positive definite function $\rho :{\mathbb{R}}^{+}\to {\mathbb{R}}^{+}$, and a nonnegative $C^0$ function μ of class $\mathcal{N}$ such that for all $(t,x)\in {\mathbb{R}}^{+}\times {\mathbb{R}}^n$, conditions (5) and (6) hold.

Proof The implication (i) → (ii) is obvious.

We should establish the implication (ii) → (iii). Suppose that there exists a $C^0$ control law $u=\varphi (t,x)$ (as in statement (ii) of the theorem) such that $S_r$ satisfies the stability in probability with respect to the closed-loop system (4). Then, by the converse Lyapunov theorem (Theorem 2) developed by Kushner [9], which proved the existence of a stochastic Lyapunov function in some neighborhood of the origin, there exists a Lyapunov function Φ in $C^{2,1}({\mathbb{R}}^{+}\times {\mathbb{R}}^n,{\mathbb{R}}^{+})$ and a continuous and positive definite function $k(t,x)$ in such a way that (5) holds and

$$\mathbf{Y}\mathrm{\Phi }(t,x)=-k(t,x)\le 0,$$

(7)

where Y is the infinitesimal generator for the stochastic process solution of closed-loop system (4). The latter inequality implies (6) with $\rho (\mathrm{\Phi }(t,x))=k(t,x)$, $\mu (t)=0$ and ${\mathbf{D}}_z\mathrm{\Phi }(t,x)=0$. Therefore, Definition 2.3 holds and the Lyapunov function $\mathrm{\Phi }(t,x)$ is a stochastic CLF for SNS (3).

Next, for (iii) → (i), suppose that there exists a $C^{2,1}$ function $\mathrm{\Phi }:{\mathbb{R}}^{+}\times {\mathbb{R}}^n\to {\mathbb{R}}^{+}$ such that both conditions (5) and (6) are fulfilled with respect to closed-loop system (4). Without any loss of generality, assume

$$\mu (t)>0\mathrm{\forall }t\ge 0.$$

(8)

Notice, by virtue of (5), that

$$\frac{\partial \mathrm{\Phi }}{\partial t}(t,0)=0,\frac{\partial \mathrm{\Phi }}{\partial x}(t,0)=0.$$

(9)

Then using (5) and (6) and by applying the converse result of Theorem 3.1 established in [2], condition (6) in conjunction with (8) and (9) enables us to build, by the standard partition of unity arguments, a $C^{\mathrm{\infty }}$ control law $u=\varphi (t,x)$ with $\varphi (t,0)=0$ such that

$$\mathbf{Y}\mathrm{\Phi }(t,x)=\mathbf{D}\mathrm{\Phi }(t,x)+\sum _{z=1}^p{\mathbf{D}}_z\mathrm{\Phi }(t,x)u_z\le -\rho (\mathrm{\Phi }(t,x))+\mu (t).$$

(10)

From (10) and $\mathrm{\Phi }(t,x)\ge 0$, we have that ${\mathrm{\Phi }}_t=\mathrm{\Phi }(t,x)$ is a supermartingale. Let $\tau =inf\{t\ge 0;|x(t)|\ge c,c>r\}$ and ${\mathrm{\Phi }}_0=\mathrm{\Phi }(0,x_0)$. Invoking Ito’s formula for SNS (3), we get

$$\begin{array}{rcl}\mathrm{\Phi }(\tau \wedge t,x(\tau \wedge t))& =& {\mathrm{\Phi }}_0+{\int }_0^{\tau \wedge t}(\mathbf{D}\mathrm{\Phi }(s,x)+\sum _{z=1}^p{\mathbf{D}}_z\mathrm{\Phi }(s,x)u_z)ds\\ +{\int }_0^{\tau \wedge t}\sum _{i=1}^n{g}^i(s,x)\frac{\partial \mathrm{\Phi }(s,x)}{\partial x_i}d{w}_s.\end{array}$$

(11)

With (10) and (11) we have

$$\mathrm{\Phi }(\tau \wedge t,x(\tau \wedge t))={\mathrm{\Phi }}_0+{\int }_0^{\tau \wedge t}\mathbf{Y}\mathrm{\Phi }(s,x)ds+{\int }_0^{\tau \wedge t}\sum _{i=1}^n{g}^i(s,x)\frac{\partial \mathrm{\Phi }(s,x)}{\partial x_i}d{w}_s.$$

(12)

From (12) we obtain

$$E\left[\mathrm{\Phi }(\tau \wedge t,x(\tau \wedge t))\right]={\mathrm{\Phi }}_0+E({\int }_0^{\tau \wedge t}\mathbf{Y}\mathrm{\Phi }(s,x)ds).$$

(13)

Assume that there exists a nonnegative $C^0$ function μ of class $\mathcal{N}$ such that

$${\int }_0^{+\mathrm{\infty }}\mu (t)dt\le K<+\mathrm{\infty }$$

(14)

for all $K>0$. Taking into account (10) and (13)-(14) yields

$$\begin{array}{rcl}E\left[\mathrm{\Phi }(\tau \wedge t,x(\tau \wedge t))\right]& \le & {\mathrm{\Phi }}_0+E\left({\int }_0^{\tau \wedge t}(-\rho (\mathrm{\Phi }(s,x))+\mu (s))ds\right)\\ \le & {\mathrm{\Phi }}_0+E({\int }_0^{\tau \wedge t}\mu (s)ds)\\ \le & {\mathrm{\Phi }}_0+E({\int }_0^{+\mathrm{\infty }}\mu (s)ds)\\ \le & {\mathrm{\Phi }}_0+K.\end{array}$$

(15)

Let for a given $ϵ\in ]0,1[$, there exist $\delta =\delta (ϵ,c)\in ]0,c[$, $c>r$, such that for $x_0\in S_{\delta }$ and $a_1(c)>\frac{K}{ϵ}$, the following inequality holds

$$\frac{{\mathrm{\Phi }}_0}{ϵ}\le a_1(c)-\frac{K}{ϵ}.$$

(16)

We note that $|x(\tau \wedge t)|=|x(\tau )|=c$ if $\tau \le t$. So,

$$E\left[\mathrm{\Phi }(\tau \wedge t,x(\tau \wedge t))\right]\ge E\left(1_{(\tau \le t)}\mathrm{\Phi }(\tau ,x(\tau ))\right).$$

(17)

Then, by taking into account (5) and (17), we have

$$\begin{array}{rl}E\left[\mathrm{\Phi }(\tau \wedge t,x(\tau \wedge t))\right]& \ge E\left(1_{(\tau \le t)}{a}_1\left(|x(\tau )|\right)\right)\\ \ge E(1_{(\tau \le t)}{a}_1(c))=a_1(c)E(1_{(\tau \le t)})\\ =a_1(c)P(\tau \le t).\end{array}$$

(18)

Thus, from (15)-(16) and (18), we have

$$a_1(c)P(\tau \le t)\le {\mathrm{\Phi }}_0+K\le ϵa_1(c).$$

(19)

The latter inequality implies that

$$P(\tau \le t)\le ϵ.$$

Letting $t\to +\mathrm{\infty }$, we obtain

$$P(\tau \le +\mathrm{\infty })\le ϵ.$$

This implies that for given $ϵ,r\ge 0$ and $c>r$, there exists $\delta \in ]0,c[$ such that

$$P\{|x(t)|<c\}\ge 1-ϵ$$

(20)

for all $t\ge t_0\ge 0$ and $|x_0|<\delta$. Therefore, from (20) and Definition 2.2, $S_r$ is stable in probability with respect to closed-loop system (4). This completes the proof of Theorem 3.1. □

4 Application

In this section we illustrate our results by designing a numerical example.

Example 4.1 Consider the SNS

$$d\left(\begin{array}{c}{x}_1\\ x_2\end{array}\right)=\left(\begin{array}{c}{x}_1+x_2^3\\ 2\end{array}\right)dt+\left(\begin{array}{c}0\\ 1\end{array}\right)udt+\left(\begin{array}{c}{x}_1\\ \frac{1}{2}\end{array}\right)d{w}_t,$$

(21)

where ${(w_t)}_{t\ge 0}$ is a standard real-valued Wiener process, u is a real-valued measurable control law, $f_1(t,x_1,x_2)=x_1+x_2^3$, $f_2(t,x_1,x_2)=2$, $h_1(t,x_1,x_2)=0$, $h_2(t,x_1,x_2)=1$, $g_1(t,x_1,x_2)=x_1$, and $g_2(t,x_1,x_2)=\frac{1}{2}$.

Define the Lyapunov function in the form

$$\mathrm{\Phi }(t,x_1,x_2)=2x_1^{2}exp(2t)+{(x_2+x_{1}exp(t))}^4.$$

A simple calculation shows that

$$\frac{\partial \mathrm{\Phi }(t,x_1,x_2)}{\partial x_2}=0\iff x_2=-x_{1}exp(t).$$

(22)

From (22) we have $\mathrm{\Phi }(t,x_1,x_2)=2x_1^{2}exp(2t)$, and so $\frac{\partial \mathrm{\Phi }}{\partial t}=4x_1^{2}exp(2t)$, $\frac{\partial \mathrm{\Phi }}{\partial x_1}=4x_{1}exp(2t)$, $\frac{{\partial }^2\mathrm{\Phi }}{\partial x_1^2}=4exp(2t)$, and $\frac{\partial \mathrm{\Phi }}{\partial x_2}=\frac{{\partial }^2\mathrm{\Phi }}{\partial x_2^2}=\frac{{\partial }^2\mathrm{\Phi }}{\partial x_1\partial x_2}=\frac{{\partial }^2\mathrm{\Phi }}{\partial x_2\partial x_1}=0$.

Thus, we get

$${\mathbf{D}}_z\mathrm{\Phi }(t,x_1,x_2)=\sum _{i=1}^n{h}_z^i(t,x)\frac{\partial \mathrm{\Phi }(t,x)}{\partial x_i}=0,$$

and

$$\begin{array}{rcl}\mathbf{D}\mathrm{\Phi }(t,x_1,x_2)& =& \frac{\partial \mathrm{\Phi }(t,x)}{\partial t}+\sum _{i=1}^n{f}^i(t,x)\frac{\partial \mathrm{\Phi }(t,x)}{\partial x_i}\\ +\frac{1}{2}\sum _{i,j=1}^n\sum _{k=1}^m{g}_k^i{g}_k^j\frac{{\partial }^2\mathrm{\Phi }(t,x)}{\partial x_i\partial x_j}\\ =& 4x_1^{2}exp(2t)+(x_1+x_2^3)2x_{1}exp(2t)+4x_1^{2}exp(2t)\\ =& 6x_1^{2}exp(2t)+(x_1-x_1^{3}exp(3t))4x_{1}exp(2t)\\ =& 10x_1^{2}exp(2t)-4x_1^{4}exp(5t).\end{array}$$

Therefore,

$$\begin{array}{rcl}\mathbf{Y}\mathrm{\Phi }(t,x_1,x_2)& =& \mathbf{D}\mathrm{\Phi }(t,x)+\sum _{z=1}^p{\mathbf{D}}_z\mathrm{\Phi }(t,x)u_z\\ =& 10x_1^{2}exp(2t)-4x_1^{4}exp(5t)\\ =& 5\mathrm{\Phi }(t,x_1,x_2)-exp(t){\mathrm{\Phi }}^2(t,x_1,x_2)\\ \le & -\frac{1}{2}{\mathrm{\Phi }}^2(t,x_1,x_2)+5exp(-t).\end{array}$$

The latter inequality implies that both (5) and (6) are fulfilled with $\rho (s)=\frac{1}{2}{s}^2$, $\mu (t)=5exp(-t)$, $a_1(s)=\frac{1}{2}{s}^2$ and $a_2(s)=6s^2$, and thus, by Theorem 3.1, there exists a $C^{\mathrm{\infty }}$-feedback law $\varphi (t,x_1,x_2)$ with $\varphi (t,0,0)=0$ such that $S_r$ is stable in probability with respect to the resulting closed-loop system deduced from (21).

5 Conclusions

Many authors have paid attention to the subject of stability in probability of trivial solutions of stochastic systems in the past years. On the other hand, it is known that there are many types of stochastic systems all of whose solutions tend to each other although they do not have the trivial solution. In this paper, we have established the stability in probability of nontrivial solutions for SNS (3). We have derived a stochastic version of CLF and provided the necessary and sufficient conditions for stability in probability of a nontrivial solution for SNS (3).