1 Introduction

Over the last four decades, Nash differential games have been extensively investigated [30]. It has attracted much attention and has been widely applied to various fields, such as control theory (see [2, 7, 15, 19], and reference therein), management science and economics [9], ecology [34], etc. Recent advances in stochastic LQ control problems have allowed us to expand the study on the Nash games for stochastic systems with state- and control-dependent noise (see [5, 6, 17, 18]).

On the other hand, the systems with Markovian jump are frequently used to describe the evolution of some physical processes subject to abrupt variations of the parameters. Partially, this is due to the fact that often dynamic systems are inherently vulnerable to component failures or repairs, changing of subsystem interconnections, or abrupt variations of the nominal operating conditions. There exists a very rich list of references of articles and books dealing with control problems for this class of systems (see, e.g., [3, 11, 12, 22] and the references therein). Now, this kind of system has proven being useful in describing hybrid dynamics arising in electric power systems [21], communications systems [1], control of nuclear power plants [27], manufacturing systems [4, 14], and economic systems (see [8, 13, 20, 37, 38], etc).

Recently, Yong [35], Mou and Yong [24], McAsey and Mou [23], and Zhu and Zhang [39] investigated a special kind of stochastic differential games for Itô systems with state- and control-dependent noise. Stochastic differential games were recently studied by many researchers, such as Wang and Yu [31, 32], Yu [36], Hui and Xiao [16], and Xu and Zhang [33], with the backward stochastic differential equation approach and stochastic maximum principle to obtain the Nash strategies. In Song, Yin, and Zhang [29], numerical methods using Markov chain approximation techniques were developed for zero-sum stochastic differential games of regime-switching diffusions. In Pan and Basar [26], the existence of a stabilizing solution for a system of game-theoretic algebraic Riccati equations associated to a linear system with Markov jump perturbations was studied in connection with piecewise deterministic differential games; and in Dragan and Morozan [10], several properties of the stabilizing solution of a class of systems of Riccati-type differential equations with indefinite sign associated to controlled systems described by differential equations with Markovian jumps were discussed.

However, we note that the results above focused on stochastic systems with only state-dependent noise. However, in some practical models, not only the state but also the control input maybe corrupted by noise. For example, a practical model with the control input-dependent noise can be found in Qian and Gajic [28], which comes from the stochastic power control in CDMA systems. In addition, in the field of mathematical finance, an optimal portfolio selection problem is modeled by a stochastic Itô equation with state- and control-dependent noise, see Example 11.2.5 of Øksendal [25]. Therefore, stochastic Nash games for Markov jump linear systems with state- and control-dependent noise deserve further study. Inspired by this, we investigate the Nash games for a class of continuous-time Markov jump linear systems with state- and control-dependent noise, which are expressed by the Itô stochastic differential equations. The main contributions of this paper are as follows. First, finite time horizon stochastic Nash games are investigated by applying the results of indefinite stochastic LQ control problems with Markovian jumps. Then, we extend the results into infinite-time horizon case. Moreover, as an important application, stochastic H 2/H ∞ control for Markov jump linear systems with state- and control-dependent noise is discussed. Finally, in order to demonstrate the validity of the obtained results, a numerical example is provided.

The rest of the paper is organized as follows: Sect. 2 discusses stochastic Nash games in finite-time horizon; Sect. 3 extends the results of finite-time horizon stochastic Nash games into infinite-time horizon case; Sect. 4 provides the application to stochastic H 2/H ∞ control; and Sect. 5 concludes the paper with some remarks.

For convenience, we will make use of the following notations throughout this paper.

The notations used in this paper are fairly standard. \(A^{\prime }\) : transpose of a matrix \(A\). \(I_n\) : the \(n\times n\) identity matrix. \(\Vert \cdot \Vert \): the Euclidean norm of a matrix. \({\mathbf{E}}\{\cdot |r_t = i\}\) : the conditional expectation operator with respect to the event \(\{r_t = i\}\). \(\chi _A\) : indicator function of a set \(A\). \({\mathbb{R}}^n\) : the \(n\)-dimensional Euclidean space. \({\mathbb{R}}^{n\times m}\) : the set of all \(n\times m\) matrices; \({\mathbf{M}}_{n, m}^l \): space of all \(A = (A(1), A(2), \cdots , A(l))\) with \(A(i)\) being \(n\times m\) matrix, \(i = 1, 2, \cdots , l\). \({\mathbf{M}}_n^l : = {\mathbf{M}}_{n,n}^l \). \({\mathbf{S}}_n\) : space of all \(n\times n\) symmetric matrices. \({\mathbf{S}}_n^l\): space of all \(A = (A(1), A(2), \cdots , A(l))\) with \(A(i)\) being \(n\times n\) symmetric matrix, \(i = 1, 2, \cdots , l\).

2 Finite-Time Horizon Stochastic Nash Games

2.1 Problem Formulation

Throughout this paper, let \((\varOmega , {{\fancyscript{F}}}, \{{{\fancyscript{F}}}_t|t\geqslant 0\}, {{\fancyscript{P}}})\) be a given filtered probability space where exists a standard one-dimensional Wiener process \(\{W(t)|t\geqslant 0\}\) and a right continuous homogeneous Markov chain \(\{r_t|t\geqslant 0\}\) with state space \(\varXi =\{1, 2, \cdots , l\}\). In a similar assumption of the existing results, it is supposed that \({r_t}\) is independent of \({W(t)}\). Furthermore, it is also assumed that the Markov process \(r_t\) has the transition probabilities given by

$$\begin{aligned} \Pr [r_{t+\Delta }=j\left| {r_t}\right. =i]=\left\{ \begin{array}{l} \pi _{ij}\Delta +o(\Delta )\;,\,\,\quad \quad \;i \ne j, \\ 1+\pi _{ii}\Delta +o(\Delta )\;,\quad i = j, \\ \end{array} \right. \end{aligned}$$
(2.1)

where \(\pi _{ij}\geqslant 0, i\ne j, \pi _{ii}=-\sum \limits _{j=1,j\ne i}^{l}\pi _{ij}\). \({{\fancyscript{F}}}_t\) stands for the smallest \(\sigma \)-algebra generated by process \(W(s), r_s, 0\leqslant s\leqslant t\), i.e., \({{\fancyscript{F}}}_t=\sigma \{W(s), r_s|0\leqslant s\leqslant t\}\).

Consider the following linear stochastic differential equations subject to Markovian jumps defined by

$$\begin{aligned} \left\{ \begin{array}{l} {{\mathrm{d}}}x(t) = \left[ {A(r_t )x(t) + B_1 (r_t )u_1 (t) + B_2 (r_t )u_2 (t)} \right] {{\mathrm{d}}}t \\ \quad \quad \quad \;\;\; + \left[ {C(r_t )x(t) + D_1 (r_t )u_1 (t) + D_2 (r_t )u_2 (t)} \right] {{\mathrm{d}}}W(t), \\ x(s) = y \in {\mathbb{R}}^n , \\ \end{array} \right. \end{aligned}$$
(2.2)

where \(x(t)\in {\mathbb{R}}^n\) is the state variable, \(u_k(t)\in {\mathbb{R}}^{m_k}\) is control strategy taken by player \({{\mathrm{P}}}_k, k=1, 2\).

Given a fixed \((s, y)\in [0, T]\times {\mathbb{R}}^n\), let \(U_k[0, T]\), \(k=1, 2\), be the set of the \({\mathbb{R}}^{m_k}\)-valued, square integrable processes adapted with the \(\sigma \)-field generated by \(W(t), r_t\), respectively. In the present paper, we suppose \(s<T\) to guarantee \([s, T]\) is an interval. Associated with each \((u_1, u_2)\in U[0, T] \equiv U_1[0, T]\times U_2[0, T]\), the cost performance \(J_k(u_1, u_2; s, y, i)\) of player \({{\mathrm{P}}}_k\) is defined by

$$ J_{k} (u_{1} ,u_{2} ;s,y,i) = {\mathbf{E}} \left\{ \int_{s}^{T} {\left[ {\begin{array}{lll} {x^{\prime } (t)} & {u_{1}^{\prime } (t)} & {u_{2}^{\prime } (t)} \\ \end{array} } \right]M_{k} (r_{t} )\left[ {\begin{array}{l} {x(t)} \\ {u_{1} (t)} \\ {u_{2} (t)} \\ \end{array} } \right]{\text{d}}t + x^{\prime } (T)H_{k} (r_{T} )x(T)\left| {r_{s} } \right. = i} \right\},\\ $$
$$ M_{k} (r_{t} ) = \left[ {\begin{array}{lll} {Q_{k} (r_{t} )} & {L_{{k1}} (r_{t} )} & {L_{{k2}} (r_{t} )} \\ {L_{{k1}}^{\prime } (r_{t} )} & {R_{{k1}} (r_{t} )} & 0 \\ {L_{{k2}}^{\prime } (r_{t} )} & 0 & {R_{{k2}} (r_{t} )} \\ \end{array} } \right],\;k = 1,\;2.$$
(2.3)

In (2.2) and (2.3), \(A(r_t)=A(i), B_k(r_t)=B_k(i), C(r_t)=C(i), D_k(r_t)=D_k(i)\) and \(M_k(r_t)=M_k(i)\) whenever \(r_t=i, i\in \varXi \). Moreover, whenever \(r_T=i, H_k(r_T)=H_k(i), k=1, 2\). Here the matrices mentioned above are given real matrices of suitable sizes. Referring to Li and Zhou [17], the value function \(V_k(s, y, i)\) is defined as

$$\begin{aligned} V_k (s, y, i) = {\mathop {\inf }\limits _{u_k \in U_k }J_k (u_k , u_\tau ^* ; s, y, i), \quad k, \tau = 1, 2, k\ne \tau , i\in \varXi }, \end{aligned}$$
(2.4)

where \( u_\tau ^*\) is the optimal control strategy of player \({{\mathrm{P}}}_\tau , \tau =1, 2\).

Since the symmetric matrices

$$\begin{aligned} M_k (i) = \left[ {\begin{array}{*{20}c} {Q_k(i)} &{} {L_{k1}(i)} &{} {L_{k2}(i)} \\ {L^{\prime }_{k1}(i)} &{} {R_{k1}(i)} &{} 0 \\ {L^{\prime }_{k2}(i)} &{} 0 &{} {R_{k2}(i)} \\ \end{array}} \right] \end{aligned}$$

are allowed to be indefinite, the above optimization problem is referred to as indefinite stochastic Nash games.

Definition 2.1

The stochastic Nash equilibrium strategy pair \((u_1^*, u_2^*)\in U[0, T]\) is defined as satisfying the following conditions:

$$\begin{aligned} J_1(u_1^*, u_2^*; s, y, i)&\leqslant J_1(u_1, u_2^*; s, y, i), \quad \forall\,u_1\in U_1, \end{aligned}$$
(2.5a)
$$\begin{aligned} J_2(u_1^*, u_2^*; s, y, i)&\leqslant J_2(u_1^*, u_2; s, y, i), \quad \forall u_2\in U_2, i\in \varXi . \end{aligned}$$
(2.5b)

Definition 2.2

The indefinite stochastic Nash games (2.2)–(2.5a,b) are well posed if

$$\begin{aligned} -\infty < V_k(s, y, i) < +\infty , \quad \forall (s, y)\in [0, T]\times {\mathbb{R}}^n, k = 1, 2, i\in \varXi . \end{aligned}$$

An admissible triple \((x^*, u_1^*, u_2^*)\) is called optimal with respect to (w.r.t.) the initial condition \((s, y, i)\) if \(u_1^*\) achieves the infimum of \(J_1(u_1, u_2^*; s, y, i)\) and \(u_2^*\) achieves the infimum of \(J_2(u_1^*, u_2; s, y, i)\).

For the indefinite stochastic Nash games (2.2)–(2.5a,b), we restrict \(u_k(t)\) to be composed of linear feedback strategies of the form: \(u_k(t)={{\mathrm{K}}}_k(r_t)x(t), k=1, 2\), and \({{\mathrm{K}}}_k \in {\mathbf{M}}_{m_k, n}^l \) are matrix-valued functions.

In the next section, we discuss the one-player case, i.e., indefinite stochastic LQ control problems [5, 6].

2.2 One-Player Case

First, one-player case is discussed. The result obtained for that particular case is used as the basis for the derivation of results for 2-player case.

Consider the linear stochastic controlled system with Markovian jumps defined by

$$\begin{aligned} \left\{ \begin{array}{l} {{\mathrm{d}}}x(t) = \left[ {A(r_t)x(t) + B_1(r_t)u_1(t)}\right] {{\mathrm{d}}}t + \left[ {C(r_t )x(t) + D_1 (r_t)u_1(t)} \right] {{\mathrm{d}}}W(t), \\ x(s) = y, \end{array} \right. \end{aligned}$$
(2.6)

where \((s, y)\in [0, T]\times {\mathbb{R}}^n\) are the initial time and initial state, respectively.

For each \((s, y)\) and \(u_1\in U[0, T]\), the associated cost is

$$\begin{aligned} \begin{array}{l} J(u_1 ;s,y,i) = {\mathbf{E}}\left\{ \displaystyle \int\nolimits_s^T {\left[ {\begin{array}{ll} {x(t)} \\ {u_1 (t)} \\ \end{array}} \right] ^\prime \left[ {\begin{array}{*{20}c} {Q_1 (r_t )} &{} {L_1 (r_t )} \\ {L^{\prime }_1 (r_t )} &{} {R_{11} (r_t )} \\ \end{array}} \right] \left[ {\begin{array}{*{20}c} {x(t)} \\ {u_1 (t)} \\ \end{array}} \right] {{\mathrm{d}}}t} \right. \\ \left. {\quad \quad \quad {} \quad {} \quad \quad \quad \quad + x^{\prime }(T)H_1 (r_t )x(T)\left| {r_s = i} \right. } \right\} ,\\ \end{array} \end{aligned}$$
(2.7)

where \(Q_1(r_t)=Q_1(i), R_{11}(r_t)=R_{11}(i)\) and \(L_1(r_t)=L_1(i)\) when \(r_t=i\), and \(H_1(r_T)=H_1(i)\) whenever \(r_T=i\), whereas \(Q_1\), etc., \(i\in \varXi \), are given matrices with suitable sizes. The objective of the optimal control problem is to minimize the cost function \(J(u_1; s, y, i)\), for a given \((s, y)\in [0, T]\times {\mathbb{R}}^n\), over all \(u_1\in U[0, T]\). The value function is defined as

$$\begin{aligned} V(s,y,i) = {\mathop {\inf }\limits _{u_1 \in U_1 } J(u_1; s, y, i)}. \end{aligned}$$
(2.8)

Note that as the symmetric matrices

$$\begin{aligned} \left[ {\begin{array}{*{20}c} {Q_1 (i)} &{} {L_1 (i)} \\ {L^{\prime }_1 (i)} &{} {R_{11} (i)} \\ \end{array}} \right] , \quad i\in \varXi \end{aligned}$$

are allowed to be indefinite; and we call the above optimization problem as an indefinite LQ problem with Markovian jumps [17, 18].

Now we introduce a type of coupled Riccati differential equations associated with the LQ problems (2.6)–(2.8) and some useful lemmas that are important in our subsequent analysis.

Definition 2.3

The following system of constrained differential equations (with the time argument \(t\) suppressed)

$$\begin{aligned} \left\{\!\!\! \! \begin{array}{llll} &{\dot{P}}(i) + P(i)A(i) + A^{\prime }(i)P(i) + C^{\prime }(i)P(i)C(i) + Q_1(i) + \sum \limits _{j=1}^l {\pi _{ij}P(j)} \\ &\quad- \left( {P(i)B_1(i) + C^{\prime }(i)P(i)D_1(i) + L_1(i)}\right) \left( {R_{11}(i) + D^{\prime }_1(i)P(i)D_1(i)} \right) ^{-1} \\& \quad\times \left( {B^{\prime }_1(i)P(i) + D^{\prime }_1(i)P(i)C(i) + L^{\prime }_1(i)}\right) = 0, \\& P(T,i) = H_1(i), \\& R_{11}(i) + D^{\prime }_1(i)P(i)D_1(i) > 0\;,\;i \in \varXi \\ \end{array} \right. \end{aligned}$$
(2.9)

is called a system of coupled stochastic Riccati differential equations (CSRDEs).

Lemma 2.4

(generalized Itô’s formula) [3]: Let \(b(t, x, i)\) and \(\sigma (t, x, i)\) be given \({\mathbb{R}}^n\) -valued, \({{{\fancyscript{F}}}}_t\) -adapted process, \(i = 1, 2, \cdots , l\) , and \(x(t)\) satisfy

$$\begin{aligned} {{\mathrm{d}}}x(t)=b(t, x(t), r_t){{\mathrm{d}}}t+\sigma (t, x(t), r_t){{\mathrm{d}}}W(t). \end{aligned}$$

Then for given \(\varphi (\cdot , \cdot , i)\in C^2([0, \infty )\times {\mathbb{R}}^n), i = 1, 2, \cdots , l\) , we have

$$\begin{aligned}&{\mathbf{E}}\left\{ {\varphi \left( {T,x(T),r_T} \right) - \varphi \left( {s,x(s),r_s}\right) \left| {r_s=i} \right. } \right\} = \\& \quad {\mathbf{E}}\left\{ \int _s^T {\left[ {\varphi _t \left( {t,x(t),r_t}\right) + \nabla \varphi \left( {t,x(t),r_t}\right) } \right] {{\mathrm{d}}}t\left| {r_s=i}\right. }\right\} , \end{aligned}$$
(2.10)

where

$$\begin{aligned} \nabla \varphi \left( {t,x,i}\right)&= b^{\prime }(t,x,i)\varphi _x (t,x,i) + \frac{1}{2}\left[ {\sigma ^{\prime }(t,x,i)\varphi _{xx}(t,x,i)\sigma (t,x,i)} \right] \\&\quad +\sum \limits _{j = 1}^l {\pi _{ij}\varphi (t,x,j)}. \end{aligned}$$

The following lemma presents the existence condition for an optimal feedback control.

Lemma 2.5

Suppose CSRDEs (2.9) admit a solution \(P:[0, T]\rightarrow {\mathbf{S}}_n^l\) , with \(P=(P(1), P(2), \cdots , P(l))\) , then the LQ problems (2.6)–(2.8) are well posed w.r.t. any initial \((s, y)\in [0, T]\times {\mathbb{R}}^n\) . Moreover, there exists an optimal control that can be represented by the state feedback form:

$$\begin{aligned} u_1^*(t) = \sum \limits _{i = 1}^l {{{\mathrm{K}}}_1(i)(t)x(t)\chi _{r_t= i}}\;,\;i \in \varXi , \end{aligned}$$
(2.11)

where

$$\begin{aligned} {{\mathrm{K}}}_1 (i) = - \left( {R_{11} (i) + D^{\prime }_1 (i)P(i)D_1 (i)} \right) ^{ - 1} \left( {B^{\prime }_1 (i)P(i) + D^{\prime }_1 (i)P(i)C(i) + L^{\prime }_1 (i)} \right) \; \end{aligned}$$

are matrix-value functions with suitable sizes. Furthermore, the following value function

$$\begin{aligned} V(s,y,i) \equiv {\mathop {\inf }\limits _{u_1 \in U_1}J(u_1;s,y,i) = y^{\prime }P(s, i)y,\;i \in \varXi} \end{aligned}$$

is uniquely determined by \(P=(P(1), P(2), \cdots , P(l))\in {\mathbf{S}}_n^l.\)

Proof

Let \(P=(P(1), P(2), \cdots , P(l))\in {\mathbf{S}}_n^l\) be a solution of the CSRDEs (2.9). Setting \(\varphi (t, x, i)=x^{\prime }P(t, i)x\) and applying the generalized Itô’s formula (Lemma 2.4) to the linear system (2.6), we have

$$\begin{aligned}&{\mathbf{E}}\left[ {x^{\prime }(T)P(r_T)x(T) - y^{\prime }P(r_s)y\left| {r_s=i} \right. } \right] \\&\quad = {\mathbf{E}}\left[ {\varphi \left( {T,x(T),r_T} \right) - \varphi \left( {s,x(s),r_s}\right) \left| {r_s=i} \right. } \right] \\&\quad = {\mathbf{E}}\left\{ \int _s^T{\nabla \varphi \left( {t,x(t),t}\right) {{\mathrm{d}}}t\left| {r_s=i} \right. } \right\} , \end{aligned}$$
(2.12)

where

$$\begin{aligned} \nabla \varphi \left( {t,x,i}\right)&= \varphi _t(t,x,i) + b(t,x,u,i)^{\prime }\varphi _x(t,x,i) \\&\quad +\frac{1}{2}\left[ {\sigma ^{\prime }(t,x,u,i)\varphi _{xx} (t,x,i)\sigma (t,x,u,i)} \right] + \sum \limits _{j = 1}^l {\pi _{ij}\varphi (t,x,j)} \\&= x^{\prime }\left[{\dot{P}}(i) + P(i)A(i) + A^{\prime }(i)P(i) + C^{\prime }(i)P(i)C(i) + \sum \limits _{j = 1}^l {\pi _{ij}P(j)} \right]x \\&\quad +2u^{\prime }_1[B^{\prime }_1(i)P(i)+D^{\prime }_1(i)P(i)C(i)]x+ u^{\prime }_1D^{\prime }_1(i)P(i)D_1(i)u_1 . \\ \end{aligned}$$

Substituting (2.12) back into (2.7), we get

$$\begin{aligned} J\left( {u_1 ;s,y,i} \right)&= y^{\prime }P(s, i)y + {\mathbf{E}}\left\{ {\int _s^T {\left[ {u_1 - {{\mathrm{K}}}_1 (r_t )x} \right] ^\prime \left[ {D^{\prime }_1 (r_t )P(r_t )D_1 (r_t )+\,R_{11} (r_t )} \right. } } \right. \\&\quad \left. {\times\left[ {u_1 - {{\mathrm{K}}}_1 (r_t )x} \right] {{\mathrm{d}}}t\left| {r_s = i} \right. } \right\} . \end{aligned}$$
(2.13)

From the definition of the CSRDEs, we have

$$\begin{aligned}&\nabla \varphi \left( {t,x,i} \right) + x^{\prime }Q_1 (i)x + 2u^{\prime }_1 L^{\prime }_1 (i)x + u^{\prime }_1 R_{11} (i)u_1 \\&\quad= x^{\prime } \Bigg[ {\dot{P}}(i) + P(i)A(i) + A^{\prime }(i)P(i) + C^{\prime }(i)P(i)C(i) + Q_1 (i) \\&\qquad + \sum \limits _{j = 1}^l {\pi _{ij} P(j)} \Bigg] x + 2u^{\prime }_1 [B^{\prime }_1 (i)P(i) + D^{\prime }_1 (i)P(i)C(i) + L_1 (i)]x \\ & \qquad + u^{\prime }_1 [R_{11} (i) + D^{\prime }_1 (i)P(i)D_1 (i)]u_1 \\ &\quad= x^{\prime }\{ [P(i)B_1 (i) + C^{\prime }(r_t )P(i)D_1 (i) + L_1 (i)][R_{11} (i) + D^{\prime }_1 (i)P(i)D_1 (i)]^{ - 1} \\ & \qquad \times \left( {B^{\prime }_1 (i)P(i) + D^{\prime }_1 (i)P(i)C(i) + L^{\prime }_1 (i)} \right) x \\ & \qquad + 2u^{\prime }_1 [B^{\prime }_1 (i)P(i) + D^{\prime }_1 (i)P(i)C(i) + L^{\prime }_1 (i)]x \\ & \qquad + u^{\prime }_1 [R_{11} (i) + D^{\prime }_1 (i)P(i)D_1 (i)]u_1 . \end{aligned}$$
(2.14)

Applying the square completion technique to (2.14), we have

$$\begin{aligned}&\nabla \varphi \left( {t,x,i} \right) + x^{\prime }Q_1 (i)x + 2u^{\prime }_1 L^{\prime }_1 (i)x + u^{\prime }_1 R_{11} (i)u_1 \\ & \quad = \left[ {u_1 - {{\mathrm{K}}}_1 (i)x} \right] ^\prime \left[ {R_{11} (i) + D^{\prime }_1 (i)P(i)D_1 (i)} \right] \left[ {u_1 - {{\mathrm{K}}}_1 (i)x} \right] . \end{aligned}$$
(2.15)

Then the equation (2.13) can be expressed as

$$ J\left( {u_{1} ;s,y,i} \right) = y^{\prime } P(s,i)y + {\mathbf{E}}\left\{ \int_{s}^{T} {\left[ {u_{1} - {\text{K}}_{1} (r_{t} )x} \right]^{\prime } \left[ {D_{1}^{\prime } (r_{t} )P(r_{t} )D_{1} (r_{t} )} \right.} \left. { + {\mkern 1mu} R_{{11}} (r_{t} )} \right]\left[ {u_{1} - {\text{K}}_{1} (r_{t} )x} \right]{\text{d}}t\left| {r_{s} = i} \right. \right\}. $$
(2.16)

From (2.16) we can see that \(J(u_1; s, y, i)\) is minimized by the control given by (2.11) with the optimal value \(y^{\prime }P(s, i)y\). This completes the proof.

2.3 Stochastic Nash Equilibrium Strategies

The solution of the stochastic Nash games is given below.

Theorem 2.6

Suppose there exist \(P=(P_1, P_2): [0, T]\rightarrow {\mathbf{S}}_n^l \times {\mathbf{S}}_n^l \) , with \(P_1=(P_1(1), \cdots , P_1(l)), P_2=(P_2(1), \cdots , P_2(l))\) that satisfy the following CSRDEs \((i, j\in \varXi )\).

$$ \left\{ \begin{array}{llll} {\dot{P}}_1(i) + P_1(i){{\bar{A}}}(i) + {\bar{A}}^{\prime }(i)P_1(i) + {\bar{C}}^{\prime }(i)P_1(i){\bar{C}}(i) + {\bar{Q}}_1(i) + \sum \limits _{j = 1}^l {\pi _{ij}P_1(j)} \\ \, \, -\left( {P_1(i)B_1(i) + {\bar{C}}^{\prime }(i)P_1(i)D_1(i) + L_{11}(i)}\right) \left( {R_{11}(i) + D^{\prime }_1(i)P_1(i)D_1(i)} \right) ^{ - 1} \\ \, \, \times \left( {B^{\prime }_1(i)P_1(i) + D^{\prime }_1(i)P_1(i){\bar{C}}(i) + L^{\prime }_{11}(i)} \right) = 0, \\ P_1(T,i) = H_1(i), \\ R_{11}(i) + D^{\prime }_1(i)P_1(i)D_1(i) > 0,\;\;i \in \varXi ,\; \\ \end{array} \right. $$
(2.17)
$$ \left\{ \begin{array}{l} {\dot{P}}_2(j)+P_2(j)\tilde{A}(j) + \tilde{A}^{\prime }(j)P_2(j) + \tilde{C}^{\prime }(j)P_2(j)\tilde{C}(j) + \tilde{Q}_2(j) + \sum \limits _{k=1}^l{\pi _{jk}P_2(k)} \\ \, \, - \left( {P_2(j)B_2(j) + \tilde{C}^{\prime }(j)P_2(j)D_2(j) + L_{22}(j)}\right) \left( {R_{22}(j) + D^{\prime }_2(j)P_2(j)D_2(j)} \right) ^{-1} \\ \, \, \times \left( {B^{\prime }_2(j)P_2(j) + D^{\prime }_2(j)P_2(j)\tilde{C}(j) + L^{\prime }_{22}(j)} \right) = 0, \\ P_2(T,j)=H_2(j), \\ R_{22}(j) + D^{\prime }_2(j)P_2(j)D_2(j) > 0,\;\;j \in \varXi ,\; \\ \end{array} \right. $$
(2.18)

where

$$ {{\mathrm{K}}}_1=-\left( {R_{11}(i) + D^{\prime }_1(i)P_1(i)D_1(i)} \right) ^{-1}\left( {B^{\prime }_1(i)P_1(i) + D^{\prime }_1(i)P_1(i){\bar{C}}(i) + L^{\prime }_{11}(i)} \right), $$
$$ {{\mathrm{K}}}_2=-\left( {R_{22}(j) + D^{\prime }_2(j)P_2(j)D_2(j)}\right) ^{-1}\left( {B^{\prime }_2(j)P_2(j) + D^{\prime }_2(j)P_2(j)\tilde{C}(j) + L^{\prime }_{22}(j)} \right), $$
$$\begin{aligned} {\bar{A}}&= A+B_2{{\mathrm{K}}}_2, {\bar{C}}=C+D_2{{\mathrm{K}}}_2, {\bar{Q}}_1=Q_1+L_{12}{{\mathrm{K}}}_2+{{\mathrm{K}}^{\prime }}_2L^{\prime }_{12}+{{\mathrm{K}}^{\prime }}_2R_{12}{{\mathrm{K}}}_2,\\ \tilde{A}&= A+B_1{{\mathrm{K}}}_1, \tilde{C}=C+D_1{{\mathrm{K}}}_1, \tilde{Q}_2=Q_2+L_{21}{{\mathrm{K}}}_1 + {{\mathrm{K}}^{\prime }}_1L^{\prime }_{21}+{{\mathrm{K}}^{\prime }}_1R_{21}{{\mathrm{K}}}_1. \end{aligned}$$

Denote \(F_1^*(i)= {{\mathrm{K}}}_1(i), F_2^*(i)= {{\mathrm{K}}}_2(i)\) , then the stochastic Nash equilibrium strategy \((u_1^*, u_2^*)\) can be represented by

$$\begin{aligned} \left\{ \begin{array}{l} u_1^* (t) = \sum \limits _{i = 1}^l {F_1^* (i)(t)x(t)\chi _{r_t=i}}, \\ u_2^* (t) = \sum \limits _{i = 1}^l {F_2^* (i)(t)x(t)\chi _{r_t=i}}. \\ \end{array} \right. \end{aligned}$$
(2.19)

Furthermore, the indefinite stochastic Nash games (2.2)–(2.5a,b) is well posed (w.r.t. \((s, y)\in [0, T]\times {\mathbb{R}}^n\)), and the optimal value is determined by

$$\begin{aligned} V_k (s,y,i)={\mathop {\inf }\limits _{u_k \in U_k }J_k(u_k,u_\tau ^* ;s,y,i)=y^{\prime }P_k (s, i)y, \quad k,\,\tau =1, 2, k\ne \tau , i\in \varXi} . \end{aligned}$$

Proof

These results can be proved by using the concept of Nash equilibrium described in definition 2.1 as follows. Given \(u_2^*=F_2^*(r_t)x(t)\) is the optimal control strategy implemented by player \({{\mathrm{P}}}_2\), player \({{\mathrm{P}}}_1\) facing the following optimization problem in which the cost function (2.20) is minimal at \(u_1^*=F_1^*(r_t)x(t)\).

$$\begin{aligned} \begin{array}{lll} &\mathop {\min }\limits _{F_1(r_t) \in U_1}{\mathbf{E}}\left\{ \displaystyle \int\nolimits_s^T {\left[ \begin{array}{l} x(t) \\ u_1(t) \\ \end{array}\right] ^\prime \left[ {\begin{array}{ll} {{\bar{Q}}_1(r_t)} &{} {L_{11}(r_t)} \\ {L^{\prime }_{11}(r_t)} &{} {R_{11}(r_t)} \\ \end{array}} \right] \left[ \begin{array}{l} x(t) \\ u_1(t) \\ \end{array} \right] {{\mathrm{d}}}t + x^{\prime }(T)H_1(r_t)x(T)\left| {r_s=i} \right. } \right\} , \\& s.t. \\& \left\{ \begin{array}{l} {{\mathrm{d}}}x(t)=\left[ {{\bar{A}}(r_t)x(t) + B_1(r_t)u_1(t)} \right] {{\mathrm{d}}}t + \left[ {{\bar{C}}(r_t)x(t) + D_1(r_t)u_1(t)} \right] {{\mathrm{d}}}W(t), \\ x(s) = y \in {\mathbb{R}}^n , \\ \end{array} \right. \\ \end{array} \end{aligned}$$
(2.20)

here \({\bar{Q}}_1= Q_1+(F_2^* )^{\prime }L^{\prime }_{12}+L_{12}F_2^* +(F_2^*)^{\prime }R_{12} F_2^* , {\bar{A}}=A+B_2F_2^* \), \({\bar{C}}=C+D_2F_2^* \).

Note that the above optimization problem defined in (2.20) is a standard indefinite stochastic LQ problem. Applying lemma 2.5 to this optimization problem as

$$\begin{aligned} \left[ {\begin{array}{*{20}c} {{\bar{Q}}_1(r_t )} &{} {L_{11}(r_t )} \\ {L^{\prime }_{11}(r_t )} &{} {R_{11}(r_t )} \\ \end{array}} \right] \Rightarrow \left[ {\begin{array}{*{20}c} {Q_1} &{} {L_1} \\ {L^{\prime }_1} &{} {R_{11}} \\ \end{array}} \right] , \quad {\bar{A}} \Rightarrow A, {\bar{C}} \Rightarrow C. \end{aligned}$$

We can easily get the optimal control

$$\begin{aligned} u_1^*(t) = F_1^*(r_t)x(t). \end{aligned}$$
(2.21)

and the optimal value function

$$\begin{aligned} V_1(s,y,i) =y^{\prime }P_1(s, i)y, i\in \varXi . \end{aligned}$$
(2.22)

Similarly, we can prove that \(u_2^*=F_2^*(r_t)x(t)\) is the optimal control strategy of player \({{\mathrm{P}}}_2\).

This completes the proof of Theorem 2.6.

3 Infinite-Time Horizon Stochastic Nash Games

3.1 Problem Formulation

In this section, we investigate the infinite-time horizon stochastic Nash games for linear Markovian jump systems with state- and control-dependent noise. In particular, infinite-time horizon stochastic Nash games for linear Markovian jump systems with state-dependent noise was considered in Zhu et al. [40].

Consider the games described by the following linear stochastic differential equation with Markovian parameter jumps

$$ \left\{ \begin{array}{ll} {{\mathrm{d}}}x(t) = \left[ {A(r_t)x(t) + B(r_t)u(t)}\right] {{\mathrm{d}}}t + \left[ {C(r_t)x(t) + D(r_t)u(t)} \right] {{\mathrm{d}}}W(t), \\ x(0) = x_0 \in {\mathbb{R}}^n , \\ \end{array} \right. $$
(3.1)

with the cost performances

$$\begin{aligned} J_k (u;x_0,i)&= \mathbb{E}\left\{ \int _0^\infty {\left[ {\begin{array}{ll} {x^{\prime }(t)} &{} {u^{\prime }(t)} \\ \end{array}} \right] M_k(r_t)\left[ {\begin{array}{l} {x(t)} \\ {u(t)} \\ \end{array}} \right] {{\mathrm{d}}}t\left| {r_0=i} \right. } \right\} ,\;k = 1, 2, \\ B&= (B_1, B_2), D=(D_1, D_2), M_k(r_t)=\left[ {\begin{array}{lll} {Q_k(r_t)} &{} {L_{k1} (r_t)} &{} {L_{k2}(r_t)} \\ {L^{\prime }_{k1}(r_t)} &{} {R_{k1}(r_t)} &{} 0 \\ {L^{\prime }_{k2}(r_t } &{} 0 &{} {R_{k2}(r_t)} \\ \end{array}} \right] , \end{aligned}$$
(3.2)

where \((x_0, i)\in {\mathbb{R}}^n \times \varXi \) is the initial state, \(x(t)\) and \(u(t)=(u_1(t), u_2(t))^{\prime }\) have similar meanings described in Sect. 2.

Referring to Li et al. [18], for each initial value \(x(0)=x_0\), the value function \(V_k(x_0, i)\) is defined as

$$\begin{aligned} V_k(x_0,i)={\mathop {\inf }\limits _{u_k\in U_k}J_k(u_k, u_\tau ^*;x_0,i)}, \end{aligned}$$
(3.3)

where \( u_\tau ^*\) is the optimal control strategy of player \({{\mathrm{P}}}_\tau , \tau =1, 2\).

We emphasize again that we are dealing with an indefinite stochastic Nash game, namely, the symmetric matrix

$$\begin{aligned} M_k (i)=\left[ {\begin{array}{*{20}c} {Q_k(i)} &{} {L_{k1}(i)} &{} {L_{k2}(i)} \\ {L^{\prime }_{k1}(i)} &{} {R_{k1}(i)} &{} 0 \\ {L^{\prime }_{k2}(i)} &{} 0 &{} {R_{k2}(i)} \\ \end{array}} \right] , \quad k=1, 2, i\in \varXi \end{aligned}$$

is possibly indefinite.

Definition 3.1

The stochastic Nash equilibrium strategy pair \((u_1^*, u_2^*)\in U[0, \infty )\) is defined as satisfying the following conditions.

$$\begin{aligned} J_1(u_1^*, u_2^*; x_0, i)&\leqslant J_1(u_1, u_2^*; x_0, i), \quad \forall u_1\in U_1,\end{aligned}$$
(3.4a)
$$\begin{aligned} J_2(u_1^*, u_2^*; x_0, i)&\leqslant J_2(u_1^*, u_2; x_0, i), \quad \forall u_2\in U_2, i\in \varXi , \end{aligned}$$
(3.4b)

where \(U[0, \infty ) = U_1[0, \infty )\times U_2[0, \infty ), U_1[0, \infty )\) and \(U_2[0, \infty )\) denote the space of all admissible strategies for player \({{\mathrm{P}}}_k, k=1, 2\) (see reference [2]).

Definition 3.2

The generalized stochastic Nash games (3.1)–(3.4a,b) are well posed if

$$\begin{aligned} -\infty < V_k(x_0, i) < +\infty , \quad \forall x_0\in {\mathbb{R}}^n, i\in \varXi , k = 1, 2. \end{aligned}$$

A well-posed problem is attainable (w.r.t. \((x_0, i)\)) if there is a control \(u_k^*(\cdot )\) achieves \(V_k(x_0, i)\). In this case the control \(u_k^*(\cdot )\) is optimal (w.r.t. \((x_0, i)\)).

3.2 Main Results

The definition of stochastic stabilizability, which was an essential assumption in the section introduced by Li et al. [18], Dragan and Morozan [11], Dragan et al. [12].

Definition 3.3

Consider the following linear stochastically controlled system with Markovian jumps

$$\begin{aligned} {{\mathrm{d}}}x(t) = \left[ {A(r_t) + B(r_t){{\mathrm{K}}}(r_t)}\right] x(t){{\mathrm{d}}}t + \left[ {C(r_t) + D(r_t){{\mathrm{K}}}(r_t)}\right] x(t){{\mathrm{d}}}W(t), \end{aligned}$$
(3.5)

which is asymptotically mean-square stable, i.e.,

$$\begin{aligned} {\mathop {\lim }\limits _{t \rightarrow \infty }{\mathbf{E}}\{{\left\| {x(t)}\right\| ^2\left| {r_0=i}\right. }\}=0}. \end{aligned}$$

Similar to the finite-time horizon stochastic Nash games discussed in Sect. 2, we can get the corresponding results of the infinite-time horizon stochastic Nash games stated as Theorem 3.4, which can be verified by following the line of Theorem 2.6.

Theorem 3.4

Assume there exist \(u_k(t), k = 1,2\) , the closed-loop system is asymptotically mean square stable. Suppose there exists a stabilizing solution \(P=(P_1, P_2): \rightarrow {\mathbf{S}}_n^l \times {\mathbf{S}}_n^l, P_1=(P_1(1), \cdots , P_1(l)), P_2=(P_2(1), \cdots , P_2(l))\) of the following CSRAEs \((i, j\in \varXi )\).

$$\begin{aligned}&\left\{ \begin{array}{llll} P_1(i){\bar{A}}(i) + {\bar{A}}^{\prime }(i)P_1(i) + {\bar{C}}^{\prime }(i)P_1(i){\bar{C}}(i) + {\bar{Q}}_1(i) + \sum \limits _{j=1}^l {\pi _{ij}P_1(j)} \\ -\left( {P_1(i)B_1(i) + {\bar{C}}^{\prime }(i)P_1(i)D_1(i) + L_{11}(i)}\right) \left( {R_{11}(i) + D^{\prime }_1(i)P_1(i)D_1(i)} \right) ^{-1} \\ \times \left( {B^{\prime }_1(i)P_1(i) + D^{\prime }_1(i)P_1(i){\bar{C}}(i) + L^{\prime }_{11}(i)} \right) = 0, \\ R_{11}(i) + D^{\prime }_1(i)P_1(i)D_1(i) > 0,\;\;i \in \varXi, \; \\ \end{array} \right. \end{aligned}$$
(3.6)
$$\begin{aligned} &\left\{ \begin{array}{llll} P_2(j)\tilde{A}(j) + \tilde{A}^{\prime }(j)P_2(j) + \tilde{C}^{\prime }(j)P_2(j)\tilde{C}(j) + \tilde{Q}_2(j) + \sum \limits _{k= 1}^l {\pi _{jk}P_2(k)} \\ \quad -\left( {P_2(j)B_2(j)+\tilde{C}^{\prime }(j)P_2(j)D_2(j) + L_{22}(j)} \right) \left( {R_{22}(j) + D^{\prime }_2(j)P_2(j)D_2(j)} \right) ^{-1} \\ \quad \times \left( {B^{\prime }_2(j)P_2(j) + D^{\prime }_2(j)P_2(j)\tilde{C}(j) + L^{\prime }_{22}(j)} \right) = 0, \\ R_{22}(j) + D^{\prime }_2(j)P_2(j)D_2(j) > 0,\;j \in \varXi,\; \\ \end{array} \right.\end{aligned}$$
(3.7)

where

$${{\mathrm{K}}}_1 = - \left( {R_{11}(i) + D^{\prime }_1(i)P_1(i)D_1(i)}\right) ^{-1} \left( {B^{\prime }_1(i)P_1(i) + D^{\prime }_1(i)P_1(i){\bar{C}}(i) + L^{\prime }_{11}(i)} \right),$$
$${{\mathrm{K}}}_2 = - \left( {R_{22}(j) + D^{\prime }_2(j)P_2(j)D_2(j)} \right) ^{-1} \left( {B^{\prime }_2(j)P_2(j) + D^{\prime }_2(j)P_2(j)\tilde{C}(j) + L^{\prime }_{22}(j)} \right),$$
$$\begin{aligned} {\bar{A}}&= A+B_2{{\mathrm{K}}}_2, {\bar{C}}=C+D_2{{\mathrm{K}}}_2, {\bar{Q}}_1=Q_1+L_{12}{{\mathrm{K}}}_2+{{\mathrm{K}}^{\prime }}_2L^{\prime }_{12}+{{\mathrm{K}}^{\prime }}_2R_{12}{{\mathrm{K}}}_2,\\ \tilde{A}&= A+B_1{{\mathrm{K}}}_1, \tilde{C}=C+D_1{{\mathrm{K}}}_1, \tilde{Q}_2=Q_2+L_{21}{{\mathrm{K}}}_1 + {{\mathrm{K}}^{\prime }}_1L^{\prime }_{21}+{{\mathrm{K}}^{\prime }}_1R_{21}{{\mathrm{K}}}_1. \end{aligned}$$

Recall that \((P_1, P_2)\) is a stabilizing solution of CSRAEs (3.6)–(3.7) if the following closed-loop system

$$\begin{aligned} {{\mathrm{d}}}x(t) =& \,\left[ {A(r_t) + B_1(r_t){{\mathrm{K}}}_1(r_t) + B_2(r_t){{\mathrm{K}}}_2(r_t)}\right] x(t){{\mathrm{d}}}t \\ & + \left[ {C(r_t) + D_1(r_t){{\mathrm{K}}}_1(r_t)+ D_2(r_t){{\mathrm{K}}}_2(r_t)}\right] x(t){{\mathrm{d}}}W(t) \end{aligned}$$

is exponentially stable in mean square, where \({{\mathrm{K}}}_1(i)\) , \({{\mathrm{K}}}_2(i)\) are defined after (3.6)–(3.7).

Denote \(F_1^*(i)= {{\mathrm{K}}}_1(i), F_2^*(i)= {{\mathrm{K}}}_2(i)\) , then the stochastic Nash equilibrium strategy \((u_1^*, u_2^*)\) can be represented by

$$\begin{aligned} \left\{ \begin{array}{l} u_1^*(t)=\sum \limits _{i = 1}^l {F_1^* (i)x(t)}\chi _{r_t = i}, \\ u_2^*(t)=\sum \limits _{i = 1}^l {F_2^* (i)x(t)}\chi _{r_t = i}. \\ \end{array} \right. \end{aligned}$$
(3.8)

Furthermore, the generalized stochastic Nash games (3.1)–(3.4a,b) are well posed (w.r.t. \((x_0, i)\) ), and the optimal value is determined by

$$\begin{aligned} V_k(x_0,i)={\mathop {\inf }\limits _{u_k\in U_k}J_k(u_k, u_\tau ^*;s,y,i)=y^{\prime }P_k(i)y, \quad k, \tau =1, 2, k\ne \tau , i\in \varXi} . \end{aligned}$$

Remark 3.5

It is worth mentioning that CSRAEs as (3.6)–(3.7) may have more solutions but not all are stabilizing solutions. It remains as a challenge for future research to find conditions which guarantee the existence of a stabilizing solution of CSRAEs like (3.6)–(3.7).

4 Application to Stochastic \({\mathbf{\it H}_2/\mathbf{\it H}_\infty} \) Control

Now, we apply the above developed theory to solve some problems related to stochastic H 2/H ∞ control. First, we state the stochastic H 2/H ∞ control problem for Markov jump linear systems; then, we demonstrate the usefulness of the above developed theory in the study of stochastic H 2/H ∞ control.

For notational simplification, we only consider the case of infinite-time horizon, which is similar for finite-time horizon. Let us now give the detailed formulation of the problem.

Consider the following stochastic controlled system with state- and control-dependent noise:

$$ \left\{ \begin{array}{ll} {{\mathrm{d}}}x(t) = \left[ {A(r_t)x(t) + B(r_t)v(t) + C(r_t)u(t)}\right] {{\mathrm{d}}}t + \left[ {D(r_t)x(t) + F(r_t)u(t)} \right] {{\mathrm{d}}}W(t), \\ z(t) = \left[ {\begin{array}{c} {L(r_t)x(t)} \\ {u(t)} \\ \end{array}} \right] \;,\;x(0) = x_0 \in {\mathbb{R}}^n, \\ \end{array} \right. $$
(4.1)

where \(u(t), v(t), z(t)\) are the control input, external disturbance, and controlled output, respectively.

Define two associated performances as follows:

$$\begin{aligned} J_1(u,v;x_0,i) = {\mathbf{E}}\left\{ \int _0^\infty {\left[ {\left\| {z(t)}\right\| ^2 - \gamma ^2 \left\| {v(t)} \right\| ^2 }\right] }{{\mathrm{d}}}t\left| {r_0= i} \right. \right\} \end{aligned}$$

and

$$\begin{aligned} J_2(u,v;x_0,i) = {\mathbf{E}}\left\{ \int _0^\infty {\left\| {z(t)}\right\| }^2 {{\mathrm{d}}}t\left| {r_0=i} \right. \right\} , \quad i\in \varXi . \end{aligned}$$

The infinite-time horizon stochastic H 2/H ∞ control problem of system (4.1) is described as follows (Huang et al. [15], Zhu et al. [40]).

Definition 4.1

For given disturbance attenuation level \(\gamma >0\), if we can find \(u^*(t)\times v^*(t) \in U[0, \infty )\), such that

  1. (1)

    \(u^*(t)\) stabilizes system (4.1) internally, i.e., when \(v(t) = 0, u=u^*\), the state trajectory of (4.1) with any initial value \((x_0, i) \in {\mathbb{R}}^n \times \varXi \) that satisfies

    $$\begin{aligned} {\mathop {\lim }\limits _{t \rightarrow \infty } {\mathbf{E}}\{{\left\| {x(t)} \right\| ^2 \left| {r_0 = i} \right. }\} = 0}. \end{aligned}$$
  2. (2)

    \(\left| {L_{u*} } \right| _\infty < \gamma \) with

    $$\begin{aligned} \left| {L_{u*} } \right| _\infty = {\mathop {\mathop {\sup }\limits _{\scriptstyle \;\;\,v \in U_2 [0,\infty ), }}\limits _{\scriptstyle v \ne 0,u = u*,x_0 = 0 }} \frac{{\left\{ {\sum \limits _{i = 1}^l {\mathbf{E}\left[ \int _0^\infty {\left\| {z(t)} \right\| ^2 {{\mathrm{d}}}t\left| {r_0 = i} \right. } \right] } } \right\} ^{1/2}}}{{\left\{ {\sum \limits _{i = 1}^l {\mathbf{E}\left[ \int _0^\infty {\left\| {v(t)} \right\| ^2 {{\mathrm{d}}}t\left| {r_0 = i} \right. } \right] }} \right\} ^{1/2}}}. \end{aligned}$$
  3. (3)

    When the worst case disturbance \(v^*(t)\in U_2[0, \infty )\), if existing, is applied to (4.1), \(u^*(t)\) minimizes the output energy

    $$\begin{aligned} J_2(u,v^*;x_0,i) = {\mathbf{E}}\left\{ \int _0^\infty {\left\| {z(t)}\right\| ^2}{{\mathrm{d}}}t\left| {r_0=i}\right. \right\} , \quad i\in \varXi . \end{aligned}$$

Then we say that the infinite-time horizon stochastic H 2/H ∞ control problem has a pair of solutions. Obviously, \((u^*, v^*)\) is the Nash equilibrium strategies [7], such that

$$\begin{aligned} J_1(u^*,v^*;x_0,i)\leqslant J_1(u^*,v;x_0,i), J_2(u^*,v^*;x_0,i)\leqslant J_2(u,v^*;x_0,i), \,i \in \varXi . \end{aligned}$$

According to Theorem 3.4 discussed in Sect. 3, the following results can be obtained straightly.

Theorem 4.2

For system (4.1), suppose the following CSRAEs (\(i, j\in \varXi \)).

$$\begin{aligned}&\left\{ \begin{array}{lll} P_1(i)\tilde{A}(i) + \tilde{A}^{\prime }(i)P_1(i) + \tilde{D}^{\prime }(i)P_1(i)\tilde{D}(i) + \tilde{Q}(i) + \sum \limits _{j=1}^l {\pi _{ij}P_1(j)} \\ \quad+\,\gamma ^{-2}P_1(i)B_1(i)B^{\prime }_1(i)P_1(i) = 0, \\ {{\mathrm{K}}}_1(i) = \gamma ^{-2}B^{\prime }_1(i)P_1(i)\;,\;\;i \in \varXi , \\ \end{array} \right. \end{aligned}$$
(4.2)
$$\begin{aligned}&\left\{ \begin{array}{l} P_2(j){\bar{A}}(j) + {\bar{A}}^{\prime }(j)P_2(j) + D^{\prime }(j)P_2(j)D(j) + L^{\prime }(j)L(j) + \sum \limits _{k=1}^l {\pi _{jk}P_2(k)} \\ \quad +\left( {P_2(j)C(j) + D^{\prime }(j)P_2(j)F(j)}\right) K_2(j) = 0, \\ \quad I + F^{\prime }(j)P_2(j)F(j) > 0, \\ {{\mathrm{K}}}_2 (j) = - \left( {I + F^{\prime }(j)P_2(j)F(j)} \right) ^{-1} \left( {C^{\prime }(j)P_2(j) + F^{\prime }(j)P_2(j)D(j)} \right) \;,\;j \in \varXi , \\ \end{array} \right. \end{aligned}$$
(4.3)

where

$$\begin{aligned} \tilde{A} = A + C{{\mathrm{K}}}_2, {\bar{A}} = A + B{{\mathrm{K}}}_1, \tilde{D} = D + F{{\mathrm{K}}}_2, \tilde{Q} = L^{\prime }L + {{\mathrm{K}}^{\prime }}_2 {{\mathrm{K}}}_2 \end{aligned}$$

have stabilizing solutions \(P=(P_1, P_2): \rightarrow {\mathbf{S}}_n^l \times {\mathbf{S}}_n^l , P_1=(P_1(1), \cdots , P_1(l)), P_2=(P_2(1), \cdots , P_2(l))\) , and \((P_1, P_2)\) is a stabilizing solution of CSRAEs (4.2)–(4.3) if the following closed-loop system

$$\begin{aligned} {{\mathrm{d}}}x(t) = \left[ {A(r_t) + B(r_t){{\mathrm{K}}}_1(r_t) + C(r_t){{\mathrm{K}}}_2(r_t)}\right] x(t){{\mathrm{d}}}t + \left[ {D(r_t) + F(r_t){{\mathrm{K}}}_2(r_t)}\right] x(t){{\mathrm{d}}}W(t) \end{aligned}$$

is exponentially stable in mean square, where \({{\mathrm{K}}}_1(i)\), \({{\mathrm{K}}}_2(i)\) are defined in (4.2)–(4.3).

Then the stochastic H 2/H ∞ control has a pair of solutions \((u^*(t), v^*(t))\) with the feedback form

$$\begin{aligned} u^*(t)&= {{\mathrm{K}}}_2(r_t)x(t),\\ v^*(t)&= {{\mathrm{K}}}_1(r_t)x(t). \end{aligned}$$

In this case, \(u^*(t) \) is a solution to the stochastic H 2/H ∞ control of system (4.1), and \(v^*(t)\) is the corresponding worst case disturbance.

Remark 4.3

Similar to remark 1, the CSRAEs as (4.2)–(4.3) may have more solutions, but not all are stabilizing solutions; so how to find conditions which guarantee the existence of a stabilizing solution of CSRAEs like (4.2)–(4.3) deserves future study.

Illustrative example—consider the following numerical example, assign the coefficients of system (4.1) as follows:

$$\begin{aligned} \varXi&= \{ 1,2\}, \varPi = \left[ {\begin{array}{*{20}c} { - 0.2} &{} {0.2} \\ {0.8} &{} { - 0.8} \\ \end{array}} \right] , A(1) = \left[ {\begin{array}{*{20}c} 0 &{} 1 \\ { - 2} &{} { - 3} \\ \end{array}} \right] , A(2) = \left[ {\begin{array}{*{20}c} 0 &{} 1 \\ 1 &{} 0 \\ \end{array}} \right] ,\\ B(1)&= \left[ {\begin{array}{*{20}c} 1 \\ 1 \\ \end{array}} \right] , B(2) = \left[ {\begin{array}{*{20}c} 0 \\ 1 \\ \end{array}} \right] , C(1) = \left[ {\begin{array}{*{20}c} 1 \\ 0 \\ \end{array}} \right] , C(2) = \left[ {\begin{array}{*{20}c} 3 \\ 1 \\ \end{array}} \right] , D(1) = \left[ {\begin{array}{*{20}c} {0.1} &{} 0 \\ 0 &{} {0.3} \\ \end{array}} \right] ,\\ D(2)&= \left[ {\begin{array}{*{20}c} {0.5} &{} 0 \\ 0 &{} {0.2} \\ \end{array}} \right] , F(1) = \left[ {\begin{array}{*{20}c} 0 \\ 0.1 \\ \end{array}} \right] , F(2) = \left[ {\begin{array}{*{20}c} 0 \\ 0 \\ \end{array}} \right] . \end{aligned}$$

Set \(\gamma =0.7\); and solving (4.2)–(4.3) by using the algorithm presented in Li et al. [18], we have

$$\begin{aligned} P(1) = \left[ {\begin{array}{*{20}c} {0.034\,8} &{} {0.024\,6} \\ {0.024\,6} &{} {0.051\,2} \\ \end{array}} \right] , \quad P(2) = \left[ {\begin{array}{*{20}c} {0.042\,7} &{} {0.068\,2} \\ {0.068\,2} &{} {0.311\,2} \\ \end{array}} \right] . \end{aligned}$$

Therefore, the stochastic H 2/H ∞ controller is given by \(u(t)=-0.035\,0x_1(t)- 0.026\,1x_2(t)\), while \(r_t=1\); and \(u(t)=-0.128\,1x_1(t)- 0.204\,6x_2(t)\), while \(r_t=2\).

Given initial values \(r_0=1, x_1(0)=2\) and \(x_2(0)=1\), using the Euler-Maruyama method with step size \(\triangle =0.00\,1\), computer simulation of the paths of \(r_t, u(t), x_1(t)\) and \(x_2(t)\) are shown in Fig. 1, 2 and 3.

Fig. 1
figure 1

Curve of \(r_t\)

Full size image
Fig. 2
figure 2

Curve of \(u(t)\)

Full size image
Fig. 3
figure 3

Curve of \(x_1(t)\) and \(x_2(t)\)

Full size image

5 Conclusion

In the present paper, stochastic Nash games of Markov jump linear systems governed by Itô’s differential equation with state- and control-dependent noises both in finite-time horizon and infinite-time horizon have been considered. The defined Nash equilibrium strategies can be calculated by solving CSRDEs (CSRAEs). Moreover, the obtained results have been applied to stochastic H 2/H ∞ control for Markov jump linear systems with state- and control-dependent noises. Finally, the numerical example has shown the validity of the proposed method.

These results are only theoretical analysis; how to extend them into practical applications, such as in the engineering/economics or anything in the social sciences, needs future investigations.