1 Introduction

The Linear Quadratic (LQ) problem was mathematically formulated and solved, as well as the filtering one, in the 1960s by Kalman [1]. An important advantage of the LQ theory is the existence of explicit feedback forms for optimal state control and the optimal cost value through the Riccati equations. The deterministic Riccati equation was essentially solved by Wonham [2] by applying Bellman’s principle of quasilinearization [3]. A detailed research of stochastic LQ control problems has been performed by Bismut [4]. The existence of a unique solution for the associated Riccati equations was studied in [5].

Switching systems are more advantageous models to describe the noninvariant phenomena with the continuous law of movement and they have gained considerable attention in science and engineering. Examples of these systems include many evolutionary processes, robotics, integrated circuit design, multimedia, manufacturing, power electronics, chaos generators, and air traffic management systems [6, 7]. Optimization problems have also received growing interest among the researchers of deterministic and for stochastic switching control systems [8–14].

Manifold problems of stochastic optimal control theory have been considered in [15–22]. Optimal control problems of switching systems have attracted considerable attention, due to the advantages, for instance, in modeling and improving the transient response on highly complex systems and systems with large uncertainties. The stochastic maximum principle via backward stochastic differential equations is derived in [23–27]. The necessary conditions of optimality for stochastic switching systems earlier have been obtained in [28–30]. In [31] the linear quadratic control problem has been investigated for a special type of stochastic systems.

In this paper, the LQ problem of stochastic switching systems with restrictions is considered. Ekeland’s variational principle [32] has been used to establish the necessary and sufficient conditions of optimality for a given problem.

2 Statement of main problem

Unless specified otherwise, throughout the paper we use the same notations as in [30].

Consider the following stochastic linear control system:

$$\begin{aligned}& \begin{aligned}[b] dx^{l}(t) ={}& \bigl[ A^{l}(t)x^{l}(t)+B^{l}(t)u^{l}(t)+g^{l}(t) \bigr]\,dt\\ &{}+ \bigl[ C^{l}(t)x^{l}(t)+D^{l}(t)u^{l}(t)+f^{l}(t) \bigr]\,dw^{l}(t),\quad t \in( {t_{l - 1} ,t_{l} }], \end{aligned} \end{aligned}$$
(1)
$$\begin{aligned}& x^{l}(t_{l - 1}) = \Phi^{l - 1}(t_{l - 1})x^{l - 1}(t_{l - 1})+K^{l - 1}(t_{l - 1}), \quad l = \overline{2,r} ; x^{1}(t_{0}) = x_{0}, \end{aligned}$$
(2)
$$\begin{aligned}& u^{l}(t) \in U_{\partial}^{l} \equiv \bigl\{ {u^{l} ( {\cdot,\cdot} )\in \mathrm{L}_{F^{l}}^{2} \vert u^{l} ( {t,\cdot} )\in U^{l}\subset\mathrm{R}^{m_{l} }, \mbox{a.c.}} \bigr\} . \end{aligned}$$
(3)

The elements of \(U_{\partial}^{l} \), are called admissible controls.

Our goal is to find an optimal solution \((x,u)= ( {x^{1},x^{2},\ldots,x^{r},u^{1},u^{2},\ldots,u^{r}} )\) and a switching sequence \(\mathbf{t}=(t_{1} ,t_{2} ,\ldots,t_{r}) \), that minimize the cost functional:

$$ J(u) = E\sum_{l = 1}^{r} \biggl[ \bigl\langle G^{l}x^{l}(t_{l}),x^{l}(t_{l}) \bigr\rangle + \int_{t_{l-1} }^{t_{l} } \bigl( \bigl\langle M^{l}(t)x^{l}(t),x^{l}(t) \bigr\rangle + \bigl\langle N^{l}(t)u^{l}(t),u^{l}(t) \bigr\rangle \bigr)\,dt \biggr], $$
(4)

on the decisions of the system (1)-(3) under the conditions:

$$ \textit{E} \bigl\langle q^{l},x^{l}(t_{l}) \bigr\rangle \in Q^{l}, \quad l = 1,\ldots,r, $$
(5)

where \(Q^{1},\ldots,Q^{r} \) are a closed convex sets in \(R^{1}\). The elements of matrices \(A^{l}\), \(B^{l}\), \(C^{l}\), \(D^{l}\), \(\Phi^{l}\), \(M^{l}\), \(N^{l}\) and vectors \(G^{l}\), \(K^{l}\), \(g^{l}\), \(f^{l} \) are continuous, bounded functions. \(G^{l}\), \(M^{l}\) are a positively semi-defined matrices, and \(N^{l}\) are positively defined matrices.

\(A_{i}\) represents the set of elements \(\pi^{i} = (t_{0} ,t_{1} ,t_{i} ,x^{1}(t) ,x^{2}(t) ,\ldots,x^{i}(t),u^{1},u^{2},\ldots,u^{i})\) for each \(i=1,\ldots,r \). To describe the main result we need to introduce some concepts, such as a solution of linear switching systems, admissible element of control problem and optimal solution for LQ problem of stochastic switching systems. For a detailed account we refer the reader to [29, 30].

3 Stochastic LQ problem of switching systems

This section is devoted to the investigation of optimal control problems for linear stochastic switching systems with constraints. The LQ problem belongs to a special class of convex control problems for which the maximum principle is a necessary as well as sufficient condition of optimality. The next theorem provides necessary and sufficient conditions of the optimality of stochastic linear switching systems.

Theorem 1

Let \(A_{r}^{0} \) be a set of admissible elements. The elements

$$\pi^{r}=\bigl(t_{0} ,t_{1}, t_{r} ,x^{1}(t) ,x^{2}(t) ,\ldots, x^{r}(t), u^{1}, u^{2}, \ldots, u^{r}\bigr) \in A_{r}^{0} $$

are an optimal solution of problem (1)-(5) if and only if:

  1. (a)

    there exist random processes \((\psi^{l}(t) ,\beta^{l}(t))\in{L}_{F}^{2} (t_{l-1} ,t_{l} ;{R}^{n_{l} })\times{L}_{F}^{2} (t_{l-1} ,t_{l} ;{R}^{n_{l}\times n_{l}})\) which are the solutions of the following stochastic backward equations:

    $$ \left \{ \textstyle\begin{array}{@{}l} d\psi^{l}(t) =- [A^{l*}(t)\psi^{l}(t)+C^{l*}(t)\beta ^{l}(t)-M^{l}(t)x(t) ] \\ \hphantom{d\psi^{l}(t) =}{}+\beta^{l}(t)\,dw^{l}(t) ,\quad t_{l-1} \le t< t_{l} , \\ \psi^{l}(t_{l}) =-\lambda^{l}_{0}G^{l} x^{l}(t_{l})-\lambda^{l}_{1}q^{l} +\psi ^{l+1}(t_{l}) \Phi^{l} (t_{l} ),\quad l=1,\ldots,r-1 , \\ \psi^{r}(t_{r}) =-\lambda^{r}_{0}G^{r} x^{r}(t_{r})-\lambda^{r}_{1}q^{r} ; \end{array}\displaystyle \right . $$
    (6)
  2. (b)

    the candidate optimal controls \({u}^{l}\in U^{l}\), \(l=\overline{1,r}\), are defined by

    $$ N^{l*}(t)u^{l}(t)=B^{l*}(t) \psi^{l}(t)+D^{l*}(t)\beta^{l}(t), \quad\textit{a.e. }\theta\in[t_{l-1} , t_{l} ] ; $$
    (7)
  3. (c)

    the following transversality conditions hold:

    $$ \psi^{l+1}(t_{l}) \bigl(\Phi_{t}^{l*} \bigl(t^{l}\bigr) x^{l}(t_{l})+K_{t}^{l*}(t_{l}) \bigr)=0, \quad \textit{a.c.}, l=1,\ldots,r-1 . $$
    (8)

Proof

First we investigate given optimal control problem without endpoint constraints (5). Let \(u^{l}(t)\) and \(\bar{u}^{l}(t)\), \(l = 1,\ldots,r \) be some admissible controls and \(x^{l}(t)\), \(\bar{x}^{l}(t) \) be corresponding trajectories. \(\Delta\bar{u}^{l}(t)\) represents the admissible increment of the control \(u^{l}(t)\). \(\mathbf{t}=(t_{0}, t_{1}, \ldots t_{r} )\) and \(\bar{\mathbf{t}}=(\bar{ t_{0}}, \bar{t_{1}}, \ldots, \bar{t_{r}})\) denote different switching laws. The increment of the cost functional (4) along the admissible control \(\bar{ \mathbf {u}}=(\bar{u}^{1}(t), \bar{u}^{2}(t), \ldots, \bar{u}^{r}(t)) \) looks like

$$\begin{aligned} \bigl\langle J^{\prime}(\mathbf{u}), \bar{\mathbf{u}}- \mathbf{u} \bigr\rangle ={}& E\sum_{l = 1}^{r} \biggl[ \bigl\langle G^{l}x^{l}(t_{l}), \bar{x}^{l}(t_{l})-x^{l}(t_{l}) \bigr\rangle \\ &{}+\int_{t_{l-1} }^{t_{l} } \bigl( \bigl\langle M^{l}(t)x^{l}(t),\bar {x}^{l}(t)-x^{l}(t) \bigr\rangle + \bigl\langle N^{l}(t)u^{l}(t),\bar {u}^{l}(t)-u^{l}(t) \bigr\rangle \bigr)\,dt \biggr]. \end{aligned}$$
(9)

By (1)-(2) the increments of the trajectories are defined as

$$ \left \{ \textstyle\begin{array}{@{}l} d(\bar{x}^{l}(t)-x^{l}(t)) \\ \quad= [ A^{l}(t)(\bar{x}^{l}(t)-x^{l}(t))+B^{l}(t)(\bar {u}^{l}(t)-u^{l}(t)) ]\,dt \\ \qquad{}+ [ C^{l}(t)(\bar{x}^{l}(t)-x^{l}(t))+B^{l}(t)(\bar{x}^{l}(t)-x^{l}(t)) ]\,dw^{l}(t),\quad t \in ( \bar{t_{l - 1}} ,\bar{t_{l}} ],\\ \Delta x^{l+1}(t_{l})=\bar{x}^{l+1}(\bar{t_{l}})-x^{l+1}(t_{l})=\Phi^{l}(\bar {t_{l}})\bar{x}^{l}(\bar{t_{l}})-\Phi^{l}(t_{l})x^{l}(t_{l}). \end{array}\displaystyle \right . $$
(10)

Let us introduce the stochastic processes \(\psi^{l}(t) \), \(l=1,\ldots,r \), as the solution of the following stochastic backward differential equations:

$$ \left \{ \textstyle\begin{array}{@{}l} d\psi^{l}(t) =- [A^{l*}(t)\psi^{l}(t)+C^{l*}(t)\beta ^{l}(t)-M^{l}(t)x(t) ] +\beta^{l}(t)\,dw^{l}(t) ,\quad t_{l-1} \le t< t_{l} , \\ \psi^{l}(t_{l}) =-G^{l} x^{l}(t_{l}) +\psi^{l+1}(t_{l}) \Phi^{l} (t_{l} ),\quad l=1,\ldots,r-1 , \\ \psi^{r}(t_{r}) =-G^{r} x^{r}(t_{r}). \end{array}\displaystyle \right . $$
(11)

According to the Ito formula for each \(l=1,\ldots,r \) the following identity is satisfied:

$$\begin{aligned} &d \bigl\langle \psi^{l}(t_{l}),(\bar{x}^{l}(t_{l})-x^{l}(t_{l}) (\Delta{t_{l}} \bigr\rangle \\ &\quad= \bigl\langle \,d\psi^{l}(t), \bigl(\bar{x}^{l}(t)-x^{l}(t)\bigr)\Delta{t_{l}} \bigr\rangle + \bigl\langle \psi^{l}(t),d\bigl(\bar{x}^{l}(t)-x^{l}(t) \bigr)\Delta{t_{l}} \bigr\rangle \\ &\qquad{}+ \bigl\langle \beta^{l}(t),C^{l}(t) \bigl(\bar{x}^{l}(t)-x^{l}(t)\bigr)\Delta {t_{l}}+D^{l}(t) \bigl(\bar{u}^{l}(t)-u^{l}(t) \bigr)\Delta{t_{l}} \bigr\rangle \,dt. \end{aligned}$$

Integrating the aforementioned equality and taking the expectation of both sides into account in (10) it follows

$$\begin{aligned} &E \bigl\langle \psi^{l}(t_{l}),(\bar{x}^{l}(t_{l})-x^{l}(t_{l}) (\Delta{t_{l}} \bigr\rangle - \bigl\langle \psi^{l}(t_{l-1}), \bigl(\bar {x}^{l}(t_{l-1})-x^{l}(t_{l-1}) \bigr)\Delta{t_{l-1}} \bigr\rangle \\ &\quad=E \int_{t_{l-1} }^{t_{l} } \bigl\langle \,d\psi^{l}(t)+A^{l*}(t) \psi ^{l}(t)+C^{l*}(t)\beta^{l}(t),\bigl( \bar{x}^{l}(t)-x^{l}(t)\bigr)\Delta t_{l} \bigr\rangle \\ &\qquad{}+E \int_{t_{l-1} }^{t_{l} } \bigl\langle B^{l*}(t)\psi ^{l}(t)+D^{l*}(t)\beta^{l}(t),\bigl( \bar{u}^{l}(t)-u^{l}(t)\bigr)\Delta t_{l} \bigr\rangle \,dt. \end{aligned}$$

Due to this equality equation (9) can be rewritten as

$$\begin{aligned} \bigl\langle J^{\prime}(\mathbf{u}),\bar{\mathbf{u}}- \mathbf{u} \bigr\rangle ={}& E\sum_{l = 1}^{r} \biggl[ \bigl\langle G^{l}x^{l}(t_{l}),\bar {x}^{l}(t_{l})-x^{l}(t_{l}) \bigr\rangle + \bigl\langle \psi^{l}(t_{l}),\bar {x}^{l}(t_{l})-x^{l}(t_{l}) \bigr\rangle \\ &{}- E \int_{t_{l-1} }^{t_{l} } \bigl\langle \,d\psi^{l}(t)+A^{l*}(t) \psi ^{l}(t)+C^{l*}(t)\beta^{l}(t),\bigl( \bar{x}^{l}(t)-x^{l}(t)\bigr)\Delta t_{l} \bigr\rangle \\ &{}-E \int_{t_{l-1} }^{t_{l} } \bigl\langle B^{l*}(t)\psi ^{l}(t)+D^{l*}(t)\beta^{l}(t),\bigl( \bar{u}^{l}(t)-u^{l}(t)\bigr)\Delta t_{l} \bigr\rangle \,dt \\ &{}+\int_{t_{l-1} }^{t_{l} } \bigl( \bigl\langle M^{l}(t)x^{l}(t),\bar {x}^{l}(t)-x^{l}(t) \bigr\rangle + \bigl\langle N^{l}(t)u^{l}(t),\bar {u}^{l}(t)-u^{l}(t) \bigr\rangle \bigr)\,dt \biggr]. \end{aligned}$$
(12)

Further, using equation (11) we get a more succinct expression:

$$ \bigl\langle J^{\prime}(\mathbf{u}), \bar{\mathbf{u}}- \mathbf{u} \bigr\rangle = E\sum_{l = 1}^{r} \int_{t_{l-1}}^{t_{l}} \bigl\langle N^{l}(t)u^{l}(t)-B^{l*}(t) \psi^{l}(t)-D^{l*}(t)\beta^{l}(t),\bigl(\bar {u}^{l}(t)-u^{l}(t)\bigr)\Delta{t_{l}} \bigr\rangle \,dt. $$
(13)

It is well known that a necessary and sufficient condition of optimality for the convex functional is given by \(J^{\prime}(\mathbf{u})=0\). The validity of (7) and (8), hence the necessary conditions of optimality for the considered unrestricted problem (1)-(4) follows from equations (11) and (13). At last, according to the independence of the increments \(\Delta\bar{x}^{l}(t)\), \(\Delta\bar {u}^{l}(t)\), \(\Delta\bar{t_{l}}\), sufficiency follows from equation (12).

To construct the optimality condition of LQ problem (1)-(4) with the right endpoint constraints (5), the above mentioned problem by using Ekeland’s variational principle [32] is converted into a sequence of unconstrained problems. Based on the results already obtained for problem (1)-(4), necessary and sufficient conditions for the sequence of switching systems are established.

To apply Ekeland’s variational principle we introduce the following approximating functional:

$$I_{j} ( {\mathbf{u}} ) = \min_{(c,y^{l}) \in\varepsilon}\sqrt {\sum_{l=1}^{r}{\bigl\vert c^{l} - \varepsilon^{l}_{j} - ES^{l} ( x ,u ,t )\bigr\vert ^{2}} + \sum_{l=1}^{r} \bigl\vert y^{l}- {Eq^{l}x^{l}(t_{l})} \bigr\vert ^{2}}. $$

Here \(S^{l} ( x ,u ,t )= \langle G^{l}x^{l}(t_{l}),x^{l}(t_{l}) \rangle+\int_{t_{l-1} }^{t_{l} } ( \langle M^{l}(t)x^{l}(t),x^{l}(t) \rangle+ \langle N^{l}(t)u^{l}(t),u^{l}(t) \rangle )\,dt\); \(\lim_{j \to\infty}\varepsilon^{l}_{j}=0 \); \(c=c^{1}+\cdots+c^{r}\); \(\varepsilon= \{ {c:c \le J^{0},y^{l}\in Q^{l}} \}\); let \(J^{0}\) be a minimal value of the functional in the problem (1)-(5).

Let \(V^{l} \equiv(U_{\partial}^{l} ,d)\) be the space of controls obtained by means of the following metric:

$$d\bigl(u^{l},v^{l}\bigr) = ( {l \otimes P} ) \bigl\{ { ( {t, \omega} ) \in [ {t_{l - 1} ,t_{l} } ]\times\Omega: \nu_{t}^{l} \ne u_{t}^{l} } \bigr\} . $$

For each \(l = 1,\ldots,r \), let \(V^{l}\) be a complete metric space [32]. □

For the following fact it is significant that we can provide a relation between the sequence of controls from the metric space \(V^{l}\) and the sequence of corresponding trajectories of system (1)-(2).

Lemma 1

([30], Lemma 4.3)

Let \((v^{1,n},\ldots,v^{r,n}) \) be the sequence of admissible controls from \((V^{1},\ldots,V^{r})\), and \((x^{1,n},\ldots,x^{r,n})\) be the sequence of corresponding trajectories of the system (1)-(2). Let for each \(l=1,\ldots,r\) the condition \(d(v^{l,n} ,u ^{l}) \to0\) when \(n\to \infty\) be met. Then

$$\lim_{n \to\infty} \Bigl\{ \sup_{t_{l - 1} \leq t \leq t_{l} } E\bigl\vert {x^{l,n}(t) - x ^{l}(t) } \bigr\vert ^{2} \Bigr\} = 0, $$

where \(x ^{l}(t) \) is a trajectory corresponding to admissible controls \(u ^{l}(t)\), \(l = 1,\ldots,r\).

Due to Ekeland’s variational principle, we see that \((x^{1,j}(t) ,\ldots,x^{r,j}(t), u^{1,j}(t) ,\ldots,u^{r,j}(t) ) \) is a solution of the following problem:

$$ \left \{ \textstyle\begin{array}{@{}l} J_{j}(u) = I_{j} (\mathbf{u})+E\sum_{l = 1}^{r} \sqrt{\varepsilon^{l} _{j}} \int_{t_{l - 1} }^{t_{l} } {\delta(u ^{l}(t) ,u^{l,j}(t) )\,dt} \\ \hphantom{J_{j}(u)}\to\min dx^{l,j}(t) = [ A^{l}(t)x^{l,j}(t)+B^{l}(t)u^{l,j}(t)+g^{l}(t) ]\,dt\\ \hphantom{J_{j}(u)=}{}+ [ C^{l}(t)x^{l,j}(t)+D^{l,j}(t)u^{l}(t) ]\,dw^{l}(t),\quad t \in ( {t_{l - 1} ,t_{l} } ], \\ x^{l,j}(t_{l - 1}) = \Phi^{l - 1}(t_{l - 1})x^{l - 1,j}(t_{l - 1})+K^{l - 1}(t_{l - 1}),\quad l = 2,\ldots,r, \\ x^{1,j}(t_{0}) = x_{0}, \\ u ^{l,j}(t) \in U_{\partial}^{l} . \end{array}\displaystyle \right . $$
(14)

\(\delta(u,v)\) is the characteristic function of the set \(\{ u,v \in V^{l}:u \ne v \}\).

Based on (11), it is found that, if \((x^{1,j}(t) ,\ldots,x^{r,j}(t),u^{1,j}(t) ,\ldots,u^{r,j}(t))\) is an optimal solution of problem (14), there exist the random processes \((\psi^{l,j}(t), \beta^{l,j}(t)) \in L_{F^{l}}^{2} (t_{l - 1} ,t_{l} ;R^{n_{l} })\times L_{F^{l}}^{2} (t_{l - 1} ,t_{l} ;R^{n_{l} \times n_{l} })\), which are solutions of the following system:

$$ \left \{ \textstyle\begin{array}{@{}l} d\psi^{l,j}(t) =- [A^{l*}(t)\psi^{l,j}(t)+C^{l*}(t)\beta ^{l,j}(t)-M^{l}(t)x^{l,j}(t) ] \\ \hphantom{d\psi^{l,j}(t) =}{}+\beta^{l}(t)\,dw^{l}(t) , \quad t_{l-1} \le t< t_{l} , \\ \psi^{l,j}(t_{l}) =-\lambda_{0}^{l,j}G^{l} x^{l,j}(t_{l}) -\lambda _{1}^{l,j}q^{l}+\psi^{l+1,j}(t_{l}) \Phi^{l} (t_{l} ),\quad l=1,\ldots,r-1 , \\ \psi^{r,j}(t_{r}) =-\lambda_{0}^{r,j}G^{r} x^{r,j}(t_{r})-\lambda_{1}^{r,j}q^{r}, \end{array}\displaystyle \right . $$
(15)

where the non-zero \((\lambda_{0}^{l,j} ,\lambda_{1}^{l,j})\), \(l = 1,\ldots,r\), are defined as

$$ \bigl( \lambda_{0}^{l,j},\lambda_{1}^{l,j} \bigr)= \frac{ ( -c^{l}+\varepsilon^{l}_{j}+ S^{l}(x^{j},u^{j},t) ,-y^{l}+ Eq^{l}x^{l,j}(t_{l}) )}{J_{j}^{0}}; $$
(16)

here

$$J_{j}^{0} = \Biggl( \sum_{l = 1}^{r} \bigl\vert y^{l}-{Eq^{l}x^{l,j}(t_{l})} \bigr\vert ^{2} + \Biggl\vert \sum_{l = 1}^{r} \bigl[{c^{l} - \varepsilon^{l}_{j} - E{S^{l}\bigl(x^{j},u^{j},t\bigr)}}\bigr] \Biggr\vert ^{2} \Biggr)^{1/2}. $$

On the one hand, due to (15), \(\forall \tilde{u}^{l} \in V^{l} \) the following necessary and sufficient condition of optimality for the unconstrained problem (14) holds:

$$ N^{l*}(t)u^{l,j}(t)=B^{l*}(t) \psi^{l,j}(t)+D^{l*}(t)\beta^{l,j}(t), \quad\textit{a.e. }t \in[t_{l-1}, t_{l} ], \textit{a.c.} $$
(17)

Besides, based on (15) and (17) we see that the optimal decision of the problem (14) satisfies the transversality condition:

$$ \psi^{l+1,j}(t_{l}) \bigl(\Phi_{t}^{l*}(t_{l}) x^{l,j}(t_{l})+K_{t}^{l*}(t_{l}) \bigr)=0, \quad \textit{a.c.}, l=1,\ldots,r-1. $$
(18)

Since \(\sum_{l = 1}^{r} |\lambda_{0}^{l,j}|^{2}+|\lambda_{1}^{l,j} |^{2} = 1 \) exists by (16) \((\lambda_{0}^{l,j} ,\lambda_{1}^{l,j} ) \to(\lambda_{0}^{l} ,\lambda _{1}^{l} )\) if \(j \to\infty\).

The truth of (6) is based upon the following lemma, which can be proved by the same method as the proof of Lemma 4 [33].

Lemma 2

Let \(\psi^{l}(t_{l} ) \) be a solution of system (6), \(\psi^{l,j}(t_{l} )\) be a solution of system (15). If the sequence of controls \((u^{1,j}(t) ,\ldots,u ^{r,j}(t) )\) satisfies the assumptions of the Lemma  1, then

$$E \int_{t_{l - 1} }^{t_{l} } {\bigl\vert \psi^{l,j}(t) - \psi^{l}(t) \bigr\vert ^{2}\,dt + } E \int_{t_{l - 1} }^{t_{l} } {\bigl\vert \beta^{l,j}(t) - \beta^{l}(t) \bigr\vert ^{2}\,dt \to0} ,\quad l = 1,\ldots,r. $$

Based on Lemma 2, we can pass to the weak limit in system (15) and obtain the fulfillment of (6). Following a similar scheme, we take the limits in (17) and (18), and justifications of (7), (8) are derived. Theorem 1 is proved.

4 Riccati equations for switching systems

In the theory of LQ problem, it is very natural to connect the LQ problem with the Riccati equation for the possible feedback design. In this section the optimal control is determined explicitly via a set of stochastic Riccati equations. First, we investigate the feedback design problem in the case \(g^{l}(t)\equiv0\); \(f^{l}(t)\equiv0\) and we search a relation in the form:

$$ \psi^{l}(t)=-p^{l}(t)x^{l}(t), \quad l = 1,\ldots,r, \mbox{a.c.} $$
(19)

To determine the stochastic processes \(p^{l}(t)\) we introduce the following theorem.

Theorem 2

Let \(\psi^{l}(t ) \) be a solution of system (6), \(p ^{l}(t) \) be a stochastic process that satisfies (19), defined as the solution of the following differential equation:

$$\begin{aligned} dp^{l}(t) ={}&{-} \bigl[p^{l}(t)A^{l}(t)+A^{l*}(t)p^{l}(t)+ \gamma ^{l}(t)C^{l}(t)+C^{l*}(t) \gamma^{l}(t)+C^{l*}p^{l}(t)C^{l} \\ &{}+M^{l} -\bigl(p^{l}(t)B^{l}(t)+\gamma ^{l}(t)D^{l}(t)+C^{l*}(t)p^{l}(t)D^{l}(t) \bigr) \bigl(N^{l}(t)+D^{l*}(t)p^{l}(t)D^{l}(t) \bigr) \\ &{}\times\bigl(B^{l*}(t)p^{l}(t)+D^{l*}(t) \gamma ^{l}(t)+D^{l*}(t)p^{l}(t)C^{l}(t) \bigr) \bigr] \\ &{}+\gamma^{l}(t)\,dw^{l}(t) ,\quad t_{l-1} \le t< t_{l} . \end{aligned}$$
(20)

Proof

Suppose that the differential of random processes \(p^{l}(t)\) is defined as

$$dp^{l}(t)=\alpha^{l}(t)\,dt+\gamma^{l}(t)\,dw^{l}(t), \quad l=1,\ldots,r. $$

According to the Ito formula:

$$d\psi^{l}(t)=-dp^{l}(t)x^{l}(t)-p^{l}(t)\,dx^{l}(t)- \gamma ^{l}(t)\bigl[C^{l}(t)x^{l}(t)+ D^{l}(t)u^{l}(t)\bigr]\,dt,\quad l = 1,\ldots,r,\mbox{a.c.} $$

Using (1) and (6) we have

$$\begin{aligned} &\bigl[-A^{l*}\psi^{l}(t)-C^{l*} \beta^{l}(t)+M^{l}(t)x^{l}(t)\bigr]+\beta ^{l}(t)\,dw^{l}(t) \\ &\quad=-\bigl[\alpha^{l}(t)x^{l}(t)\,dt +\gamma ^{l}(t)x^{l}(t)\,dw^{l}(t)+p^{l}(t)A^{l}(t)x^{l}(t)\,dt \\ &\qquad{}+p^{l}(t)B^{l}(t)u^{l}(t)\,dt+p^{l} \bigl(C^{l}(t)x^{l}(t) \\ &\qquad{}+D^{l}(t)u^{l}(t)\bigr)\,dw^{l}(t)+ \gamma^{l}(t) \bigl(C^{l}(t)x^{l}(t)+ D^{l}(t)u^{l}(t)\bigr)\,dt\bigr],\quad l = 1,\ldots,r. \end{aligned}$$
(21)

Taking expectation from both side we obtain the following expression for \(\beta^{l}(t)\), \(l = 1,\ldots,r\):

$$ \beta^{l}(t)=-\gamma ^{l}(t)x^{l}-p^{l}(t)C^{l}(t)x^{l}(t)-p^{l}(t)D^{l}(t)u^{l}(t), \quad t\in [t_{l-1},t_{l} ]. $$
(22)

By means of simple transformations taking into account (22) equation (21) can be rewritten as follows:

$$\begin{aligned} & \bigl[\alpha^{l}(t)+p^{l}(t)A^{l}(t)+A^{l*}(t)p^{l}(t)+ \gamma ^{l}(t)C^{l}(t) \\ &\quad{}+C^{l*}(t)\gamma^{l}(t)+C^{l*}p^{l}(t)C^{l}+ M^{l} \bigr]x^{l}(t) \\ &\quad{}+\bigl[p^{l}(t)B^{l}(t)+ \gamma ^{l}(t)D^{l}(t)+C^{l*}(t)p^{l}(t)D^{l}(t) \bigr]u^{l}(t)=0. \end{aligned}$$
(23)

Considering (19) in equation (6) the optimal control can be defined explicitly:

$$\begin{aligned} &\bigl[ N^{l}(t)+D^{l*}(t)p^{l}(t)D^{l}(t) \bigr]u^{l}(t) \\ &\quad{}+ \bigl[B^{l*}(t)p^{l}(t)+D^{l*}(t) \gamma ^{l}(t)+D^{l*}(t)p^{l}(t)C^{l}(t) \bigr]x^{l}(t)=0. \end{aligned}$$
(24)

Hence, Theorem 2 is proved. □

Finally, the feedback design for LQ problem (1)-(5) is obtained by means of the next theorem.

Theorem 3

Let \(\psi^{l}(t )\) be a solution of system (6), \(p ^{l}(t)\), \(\nu^{l}(t) \) be a stochastic processes satisfy to \(\psi^{l}(t)=-[p^{l}(t)x^{l}(t)+\nu^{l}(t)]\), a.c. Then for each l let the random process \(\nu^{l}(t) \) be a solution of the following differential equation:

$$\begin{aligned} & \bigl[\bigl(p^{l}(t)B^{l}(t)+\gamma ^{l}(t)D^{l}(t)+C^{l*}p^{l}(t)D^{l} \bigr) \bigl(N^{l}(t)+D^{l*}(t)p^{l}(t)D^{l}(t) \bigr)^{-1}B^{l*}(t)-A^{l*}(t) \bigr] \nu^{l}(t)\,dt \\ &\quad{} + \bigl[\bigl(p^{l}(t)B^{l}(t)+\gamma ^{l}(t)D^{l}(t)+C^{l*}p^{l}(t)D^{l} \bigr) \bigl(N^{l}(t)+D^{l*}(t)p^{l}(t)D^{l}(t) \bigr)^{-1}D^{l*}(t)-C^{l*}(t) \bigr] \\ &\qquad{}\times \bigl[p^{l*}(t)f^{l}(t)+ \varphi^{l}(t)\bigr]\,dt \\ &\quad= d\nu ^{l}(t) +\bigl[p^{l*}(t)g^{l}(t)- \gamma^{l*}(t)f^{l}(t)\bigr]\,dt+\varphi ^{l}(t)\,dw^{l}(t), \quad \nu^{l}(t_{l})=0 . \end{aligned}$$
(25)

Here for each l stochastic process \(p^{l}(t)\) is the solution of (20).

Proof

Suppose that random processes \(\nu^{l}(t)\) are defined in the following way:

$$\nu^{l}(t_{l})-\nu^{l}(t_{l-1})= \int_{t_{l-1}}^{t_{l}}\kappa ^{l}(t)\,dt+ \int_{t_{l-1}}^{t_{l}}\varphi^{l}(t)\,dw^{l}(t), \quad l=1,\ldots,r. $$

According to Ito’s formula for each l:

$$\begin{aligned} &d\psi^{l}(t)+\kappa^{l}(t)\,dt+\varphi ^{l}(t)\,dw^{l}(t) \\ &\quad=-dp^{l}(t)x^{l}(t)-p^{l}(t)\,dx^{l}(t) -\gamma^{l*}(t)\bigl[C^{l}(t)x^{l}(t)+ D^{l}(t)u^{l}(t)+f^{l}(t)\bigr]\,dt,\quad \mbox{a.c.} \end{aligned}$$
(26)

In view of (6) we obtain the following expression:

$$\beta^{l}(t)=-\gamma ^{l}(t)x^{l}(t)-p^{l}(t)C^{l}(t)x^{l}(t)+p^{l}(t)D^{l}(t)u^{l}(t) +p^{l*}(t)f^{l}(t)+\varphi ^{l}(t), \quad l = 1, \ldots,r,\mbox{a.c.} $$

Substituting this expression for \(\beta^{l}(t)\) into (10), in view of \([N^{l}(t)+D^{l*}(t)p^{l}(t)D^{l}(t)]\) being a positively defined matrix, we have

$$\begin{aligned} u^{l}(t)={}&{-}\bigl[N^{l}(t)+D^{l*}(t)p^{l}(t)D^{l}(t) \bigr]^{-1} \bigl\{ B^{l*}(t)\nu^{l}(t) +p^{l}(t)D^{l}(t)f^{l}(t) \\ &{} +D^{l*}(t)\varphi^{l}(t)+ \bigl[(p^{l}(t)B^{l}(t)+D^{l*}(t) \gamma ^{l}(t)+p^{l}(t)D^{l}(t)C^{l}(t) \bigr] x^{l}(t) \bigr\} . \end{aligned}$$
(27)

Integrating both sides of (26) and using equation (27) of \(u^{l}(t)\), bearing in mind that stochastic processes \(p^{l}(t)\), \(\gamma^{l}(t)\) are the solutions of differential equation (20):

$$\begin{aligned} \kappa^{l}(t)={}& \bigl[ \bigl(p^{l}(t)B^{l}(t)+ \gamma ^{l}(t)D^{l}(t)+C^{l*}(t)p^{l}(t)D^{l}(t) \bigr) \bigl(N^{l}(t)+D^{l*}(t)p^{l}(t)D^{l}(t) \bigr)^{-1} \\ &{}\times B^{l*}(t)-A^{l*}(t)\bigr] \nu^{l}(t)+ \bigl[\bigl(p^{l}(t)B^{l}(t)+\gamma ^{l}(t)D^{l}(t)+C^{l*}(t)p^{l}(t)D^{l}(t) \bigr) \\ &{}\times \bigl(N^{l}(t)+D^{l*}(t)p^{l}(t)D^{l}(t) \bigr)^{-1}D^{l*}(t)-C^{l*}(t)\bigr] \bigl[p^{l*}(t)f^{l}(t)+\varphi ^{l}(t) \bigr]-p^{l*}(t)g^{l}(t) \\ &{}-\gamma^{l*}(t)f^{l}(t). \end{aligned}$$
(28)

Therefore, the assertion of the theorem is true. □

5 Conclusion

There are a lot relevant applications of LQ problems in fields such as aerospace, biology, economics, management sciences, etc. [34–38].

Switching systems provide a natural and convenient theoretical account for mathematical modeling of many complex real phenomena and practical applications. A broad spectrum of the latest research is concerned with optimal control problems of stochastic switching systems [39–41].

The LQ problem of switching systems in which the endpoint restrictions are defined with the help of convex closed sets has been investigated. The objective of the present research is to give an explicit solution to the LQ problem of stochastic switching systems of which drift and diffusion coefficients comprise non-homogeneous terms. The results developed in this study can be viewed as an extension of the problems formulated in [42, 43] for stochastic switching systems.