1 Introduction

The fractional order differential equations has been gaining much attention due to their various applications in science and engineering such as fluid dynamics, heat conduction, control theory, electroanalytical chemistry, economics, fractal theory, fractional biological neurons, etc. It is proved that the fractional order differential equation is a better tool for the description of hereditary properties of various materials and processes than the corresponding integer order differential equation. With this advantage, fractional-order models have become more realistic and practical than the corresponding classical integer-order models, that is, there are more degrees of freedom in the fractional-order models. Westerlund [1] utilized the fractional differential equation to depict the transmission of electromagnetic waves; the one dimensional model is

$$\mu\varepsilon\frac{\partial^{2}E(x, t)}{\partial x^{2}}+ \mu\varepsilon\zeta D^{\nu}_{t}E(x, t)+\frac{\partial^{2}E(x, t)}{\partial t^{2}}=0, $$

where μ, ε, ζ are constants, \(D^{\nu}_{t}E(x, t)=\frac{\partial^{\nu}E(x, t)}{\partial t^{\nu}}\) is a fractional derivative.

This paper deals with the positive solutions for a class of singular fractional differential system involving the coupled integral boundary conditions

$$ \textstyle\begin{cases} D^{\alpha_{1} }_{0^{+}}u(t)+ f_{1}(t,u(t), v(t))=0,\qquad D^{\alpha_{2} }_{0^{+}}v(t)+ f_{2}(t,u(t), v(t))=0,\quad 0< t< 1,\\ u(0)=u'(0)=\cdots=u^{(n-2)}(0)=0, \qquad u(1)=\mu_{1}\int _{0}^{1}g_{1}(s)v(s)\,dA_{1}(s),\\ v(0)=v'(0)=\cdots= v^{(n-2)}(0)=0,\qquad v(1)=\mu_{2}\int_{0}^{1}g_{2}(s)u(s)\,dA_{2}(s), \end{cases} $$
(1.1)

where \(n-1<\alpha_{1}\), \(\alpha_{2}\leq n \), \(n\geq2\), \(D^{\alpha_{i}}_{0^{+}}\) is the standard Riemann-Liouville derivative. \(\mu_{i}>0\) is a constant, \(g_{i}:(0,1) \rightarrow[0, +\infty)\) is continuous with \(g_{i}\in L^{1}(0, 1)\), \(A_{i}\) is right continuous on \([0, 1)\), left continuous at \(t = 1\), and nondecreasing on \([0, 1]\), \(A_{i}(0) = 0\), \(\int_{0}^{1}x(s)\,dA_{i}(s)\) denotes the Riemann-Stieltjes integrals of x with respect to \(A_{i}\), \(f_{i}: (0,1)\times(0,+\infty)\times (0,+\infty) \rightarrow[0, +\infty)\) is a continuous function, \(f_{i}(t, u, v)\) may be singular at \(t = 0, 1\) and \(u=v=0\) (\(i=1, 2\)). The positive solution of the system (1.1) means that \((u, v) \in C[0, 1]\times C[0, 1]\), \((u, v)\) satisfies (1.1) and \(u(t)>0\), \(v(t)>0\), for all \(t\in(0, 1]\).

Many researchers have shown interest in fractional differential equations. The motivation for those works stems from both the intensive development of the theory of fractional calculus itself and the applications, so a large number of articles on fractional differential equations have appeared, for example, [215] have established the existence of solutions or positive solutions of initial or boundary value problems of some systems of nonlinear fractional differential equations by the use of techniques of nonlinear analysis (fixed point index theorems, Leray-Schauder theory, Guo-Krasnosel’skii fixed point theorem, the upper and lower solution method, Adomian decomposition method, etc.), for the case where \(\alpha_{i}\) is an integer, a lot of work has been done by many authors; see [16, 17] and the references therein. Wang et al. [18] studied the following system of nonlinear fractional differential equations:

$$ \textstyle\begin{cases} D^{\alpha}_{0^{+}}u(t)+f(t, v(t))=0,\qquad D^{\beta}_{0^{+}}v(t)+g(t, u(t))=0,\quad 0< t< 1,\\ u(0)=v{(0)}=0,\qquad u(1)=au(\xi),\qquad v(1)=bv(\xi), \end{cases} $$
(1.2)

where \(1 < \alpha, \beta< 2\), \(0\leq a, b < 1\), \(0 < \xi< 1\), \(f, g: [0, 1]\times[0, +\infty)\rightarrow[0, +\infty)\) are continuous functions, \(D^{\alpha}_{0^{+}}\), \(D^{\beta}_{0^{+}}\) are two standard Riemann-Liouville fractional derivatives. By using the Banach fixed point theorem and the nonlinear alternative of Leray-Schauder type, the existence and uniqueness of a positive solution for system (1.2) are obtained.

The system

$$ \textstyle\begin{cases} D^{\alpha}u(t)+a(t)f(t,v(t))=0,\qquad D^{\beta}v(t)+b(t)g(t,u(t))=0, \quad 0< t< 1,\\ u(0)=0,\qquad u(1)=\int_{0}^{1}\kappa(t)u(t)\,dt,\qquad v(0)=0,\qquad v(1)=\int_{0}^{1}\mu(t)v(t)\,dt, \end{cases} $$
(1.3)

was discussed in [19], in which \(1<\alpha, \beta\leq2\), \(D^{\alpha}\), \(D^{\beta}\) are standard Riemann-Liouville fractional derivatives, \(a, b :(0, 1)\rightarrow[0,+\infty)\) are continuous, \(\kappa, \mu: [0, 1]\rightarrow[0,+\infty)\) are nonnegative and integrable functions, and \(f, g : [0, 1]\times[0,+\infty)\rightarrow[0,+\infty)\) are continuous. By applying the Banach fixed point theorem and the fixed point theorems of cone expansion and compression of norm type, the authors in [19] get the sufficient conditions for the existence and nonexistence of positive solutions on system (1.3).

In this paper, we consider the system under the condition that \(f_{i}\) can be singular at \(t=0\) and \(u=v=0\). It is well known in linear elastic fracture mechanics that the stress near the crack tip exhibits a power singularity of \(r^{-0.5}\) [20], where r is the distance measured from the crack tip, this classical singularity also exists in nonlocal nonlinear problems. Owing to the singularity of \(f_{i}\), in this work, we shall devote our efforts to finding the suitable upper and lower solution of the system (1.1), and establishing the criterion of the existence of positive solutions for the system (1.1) by the virtue of the Schauder fixed point theorem. To our knowledge, very few authors studied the existence of positive solutions for the singular phenomena in coupled integral condition for fractional differential system, and this work improves and further develops results of previous work in this field to a certain degree.

2 Preliminaries and lemmas

The basic space used in this paper is \(X=C([0, 1], \mathbb{R})\times C([0, 1], \mathbb{R})\), where \(\mathbb{R}\) is a real number set, then X is a Banach space with the norm

$$\bigl\Vert (u,v)\bigr\Vert =\max\bigl\{ \Vert u\Vert , \Vert v\Vert \bigr\} ,\qquad \Vert u\Vert =\max_{t\in[0, 1]}\bigl\vert u(t)\bigr\vert , \qquad \Vert v\Vert =\max_{t\in[0, 1]}\bigl\vert v(t) \bigr\vert . $$

Now we begin our work based on the theory of fractional calculus, and for definitions and related properties of Riemann-Liouville fractional derivatives and integrals, we refer the reader to [21, 22]. In the following, we give the definition on the lower and upper solution of the system (1.1).

Definition 2.1

A pair of continuous functions \((\phi_{1} (t), \varphi_{1}(t))\) is called a lower solution of the system (1.1), if

$$\textstyle\begin{cases} D^{\alpha_{1} }_{0^{+}}\phi_{1} (t)+ f_{1}(t, \phi_{1} (t), \varphi_{1}(t))\geq0,\\ D^{\alpha_{2} }_{0^{+}}\varphi_{1}(t)+ f_{2}(t, \phi_{1} (t), \varphi_{1}(t))\geq0,\quad 0< t< 1,\\ \phi_{1} (0)=\phi_{1} '(0)=\cdots= \phi_{1} ^{(n-2)}(0)\geq0,\qquad \phi_{1} (1)\geq\mu_{1}\int_{0}^{1}g_{1}(s)\varphi_{1}(s)\,dA_{1}(s),\\ \varphi_{1}(0)=\varphi_{1}'(0)=\cdots= \varphi_{1}^{(n-2)}(0)\geq0,\qquad \varphi_{1}(1)\geq\mu_{2}\int_{0}^{1}g_{2}(s)\phi_{1} (s)\,dA_{2}(s). \end{cases} $$

Definition 2.2

A pair of continuous functions \((\phi_{2} (t), \varphi_{2}(t))\) is called an upper solution of the system (1.1), if it satisfies

$$\textstyle\begin{cases} D^{\alpha_{1} }_{0^{+}}\phi_{2} (t)+ f_{1}(t, \phi_{2} (t), \varphi_{2}(t))\leq0,\\ D^{\alpha_{2} }_{0^{+}}\varphi_{2}(t)+ f_{2}(t, \phi_{2} (t), \varphi_{2}(t))\leq0,\quad 0< t< 1,\\ \phi_{2} (0)=\phi_{2} '(0)=\cdots=\phi_{2} ^{(n-2)}(0)\leq0,\qquad \phi_{2} (1)\leq\mu_{1}\int_{0}^{1}g_{1}(s)\varphi_{2}(s)\,dA_{1}(s),\\ \varphi_{2}(0)=\varphi_{2}'(0)=\cdots= \varphi_{2}^{(n-2)}(0)\leq0,\qquad \varphi_{2}(1)\leq\mu_{2}\int_{0}^{1}g_{2}(s)\phi_{2} (s)\,dA_{2}(s). \end{cases} $$

Similarly to the proof in [5, 6], it enables us to obtain Lemmas 2.1 and 2.2.

Lemma 2.1

Assume condition (\(\mathbf{H}_{0}\)) holds:

(\(\mathbf{H}_{0}\)):
$$k_{1}= \int_{0}^{1}g_{1}(t)t^{\alpha_{2}-1} \,dA_{1}(t)>0,\qquad k_{2}= \int_{0}^{1}g_{2}(t)t^{\alpha _{1}-1} \,dA_{2}(t)>0, \qquad 1-\mu_{1}\mu_{2}k_{1}k_{2}>0. $$

Let \(h_{i}\in C(0, 1)\cap L(0, 1)\) (\(i=1, 2\)), then the system with the coupled boundary conditions

$$ \textstyle\begin{cases} D^{\alpha_{1}}_{0^{+}}u(t)+h_{1}(t)=0,\qquad D^{\alpha_{2}}_{0^{+}}v(t)+h_{2}(t)=0,\quad 0< t< 1,\\ u(0)=u'(0)=\cdots= u^{(n-2)}=0,\qquad u(1)=\mu_{1}\int _{0}^{1}g_{1}(s)v(s)\,dA_{1}(s),\\ v(0)=v'(0)=\cdots=v^{(n-2)}=0,\qquad v(1)=\mu_{2}\int_{0}^{1}g_{2}(s)u(s)\,dA_{2}(s), \end{cases} $$
(2.1)

has a unique integral representation,

$$ \textstyle\begin{cases} u(t)=\int_{0}^{1} K_{1}(t, s)h_{1}(s)\,ds+\int_{0}^{1} H_{1}(t, s)h_{2}(s)\,ds,\\ v(t)=\int_{0}^{1} K_{2}(t, s)h_{2}(s)\,ds+\int_{0}^{1} H_{2}(t, s)h_{1}(s)\,ds, \end{cases} $$
(2.2)

where

$$\begin{aligned}& \begin{aligned} &K_{1}(t, s)=\frac{\mu_{1}\mu_{2}k_{1}t^{\alpha_{1}-1}}{1-\mu_{1}\mu_{2}k_{1}k_{2}} \int_{0}^{1} g_{2}(t)\overline{G}_{1}(t, s)\,dA_{2}(t)+ \overline{G}_{1}(t, s), \\ &H_{1}(t, s)= \frac{\mu_{1}t^{\alpha_{1}-1}}{1-\mu_{1}\mu_{2}k_{1}k_{2}} \int_{0}^{1} g_{1}(t)\overline{G}_{2}(t, s)\,dA_{1}(t), \\ &K_{2}(t, s)=\frac{\mu_{2}\mu_{1}k_{2}t^{\alpha_{2}-1}}{1-\mu_{1}\mu_{2}k_{1}k_{2}} \int_{0}^{1}g_{1}(t) \overline{G}_{2}(t, s)\,dA_{1}(t)+ \overline{G}_{2}(t, s), \\ & H_{2}(t, s)= \frac{\mu_{2}t^{\alpha_{2}-1}}{1-\mu_{1}\mu_{2}k_{1}k_{2}} \int_{0}^{1} g_{2}(t)\overline{G}_{1}(t, s)\,dA_{2}(t), \end{aligned} \end{aligned}$$
(2.3)

and

$$\overline{G}_{i}(t,s)=\frac{1}{\Gamma(\alpha_{i})} \textstyle\begin{cases} [t(1-s)]^{\alpha_{i}-1}-(t-s)^{\alpha_{i}-1},& 0\leq s\leq t\leq1,\\ [t(1-s)]^{\alpha_{i}-1},& 0\leq t\leq s\leq1, \end{cases}\displaystyle \quad i=1,2. $$

Lemma 2.2

For \(t, s\in[0, 1]\), the functions \(K_{i}(t, s)\) and \(H_{i}(t, s)\) (\(i=1, 2\)) defined as (2.3) satisfy

$$\begin{aligned} &K_{1}(t, s), H_{1}(t, s)\leq\rho t^{\alpha_{1}-1}, \qquad K_{2}(t, s), H_{2}(t, s)\leq\rho t^{\alpha_{2}-1}, \\ &K_{1}(t, s)\geq\varrho t^{{\alpha_{1}-1}} s(1-s)^{\alpha_{1}-1},\qquad H_{2}(t, s)\geq\varrho t^{{\alpha_{2}-1}} s(1-s)^{\alpha_{1}-1}, \\ &K_{2}(t, s)\geq\varrho t^{{\alpha_{2}-1}} s(1-s)^{\alpha_{2}-1}, \qquad H_{1}(t, s)\geq\varrho t^{{\alpha_{1}-1}} s(1-s)^{\alpha_{2}-1}, \end{aligned}$$

where

$$\begin{aligned} &\rho=\max \left \{\begin{matrix} {}\frac{1}{\Gamma(\alpha_{1}-1)} (\frac{\mu_{1}\mu _{2}k_{1}}{1-\mu_{1}\mu_{2}k_{1}k_{2}} \int_{0}^{1} g_{2}(t)\,dA_{2}(t)+ 1 ), \frac{\mu_{1}}{\Gamma(\alpha_{2}-1)(1-\mu_{1}\mu_{2}k_{1}k_{2})}\int_{0}^{1} g_{1}(t)\,dA_{1}(t),\\ \frac{1}{\Gamma(\alpha_{2}-1)} (\frac{\mu_{2}\mu_{1}k_{2}}{1-\mu_{1}\mu _{2}k_{1}k_{2}}\int_{0}^{1} g_{1}(t)\,dA_{1}(t)+1 ), \frac{\mu_{2}}{\Gamma(\alpha_{1}-1)(1-\mu_{1}\mu_{2}k_{1}k_{2})}\int_{0}^{1} g_{2}(t)\,dA_{2}(t) \end{matrix} \right \}, \\ &\varrho=\min \left \{\begin{matrix} {}\frac{\mu_{1}\mu_{2}k_{1}}{\Gamma(\alpha_{1})(1-\mu_{1}\mu_{2}k_{1}k_{2})} \int_{0}^{1} (1-t)t^{\alpha_{1}-1}g_{2}(t)\,dA_{2}(t), \\ \frac{\mu_{2}\mu_{1}k_{2}}{\Gamma(\alpha_{2})(1-\mu_{1}\mu_{2}k_{1}k_{2})}\int_{0}^{1} (1-t)t^{\alpha_{2}-1}g_{1}(t)\,dA_{1}(t), \\ \frac{\mu_{1}}{\Gamma(\alpha_{2})(1-\mu_{1}\mu_{2}k_{1}k_{2})}\int_{0}^{1} (1-t)t^{\alpha_{2}-1}g_{1}(t)\,dA_{1}(t),\\ \frac{\mu_{2}}{\Gamma(\alpha_{1})(1-\mu_{1}\mu_{2}k_{1}k_{2})}\int_{0}^{1} (1-t)t^{\alpha_{1}-1}g_{2}(t)\,dA_{2}(t) \end{matrix} \right \}. \end{aligned}$$

Lemmas 2.1 and 2.2 can lead to the following maximum principle.

Lemma 2.3

If \((u, v)\in X\) satisfies

$$\begin{aligned}& u(0)=u'(0)=\cdots= u^{(n-2)}=0,\qquad u(1)=\mu_{1} \int_{0}^{1}g_{1}(s)v(s) \,dA_{1}(s), \\& v(0)=v'(0)=\cdots= v^{(n-2)}=0,\qquad v(1)=\mu_{2} \int_{0}^{1}g_{2}(s)u(s) \,dA_{2}(s), \end{aligned}$$

and \(D^{\alpha_{1}}_{0^{+}}u(t)\leq0\), \(D^{\alpha_{2}}_{0^{+}}v(t)\leq0\), \(0< t<1\), then \(u(t)\geq0\), \(v(t)\geq0\), \(0\leq t\leq1\).

3 Main results

We make the following assumptions throughout this paper:

(\(\mathbf{H}_{1}\)):

\(f_{i}\in C( (0,1)\times(0,+\infty)\times (0,+\infty), [0, +\infty))\) is decreasing in second and third variables and \(f_{i}(s, s^{\alpha_{1}-1}, s^{\alpha_{2}-1})\in L^{1}(0, 1)\), \(i=1, 2\).

(\(\mathbf{H}_{2}\)):

For all \(r\in(0, 1)\), there exist \(0<\varepsilon, \sigma<1\), for any \((t, x, y)\in(0,1)\times (0,+\infty)\times(0,+\infty)\),

$$f_{1}(t, rx, ry)\leq r^{-\varepsilon}f_{1}(t, x, y),\qquad f_{2}(t, rx, ry)\leq r^{-\sigma}f_{2}(t, x, y). $$

Theorem 3.1

Assume (\(\mathbf{H}_{0}\))-(\(\mathbf{H}_{2}\)) hold. Then system (1.1) has at least one positive solution \((u^{*}, v^{*})\), which satisfies \((L^{-1}t^{\alpha_{1}-1}, L^{-1}t^{\alpha_{2}-1})\leq(u^{*}, v^{*}) \leq(L t^{\alpha_{1}-1}, L t^{\alpha_{2}-1})\), where

$$L=\max \left \{\begin{matrix} (2\rho\int_{0}^{1} f_{1}(s, s^{\alpha_{1}-1}, s^{\alpha_{2}-1})\,ds )^{\frac{1}{1-\varepsilon}},\\ (2\rho\int_{0}^{1} f_{2}(s, s^{\alpha_{1}-1}, s^{\alpha_{2}-1})\,ds )^{\frac{1}{1-\sigma}}, \\ 1, (2\varrho\int_{0}^{1} s(1-s)^{\alpha_{1}-1}f_{1}(s, s^{\alpha_{1}-1}, s^{\alpha_{2}-1})\,ds )^{\frac{-1}{1-\varepsilon}},\\ (2\varrho\int_{0}^{1}s(1-s)^{\alpha_{2}-1} f_{2}(s, s^{\alpha_{1}-1}, s^{\alpha_{2}-1})\,ds )^{\frac{-1}{1-\sigma}} \end{matrix} \right \}. $$

In particular, if \(L= 1\), then \((t ^{\alpha_{1}-1}, t^{\alpha_{2}-1})\) is the positive solution of system (1.1).

Proof

Denote a cone as follows:

$$P= \bigl\{ (u, v)\in X: L^{-1}t^{\alpha_{1}-1}\leq u(t) \leq L t^{\alpha_{1}-1}, L^{-1}t^{\alpha_{2}-1}\leq v(t) \leq L t^{\alpha_{2}-1}, t\in[0, 1] \bigr\} , $$

then P is nonempty since \((t^{\alpha_{1}-1}, t^{\alpha_{2}-1})\in P\). Now let us denote an operator T by

$$ T(u,v) (t)=\bigl(T_{1}(u,v) (t), T_{2}(u,v) (t)\bigr), \quad 0\leq t \leq1, (u, v)\in P, $$
(3.1)

where

$$\begin{aligned}& \begin{aligned} &T_{1}(u,v) (t)= \int_{0}^{1} K_{1}(t, s)f_{1} \bigl(s, u(s), v(s)\bigr)\,ds + \int_{0}^{1} H_{1}(t, s)f_{2} \bigl(s, u(s), v(s)\bigr)\,ds, \\ &T_{2}(u,v) (t)= \int_{0}^{1} K_{2}(t, s)f_{2} \bigl(s, u(s), v(s)\bigr)\,ds + \int_{0}^{1} H_{2}(t, s)f_{1} \bigl(s, u(s), v(s)\bigr)\,ds. \end{aligned} \end{aligned}$$
(3.2)

We assert that T is well defined and \(T(P)\subset P\). In fact, for any \((u, v)\in P\), we have

$$L^{-1}t^{\alpha_{1}-1}\leq u(t) \leq L t^{\alpha_{1}-1},\qquad L^{-1}t^{\alpha_{2}-1}\leq v(t) \leq L t^{\alpha_{2}-1},\quad t\in[0, 1]. $$

Thus, by Lemma 2.2 and (\(\mathbf{H}_{0}\))-(\(\mathbf{H}_{2}\)), we get

$$\begin{aligned} T_{1}(u,v) (t) =& \int_{0}^{1} K_{1}(t, s)f_{1} \bigl(s, u(s), v(s)\bigr)\,ds + \int_{0}^{1} H_{1}(t, s)f_{2} \bigl(s, u(s), v(s)\bigr)\,ds \\ \leq& \int_{0}^{1} \rho t^{\alpha_{1}-1}f_{1} \bigl(s, L^{-1}s^{\alpha_{1}-1}, L^{-1}s^{\alpha_{2}-1}\bigr) \,ds \\ {}&+ \int_{0}^{1} \rho t^{\alpha_{1}-1}f_{2} \bigl(s, L^{-1}s^{\alpha_{1}-1}, L^{-1}s^{\alpha_{2}-1}\bigr) \,ds \\ \leq&L^{\varepsilon}\rho t^{\alpha_{1}-1} \int_{0}^{1} f_{1}\bigl(s, s^{\alpha_{1}-1}, s^{\alpha_{2}-1}\bigr)\,ds +L^{\sigma}\rho t^{\alpha_{1}-1} \int_{0}^{1} f_{2}\bigl(s, s^{\alpha_{1}-1}, s^{\alpha_{2}-1}\bigr)\,ds \\ \leq& L t^{\alpha_{1}-1} \end{aligned}$$
(3.3)

and

$$\begin{aligned} T_{2}(u,v) (t) =& \int_{0}^{1} K_{2}(t, s)f_{2} \bigl(s, u(s), v(s)\bigr)\,ds + \int_{0}^{1} H_{2}(t, s)f_{1} \bigl(s, u(s), v(s)\bigr)\,ds \\ \leq& \int_{0}^{1} \rho t^{\alpha_{2}-1}f_{2} \bigl(s, L^{-1}s^{\alpha_{1}-1}, L^{-1}s^{\alpha_{2}-1}\bigr) \,ds \\ &{} + \int_{0}^{1} \rho t^{\alpha_{2}-1}f_{1} \bigl(s, L^{-1}s^{\alpha_{1}-1}, L^{-1}s^{\alpha_{2}-1}\bigr) \,ds \\ \leq& L^{\sigma}\rho t^{\alpha_{2}-1} \int_{0}^{1} f_{2}\bigl(s, s^{\alpha_{1}-1}, s^{\alpha_{2}-1}\bigr)\,ds +L^{\varepsilon}\rho t^{\alpha_{2}-1} \int_{0}^{1} f_{1}\bigl(s, s^{\alpha_{1}-1}, s^{\alpha_{2}-1}\bigr)\,ds \\ \leq& Lt^{\alpha_{2}-1}. \end{aligned}$$
(3.4)

On the other hand, by Lemma 2.2 and (\(\mathbf{H}_{0}\))-(\(\mathbf{H}_{2}\)), we also get

$$\begin{aligned} T_{1}(u,v) (t) \geq& \int_{0}^{1} \varrho t^{\alpha_{1}-1}s(1-s)^{\alpha_{1}-1}f_{1} \bigl(s, L s^{\alpha_{1}-1}, L s^{\alpha_{2}-1}\bigr)\,ds \\ &{}+ \int_{0}^{1} \varrho t^{\alpha_{1}-1}s(1-s)^{\alpha_{2}-1}f_{2} \bigl(s, L s^{\alpha_{1}-1}, L s^{\alpha _{2}-1}\bigr)\,ds \\ \geq&L^{-\varepsilon}\varrho t^{\alpha_{1}-1} \int_{0}^{1} s(1-s)^{\alpha_{1}-1}f_{1} \bigl(s, s^{\alpha_{1}-1}, s^{\alpha_{2}-1}\bigr)\,ds \\ &{}+L^{-\sigma}\varrho t^{\alpha_{1}-1} \int_{0}^{1}s(1-s)^{\alpha_{2}-1} f_{2} \bigl(s, s^{\alpha_{1}-1}, s^{\alpha_{2}-1}\bigr)\,ds \\ \geq&L^{-1} t^{\alpha_{1}-1} \end{aligned}$$
(3.5)

and

$$\begin{aligned} T_{2}(u,v) (t) \geq& \int_{0}^{1} \varrho t^{\alpha_{2}-1}s(1-s)^{\alpha_{2}-1}f_{2} \bigl(s, Ls^{\alpha_{1}-1}, Ls^{\alpha_{2}-1}\bigr)\,ds \\ &{} + \int_{0}^{1} \varrho t^{\alpha_{2}-1}s(1-s)^{\alpha_{1}-1}f_{1} \bigl(s, L^{-1}s^{\alpha_{1}-1}, L^{-1}s^{\alpha_{2}-1}\bigr) \,ds \\ \geq& L^{-\sigma}\varrho t^{\alpha_{2}-1} \int_{0}^{1} s(1-s)^{\alpha_{2}-1}f_{2} \bigl(s, s^{\alpha_{1}-1}, s^{\alpha_{2}-1}\bigr)\,ds \\ &{} +L^{-\varepsilon}\varrho t^{\alpha_{2}-1} \int_{0}^{1} s(1-s)^{\alpha_{1}-1}f_{1} \bigl(s, s^{\alpha_{1}-1}, s^{\alpha_{2}-1}\bigr)\,ds \\ \geq&L^{-1}t^{\alpha_{2}-1}. \end{aligned}$$
(3.6)

It follows from (3.3)-(3.6) that T is well defined and \(T(P)\subset P\). Moreover, we have

$$ \textstyle\begin{cases} D^{\alpha_{1} }_{0^{+}}T_{1}(u, v)(t)+ f_{1}(t,T_{1}(u, v)(t), T_{2}(u, v)(t))=0,\\ D^{\alpha_{2} }_{0^{+}}T_{2}(u, v)(t)+ f_{2}(t,T_{1}(u, v)(t), T_{2}(u, v)(t))=0,\quad 0< t< 1,\\ T_{1}(u, v)(0)=T'_{1}(u, v)(0)=\cdots=T^{(n-2)}_{1}(u, v)(0)=0,\\ T_{1}(u, v)(1)=\mu_{1}\int_{0}^{1}g_{1}(s)T_{2}(u, v)(s)\,dA_{1}(s),\\ T_{2}(u, v)(0)=T'_{2}(u, v)(0)=\cdots= T^{(n-2)}_{2}(u, v)(0)=0,\\ T_{2}(u, v)(1)=\mu_{2}\int_{0}^{1}g_{2}(s)T_{1}(u, v)(s)\,dA_{2}(s). \end{cases} $$
(3.7)

Take

$$\begin{aligned} &\underline{\phi}(t)=\min\bigl\{ t^{\alpha_{1}-1}, T_{1} \bigl(t^{\alpha_{1}-1}, t^{\alpha_{2}-1}\bigr) \bigr\} ,\qquad \overline{\phi}(t)= \max\bigl\{ t^{\alpha_{1}-1}, T_{1}\bigl(t^{\alpha_{1}-1}, t^{\alpha_{2}-1}\bigr) \bigr\} , \end{aligned}$$
(3.8)
$$\begin{aligned} &\underline{\varphi}(t)=\min\bigl\{ t^{\alpha_{2}-1}, T_{2} \bigl(t^{\alpha_{1}-1}, t^{\alpha_{2}-1}\bigr) \bigr\} ,\qquad \overline{ \varphi}(t)=\max\bigl\{ t^{\alpha_{2}-1}, T_{2}\bigl(t^{\alpha_{1}-1}, t^{\alpha_{2}-1}\bigr) \bigr\} , \end{aligned}$$
(3.9)

since \((t^{\alpha_{1}-1}, t^{\alpha_{2}-1})\in P\), \((T_{1}(t^{\alpha_{1}-1}, t^{\alpha_{2}-1}), T_{2}(t^{\alpha_{1}-1}, t^{\alpha_{2}-1}))\in P\), we have

$$\begin{aligned} (\underline{\phi}, \underline{\varphi})\in P, (\overline{\phi}, \overline{ \varphi})\in P,\qquad \underline{\phi}(t)\leq t^{\alpha_{1}-1}\leq\overline{ \phi}(t),\qquad \underline{\varphi}(t)\leq t^{\alpha_{2}-1}\leq\overline{ \varphi}(t). \end{aligned}$$
(3.10)

Let

$$\begin{aligned} (\phi_{1}, \varphi_{1})= \bigl(T_{1}( \underline{\phi}, \underline{\varphi}), T_{2}(\underline{\phi}, \underline{\varphi}) \bigr),\qquad (\phi_{2}, \varphi_{2})= \bigl(T_{1}(\overline{\phi}, \overline{\varphi}), T_{2}( \overline{\phi}, \overline{\varphi}) \bigr). \end{aligned}$$
(3.11)

By (3.8)-(3.11) and (\(\mathbf{H}_{1}\)), we have

$$\begin{aligned}& \begin{aligned}[b] (\phi_{2}, \varphi_{2})&= \bigl(T_{1}(\overline{\phi}, \overline{\varphi}), T_{2}( \overline{\phi}, \overline{\varphi}) \bigr) \\ &\leq \bigl(T_{1}\bigl(t^{\alpha_{1}-1}, t^{\alpha_{2}-1}\bigr), T_{2}\bigl(t^{\alpha_{1}-1}, t^{\alpha_{2}-1}\bigr) \bigr)\leq \bigl(T_{1}(\underline{\phi}, \underline{\varphi}), T_{2}( \underline{\phi}, \underline{\varphi}) \bigr)=(\phi_{1}, \varphi_{1}), \end{aligned} \end{aligned}$$
(3.12)
$$\begin{aligned}& \begin{aligned} &(\phi_{2}, \varphi_{2})\leq \bigl(T_{1}\bigl(t^{\alpha_{1}-1}, t^{\alpha_{2}-1}\bigr), T_{2}\bigl(t^{\alpha_{1}-1}, t^{\alpha_{2}-1}\bigr) \bigr)\leq( \overline{\phi}, \overline{\varphi}), \\ &(\phi_{1}, \varphi_{1})\geq \bigl(T_{1} \bigl(t^{\alpha_{1}-1}, t^{\alpha_{2}-1}\bigr), T_{2} \bigl(t^{\alpha_{1}-1}, t^{\alpha_{2}-1}\bigr) \bigr)\geq(\underline{\phi}, \underline{\varphi}). \end{aligned} \end{aligned}$$
(3.13)

Consequently, it follows from (3.7) and (3.10)-(3.13) that

$$\begin{aligned}& \begin{aligned} &D^{\alpha_{1} }_{0^{+}} \phi_{1}(t)+ f_{1}\bigl(t, \phi_{1}(t), \varphi_{1}(t)\bigr)=D^{\alpha_{1} }_{0^{+}}T_{1}( \underline{\phi}, \underline{\varphi}) (t)+ f_{1}\bigl(t, \phi_{1}(t), \varphi_{1}(t)\bigr) \\ &\hphantom{D^{\alpha_{1} }_{0^{+}}\phi_{1}(t)+ f_{1}(t, \phi_{1}(t), \varphi_{1}(t))}=-f_{1}\bigl(t, \underline{\phi}(t), \underline{ \varphi}(t)\bigr)+ f_{1}\bigl(t, \phi_{1}(t), \varphi_{1}(t)\bigr) \\ &\hphantom{D^{\alpha_{1} }_{0^{+}}\phi_{1}(t)+ f_{1}(t, \phi_{1}(t), \varphi_{1}(t))}\leq-f_{1}\bigl(t, \underline{\phi}(t), \underline{ \varphi}(t)\bigr)+f_{1}\bigl(t, \underline{\phi}(t), \underline{ \varphi}(t)\bigr)=0, \\ &\phi_{1}(0)=\phi'_{1}(0)=\cdots = \phi^{(n-2)}_{1}(0)=0,\qquad \phi_{1}(1)= \mu_{1} \int_{0}^{1}g_{1}(s)\varphi_{1}(s) \,dA_{1}(s), \end{aligned} \end{aligned}$$
(3.14)
$$\begin{aligned}& \begin{aligned} & D^{\alpha_{2} }_{0^{+}} \varphi_{1}(t)+ f_{2}\bigl(t, \phi_{1}(t), \varphi_{1}(t)\bigr)=D^{\alpha_{2} }_{0^{+}}T_{2}( \underline{\phi}, \underline{\varphi}) (t)+ f_{2}\bigl(t, \phi_{1}(t), \varphi_{1}(t)\bigr) \\ &\hphantom{D^{\alpha_{2} }_{0^{+}}\varphi_{1}(t)+ f_{2}(t, \phi_{1}(t), \varphi_{1}(t))}=-f_{2}\bigl(t, \underline{\phi}(t), \underline{ \varphi}(t)\bigr)+ f_{2}\bigl(t, \phi_{1}(t), \varphi_{1}(t)\bigr) \\ &\hphantom{D^{\alpha_{2} }_{0^{+}}\varphi_{1}(t)+ f_{2}(t, \phi_{1}(t), \varphi_{1}(t))}\leq-f_{2}\bigl(t, \underline{\phi}(t), \underline{ \varphi}(t)\bigr)+f_{2}\bigl(t, \underline{\phi}(t), \underline{ \varphi}(t)\bigr)=0, \\ &\varphi_{1}(0)=\varphi'_{1}(0)=\cdots= \varphi^{(n-2)}_{1}(0)=0,\qquad \varphi_{1}(1)= \mu_{2} \int_{0}^{1}g_{2}(s)\phi_{1}(s) \,dA_{2}(s), \end{aligned} \end{aligned}$$
(3.15)

and

$$\begin{aligned}& \begin{aligned} & D^{\alpha_{1} }_{0^{+}} \phi_{2}(t)+ f_{1}\bigl(t, \phi_{2}(t), \varphi_{2}(t)\bigr)=D^{\alpha_{1} }_{0^{+}}T_{1}( \overline{\phi}, \overline{\varphi}) (t)+ f_{1}\bigl(t, \phi_{2}(t), \varphi_{2}(t)\bigr) \\ &\hphantom{D^{\alpha_{1} }_{0^{+}}\phi_{2}(t)+ f_{1}(t, \phi_{2}(t), \varphi_{2}(t))}=-f_{1}\bigl(t, \overline{\phi}(t), \overline{ \varphi}(t)\bigr)+ f_{1}\bigl(t, \phi_{2}(t), \varphi_{2}(t)\bigr) \\ &\hphantom{D^{\alpha_{1} }_{0^{+}}\phi_{2}(t)+ f_{1}(t, \phi_{2}(t), \varphi_{2}(t))}\geq-f_{1}\bigl(t, \overline{\phi}(t), \overline{ \varphi}(t)\bigr)+f_{1}\bigl(t, \overline{\phi}(t), \overline{ \varphi}(t)\bigr)=0, \\ & \phi_{2}(0)=\phi'_{2}(0)=\cdots = \phi^{(n-2)}_{2}(0)=0,\qquad \phi_{2}(1)= \mu_{1} \int_{0}^{1}g_{1}(s)\varphi_{2}(s) \,dA_{1}(s), \end{aligned} \end{aligned}$$
(3.16)
$$\begin{aligned}& \begin{aligned} &D^{\alpha_{2} }_{0^{+}} \varphi_{2}(t)+ f_{2}\bigl(t, \phi_{2}(t), \varphi_{2}(t)\bigr)=D^{\alpha_{2} }_{0^{+}}T_{2}( \overline{\phi}, \overline{\varphi}) (t)+ f_{2}\bigl(t, \phi_{2}(t), \varphi_{2}(t)\bigr) \\ &\hphantom{D^{\alpha_{2} }_{0^{+}}\varphi_{2}(t)+ f_{2}(t, \phi_{2}(t), \varphi_{2}(t))}=-f_{2}\bigl(t, \overline{\phi}(t), \overline{ \varphi}(t)\bigr)+ f_{2}\bigl(t, \phi_{2}(t), \varphi_{2}(t)\bigr) \\ &\hphantom{D^{\alpha_{2} }_{0^{+}}\varphi_{2}(t)+ f_{2}(t, \phi_{2}(t), \varphi_{2}(t))}\geq-f_{2}\bigl(t, \overline{\phi}(t), \overline{ \varphi}(t)\bigr)+f_{2}\bigl(t, \overline{\phi}(t), \overline{ \varphi}(t)\bigr)=0, \\ & \varphi_{2}(0)=\varphi'_{2}(0)=\cdots= \varphi^{(n-2)}_{2}(0)=0,\qquad \varphi_{2}(1)= \mu_{2} \int_{0}^{1}g_{2}(s)\phi_{2}(s) \,dA_{2}(s). \end{aligned} \end{aligned}$$
(3.17)

From (3.12), (3.14)-(3.17), we obtain \((\phi_{2}, \varphi_{2} )\), \((\phi_{1}, \varphi_{1})\) are lower and upper solutions of the system (1.1), and \((\phi_{2}, \varphi_{2} ), (\phi_{1}, \varphi_{1})\in P\).

Define the function \(F_{1}\), \(F_{2}\) by

$$ F_{i}(t, u, v)= \textstyle\begin{cases} f_{i}(t, \phi_{2}(t),\varphi_{2}(t)),\\ \quad 0< u< \phi_{2}(t), 0< v< \varphi_{1}(t), \mbox{ or } 0< u< \phi_{1}(t), 0< v< \varphi_{2}(t), \\ f_{i}(t, u, v),\quad \phi_{2}(t)\leq u\leq\phi_{1}(t), \varphi_{2}(t)\leq v\leq\varphi_{1}(t), \\ f_{i}(t, \phi_{1}(t), \varphi_{1}(t)),\quad u> \phi_{1}(t), \mbox{ or } v>\varphi_{1}(t), i=1, 2.\end{cases} $$
(3.18)

It then follows from (\(\mathbf{H}_{1}\)) and (3.18) that \(F_{i}: (0, 1)\times[0, +\infty)\times[0, +\infty)\rightarrow[0, +\infty)\) is continuous. We now show that the fractional boundary value system

$$ \textstyle\begin{cases} D^{\alpha_{1} }_{0^{+}}u(t)+ F_{1}(t,u(t), v(t))=0,\qquad D^{\alpha_{2} }_{0^{+}}v(t)+ F_{2}(t,u(t), v(t))=0,\quad 0< t< 1,\\ u(0)=u'(0)=\cdots=u^{(n-2)}(0)=0,\qquad u(1)=\mu_{1}\int _{0}^{1}g_{1}(s)v(s)\,dA_{1}(s),\\ v(0)=v'(0)=\cdots= v^{(n-2)}(0)=0,\qquad v(1)=\mu_{2}\int_{0}^{1}g_{2}(s)u(s)\,dA_{2}(s), \end{cases} $$
(3.19)

has a positive solution. Define the operator in X

$$\widetilde{T}(u,v) (t)=\bigl(\widetilde{T}_{1}(u,v) (t), \widetilde{T}_{2}(u,v) (t)\bigr),\quad 0\leq t \leq1, $$

where

$$\begin{aligned}& \widetilde{T}_{1}(u,v) (t)= \int_{0}^{1} K_{1}(t, s)F_{1} \bigl(s, u(s), v(s)\bigr)\,ds + \int_{0}^{1} H_{1}(t, s)F_{2} \bigl(s, u(s), v(s)\bigr)\,ds, \\& \widetilde{T}_{2}(u,v) (t)= \int_{0}^{1} K_{2}(t, s)F_{2} \bigl(s, u(s), v(s)\bigr)\,ds + \int_{0}^{1} H_{2}(t, s)F_{1} \bigl(s, u(s), v(s)\bigr)\,ds. \end{aligned}$$

Then a fixed point of the operator is a solution of the system (3.19). For any \((u, v)\in X\), by (3.18), we have

$$\begin{aligned} \widetilde{T}_{1}(u,v) (t) \leq& \int_{0}^{1} \rho t^{\alpha_{1}-1}F_{1} \bigl(s, u(s), v(s)\bigr)\,ds + \int_{0}^{1} \rho t^{\alpha_{1}-1}F_{2} \bigl(s, u(s), v(s)\bigr)\,ds \\ \leq& \int_{0}^{1} \rho f_{1}\bigl(s, \phi_{2}(s), \varphi_{2}(s)\bigr)\,ds + \int_{0}^{1} \rho f_{2}\bigl(s, \phi_{2}(s), \varphi_{2}(s)\bigr)\,ds \\ \leq& \int_{0}^{1} \rho f_{1}\bigl(s, L^{-1}s^{\alpha_{1}-1}, L^{-1}s^{\alpha_{2}-1}\bigr)\,ds + \int_{0}^{1} \rho f_{2}\bigl(s, L^{-1}s^{\alpha_{1}-1}, L^{-1}s^{\alpha_{2}-1}\bigr)\,ds \\ \leq&L^{\varepsilon}\rho \int_{0}^{1} f_{1}\bigl(s, s^{\alpha_{1}-1}, s^{\alpha_{2}-1}\bigr)\,ds +L^{\sigma}\rho \int_{0}^{1} f_{2}\bigl(s, s^{\alpha_{1}-1}, s^{\alpha_{2}-1}\bigr)\,ds< +\infty \end{aligned}$$

and

$$\begin{aligned} \widetilde{T}_{2}(u,v) (t) \leq& \int_{0}^{1} \rho t^{\alpha_{2}-1}F_{2} \bigl(s, u(s), v(s)\bigr)\,ds + \int_{0}^{1} \rho t^{\alpha_{2}-1}F_{1} \bigl(s, u(s), v(s)\bigr)\,ds \\ \leq& \int_{0}^{1} \rho f_{2}\bigl(s, \phi_{2}(s), \varphi_{2}(s)\bigr)\,ds + \int_{0}^{1} \rho f_{1}\bigl(s, \phi_{2}(s), \varphi_{2}(s)\bigr)\,ds \\ \leq& \int_{0}^{1} \rho f_{2}\bigl(s, L^{-1}s^{\alpha_{1}-1}, L^{-1}s^{\alpha_{2}-1}\bigr)\,ds + \int_{0}^{1} \rho f_{1}\bigl(s, L^{-1}s^{\alpha_{1}-1}, L^{-1}s^{\alpha_{2}-1}\bigr)\,ds \\ \leq&L^{\sigma}\rho \int_{0}^{1} f_{2}\bigl(s, s^{\alpha_{1}-1}, s^{\alpha_{2}-1}\bigr)\,ds +L^{\varepsilon}\rho \int_{0}^{1} f_{1}\bigl(s, s^{\alpha_{1}-1}, s^{\alpha_{2}-1}\bigr)\,ds< +\infty. \end{aligned}$$

So \(\Vert \widetilde{T}\Vert =\max\{\Vert \widetilde{T}_{1}\Vert , \Vert \widetilde{T}_{2}\Vert \} <+\infty\); this implies that \(\Vert \widetilde{T}\Vert \) is uniformly bounded. In addition, it follows from the continuity of \(F_{1}\), \(F_{2}\), the uniform continuity of \(K_{i}(t, s)\), \(H_{i}(t, s)\) on \([0, 1]\times[0, 1]\), and (\(\mathbf{H}_{1}\)) that \(\widetilde{T}: X \rightarrow X\) is continuous.

Let \(\Omega\in X\times X\) be bounded, by a standard discussion and the Arzela-Ascoli theorem, we know \(\widetilde{T}_{i}(\Omega)\) is equicontinuous. Thus \(\widetilde{T}: X \rightarrow X\) is completely continuous, by using the Schauder fixed point theorem, has at least a fixed point \((u^{*}, v^{*} )\) such that \((u^{*}, v^{*} )=\widetilde{T}(u^{*}, v^{*})\).

Now we prove

$$ \phi_{2}(t)\leq u^{*}(t)\leq\phi_{1}(t),\qquad \varphi_{2}(t)\leq v^{*}(t)\leq\varphi_{1}(t),\quad t\in[0, 1]. $$
(3.20)

We first of all prove \(u^{*}(t)\leq\phi_{1}(t)\), \(v^{*}(t)\leq\varphi_{1}(t)\). Otherwise, suppose \(u^{*}(t)>\phi_{1}(t)\), \(v^{*}(t)> \varphi_{1}(t)\). According to the definition of \(F_{i}\), we have

$$\begin{aligned}& \begin{aligned} D^{\alpha_{1} }_{0^{+}}u^{*}(t)+ F_{1}\bigl(t,u^{*}(t), v^{*}(t)\bigr)=D^{\alpha_{1} }_{0^{+}}u^{*}(t)+ f_{1}\bigl(t,\phi_{1}(t), \varphi_{1}(t)\bigr)=0, \\ D^{\alpha_{2} }_{0^{+}}v^{*}(t)+ F_{2}\bigl(t,u^{*}(t), v^{*}(t) \bigr)=D^{\alpha_{2} }_{0^{+}}v^{*}(t)+ f_{2}\bigl(t, \phi_{1}(t), \varphi_{1}(t)\bigr)=0. \end{aligned} \end{aligned}$$
(3.21)

On the other hand, since \((\phi_{1}(t), \varphi_{1}(t))\) is an upper solution of system (1.1), we have

$$ D^{\alpha_{1} }_{0^{+}}\phi_{1}(t)+ f_{1}\bigl(t,\phi_{1}(t), \varphi_{1}(t)\bigr) \leq0,\qquad D^{\alpha_{2} }_{0^{+}}\varphi_{1}(t)+ f_{2}\bigl(t, \phi_{1}(t), \varphi_{1}(t)\bigr) \leq0. $$
(3.22)

Let \(x(t)=\phi_{1}(t)-u^{*}(t)\), \(y(t)=\varphi_{1}(t)-v^{*}(t)\), by (3.21)-(3.22), we get

$$D^{\alpha_{1} }_{0^{+}}x(t)=D^{\alpha_{1} }_{0^{+}} \phi_{1}(t)-D^{\alpha_{1} }_{0^{+}}u^{*}(t)\leq0,\qquad D^{\alpha_{2} }_{0^{+}}y(t)=D^{\alpha_{2} }_{0^{+}} \varphi_{1}(t)-D^{\alpha_{2} }_{0^{+}}v^{*}(t)\leq0. $$

Since \((\phi_{1}(t), \varphi_{1}(t))\) is upper solution of the system (1.1) and \((u^{*}, v^{*} )\) is the fixed point of , we know

$$\begin{aligned}& x(0)=x'(0)=\cdots=x^{(n-2)}(0)=0,\qquad x(1)= \mu_{1} \int _{0}^{1}g_{1}(s)y(s) \,dA_{1}(s), \\& y(0)=y'(0)=\cdots= y^{(n-2)}(0)=0,\qquad y(1)= \mu_{2} \int_{0}^{1}g_{2}(s)x(s) \,dA_{2}(s). \end{aligned}$$

It follows from Lemma 2.3 that \(x(t)\geq0\), \(y(t)\geq0\), i.e., \(u^{*}(t)\leq\phi_{1}(t)\), \(v^{*}(t)\leq\varphi_{1}(t)\) on \([0, 1]\), which contradicts \(u^{*}(t)> \phi_{1}(t)\), \(v^{*}(t)>\varphi_{1}(t)\) on \([0, 1]\). In the same way, we obtain \(u^{*}(t)\geq\phi_{2}(t)\), \(v^{*}(t)\geq \varphi_{2}(t)\) on \([0, 1]\). Consequently, (3.20) is satisfied, then \((u^{*}, v^{*})\) is a positive solution of the system (1.1).

Owing to the fact \((\phi_{2}, \varphi_{2}), (\phi_{1}, \varphi_{1})\in P\) and (3.20), we get \((L^{-1}t^{\alpha_{1}-1}, L^{-1}t^{\alpha_{2}-1} )\leq(u^{*}, v^{*})\leq(Lt^{\alpha_{1}-1}, Lt^{\alpha_{2}-1} )\). The proof is completed. □

Theorems 3.2 and 3.3 can be obtained by the same method as Theorem 3.1.

Theorem 3.2

Assume (\(\mathbf{H}_{0}\)) holds and for \(i=1, 2\), \(f_{i}\) satisfies:

(\(\mathbf{H}^{*}_{1}\)):

\(f_{i}\in C( (0,1)\times[0,+\infty)\times [0,+\infty), [0, +\infty))\) is decreasing in second and third variables and

$$0< \int_{0}^{1}f_{i}(t, 0, 0)< +\infty. $$

Then system (1.1) has at least one positive solution \((u^{*}, v^{*})\), which satisfies \((0, 0)\leq(u^{*}, v^{*}) \leq(L t^{\alpha_{1}-1}, L t^{\alpha_{2}-1})\), where

$$L=\max\biggl\{ 2\rho \int_{0}^{1} f_{1}(s, 0, 0)\,ds, 2\rho \int_{0}^{1} f_{2}(s, 0, 0)\,ds \biggr\} . $$

Theorem 3.3

Assume (\(\mathbf{H}_{0}\)) holds and, for \(i=1, 2\), \(f_{i}\) satisfies:

(\(\mathbf{H}'_{1}\)):

\(f_{i}\in C( [0,1]\times[0,+\infty)\times [0,+\infty), [0, +\infty))\) is decreasing in second and third variables, \(f_{i}(t, 0, 0)\not\equiv0\), \(t\in[0, 1]\).

Then system (1.1) has at least one positive solution \((u^{*}, v^{*})\), which satisfies \((0, 0)\leq(u^{*}, v^{*}) \leq(L t^{\alpha_{1}-1}, L t^{\alpha_{2}-1})\), where

$$L=\max\biggl\{ 2\rho \int_{0}^{1} f_{1}(s, 0, 0)\,ds, 2\rho \int_{0}^{1} f_{2}(s, 0, 0)\,ds \biggr\} . $$

4 Example

Consider the fractional differential system

$$ \textstyle\begin{cases} D^{\frac{5}{2}}_{0^{+}}u(t)+ f_{1}(t,u(t), v(t)) =0,\qquad D^{\frac{5}{2}}_{0^{+}}v(t)+ f_{2}(t,u(t), v(t))=0, \quad 0< t< 1, \\ u(0)=u'(0)=0,\qquad u(1)=\frac{1}{2}\int_{0}^{1}t^{-\frac{1}{2}}v(t)\,dt,\\ v(0)=v'(0)=0,\qquad v(1)=\int_{0}^{1}u(t)\,dt^{\frac{1}{2}}, \end{cases} $$
(4.1)

where \(\alpha_{1}=\alpha_{2}=\frac{5}{2}\), \(\mu_{1}=\frac{1}{2}\), \(\mu_{2}=1\), \(A_{1}(t)=t\), \(A_{2}(t)=t^{\frac{1}{2}}\), \(g_{1}(t)=t^{-\frac{1}{2}}\), \(g_{2}(t)=1\),

$$\begin{aligned}& k_{1}= \int_{0}^{1}g_{1}(t)t^{\alpha_{2}-1} \,dA_{1}(t)= \int_{0}^{1}t^{-\frac{1}{2}}t^{\frac{3}{2}}\,dt= \frac{1}{2}>0, \\& k_{2}= \int_{0}^{1}g_{2}(t)t^{\alpha_{1}-1} \,dA_{2}(t)= \int_{0}^{1}t^{\frac{3}{2}}\,dt^{\frac{1}{2}} = \frac{1}{2} \int_{0}^{1}t\,dt=\frac{1}{4}>0, \\& 1-\mu_{1}\mu_{2}k_{1}k_{2}= \frac{15}{16}>0. \end{aligned}$$

So, the condition (\(\mathbf{H}_{0}\)) holds. Let

$$f_{i}(t, u, v)=t^{-\gamma_{i}}\bigl(u^{-\delta_{i}}+v^{-\epsilon_{i}} \bigr),\quad 0< \gamma_{i}+\frac{3}{2}\delta_{i}< 1, 0< \gamma_{i}+\frac{3}{2}\epsilon_{i}< 1, i=1, 2, $$

then \(f_{i}\) is decreasing in u and v, and \(f_{i}(s, s^{\alpha_{1}-1}, s^{\alpha_{2}-1})=s^{-\gamma_{i}-\frac{3}{2}\delta_{i}}+s^{-\gamma_{i}-\frac {3}{2}\delta_{i}}\in L^{1}(0, 1)\), \(i=1, 2\). Moreover, for all \(r\in(0, 1)\), \((t, u, v)\in (0,1)\times(0,+\infty)\times(0,+\infty)\),

$$f_{i}(t, ru, rv)\leq r^{-\max\{\delta_{i}, -\epsilon_{i}\}}f_{i}(t, u, v),\quad i=1, 2. $$

So, all the conditions of Theorem 3.1 hold, and, by Theorem 3.1, the system (4.1) at least has a positive solution.