1 Introduction

Fractional-order differential equations is a natural generalization of the case of integer order, which has become the focus of attention involving various kinds of boundary conditions because of the wide application in mathematical models and applied sciences. Some latest results on the topic can be found in a series of papers [115] and the references therein. In particular, a monotone iterative technique is believed to be an efficient and important method to deal with sequences of monotone solutions for initial and boundary value problems. For some applications of this method to nonlinear fractional differential equations, see [1624]. We also note that there are some results about monotone iterative solution of a single fractional order equation on a half-line, see [2529].

In [25] Zhang considered a nonlinear fractional boundary value problem on a half-line

$$ \textstyle\begin{cases} D^{\alpha }u(t)+f(t,u(t),D^{\alpha -1}u(t))=0, \quad t\in {(0,+ \infty )}, \alpha \in (1,2], \\ u(0)=0,\qquad D^{\alpha -1}u(+\infty )=\beta u(\xi ),\qquad \beta >0, \end{cases} $$
(1.1)

where \(f\in C(J\times \mathbb{R}\times \mathbb{R},\mathbb{R})\). By utilizing the monotone iterative technique, iterative sequences of the positive extremal solutions were acquired.

In [27] Pei et al. studied the Hadamard fractional integro-differential equation on an infinite interval

$$ \textstyle\begin{cases} D^{\alpha } u(t)+f (t, u(t),I^{r} u(t), D^{\alpha -1} u(t) )=0, \quad 1< \alpha < 2, t \in (1,+\infty ), \\ u(1)=0,\qquad D^{\alpha -1} u(\infty )=\sum_{i=1}^{m} \lambda _{i} I^{ \beta _{i}} u(\eta ), \end{cases} $$
(1.2)

where \(I^{(\cdot )}\) is the Hadamard fractional integral and α, η, \(\beta _{i}\), \(\lambda _{i}\) (\(i=1,2,\ldots,m\)) are some given constants with \(\varGamma (\alpha )>\sum_{i=1}^{m} \frac{\lambda _{i} \varGamma (\alpha )}{\varGamma (\alpha +\beta _{i} )}(\log \eta )^{\alpha +\beta _{i}-1}\). By using the monotone iterative technique, the existence of positive solutions was established.

On the other hand, integral boundary conditions are considered to be more reasonable than the local boundary conditions, which can depict phenomena of heat transmission, population dynamics, blood flow, etc. A large number of results about fractional differential equations with integral boundary condition have been obtained, see [9, 10, 3043] and the references cited therein. Meanwhile, we note that the coupled systems of fractional-order differential equations have also attracted much attention due to their extensive applications, we refer to [3, 9, 10, 14, 22, 29, 3243].

In [9] Jiang et al. utilized the fixed point index to construct the existence of positive solutions for the following system on a finite interval:

$$ \textstyle\begin{cases} {D^{\beta } u(t)+f_{1}(t, u(t), v(t))=0,} \quad {1< t< e,} \\ {D^{\beta } v(t)+f_{2}(t, u(t), v(t))=0,} \quad {1< t< e,} \\ {u(1)=v(1)=u^{\prime }(1)=v^{\prime }(1)=0,} \\ {u(e)=\int _{1}^{e} h(s) v(s) \frac{d s}{s},} \\ {v(e)=\int _{1}^{e} g(s) u(s) \frac{d s}{s},} \end{cases} $$
(1.3)

where the nonlinearities \(f_{i}\) (\(i=1,2\)) can grow superlinearly and sublinearly, and boundary value conditions depend on the coupled integral of unknown functions.

In [29] a coupled system of fractional differential equations on an infinite interval is studied

$$ \textstyle\begin{cases} D^{\alpha }u(t)+\varphi (t,v(t),D^{\gamma _{1}}v(t))=0, \quad \alpha \in (2,3], \gamma _{1}\in (0,1), \\ D^{\beta }v(t)+\psi (t,u(t),D^{\gamma _{2}}u(t))=0, \quad \beta \in (2,3], \gamma _{2}\in (0,1), \\ I^{3-\alpha }u(0)=0,\qquad D^{\alpha -2}u(0)=\int _{0}^{h}g_{1}(s)u(s)\,\mathrm{d}s, \qquad D^{\alpha -1}u(+\infty )=M u(\xi )+a, \\ I^{3-\beta }v(0)=0,\qquad D^{\beta -2}v(0)=\int _{0}^{h}g_{2}(s)v(s)\,\mathrm{d}s, \qquad D^{\beta -1}v(+\infty )=N v(\eta )+b, \end{cases} $$
(1.4)

where \(t\in J=[0,+\infty )\), \(\varphi ,\psi \in C(J\times \mathbb{R}\times \mathbb{R},J)\), M, N are real numbers satisfying \(0< M\xi ^{\alpha -1}<\varGamma (\alpha )\), \(0< N\eta ^{\beta -1}<\varGamma ( \beta )\), \(\xi ,\eta , h>0\), and \(a,b\in \mathbb{\mathbb{R}^{+}}\), \(g_{1},g_{2}\in L^{1}[0,h]\) are nonnegative functions, the nonlinear terms φ, ψ and boundary conditions of the system are not coupled.

In [32] Aljoudi et al. studied the sequential fractional differential equations on a finite interval

$$ \textstyle\begin{cases} {(D^{q}+kD^{q-1})u(t)=f(t, u(t), v(t),D^{\alpha }v(t)),\quad k>0,} \\ { 1< q\leq 2,\qquad 0< \alpha < 1,} \\ {(D^{p}+kD^{p-1})u(t)=g(t, u(t), v(t),D^{\delta }u(t)),} \\ { 1< p\leq 2,\qquad 0< \delta < 1,} \\ {u(1)=0,\qquad u(e)=I^{\gamma }v(\eta ),} \qquad {\gamma >0,\qquad 1< \eta < e,} \\ {v(1)=0,\qquad v(e)=I^{\beta }u(\xi ),} \qquad {\beta >0,\qquad 1< \xi < e,} \end{cases} $$
(1.5)

where \(D^{(\cdot )}\) and \(I^{(\cdot )}\) denote the Hadamard fractional derivative and Hadamard fractional integral, \(f, g : [1, e]\times \mathbb{R}^{3} \rightarrow \mathbb{R}\) are given continuous functions, and boundary value conditions depend on the coupled fractional integral of unknown functions.

Inspired by the works above, in this paper we utilize the monotone iterative technique to study the existence of positive extremal solutions of a fractional differential system on a half-line

$$ \textstyle\begin{cases} D^{\alpha }u(t)+\varphi (t,u(t),v(t),D^{\beta -1}v(t))=0,\quad 2< \alpha \leq 3, \\ D^{\beta }v(t)+\psi (t,u(t),v(t),D^{\alpha -1}u(t))=0,\quad 2< \beta \leq 3, \\ u(0)=u'(0)=0,\qquad D^{\alpha -1}u(+\infty )= \int _{0}^{+\infty }h(t)v(t) \,\mathrm{d}t, \\ v(0)=v'(0)=0,\qquad D^{\beta -1}v(+\infty )= \int _{0}^{+\infty }g(t)u(t) \,\mathrm{d}t, \end{cases} $$
(1.6)

where \(D^{\alpha }\), \(D^{\beta }\) are the Riemann–Liouville fractional derivatives. Here we emphasize that the nonlinearity terms φ, ψ include not only unknown functions, but also the lower-order fractional derivative of unknown functions. By the way, the fractional infinite boundary value conditions depend on the coupled infinite integral of unknown functions. To the best of the authors’ knowledge, the system with coupled infinite integral boundary value conditions is yet to be investigated. φ, ψ satisfy the following assumptions:

(C0):

\(\varphi ,\psi \in C( J \times \mathbb{R}\times \mathbb{R}\times \mathbb{R} ,J)\), \(J=[0,+\infty )\).

(C1):

\(h(t),g(t)\in L[0,+\infty )\) with \(\int ^{+\infty }_{0}h(t)t^{\beta -1}\,\mathrm{d}t=\varUpsilon _{1}\), \(\int ^{+ \infty }_{0}g(t)t^{\alpha -1}\,\mathrm{d}t=\varUpsilon _{2}\), \(\varUpsilon _{1} \varUpsilon _{2}< \varGamma (\alpha )\varGamma (\beta )\).

(C2):

The nonnegative functions \(a_{i}(t),b_{i}(t)\in L[0,+\infty )\) (\(i=0,1,2,3\)) and constants \(0\leq \lambda _{k}\), \(\tau _{k}<1\) (\(k=1,2,3\)) satisfy

$$ \bigl\vert \varphi (t,u,v,w) \bigr\vert \leq a_{0}(t)+a_{1}(t) \vert u \vert ^{\lambda _{1}}+a_{2}(t) \vert v \vert ^{ \lambda _{2}}+a_{3}(t) \vert w \vert ^{\lambda _{3}},\quad u,v,w\in \mathbb{R}, \forall t\in J, $$

with

$$\begin{aligned}& \int ^{+\infty }_{0}a_{0}(t) \,\mathrm{d}t=a_{0}^{\ast }< +\infty ,\qquad \int ^{+ \infty }_{0}a_{1}(t) \bigl(1+t^{\alpha +\beta -1}\bigr)^{\lambda _{1}}\,\mathrm{d}t=a_{1}^{ \ast }< + \infty , \\& \int ^{+\infty }_{0}a_{2}(t) \bigl(1+t^{\alpha +\beta -1}\bigr)^{\lambda _{2}} \,\mathrm{d}t=a_{2}^{\ast }< + \infty ,\qquad \int ^{+\infty }_{0}a_{3}(t) \,\mathrm{d}t=a_{3}^{ \ast }< +\infty , \end{aligned}$$

and

$$ \bigl\vert \psi (t,u,v,z) \bigr\vert \leq b_{0}(t)+b_{1}(t) \vert u \vert ^{\tau _{1}}+b_{2}(t) \vert v \vert ^{ \tau _{2}}+a_{3}(t) \vert z \vert ^{\tau _{3}},\quad u,v,z\in \mathbb{R}, \forall t \in J, $$

with

$$\begin{aligned}& \int ^{+\infty }_{0}b_{0}(t) \,\mathrm{d}t=b_{0}^{\ast }< +\infty ,\qquad \int ^{+ \infty }_{0}b_{1}(t) \bigl(1+t^{\alpha +\beta -1}\bigr)^{\tau _{1}}\,\mathrm{d}t=b_{1}^{ \ast }< + \infty , \\& \int ^{+\infty }_{0}b_{2}(t) \bigl(1+t^{\alpha +\beta -1}\bigr)^{\tau _{2}} \,\mathrm{d}t=b_{2}^{\ast }< + \infty ,\qquad \int ^{+\infty }_{0}b_{3}(t) \,\mathrm{d}t=b_{3}^{ \ast }< +\infty . \end{aligned}$$
(C3):

\(\varphi (t,u,v,w)\) and \(\psi (t,u,v,z)\) are increasing with respect to the variables u, v, w and u, v, z, and \(\varphi (t,0,0,0)\not \equiv 0\), \(\psi (t,0,0,0)\not \equiv 0\), \(\forall t \in J\).

For convenience, we set \(\Delta =\varGamma (\alpha )\varGamma (\beta )-\varUpsilon _{1}\varUpsilon _{2}\), \(\varTheta =\max \{\varGamma (\beta )+\varUpsilon _{1},\varGamma ( \alpha )+\varUpsilon _{2}, \varGamma (\alpha )(\varGamma (\beta )+\varUpsilon _{1}), \varGamma (\beta )(\varGamma (\alpha )+\varUpsilon _{2})\}\), \(\varLambda =\frac{\varTheta }{\Delta }\).

2 Preliminaries

In this section we only list some definitions and lemmas of the Riemann–Liouville fractional integral and derivative; for more details, we refer the readers to [1].

Definition 2.1

(see [1])

The Riemann–Liouville fractional integral of order \(q>0\) for an integrable function g is defined as

$$ I^{q}g(x)=\frac{1}{\varGamma (q)} \int ^{x}_{0}(x-t)^{q-1}g(t)\,\mathrm{d} t, $$

provided that the integral exists.

Definition 2.2

(see [1])

The Riemann–Liouville fractional derivative of order \(q>0\) for an integrable function g is defined as

$$ D^{q} g(x)=\frac{1}{\varGamma (n-q)} \biggl(\frac{d}{dx} \biggr)^{n} \int ^{x}_{0}(x-t)^{n-q-1}g(t) \,\mathrm{d} t, $$

where \(n=[q]+1, { [q]}\) is the smallest integer greater than or equal to q, provided that the right-hand side is pointwise defined on \((0,+\infty )\).

Lemma 2.3

(see [1])

Let\(q>0\)and\(u\in C(0,+\infty )\cap L(0,+\infty )\). Then the general solution of fractional differential equation\(D^{q}u(t)=0\)is

$$ u(t)=c_{1}t^{q-1}+c_{2}t^{q-2}+\cdots +c_{n}t^{q-n}, $$

where\(c_{i}\in \mathbb{R}\), \(i=1,2,\ldots ,n\), and\(n-1< q< n\).

Lemma 2.4

Let\(x,y\in C(0,+\infty )\cap L(0,+\infty )\)and assumption (C1) be satisfied. Then the fractional differential system with coupled integral boundary conditions

$$ \textstyle\begin{cases} D^{\alpha }u(t)+x(t)=0,\quad 2< \alpha \leq 3, \\ D^{\beta }v(t)+y(t)=0,\quad 2< \beta \leq 3, \\ u(0)=u'(0)=0,\qquad D^{\alpha -1}u(+\infty )= \int _{0}^{+\infty }h(t)v(t) \,\mathrm{d}t, \\ v(0)=v'(0)=0,\qquad D^{\beta -1}v(+\infty )= \int _{0}^{+\infty }g(t)u(t) \,\mathrm{d}t, \end{cases}$$
(2.1)

has a solution which can take the integral representation

$$ \textstyle\begin{cases} u(t)= \int ^{+\infty }_{0}K_{1}(t,s)x(s)\,\mathrm{d}s+\int ^{+ \infty }_{0}K_{3}(t,s)y(s)\,\mathrm{d}s, \\ v(t)= \int ^{+\infty }_{0}K_{2}(t,s)y(s)\,\mathrm{d}s+\int ^{+ \infty }_{0}K_{4}(t,s)x(s)\,\mathrm{d}s, \end{cases} $$
(2.2)

where

$$ \begin{aligned} &K_{1}(t,s)=K_{11}(t,s)+K_{12}(t,s), \qquad K_{2}(t,s)=K_{21}(t,s)+K_{22}(t,s), \\ &K_{3}(t,s)= \frac{\varGamma (\beta )t^{\alpha -1}}{\Delta } \int ^{+ \infty }_{0}h(t)K_{21}(t,s)\,\mathrm{d}t, \\ & K_{4}(t,s)= \frac{\varGamma (\alpha )t^{\beta -1}}{\Delta } \int ^{+ \infty }_{0}g(t)K_{11}(t,s)\,\mathrm{d}t \end{aligned} $$
(2.3)

with

$$ \begin{aligned} & K_{11}(t,s)=\frac{1}{\varGamma (\alpha )} \textstyle\begin{cases} t^{\alpha -1}-(t-s)^{\alpha -1},&0\leq s\leq t\leq +\infty , \\ t^{\alpha -1},&0\leq t\leq s\leq +\infty , \end{cases}\displaystyle \\ & K_{12}(t,s)= \frac{\varUpsilon _{1}t^{\alpha -1}}{\Delta } \int ^{+ \infty }_{0}g(t)K_{11}(t,s)\,\mathrm{d}t, \\ &K_{21}(t,s)=\frac{1}{\varGamma (\beta )} \textstyle\begin{cases} t^{\beta -1}-(t-s)^{\beta -1},&0\leq s\leq t\leq +\infty , \\ t^{\beta -1},&0\leq t\leq s\leq +\infty , \end{cases}\displaystyle \\ &K_{22}(t,s)= \frac{\varUpsilon _{2}t^{\beta -1}}{\Delta } \int ^{+\infty }_{0}h(t)K_{21}(t,s) \,\mathrm{d}t. \end{aligned} $$

Proof

By Lemma 2.3, we can turn system (2.1) into an equivalent integral system

$$ \textstyle\begin{cases} u(t)=-\frac{1}{\varGamma (\alpha )}\int _{0}^{t}(t-s)^{ \alpha }x(s)\,\mathrm{d}s+ c_{1}t^{\alpha -1}+c_{2}t^{\alpha -2}+c_{3}t^{ \alpha -3}, \\ v(t)=-\frac{1}{\varGamma (\beta )}\int _{0}^{t}(t-s)^{ \beta }y(s)\,\mathrm{d}s+ d_{1}t^{\beta -1}+d_{2}t^{\beta -2}+d_{3}t^{ \beta -3}, \end{cases} $$
(2.4)

where \(c_{i},d_{i}\in \mathbb{R} \) (\(i=1,2,3\)). Notice that \(u(0)=u'(0)=0\) and \(v(0)=v'(0)=0\), we have \(c_{2}=c_{3}=d_{2}=d_{3}=0\). From (2.4) we have

$$ \textstyle\begin{cases} u(t)=-\frac{1}{\varGamma (\alpha )}\int _{0}^{t}(t-s)^{ \alpha }x(s)\,\mathrm{d}s+ c_{1}t^{\alpha -1}, \\ v(t)=-\frac{1}{\varGamma (\beta )}\int _{0}^{t}(t-s)^{ \beta }y(s)\,\mathrm{d}s+ d_{1}t^{\beta -1}. \end{cases} $$
(2.5)

As a result,

$$ \textstyle\begin{cases} D^{\alpha -1}u(t)= c_{1}\varGamma (\alpha )-\int _{0}^{t}x(s) \,\mathrm{d}s, \\ D^{\beta -1}v(t)=d_{1}\varGamma (\beta )-\int _{0}^{t}y(s) \,\mathrm{d}s. \end{cases} $$
(2.6)

That is,

$$ \textstyle\begin{cases} D^{\alpha -1}u(+\infty )= c_{1}\varGamma (\alpha )-\int _{0}^{+ \infty }x(s)\,\mathrm{d}s, \\ D^{\beta -1}v(+\infty )=d_{1}\varGamma (\beta )-\int _{0}^{+ \infty }y(s)\,\mathrm{d}s. \end{cases} $$
(2.7)

By means of conditions \(D^{\alpha -1}u(+\infty )= \int _{0}^{+\infty }h(t)v(t)\,\mathrm{d}t\) and \(D^{\beta -1}v(+\infty )= \int _{0}^{+\infty }g(t)u(t)\,\mathrm{d}t\), we have

$$ \textstyle\begin{cases} c_{1}=\frac{1}{\varGamma (\alpha )}\int _{0}^{+\infty }h(t)v(t) \,\mathrm{d}t +\frac{1}{\varGamma (\alpha )}\int _{0}^{+\infty }x(s)\,\mathrm{d}s, \\ d_{1}=\frac{1}{\varGamma (\beta )}\int _{0}^{+\infty }g(t)u(t) \,\mathrm{d}t +\frac{1}{\varGamma (\beta )}\int _{0}^{+\infty }y(s)\,\mathrm{d}s. \end{cases} $$
(2.8)

Submitting (2.8) to (2.5), we obtain

$$ \textstyle\begin{cases} u(t)=-\frac{1}{\varGamma (\alpha )}\int _{0}^{t}(t-s)^{ \alpha }x(s)\,\mathrm{d}s+ \frac{t^{\alpha -1}}{\varGamma (\alpha )} [\int _{0}^{+ \infty }h(t)v(t)\,\mathrm{d}t+\int _{0}^{+\infty }x(s)\,\mathrm{d}s ] \\ \hphantom{u(t)}=\int _{0}^{\infty }K_{11}(t,s)x(s)\,\mathrm{d}s + \frac{t^{\alpha -1}}{\varGamma (\alpha )}\int _{0}^{+\infty }h(t)v(t) \,\mathrm{d}t, \\ v(t)=-\frac{1}{\varGamma (\beta )}\int _{0}^{t}(t-s)^{ \beta }y(s)\,\mathrm{d}s+ \frac{t^{\beta -1}}{\varGamma (\beta )} [\int _{0}^{+ \infty }g(t)u(t)\,\mathrm{d}t+\int _{0}^{+\infty }y(s)\,\mathrm{d}s ] \\ \hphantom{v(t)} =\int _{0}^{\infty }K_{21}(t,s)y(s)\,\mathrm{d}s+ \frac{t^{\beta -1}}{\varGamma (\beta )}\int _{0}^{+\infty }g(t)u(t) \,\mathrm{d}t. \end{cases}$$
(2.9)

Multiplying both sides of equality (2.9) by \(g(t)\) and \(h(t)\) and integrating from 0 to +∞, we have

$$ \textstyle\begin{cases} \int _{0}^{+\infty }g(t)u(t)\,\mathrm{d}t= \int _{0}^{+\infty }g(t) \int _{0}^{+\infty }K_{11}(t,s)x(s)\,\mathrm{d}s\,\mathrm{d}t + \frac{\varUpsilon _{2}}{\varGamma (\alpha )}\int _{0}^{+\infty }h(t)v(t) \,\mathrm{d}t, \\ \int _{0}^{+\infty }h(t)v(t)\,\mathrm{d}t = \int _{0}^{+ \infty }h(t)\int _{0}^{+\infty }K_{21}(t,s)y(s)\,\mathrm{d}s\,\mathrm{d}t + \frac{\varUpsilon _{1}}{\varGamma (\beta )}\int _{0}^{+\infty }g(t)u(t) \,\mathrm{d}t. \end{cases} $$

Then

$$ \textstyle\begin{cases} \int _{0}^{+\infty }g(t)u(t)\,\mathrm{d}t= \frac{\varGamma (\alpha )\varGamma (\beta )}{\Delta } [\int _{0}^{+ \infty }g(t)\int _{0}^{+\infty }K_{11}(t,s)x(s)\,\mathrm{d}s\,\mathrm{d}t \\ \hphantom{\int _{0}^{+\infty }g(t)u(t)\,\mathrm{d}t=}{}+ \frac{\varUpsilon _{2}}{\varGamma (\alpha )} \int _{0}^{+ \infty }h(t)\int _{0}^{+\infty }K_{21}(t,s)y(s)\,\mathrm{d}s\,\mathrm{d}t ], \\ \int _{0}^{+\infty }h(t)v(t)\,\mathrm{d}t = \frac{\varGamma (\alpha )\varGamma (\beta )}{\Delta } [ \frac{\varUpsilon _{1}}{\varGamma (\beta )}\int _{0}^{+\infty }g(t)\int _{0}^{+ \infty }K_{11}(t,s)x(s)\,\mathrm{d}s\,\mathrm{d}t \\ \hphantom{\int _{0}^{+\infty }h(t)v(t)\,\mathrm{d}t =}{}+\int _{0}^{+\infty }h(t)\int _{0}^{+\infty }K_{21}(t,s)y(s) \,\mathrm{d}s\,\mathrm{d}t ]. \end{cases} $$
(2.10)

Submitting (2.10) to (2.9), we have

$$ \textstyle\begin{cases} u(t)=\int _{0}^{+\infty }K_{11}(t,s)x(s)\,\mathrm{d}s + \frac{t^{\alpha -1}\varUpsilon _{1}}{\Delta } \int _{0}^{+ \infty }g(t)\int _{0}^{+\infty }K_{11}(t,s)x(s)\,\mathrm{d}s\,\mathrm{d}t \\ \hphantom{u(t)=}{}+\frac{t^{\alpha -1}\varGamma (\beta )}{\Delta } \int _{0}^{+ \infty }h(t)\int _{0}^{+\infty }K_{21}(t,s)y(s)\,\mathrm{d}s\,\mathrm{d}t \\ \hphantom{u(t)}=\int _{0}^{\infty }K_{11}(t,s)x(s)\,\mathrm{d}s+ \int _{0}^{\infty }K_{12}(t,s)x(s)\,\mathrm{d}s +\int _{0}^{ \infty }K_{3}(t,s)y(s)\,\mathrm{d}s \\ \hphantom{u(t)}=\int _{0}^{\infty }K_{1}(t,s)x(s)\,\mathrm{d}s +\int _{0}^{ \infty }K_{3}(t,s)y(s)\,\mathrm{d}s, \\ v(t)=\int _{0}^{+\infty }K_{21}(t,s)y(s)\,\mathrm{d}s+ \frac{t^{\beta -1}\varGamma (\alpha )}{\Delta } \int _{0}^{+\infty }g(t) \int _{0}^{+\infty }K_{11}(t,s)x(s)\,\mathrm{d}s\,\mathrm{d}t \\ \hphantom{v(t)=}{}+\frac{t^{\beta -1}\varUpsilon _{2}}{\Delta } \int _{0}^{+ \infty }h(t)\int _{0}^{+\infty }K_{21}(t,s)y(s)\,\mathrm{d}s\,\mathrm{d}t \\ \hphantom{v(t)}=\int _{0}^{+\infty }K_{21}(t,s)y(s)\,\mathrm{d}s+\int _{0}^{+ \infty }K_{22}(t,s)y(s)\,\mathrm{d}s+ \int _{0}^{+\infty }K_{4}(t,s)x(s) \,\mathrm{d}s \\ \hphantom{v(t)}=\int _{0}^{+\infty }K_{2}(t,s)y(s)\,\mathrm{d}s+ \int _{0}^{+ \infty }K_{4}(t,s)x(s)\,\mathrm{d}s. \end{cases} $$
(2.11)

The proof is completed. □

Lemma 2.5

For\((s,t)\in J\times J\), if assumption (C1) is satisfied, then

$$\begin{aligned}& \begin{aligned} 0\leq K_{1}(t,s)\leq \frac{\varGamma (\beta )t^{\alpha -1}}{\Delta }, \qquad 0\leq \frac{K_{1}(t,s)}{1+t^{\alpha +\beta -1}}\leq \frac{\varGamma (\beta )}{\Delta },\end{aligned} \\& \begin{aligned} 0\leq K_{2}(t,s)\leq \frac{\varGamma (\alpha )t^{\beta -1}}{\Delta }, \qquad 0\leq \frac{K_{2}(t,s)}{1+t^{\alpha +\beta -1}}\leq \frac{\varGamma (\alpha )}{\Delta },\end{aligned} \\& \begin{aligned} 0\leq K_{3}(t,s)\leq \frac{\varUpsilon _{1}t^{\alpha -1}}{\Delta }, \qquad 0\leq \frac{K_{3}(t,s)}{1+t^{\alpha +\beta -1}}\leq \frac{\varUpsilon _{1}}{\Delta },\end{aligned} \\& \begin{aligned} 0\leq K_{4}(t,s)\leq \frac{\varUpsilon _{2}t^{\beta -1}}{\Delta }, \qquad 0\leq \frac{K_{4}(t,s)}{1+t^{\alpha +\beta -1}}\leq \frac{\varUpsilon _{2}}{\Delta }.\end{aligned} \end{aligned}$$

Proof

From (2.3), it is obvious that

$$ \begin{aligned} 0\leq K_{11}(t,s)\leq \frac{t^{\alpha -1}}{\varGamma (\alpha )},\quad \forall (t,s) \in J\times J,\end{aligned} $$

and

$$ \begin{aligned} 0\leq K_{12}(t,s)\leq \frac{\varUpsilon _{1}t^{\alpha -1}}{\Delta } \int ^{+ \infty }_{0} \frac{g(t)t^{\alpha -1}}{\varGamma (\alpha )}\,\mathrm{d}t = \frac{\varUpsilon _{1}\varUpsilon _{2} t^{\alpha -1}}{\varGamma (\alpha )\Delta },\quad \forall (t,s)\in J\times J.\end{aligned} $$

Thus

$$ \begin{aligned} 0\leq K_{1}(t,s)=K_{11}(t,s)+K_{12}(t,s) \leq \frac{\varGamma (\beta )t^{\alpha -1}}{\Delta },\quad \forall (t,s)\in J\times J.\end{aligned} $$

Furthermore,

$$ \begin{aligned} 0\leq \frac{K_{1}(t,s)}{1+t^{\alpha +\beta -1}}\leq \frac{\varGamma (\beta )}{\Delta },\quad \forall (t,s)\in J\times J.\end{aligned} $$

By a similar calculation, we can prove other inequality results about \(K_{2}(t,s)\), \(K_{3}(t,s)\), and \(K_{4}(t,s)\). So the proof is completed. □

Remark 2.6

From (2.5), (2.8), and (2.10), by a direct calculation, we have

$$ \textstyle\begin{cases} D^{\alpha -1}u(t)=\int _{0}^{+\infty }H_{1}(t,s)x(s) \,\mathrm{d}s+\int _{0}^{+\infty }H_{3}(t,s)y(s)\,\mathrm{d}s, \\ D^{\beta -1}v(t)= \int _{0}^{+\infty }H_{2}(t,s)y(s) \,\mathrm{d}s+\int _{0}^{+\infty }H_{4}(t,s)x(s)\,\mathrm{d}s, \end{cases} $$
(2.12)

where

$$ \begin{aligned} &H_{1}(t,s)=H_{11}(t,s)+H_{12}(t,s),\qquad H_{2}(t,s)=H_{11}(t,s)+H_{22}(t,s), \\ &H_{3}(t,s)= \frac{\varGamma (\alpha )\varGamma (\beta )}{\Delta } \int ^{+ \infty }_{0}h(t)K_{21}(t,s)\,\mathrm{d}t,\\ & H_{4}(t,s)= \frac{\varGamma (\alpha )\varGamma (\beta )}{\Delta } \int ^{+ \infty }_{0}g(t)K_{11}(t,s) \,\mathrm{d}t\end{aligned} $$

with

$$\begin{aligned}& \begin{aligned} & H_{11}(t,s)= \textstyle\begin{cases} 0,&0\leq s\leq t\leq +\infty , \\ 1,&0\leq t\leq s\leq +\infty , \end{cases}\displaystyle \qquad H_{12}(t,s)= \frac{\varUpsilon _{1}\varGamma (\alpha )}{\Delta } \int ^{+ \infty }_{0}g(t)K_{11}(t,s) \,\mathrm{d}t.\end{aligned} \\& \begin{aligned} H_{22}(t,s)= \frac{\varUpsilon _{2}\varGamma (\beta )}{\Delta } \int ^{+ \infty }_{0}h(t)K_{21}(t,s) \,\mathrm{d}t.\end{aligned} \end{aligned}$$

Remark 2.7

From Lemma 2.5, by a direct calculation, we can easily obtain that

$$\begin{aligned}& 0\leq H_{1}(t,s)=H_{11}(t,s)+H_{12}(t,s)\leq 1+ \frac{\varUpsilon _{1}\varUpsilon _{2}}{\Delta } = \frac{\varGamma (\alpha )\varGamma (\beta )}{\Delta },\quad \forall (t,s)\in J \times J, \\& 0\leq H_{2}(t,s)=H_{11}(t,s)+H_{22}(t,s)\leq 1+ \frac{\varUpsilon _{1}\varUpsilon _{2}}{\Delta } = \frac{\varGamma (\alpha )\varGamma (\beta )}{\Delta },\quad \forall (t,s)\in J \times J, \\& 0\leq H_{3}(t,s)\leq \frac{\varGamma (\alpha )\varUpsilon _{1}}{\Delta }, \quad \forall (t,s)\in J\times J, \\& 0\leq H_{4}(t,s)\leq \frac{\varGamma (\beta )\varUpsilon _{2}}{\Delta },\quad \forall (t,s)\in J\times J. \end{aligned}$$

Define two spaces

$$\begin{aligned}& X= \biggl\{ u\in C(J),D^{\alpha -1}u\in C(J)\Big|\sup_{t\in J} \frac{ \vert u(t) \vert }{1+t^{\alpha +\beta -1}}< +\infty ,\sup_{t\in J} \bigl\vert D^{ \alpha -1}u(t) \bigr\vert < +\infty \biggr\} , \\& Y= \biggl\{ v\in C(J),D^{\beta -1}v\in C(J)\Big|\sup_{t\in J} \frac{ \vert v(t) \vert }{1+t^{\alpha +\beta -1}}< +\infty ,\sup_{t\in J} \bigl\vert D^{ \beta -1}v(t) \bigr\vert < +\infty \biggr\} , \end{aligned}$$

equipped with the norms

$$ \begin{gathered} \Vert u \Vert _{X}=\max \biggl\{ \sup_{t\in J} \frac{ \vert u(t) \vert }{1+t^{\alpha +\beta -1}}, \sup_{t\in J} \bigl\vert D^{\alpha -1}u(t) \bigr\vert \biggr\} ,\\ \Vert v \Vert _{Y}=\max \biggl\{ \sup _{t\in J} \frac{ \vert v(t) \vert }{1+t^{\alpha +\beta -1}}, \sup_{t\in J} \bigl\vert {D^{ \beta -1}v(t)} \bigr\vert \biggr\} , \end{gathered} $$

where \(2<\alpha ,\beta \leq 3\). \(C(J)\) denotes the space of all continuous functions defined on \([0, +\infty )\).

Lemma 2.8

\((X, \|\cdot \|_{X})\)and\((Y, \|\cdot \|_{Y})\)are two Banach spaces.

Proof

The proof is similar to that of Lemma 2.4 in [29], so we omit it. □

Lemma 2.9

(see [44])

Let\(U\subset X\)be a bounded set. ThenUis relatively compact inXif the following conditions hold:

  1. (i)

    For any\(u\in U\), \(\frac{u(t)}{1+t^{\alpha -1}}\)and\(D^{\alpha -1}u(t)\)are equicontinuous on any compact interval of J;

  2. (ii)

    For any\(\varepsilon >0\), there is a constant\(C=C(\varepsilon )>0\)such that\(|\frac{u(t_{1})}{1+t_{1}^{\alpha -1}}- \frac{u(t_{2})}{1+t_{2}^{\alpha -1}}|<\varepsilon \)and\(|D^{\alpha -1}u(t_{1})-D^{\alpha -1}u(t_{2})|<\varepsilon \)for any\(t_{1},t_{2}\geq C\)and\(u\in U\).

Remark 2.10

Let \(U\subset X\) be a bounded set. According to Lemmas 2.8 and 2.9, it is clear that U is relatively compact in X if the following conditions hold:

  1. (i)

    For any \(u\in U,\frac{u(t)}{1+t^{\alpha +\beta -1}}\) and \(D^{\alpha -1}u(t)\) are equicontinuous on any compact interval of J;

  2. (ii)

    For any \(\varepsilon >0\), there is a constant \(C=C(\varepsilon )>0\) such that \(|\frac{u(t_{1})}{1+t_{1}^{\alpha +\beta -1}}- \frac{u(t_{2})}{1+t_{2}^{\alpha +\beta -1}}|<\varepsilon \) and \(|D^{\alpha -1}u(t_{1})-D^{\alpha -1}u(t_{2})|<\varepsilon \) for any \(t_{1},t_{2}\geq C\) and \(u\in U\).

3 Main results

We define the cone \(P\subset X\times Y\) as \(P=\{(u,v)\in X\times Y| u(t)\geq 0,v(t)\geq 0, D^{\alpha -1}u(t) \geq 0,D^{\beta -1}v(t)\geq 0,t\in J\}\). From Lemma 2.4 it is easy to know that the fractional differential system (1.6) is equivalent to the following system of Hammerstein-type integral equations:

( u ( t ) v ( t ) ) = ( 0 + K 1 ( t , s ) φ ( u , v ) ( s ) d s + 0 + K 3 ( t , s ) ψ ( u , v ) ( s ) d s 0 + K 2 ( t , s ) ψ ( u , v ) ( s ) d s + 0 + K 4 ( t , s ) φ ( u , v ) ( s ) d s ) : = ( Ϝ 1 ( u , v ) ( t ) Ϝ 2 ( u , v ) ( t ) ) for  u , v P , t J ,
(3.1)

and for convenience, we set

$$ \textstyle\begin{cases} \varphi _{(u,v)}(s)=\varphi (s,u(s),v(s),D^{\beta -1}v(s)), \\ \psi _{(u,v)}(s)= \psi (s,u(s),v(s),D^{\alpha -1}u(s)). \end{cases} $$

Then we can define an operator \(\digamma :P\times P \to P\times P\) as follows:

$$ \digamma (u, v) (t)= (\digamma _{1}, \digamma _{2} ) (u, v) (t) \quad \text{for } u, v \in P, t\in J. $$

Therefore, if \((u, v) \in (P \times P)\backslash \{ \mathbf{0}\} \) is a fixed point of Ϝ, then \((u, v)\) is a positive solution for the fractional differential system (1.6). Next, we will directly study the existence of fixed points of the operator Ϝ.

By Remark 2.6 and (3.1), we have

( D α 1 Ϝ 1 ( u , v ) ( t ) D β 1 Ϝ 2 ( u , v ) ( t ) ) = ( 0 + H 1 ( t , s ) φ ( u , v ) ( s ) d s + 0 + H 3 ( t , s ) ψ ( u , v ) ( s ) d s 0 + H 2 ( t , s ) ψ ( u , v ) ( s ) d s + 0 + H 4 ( t , s ) φ ( u , v ) ( s ) d s ) for  u , v P , t J .
(3.2)

Lemma 3.1

If assumptions (C0) and (C2) are satisfied, then

$$ \int _{0}^{+\infty } \bigl\vert \varphi _{(u,v)}(s) \bigr\vert \,\mathrm{d}s \leq a_{0}^{\ast }+ \sum_{k=1}^{3}a_{k}^{\ast } \bigl\Vert (u,v) \bigr\Vert _{X\times Y}^{\lambda _{k}},\quad \forall (u,v)\in X\times Y, $$

and

$$ \int _{0}^{+\infty } \bigl\vert \psi _{(u,v)}(s) \bigr\vert \,\mathrm{d}s \leq b_{0}^{\ast }+\sum _{k=1}^{3}b_{k}^{ \ast } \bigl\Vert (u,v) \bigr\Vert _{X\times Y}^{\tau _{k}},\quad \forall (u,v)\in X \times Y. $$

Proof

For \(\forall (u,v)\in X\times Y\), by assumptions (C0) and (C2), we have

$$ \begin{aligned} &\int _{0}^{+\infty } \bigl\vert \varphi _{(u,v)}(s) \bigr\vert \,\mathrm{d}s \\ &\quad \leq \int _{0}^{+ \infty } \bigl(a_{0}(s)+a_{1}(s) \bigl\vert u(s) \bigr\vert ^{\lambda _{1}}+a_{2}(s) \bigl\vert v(s) \bigr\vert ^{ \lambda _{2}}+a_{3}(s) \bigl\vert D^{\beta -1}v(s) \bigr\vert ^{\lambda _{3}} \bigr)\,\mathrm{d}s \\ &\quad \leq a_{0}^{\ast }+ \int _{0}^{+\infty }a_{1}(s)) \bigl(1+s^{\alpha +\beta -1}\bigr)^{ \lambda _{1}} \frac{ \vert u(s) \vert ^{\lambda _{1}}}{(1+s^{\alpha +\beta -1})^{\lambda _{1}}}\,\mathrm{d}s \\ &\qquad {}+ \int _{0}^{+\infty }a_{2}(s)) \bigl(1+s^{\alpha +\beta -1}\bigr)^{\lambda _{2}} \frac{ \vert v(s) \vert ^{\lambda _{2}}}{(1+s^{\alpha +\beta -1})^{\lambda _{2}}}\,ds + \int _{0}^{+\infty }a_{3}(s) \bigl\vert D^{\beta -1}v(s) \bigr\vert ^{\lambda _{3}}\,\mathrm{d}s \\ &\quad \leq a_{0}^{\ast }+a_{1}^{\ast } \Vert u \Vert _{X}^{\lambda _{1}}+a_{2}^{\ast } \Vert v \Vert _{Y}^{ \lambda _{2}} +a_{3}^{\ast } \Vert v \Vert _{Y}^{\lambda _{3}} \\ &\quad \leq a_{0}^{\ast }+\sum_{k=1}^{3}a_{k}^{\ast } \bigl\Vert (u,v) \bigr\Vert _{X\times Y}^{ \lambda _{k}}\end{aligned} $$

and

$$ \begin{aligned} &\int _{0}^{+\infty } \bigl\vert \psi _{(u,v)}(s) \bigr\vert \,\mathrm{d}s \\ &\quad \leq \int _{0}^{+ \infty } \bigl(b_{0}(s)+b_{1}(s) \bigl\vert u(s) \bigr\vert ^{\tau _{1}}+b_{2}(s) \bigl\vert v(s) \bigr\vert ^{ \tau _{2}}+b_{3}(s) \bigl\vert D^{\alpha -1}u(s) \bigr\vert ^{\tau _{3}} \bigr)\,\mathrm{d}s \\ &\quad \leq b_{0}^{\ast }+ \int _{0}^{+\infty }b_{1}(s)) \bigl(1+s^{\alpha +\beta -1}\bigr)^{ \tau _{1}} \frac{ \vert u(s) \vert ^{\tau _{1}}}{(1+s^{\alpha +\beta -1})^{\tau _{1}}}\,\mathrm{d}s \\ &\qquad{}+ \int _{0}^{+\infty }b_{2}(s)) \bigl(1+s^{\alpha +\beta -1}\bigr)^{\tau _{2}} \frac{ \vert v(s) \vert ^{\tau _{2}}}{(1+s^{\alpha +\beta -1})^{\tau _{2}}}\,ds + \int _{0}^{+\infty }b_{3}(s) \bigl\vert D^{\alpha -1}u(s) \bigr\vert ^{\tau _{3}}\,\mathrm{d}s \\ &\quad \leq b_{0}^{\ast }+b_{1}^{\ast } \Vert u \Vert _{X}^{\tau _{1}}+b_{2}^{\ast } \Vert v \Vert _{Y}^{ \tau _{2}} +b_{3}^{\ast } \Vert u \Vert _{X}^{\tau _{3}} \\ &\quad \leq b_{0}^{\ast }+\sum_{k=1}^{3}b_{k}^{\ast } \bigl\Vert (u,v) \bigr\Vert _{X\times Y}^{ \tau _{k}}. \end{aligned} $$

 □

Lemma 3.2

If assumptions (C0), (C1), and (C2) are satisfied, then the operator\(\digamma :P\rightarrow P\)is completely continuous.

Proof

Since \(K_{i}(t,s)\geq 0\), \(\forall (t,s)\in J\times J\), \(i=1,2,3,4\), and \(\varphi \geq 0\), \(\psi \geq 0\), \(\forall (u,v)\in P\times P\), we have \(\digamma _{1}(u,v)(t)\geq 0\), \(\digamma _{2}(u,v)(t)\geq 0\), \(\forall (u,v) \in P\), \(t\in J\). So it is obvious that \(\digamma :P\rightarrow P\).

Next we show that the operator \(\digamma :P\rightarrow P\) is relatively compact. First let \(U=\{(u,v)| (u,v)\in P,\|(u,v)\|_{X\times Y}\leq M\}\). For \(\forall (u,v)\in U\), by Lemma 2.4, Lemma 2.5, and Lemma 3.1, we have

$$ \begin{aligned}[b] &\sup_{t\in J} \frac{ \vert \digamma _{1}(u,v)(t) \vert }{1+t^{\alpha +\beta -1}}\\ &\quad \leq \sup_{t\in J} \biggl\vert \int _{0}^{+\infty } \frac{K_{1}(t,s)}{1+t^{\alpha +\beta -1}} \varphi _{(u,v)}(s)\,\mathrm{d}s \biggr\vert +\sup_{t\in J} \biggl\vert \int _{0}^{+\infty } \frac{K_{3}(t,s)}{1+t^{\alpha +\beta -1}} \psi _{(u,v)}(s)\,\mathrm{d}s \biggr\vert \\ &\quad \leq \frac{\varGamma (\beta )}{\Delta } \int _{0}^{+\infty } \bigl\vert \varphi _{(u,v)}(s) \bigr\vert \,\mathrm{d}s+\frac{\varUpsilon _{1}}{\Delta } \int _{0}^{+\infty } \bigl\vert \psi _{(u,v)}(s) \bigr\vert \,\mathrm{d}s \\ &\quad \leq \frac{\varGamma (\beta )+\varUpsilon _{1}}{\Delta } \Biggl[a_{0}^{\ast }+\sum _{k=1}^{3}a_{k}^{\ast } \bigl\Vert (u,v) \bigr\Vert _{X\times Y}^{ \lambda _{k}}+b_{0}^{\ast }+ \sum_{k=1}^{3}b_{k}^{\ast } \bigl\Vert (u,v) \bigr\Vert _{X \times Y}^{\tau _{k}} \Biggr] \\ &\quad \leq \frac{\varGamma (\beta )+\varUpsilon _{1}}{\Delta } \Biggl[a_{0}^{\ast }+b_{0}^{\ast }+ \sum_{k=1}^{3}\bigl(a_{k}^{\ast }M^{ \lambda _{k}}+b_{k}^{\ast }M^{\tau _{k}} \bigr) \Biggr]. \end{aligned} $$
(3.3)

By Remark 2.6, Remark 2.7, and Lemma 3.1, we have

$$\begin{aligned} & \sup_{t\in J} \bigl\vert D^{\alpha -1}\digamma _{1}(u,v) (t) \bigr\vert \\ &\quad \leq \sup _{t\in J} \biggl\vert \int _{0}^{\infty }H_{1}(t,s) \varphi _{(u,v)}(s)\,\mathrm{d}s \biggr\vert +\sup_{t\in J} \biggl\vert \int _{0}^{\infty }H_{3}(t,s) \psi _{(u,v)}(s)\,\mathrm{d}s \biggr\vert \\ &\quad \leq \frac{\varGamma (\alpha )\varGamma (\beta )}{\Delta } \int _{0}^{+\infty } \bigl\vert \varphi _{(u,v)}(s) \bigr\vert \,\mathrm{d}s+\frac{\varGamma (\alpha )\varUpsilon _{1}}{\Delta } \int _{0}^{+\infty } \bigl\vert \psi _{(u,v)}(s) \bigr\vert \,\mathrm{d}s \\ &\quad \leq \frac{\varGamma (\alpha )(\varGamma (\beta )+\varUpsilon _{1})}{\Delta } \Biggl[a_{0}^{ \ast }+\sum _{k=1}^{3}a_{k}^{\ast } \bigl\Vert (u,v) \bigr\Vert _{X\times Y}^{\lambda _{k}}+b_{0}^{ \ast }+ \sum_{k=1}^{3}b_{k}^{\ast } \bigl\Vert (u,v) \bigr\Vert _{X\times Y}^{\tau _{k}} \Biggr] \\ &\quad \leq \frac{\varGamma (\alpha )(\varGamma (\beta )+\varUpsilon _{1})}{\Delta } \Biggl[a_{0}^{ \ast }+b_{0}^{\ast }+ \sum_{k=1}^{3}\bigl(a_{k}^{\ast }M^{\lambda _{k}}+b_{k}^{ \ast }M^{\tau _{k}} \bigr) \Biggr]. \end{aligned}$$
(3.4)

Thus

$$ \begin{aligned}[b] &\bigl\Vert \digamma _{1}(u,v) \bigr\Vert _{X}\\ &\quad =\max \biggl\{ \sup _{t\in J} \frac{ \vert \digamma _{1}(u,v)(t) \vert }{1+t^{\alpha +\beta -1}},\sup_{t\in J} \bigl\vert D^{ \alpha -1}\digamma _{1}(u,v) (t) \bigr\vert \biggr\} \\ &\quad \leq \frac{\max \{\varGamma (\beta )+\varUpsilon _{1},\varGamma (\alpha )(\varGamma (\beta )+\varUpsilon _{1})\}}{\Delta } \Biggl[a_{0}^{\ast }+b_{0}^{\ast }+ \sum_{k=1}^{3}\bigl(a_{k}^{\ast }M^{ \lambda _{k}}+b_{k}^{\ast }M^{\tau _{k}} \bigr) \Biggr]. \end{aligned} $$
(3.5)

Similarly

$$ \bigl\Vert \digamma _{2}(u,v) \bigr\Vert _{Y}\leq \frac{\max \{\varGamma (\alpha )+\varUpsilon _{2},\varGamma (\beta )(\varGamma (\alpha )+\varUpsilon _{2})\}}{\Delta } \Biggl[a_{0}^{\ast }+b_{0}^{\ast }+ \sum_{k=1}^{3}\bigl(a_{k}^{\ast }M^{ \lambda _{k}}+b_{k}^{\ast }M^{\tau _{k}} \bigr) \Biggr]. $$

Therefore

$$ \bigl\Vert \digamma (u,v) \bigr\Vert _{X\times Y}=\max \bigl\{ \bigl\Vert \digamma _{1}(u,v) \bigr\Vert _{X}, \bigl\Vert \digamma _{2}(u,v) \bigr\Vert _{Y} \bigr\} \leq \frac{\varTheta }{\Delta } \Biggl[a_{0}^{\ast }+b_{0}^{\ast }+ \sum_{k=1}^{3}\bigl(a_{k}^{\ast }M^{ \lambda _{k}}+b_{k}^{\ast }M^{\tau _{k}} \bigr) \Biggr], $$

which implies that TU is uniformly bounded.

Second, let \(I\subset J\) be any compact interval. Then, for all \(t_{1},t_{2}\in I\), \(t_{2}>t_{1}\) and \((u,v)\in U\), we have

$$ \begin{aligned}[b] & \biggl\vert \frac{\digamma _{1}(u,v)(t_{2})}{1+t^{\alpha +\beta -1}_{2}} - \frac{\digamma _{1}(u,v)(t_{1})}{1+t^{\alpha +\beta -1}_{1}} \biggr\vert \\ &\quad \leq \int _{0}^{+\infty } \biggl\vert \frac{K_{1}(t_{2},s)}{1+t^{\alpha +\beta -1}_{2}} - \frac{K_{1}(t_{1},s)}{1+t^{\alpha +\beta -1}_{1}} \biggr\vert \bigl\vert \varphi _{(u,v)}(s) \bigr\vert \,\mathrm{d}s\\ &\qquad {}+ \int _{0}^{+\infty } \biggl\vert \frac{K_{3}(t_{2},s)}{1+t^{\alpha +\beta -1}_{2}} - \frac{K_{3}(t_{1},s)}{1+t^{\alpha +\beta -1}_{1}} \biggr\vert \bigl\vert \psi _{(u,v)}(s) \bigr\vert \,\mathrm{d}s. \end{aligned} $$
(3.6)

Notice that \(K_{1}(t,s)/(1+t^{\alpha +\beta -1})\), \(K_{3}(t,s)/(1+t^{\alpha +\beta -1})\) are uniformly continuous for any \((t,s)\in I\times I\). Furthermore, \(K_{1}(t,s)/(1+t^{\alpha +\beta -1})\), \(K_{3}(t,s)/(1+t^{\alpha +\beta -1})\) only depend on t for \(s\geq t\), which implies that \(K_{1}(t,s)/(1+t^{\alpha +\beta -1})\), \(K_{3}(t,s)/(1+t^{\alpha +\beta -1})\) are uniformly continuous on \(I\times (J\setminus I)\). Thus, for all \(s\in J\) and \(t_{1},t_{2}\in I\), we have

$$ \begin{aligned}[b] &\forall \epsilon >0, \exists \delta (\epsilon ) \text{ such that if } \vert t_{1}-t_{2} \vert < \delta , \text{ then}\\ &\biggl\vert \frac{K_{1}(t_{2},s)}{1+t^{\alpha +\beta -1}_{2}} - \frac{K_{1}(t_{1},s)}{1+t^{\alpha +\beta -1}_{1}} \biggr\vert < \epsilon ,\qquad \biggl\vert \frac{K_{3}(t_{2},s)}{1+t^{\alpha +\beta -1}_{2}} - \frac{K_{3}(t_{1},s)}{1+t^{\alpha +\beta -1}_{1}} \biggr\vert < \epsilon . \end{aligned} $$
(3.7)

Combining (3.6) and (3.7) with Lemma 3.1, for all \(s\in J\), \((u,v)\in U\), and \(t_{1},t_{2}\in I\), we have

$$ \biggl\vert \frac{\digamma _{1}(u,v)(t_{2})}{1+t^{\alpha +\beta -1}_{2}} - \frac{\digamma _{1}(u,v)(t_{1})}{1+t^{\alpha +\beta -1}_{1}} \biggr\vert \leq \Biggl[a_{0}^{\ast }+\sum_{k=1}^{3}a_{k}^{\ast }M^{\lambda _{k}}+ b_{0}^{ \ast }+\sum_{k=1}^{3}b_{k}^{\ast }M^{\tau _{k}} \Biggr]\epsilon , $$

which implies that \(\digamma _{1}(u,v)(t)/(1+t^{\alpha +\beta -1})\) is equicontinuous on I.

Note that

$$ D^{\alpha -1}\digamma _{1}(u,v) (t)= \int _{0}^{+\infty }H_{1}(t,s) \varphi _{(u,v)}(s)\,\mathrm{d}s+ \int _{0}^{+\infty }H_{3}(t,s)\psi _{(u,v)}(s) \,\mathrm{d}s $$

and \(H_{1}(t,s),H_{3}(t,s)\in C(J\times J)\) do not depend on t, which infers that \(D^{\alpha -1}\digamma _{1}(u,v)(t)\) is equicontinuous on I. In the same way, we can show that \(\digamma _{2}(u,v)(t)/(1+t^{\alpha +\beta -1})\) and \(D^{\beta -1}\digamma _{2}(u,v)(t)\) are equicontinuous. Thus condition (i) of Remark 2.10 is satisfied.

Then we show that operators \(\digamma _{1}\), \(\digamma _{2}\) are equiconvergent at +∞. Since

$$ \lim_{t\rightarrow +\infty }\frac{K_{1}(t,s)}{1+t^{\alpha +\beta -1}} { =0}, \qquad \lim_{t\rightarrow +\infty } \frac{K_{3}(t,s)}{1+t^{\alpha +\beta -1}} {=0}, $$

we can infer that, for any \(\epsilon >0\), there exists a sufficiently large constant \(C=C(\epsilon )>0\), for any \(t_{1},t_{2}\geq C\) and \(s\in J\), such that

$$ \biggl\vert \frac{K_{1}(t_{2},s)}{1+t^{\alpha +\beta -1}_{2}} - \frac{K_{1}(t_{1},s)}{1+t^{\alpha +\beta -1}_{1}} \biggr\vert < \epsilon ,\qquad \biggl\vert \frac{K_{3}(t_{2},s)}{1+t^{\alpha +\beta -1}_{2}} - \frac{K_{3}(t_{1},s)}{1+t^{\alpha +\beta -1}_{1}} \biggr\vert < \epsilon . $$

Therefore, by Lemma 3.1 and (3.6), we conclude that \(\digamma _{1}(u,v)(t)/1+t^{\alpha +\beta -1}\) is equiconvergent at +∞. On the other hand, the functions \(H_{1}(t,s)\), \(H_{3}(t,s)\) do not depend on t, it is obvious that \(D^{\alpha -1}\digamma _{1}(u,v)(t)\) is equiconvergent at +∞. Similarly, \(\digamma _{2}(u,v)(t)/1+t^{\alpha +\beta -1}\) and \(D^{\alpha -1}\digamma _{2}(u,v)(t)\) are equiconvergent at +∞. Thus condition (ii) of Remark 2.10 is satisfied.

As can be seen from the above discussion, all the conditions of Remark 2.10 are satisfied. Thus the operator \(\digamma :P\rightarrow P\) is relatively compact.

Finally, we prove that the operator \(\digamma :P\rightarrow P\) is continuous. Let \((u_{n},v_{n}),(u,v)\in P\) such that \((u_{n},v_{n})\rightarrow (u,v)(n\rightarrow \infty )\). Then \(\|(u_{n},v_{n})\|_{X\times Y}<+\infty \), \(\|(u,v)\|_{X\times Y}<+ \infty \). Similar to (3.3) and (3.4), we can obtain

$$ \begin{aligned} &\sup_{t\in J} \frac{ \vert \digamma _{1}(u_{n},v_{n})(t) \vert }{1+t^{\alpha +\beta -1}} \\ &\quad \leq \frac{\varGamma (\beta )+\varUpsilon _{1}}{\Delta } \Biggl[a_{0}^{\ast }+b_{0}^{ \ast }+ \sum_{k=1}^{3} \bigl(a_{k}^{\ast } \bigl\Vert (u_{n},v_{n}) \bigr\Vert _{X\times Y}^{ \lambda _{k}}+b_{k}^{\ast } \bigl\Vert (u_{n},v_{n}) \bigr\Vert _{X\times Y}^{\tau _{k}} \bigr) \Biggr]< +\infty \end{aligned} $$

and

$$ \begin{aligned} &\sup_{t\in J} \bigl\vert D^{\alpha -1}\digamma _{1}(u_{n},v_{n}) (t) \bigr\vert \\ &\quad \leq \frac{\varGamma (\alpha )(\varGamma (\beta )+\varUpsilon _{1})}{\Delta } \Biggl[a_{0}^{\ast }+b_{0}^{\ast }+ \sum_{k=1}^{3} \bigl(a_{k}^{\ast } \bigl\Vert (u_{n},v_{n}) \bigr\Vert _{X \times Y}^{\lambda _{k}}+b_{k}^{\ast } \bigl\Vert (u_{n},v_{n}) \bigr\Vert _{X\times Y}^{ \tau _{k}} \bigr) \Biggr]< +\infty . \end{aligned} $$

By the continuity of function φ, ψ and the Lebesgue dominated convergence theorem, we have

$$ \begin{aligned} &\lim_{n\rightarrow \infty }\frac{\digamma _{1}(u_{n},v_{n})(t)}{1+t^{\alpha +\beta -1}}\\ &\quad = \lim_{n\rightarrow \infty } \biggl[ \int _{0}^{+\infty } \frac{K_{1}(t,s)}{1+t^{\alpha +\beta -1}}\varphi _{(u_{n},v_{n})}(s) \,\mathrm{d}s + \int _{0}^{+\infty } \frac{K_{3}(t,s)}{1+t^{\alpha +\beta -1}}\psi _{(u_{n},v_{n})}(s) \,\mathrm{d}s \biggr] \\ &\quad = \int _{0}^{+\infty }\frac{K_{1}(t,s)}{1+t^{\alpha +\beta -1}} \varphi _{(u,v)}(s)\,\mathrm{d}s + \int _{0}^{+\infty } \frac{K_{3}(t,s)}{1+t^{\alpha +\beta -1}}\psi _{(u,v)}(s)\,\mathrm{d}s= \frac{\digamma _{1}(u,v)(t)}{1+t^{\alpha +\beta -1}} \end{aligned} $$

and

$$ \begin{aligned}& \lim_{n\rightarrow \infty }D^{\alpha -1}\digamma _{1}(u_{n},v_{n}) (t)\\ &\quad = \lim_{n\rightarrow \infty } \biggl[ \int _{0}^{+\infty }H_{1}(t,s)\varphi _{(u_{n},v_{n})}(s) \,\mathrm{d}s+ \int _{0}^{+\infty }H_{3}(t,s)\psi _{(u_{n},v_{n})}(s) \,\mathrm{d}s \biggr] \\ &\quad = \int _{0}^{\infty }H_{1}(t,s)\varphi _{(u,v)}(s)\,\mathrm{d}s+ \int _{0}^{+ \infty }H_{3}(t,s)\psi _{(u,v)}(s)\,\mathrm{d}s=D^{\alpha -1}\digamma _{1}(u,v) (t). \end{aligned} $$

Then, as \(n\rightarrow \infty \),

$$ \begin{aligned} &\sup_{t\in J} \frac{ \vert \digamma _{1}(u_{n},v_{n})(t)-\digamma _{1}(u,v)(t) \vert }{1+t^{\alpha +\beta -1}} \\ &\quad \leq \sup_{t\in J} \int _{0}^{+\infty } \frac{K_{1}(t,s)}{1+t^{\alpha +\beta -1}} \bigl\vert \varphi _{(u_{n},v_{n})}(s)- \varphi _{(u,v)}(s) \bigr\vert \,\mathrm{d}s\\ &\qquad {} + \sup_{t\in J} \int _{0}^{+\infty } \frac{K_{3}(t,s)}{1+t^{\alpha +\beta -1}} \bigl\vert \psi _{(u_{n},v_{n})}(s)- \psi _{(u,v)}(s) \bigr\vert \,\mathrm{d}s \\ &\quad \leq \frac{\varGamma (\beta )+\varUpsilon _{1}}{\Delta } \biggl[ \int _{0}^{+ \infty } \bigl\vert \varphi _{(u_{n},v_{n})}(s)-\varphi _{(u,v)}(s) \bigr\vert \,\mathrm{d}s + \int _{0}^{+\infty } \bigl\vert \psi _{(u_{n},v_{n})}(s)-\psi _{(u,v)}(s) \bigr\vert \,\mathrm{d}s \biggr] \\ &\quad \rightarrow 0,\end{aligned} $$

and as \(n\rightarrow \infty \),

$$ \begin{aligned} &\sup_{t\in J} \bigl\vert D^{\alpha -1}\digamma _{1}(u_{n},v_{n}) (t)-D^{\alpha -1} \digamma _{1}(u,v) (t) \bigr\vert \\ &\quad \leq \sup_{t\in J} \int _{0}^{+\infty }H_{1}(t,s) \bigl\vert \varphi _{(u_{n},v_{n})}(s)- \varphi _{(u,v)}(s) \bigr\vert \,\mathrm{d}s \\ &\qquad {}+ \sup_{t\in J} \int _{0}^{+\infty }H_{3}(t,s) \bigl\vert \psi _{(u_{n},v_{n})}(s)-\varphi _{(u,v)}(s) \bigr\vert \,\mathrm{d}s \\ &\quad \leq \frac{\varGamma (\alpha )(\varGamma (\beta )+\varUpsilon _{1})}{\Delta } \biggl[ \int _{0}^{+\infty } \bigl\vert \varphi _{(u_{n},v_{n})}(s)-\varphi _{(u,v)}(s) \bigr\vert \,\mathrm{d}s + \int _{0}^{+\infty } \bigl\vert \psi _{(u_{n},v_{n})}(s)- \varphi _{(u,v)}(s) \bigr\vert \,\mathrm{d}s \biggr] \\ &\quad \rightarrow 0 . \end{aligned} $$

So, as \(n\rightarrow \infty \),

$$ \begin{aligned} & \bigl\Vert \digamma _{1}(u_{n},v_{n})- \digamma _{1}(u,v) \bigr\Vert _{X} \\ &\quad =\max \biggl\{ \sup_{t\in J} \frac{ \vert \digamma _{1}(u_{n},v_{n})(t)-\digamma _{1}(u,v)(t) \vert }{1+t^{\alpha +\beta -1}}, \sup _{t\in J} \bigl\vert D^{\alpha -1}\digamma _{1}(u_{n},v_{n}) (t)-D^{\alpha -1} \digamma _{1}(u,v) (t) \bigr\vert \biggr\} \\ &\quad \rightarrow 0, \end{aligned} $$

which implies that the operator \(\digamma _{1}\) is continuous. By the same way, we can obtain that the operator \(\digamma _{2}\) is continuous. That is, the operator Ϝ is continuous.

In view of all above arguments, the operator \(\digamma :P\rightarrow P\) is completely continuous. So the proof is completed.

Define a partial order over the product space:

$$ \begin{pmatrix} u_{1} \\ v_{1} \end{pmatrix} \geq \begin{pmatrix} u_{2} \\ v_{2} \end{pmatrix} $$

if \(u_{1}(t)\geq u_{2}(t)\), \(v_{1}(t)\geq v_{2}(t)\), \(D^{\alpha -1}u_{1}(t) \geq D^{\alpha -1}u_{2}(t)\), \(D^{\beta -1}v_{1}(t)\geq D^{\beta -1}v_{2}(t)\), \(t \in J\). □

Theorem 3.3

Suppose that (C0), (C1), (C2), and (C3) are satisfied. Then there exists a positive constantRsuch that system (1.6) has two positive solutions\((u^{\ast },v^{\ast })\)and\((w^{\ast },z^{\ast })\)satisfying\(0\leq \|(u^{\ast },v^{\ast })\|_{X\times Y}\leq R\)and\(0\leq \|(w^{\ast },z^{\ast })\|_{X\times Y}\leq R\)with\(\lim_{n\rightarrow \infty }(u_{n},v_{n})=(u^{\ast },v^{\ast })\)and\(\lim_{n\rightarrow \infty }(w_{n},z_{n})=(w^{\ast },z^{\ast })\), where\((u_{n},v_{n})\)and\((w_{n},z_{n})\)can be given by the following monotone iterative sequences:

( u n ( t ) v n ( t ) ) = ( Ϝ 1 ( u n 1 , v n 1 ) ( t ) Ϝ 2 ( u n 1 , v n 1 ) ( t ) ) ,n=1,2,, with  ( u 0 ( t ) v 0 ( t ) ) = ( R t α R t β )
(3.8)

and

( w n ( t ) z n ( t ) ) = ( Ϝ 1 ( w n 1 , z n 1 ) ( t ) Ϝ 2 ( w n 1 , z n 1 ) ( t ) ) ,n=1,2,, with  ( w 0 ( t ) z 0 ( t ) ) = ( 0 0 ) .
(3.9)

In addition,

( w 0 ( t ) z 0 ( t ) ) ( w 1 ( t ) z 1 ( t ) ) ( w n ( t ) z n ( t ) ) ( w ( t ) z ( t ) ) ( u ( t ) v ( t ) ) ( u n ( t ) v n ( t ) ) ( u 2 ( t ) v 2 ( t ) ) ( u 1 ( t ) v 1 ( t ) ) ( u 0 ( t ) v 0 ( t ) )
(3.10)

and

( D α 1 w 0 ( t ) D β 1 z 0 ( t ) ) ( D α 1 w 1 ( t ) D β 1 z 1 ( t ) ) ( D α 1 w n ( t ) D β 1 z n ( t ) ) ( D α 1 w ( t ) D β 1 z ( t ) ) ( D α 1 1 u ( t ) D α 2 1 v ( t ) ) ( D α 1 u n ( t ) D β 1 v n ( t ) ) ( D α 1 u 2 ( t ) D β 1 v 2 ( t ) ) ( D α 1 u 1 ( t ) D β 1 v 1 ( t ) ) ( D α 1 u 0 ( t ) D β 1 v 0 ( t ) ) .
(3.11)

Proof

First, Lemma 3.2 brings about the fact that \(\digamma (P) \subset P\) for any \((u,v)\in P\), \(t\in J\).

Next, let

$$ R\geq \max \bigl\{ 8\varLambda a_{0}^{\ast },8\varLambda b_{0}^{\ast },\bigl(8 \varLambda a_{k}^{\ast } \bigr)^{1/(1-\lambda _{k})}, \bigl(8\varLambda b_{k}^{\ast } \bigr)^{1/(1- \tau _{k})},k=1,2,3 \bigr\} $$

and define \(U_{R}=\{(u,v)\in P:\|(u,v)\|_{X\times Y}\leq R\}\). For any \((u,v)\in U_{R}\), similar to (3.3) and (3.4), we obtain

$$ \sup_{t\in J}\frac{ \vert \digamma _{1}(u,v)(t) \vert }{1+t^{\alpha +\beta -1}} \leq \varLambda \Biggl[a_{0}^{\ast }+b_{0}^{\ast }+\sum _{k=1}^{3}\bigl(a_{k}^{ \ast }R^{\lambda _{k}}+b_{k}^{\ast }R^{\tau _{k}} \bigr) \Biggr]\leq R $$

and

$$ \sup_{t\in J} \bigl\vert D^{\alpha -1}u(t) \bigr\vert \leq \varLambda \Biggl[a_{0}^{\ast }+b_{0}^{ \ast }+ \sum_{k=1}^{3}\bigl(a_{k}^{\ast }R^{\lambda _{k}}+b_{k}^{\ast }R^{ \tau _{k}} \bigr) \Biggr]\leq R. $$

This means that \(\|\digamma _{1}(u,v)\|_{X}\leq R\). In the same way, \(\|\digamma _{2}(u,v)\|_{Y}\leq R\) for all \((u,v)\in U_{R}\). Thus we know

$$ \bigl\Vert \digamma (u,v) \bigr\Vert _{X\times Y}= \bigl\{ \bigl\Vert \digamma _{1}(u,v) \bigr\Vert _{X}, \bigl\Vert \digamma _{2}(u,v) \bigr\Vert _{Y} \bigr\} \leq R. $$

That is, \(\digamma (U_{R})\subset U_{R}\).

Through (3.8) and (3.9), it is obvious that \((u_{0}(t),v_{0}(t)),(w_{0}(t),z_{0}(t))\in U_{R}\). By the complete continuity of the operator Ϝ, we define the sequences \((u_{n},v_{n})\) and \((w_{n},z_{n})\) as \((u_{n},v_{n})= \digamma (u_{n-1},v_{n-1})\), \((w_{n},z_{n})= \digamma (w_{n-1},z_{n-1})\) for \(n=1,2,\ldots \) . Since \(\digamma (U_{R})\subset U_{R}\), we can obtain that \((u_{n},v_{n}),(w_{n},z_{n})\in \digamma (U_{R})\) for \(n=1,2,\ldots \) . Hence we need to prove that there exist \((u^{\ast },v^{\ast })\) and \((w^{\ast },z^{\ast })\) satisfying \(\lim_{n\rightarrow \infty }(u_{n},v_{n})=(u^{\ast },v^{\ast })\) and \(\lim_{n\rightarrow \infty }(w_{n},z_{n})=(w^{\ast },z^{\ast })\), which are two monotone sequences for positive solutions of the fractional differential system (1.6).

For \(t\in J\), by Lemma 2.5, (3.1), and (3.8), we have

$$ \begin{aligned} u_{1}(t)=\digamma _{1}(u_{0},v_{0}) (t) \leq t^{\alpha -1} \varLambda \Biggl[a_{0}^{\ast }+\sum _{k=1}^{3}a_{k}^{\ast }R^{\lambda _{k}}+b_{0}^{ \ast }+ \sum_{k=1}^{3}b_{k}^{\ast }R^{\tau _{k}} \Biggr] \leq Rt^{\alpha -1}=u_{0}(t)\end{aligned} $$

and

$$ \begin{aligned} v_{1}(t)=\digamma _{2}(u_{0},v_{0}) (t) \leq t^{\beta -1} \varLambda \Biggl[b_{0}^{\ast }+\sum _{k=1}^{3}b_{k}^{\ast }R^{\tau _{k}}+a_{0}^{ \ast }+ \sum_{k=1}^{3}a_{k}^{\ast }R^{\lambda _{k}} \Biggr] \leq Rt^{ \beta -1}=v_{0}(t),\end{aligned} $$

that is,

( u 1 ( t ) v 1 ( t ) ) = ( Ϝ 1 ( u 0 , v 0 ) ( t ) Ϝ 2 ( u 0 , v 0 ) ( t ) ) ( R t α 1 R t β 1 ) = ( u 0 ( t ) v 0 ( t ) ) .
(3.12)

Then we consider the monotonicity of the fractional derivative of \((u,v)\). By (3.12) and Remark 2.6, we know

$$ \begin{aligned} D^{\alpha -1}u_{1}(t)&=D^{\alpha -1} \digamma _{1}(u_{0},v_{0}) (t) = \int ^{+\infty }_{0}H_{1}(t,s)\varphi _{(u_{0},v_{0})}(s) \,\mathrm{d}s+ \int ^{+\infty }_{0}H_{3}(t,s)\psi _{(u_{0},v_{0})}(s) \,\mathrm{d}s \\ &\leq \varGamma (\alpha )\varLambda \Biggl[a_{0}^{\ast }+\sum _{k=1}^{3}a_{k}^{ \ast }R^{\lambda _{k}}+ b_{0}^{\ast }+\sum_{k=1}^{3}b_{k}^{\ast }R^{ \tau _{k}} \Biggr] \leq \varGamma (\alpha )R=D^{\alpha -1}u_{0}(t), \\ D^{\beta -1}v_{1}(t)&=D^{\beta -1}T_{2}(u_{0},v_{0}) (t) = \int ^{+\infty }_{0}H_{2}(t,s)\psi _{(u_{0},v_{0})}(s) \,\mathrm{d}s+ \int ^{+\infty }_{0}H_{4}(t,s)\varphi _{(u_{0},v_{0})}(s) \,\mathrm{d}s \\ &\leq \varGamma (\beta )\varLambda \Biggl[a_{0}^{\ast }+\sum _{k=1}^{3}a_{k}^{ \ast }R^{\lambda _{k}}+ b_{0}^{\ast }+\sum_{k=1}^{3}b_{k}^{\ast }R^{ \tau _{k}} \Biggr] \leq \varGamma (\beta )R=D^{\beta -1}v_{0}(t),\end{aligned} $$

that is,

( D α 1 u 1 ( t ) D β 1 v 1 ( t ) ) = ( D α 1 Ϝ 1 ( u 0 , v 0 ) ( t ) D β 1 Ϝ 2 ( u 0 , v 0 ) ( t ) ) ( Γ ( α ) R Γ ( β ) R ) = ( D α 1 u 0 ( t ) D β 1 v 0 ( t ) ) .
(3.13)

By the monotonicity assumption (C3) of functions φ and ψ, similar to (3.12) and (3.13), for \(\forall t\in J\), we do the second iteration:

$$\begin{aligned}& \begin{pmatrix} u_{2}(t) \\ v_{2}(t) \end{pmatrix} = \begin{pmatrix} \digamma _{1}(u_{1},v_{1})(t) \\ \digamma _{2}(u_{1},v_{1})(t) \end{pmatrix}\leq \begin{pmatrix} \digamma _{1}(u_{0},v_{0})(t) \\ \digamma _{2}(u_{0},v_{0})(t) \end{pmatrix} = \begin{pmatrix} u_{1}(t) \\ v_{1}(t) \end{pmatrix} , \\& \begin{pmatrix} D^{\alpha -1}u_{2}(t) \\ D^{\beta -1}v_{2}(t) \end{pmatrix} = \begin{pmatrix} D^{\alpha -1}\digamma _{1}(u_{1},v_{1})(t) \\ D^{\beta -1}\digamma _{2}(u_{1},v_{1})(t) \end{pmatrix}\leq \begin{pmatrix} D^{\alpha -1}\digamma _{1}(u_{0},v_{0})(t) \\ D^{\beta -1}\digamma _{2}(u_{0},v_{0})(t) \end{pmatrix} = \begin{pmatrix} D^{\alpha -1}u_{1}(t) \\ D^{\beta -1}v_{1}(t) \end{pmatrix} . \end{aligned}$$

By recursion, for \(t\in J\), the sequence \(\{(u_{n},v_{n})\}_{n=0}^{\infty }\) satisfies

$$ \begin{pmatrix} u_{n+1}(t) \\ v_{n+1}(t) \end{pmatrix} \leq \begin{pmatrix} u_{n}(t) \\ v_{n}(t) \end{pmatrix}, \qquad \begin{pmatrix} D^{\alpha -1}u_{n+1}(t) \\ D^{\beta -1}v_{n+1}(t) \end{pmatrix}\leq \begin{pmatrix} D^{\alpha -1}u_{n}(t) \\ D^{\beta -1}v_{n}(t) \end{pmatrix} . $$

Applying the iterative sequence \((u_{n+1},v_{n+1})=\digamma (u_{n},v_{n})\) and the complete continuity of the operator Ϝ, it is easy to infer that \((u_{n},v_{n})\rightarrow (u^{\ast },v^{\ast })\) and \(\digamma (u^{\ast },v^{\ast })=(u^{\ast },v^{\ast })\). Thus \((u^{\ast },v^{\ast })\) is a fixed point of Ϝ.

For the sequence \(\{(w_{n},z_{n})\}_{n=0}^{\infty }\), we take a similar discussion. For \(t\in J\), we attain

$$\begin{aligned}& \begin{aligned} \begin{pmatrix} w_{1}(t) \\ z_{1}(t) \end{pmatrix} &= \begin{pmatrix} \digamma _{1}(w_{0},z_{0})(t) \\ \digamma _{2}(w_{0},z_{0})(t) \end{pmatrix} = \begin{pmatrix} \int ^{+\infty }_{0}K_{1}(t,s)\varphi _{(w_{0},z_{0})}(s) \,\mathrm{d}s +\int ^{+\infty }_{0}K_{3}(t,s)\psi _{(w_{0},z_{0})}(s) \,\mathrm{d}s \\ \int ^{+\infty }_{0}K_{2}(t,s)\psi _{(w_{0},z_{0})}(s) \,\mathrm{d}s +\int ^{+\infty }_{0}K_{4}(t,s)\varphi _{(w_{0},z_{0})}(s) \,\mathrm{d}s \end{pmatrix}\\ &\geq \begin{pmatrix} 0 \\ 0 \end{pmatrix} = \begin{pmatrix} w_{0}(t) \\ z_{0}(t) \end{pmatrix} , \end{aligned} \\& \begin{aligned} \begin{pmatrix} D^{\alpha -1}w_{1}(t) \\ D^{\beta -1}z_{1}(t) \end{pmatrix} &= \begin{pmatrix} \int ^{+\infty }_{0}H_{1}(t,s)\varphi _{(w_{0},z_{0})}(s) \,\mathrm{d}s +\int ^{+\infty }_{0}H_{3}(t,s)\psi _{(w_{0},z_{0})}(s) \,\mathrm{d}s \\ \int ^{+\infty }_{0}H_{2}(t,s)\psi _{(w_{0},z_{0})}(s) \,\mathrm{d}s +\int ^{+\infty }_{0}H_{4}(t,s)\varphi _{(w_{0},z_{0})}(s) \,\mathrm{d}s \end{pmatrix}\\ &\geq \begin{pmatrix} 0 \\ 0 \end{pmatrix} = \begin{pmatrix} D^{\alpha -1}w_{0}(t) \\ D^{\beta -1}z_{0}(t) \end{pmatrix} . \end{aligned} \end{aligned}$$

Using the the monotonicity assumption (C3) of functions φ and ψ, we can obtain

$$\begin{aligned}& \begin{pmatrix} w_{2}(t) \\ z_{2}(t) \end{pmatrix} = \begin{pmatrix} \digamma _{1}(w_{1},z_{1})(t) \\ \digamma _{2}(w_{1},z_{1})(t) \end{pmatrix}\geq \begin{pmatrix} \digamma _{1}(w_{0},z_{0})(t) \\ \digamma _{2}(w_{0},z_{0})(t) \end{pmatrix} = \begin{pmatrix} w_{1}(t) \\ z_{1}(t) \end{pmatrix} , \\& \begin{pmatrix} D^{\alpha -1}w_{2}(t) \\ D^{\beta -1}z_{2}(t) \end{pmatrix} = \begin{pmatrix} D^{\alpha -1}\digamma _{1}(w_{1},z_{1})(t) \\ D^{\beta -1} \digamma _{2}(w_{1},z_{1})(t) \end{pmatrix}\geq \begin{pmatrix} D^{\alpha -1}\digamma _{1}(w_{0},z_{0})(t) \\ D^{\beta -1}\digamma _{2}(w_{0},z_{0})(t) \end{pmatrix} = \begin{pmatrix} D^{\alpha -1}w_{1}(t) \\ D^{\beta -1}z_{1}(t) \end{pmatrix} . \end{aligned}$$

Analogously, for \(n=0,1,2,\ldots \) and \(t\in J\), we know

$$ \begin{pmatrix} w_{n+1}(t) \\ z_{n+1}(t) \end{pmatrix} \geq \begin{pmatrix} w_{n}(t) \\ z_{n}(t) \end{pmatrix}, \qquad \begin{pmatrix} D^{\alpha -1}w_{n+1}(t) \\ D^{\beta -1}z_{n+1}(t) \end{pmatrix}\geq \begin{pmatrix} D^{\alpha -1}w_{n}(t) \\ D^{\beta -1}z_{n}(t) \end{pmatrix} . $$

Applying the iterative sequence \((w_{n+1},z_{n+1})=\digamma (w_{n},z_{n})\) and the complete continuity of the operator Ϝ, it is easy to acquire that \((w_{n},z_{n})\rightarrow (w^{\ast },z^{\ast })\) and \(\digamma (w^{\ast },z^{\ast })=(w^{\ast },z^{\ast })\). Thus \((w^{\ast },z^{\ast })\) is also a fixed point of Ϝ.

Finally we prove that \((u^{\ast },v^{\ast })\) and \((w^{\ast },z^{\ast })\) are the minimal and maximal positive solutions of system (1.6). Suppose that \((\xi (t),\eta (t))\) is any positive solution of system (1.6), then \(\digamma (\xi (t),\eta (t))=(\xi (t),\eta (t))\) and

$$\begin{aligned}& \begin{pmatrix} w_{0}(t) \\ z_{0}(t) \end{pmatrix} = \begin{pmatrix} 0 \\ 0 \end{pmatrix} \leq \begin{pmatrix} \xi (t) \\ \eta (t) \end{pmatrix}\leq \begin{pmatrix} Rt^{\alpha -1} \\ Rt^{\beta -1} \end{pmatrix} = \begin{pmatrix} u_{0}(t) \\ v_{0}(t) \end{pmatrix} , \\& \begin{pmatrix} D^{\alpha -1}w_{0}(t) \\ D^{\beta -1}z_{0}(t) \end{pmatrix} \leq \begin{pmatrix} D^{\alpha -1}\xi (t) \\ D^{\beta -1}\eta (t) \end{pmatrix}\leq \begin{pmatrix} D^{\alpha -1}u_{0}(t) \\ D^{\beta -1}v_{0}(t) \end{pmatrix} . \end{aligned}$$

Using the monotone property of the operator Ϝ, we obtain that

$$\begin{aligned}& \begin{pmatrix} w_{1}(t) \\ z_{1}(t) \end{pmatrix} = \begin{pmatrix} \digamma _{1}(w_{0},z_{0})(t) \\ \digamma _{2}(w_{0},z_{0})(t) \end{pmatrix} \leq \begin{pmatrix} \xi (t) \\ \eta (t) \end{pmatrix} \leq \begin{pmatrix} \digamma _{1}(u_{0},v_{0})(t) \\ \digamma _{2}(u_{0},v_{0})(t) \end{pmatrix} = \begin{pmatrix} u_{1}(t) \\ v_{1}(t) \end{pmatrix} , \\& \begin{pmatrix} D^{\alpha -1} w_{1}(t) \\ D^{\beta -1}z_{1}(t) \end{pmatrix}\leq \begin{pmatrix} D^{\alpha -1} \xi (t) \\ D^{\beta -1}\eta (t) \end{pmatrix} \leq \begin{pmatrix} D^{\alpha -1} u_{1}(t) \\ D^{\beta -1} v_{1}(t) \end{pmatrix} . \end{aligned}$$

Repeating the above process, we have

$$\begin{aligned}& \begin{pmatrix} w_{n}(t) \\ z_{n}(t) \end{pmatrix}\leq \begin{pmatrix} \xi (t) \\ \eta (t) \end{pmatrix} \leq \begin{pmatrix} u_{n}(t) \\ v_{n}(t) \end{pmatrix}, \\& \begin{pmatrix} D^{\alpha -1}w_{n}(t) \\ D^{\beta -1}z_{n}(t) \end{pmatrix} \leq \begin{pmatrix} D^{\alpha -1}\xi (t) \\ D^{\beta -1}\eta (t) \end{pmatrix}\leq \begin{pmatrix} D^{\alpha -1}u_{n}(t) \\ D^{\beta -1}v_{n}(t) \end{pmatrix} . \end{aligned}$$

Combining \(\lim_{n\rightarrow \infty } (w_{n},z_{n})=(w^{\ast },z^{\ast })\) and \(\lim_{n\rightarrow \infty } (u_{n},v_{n})=(u^{\ast },v^{\ast })\), the results (3.10) and (3.11) come naturally.

Again \(\varphi (t,0,0,0)\neq 0\) and \(\psi (t,0,0,0)\neq 0\) for all \(t\in J\), we know that \((0,0)\) is not a solution of system (1.6). By (3.10) and (3.11), it is obvious that \((w^{\ast },z^{\ast })\) and \((u^{\ast },v^{\ast })\) are the extreme positive solutions of system (1.6), which can be constructed by means of two monotone iterative sequences in (3.8) and (3.9). Thus the proof is completed. □

Remark 3.4

When the parameters \(\lambda _{k}\), \(\tau _{k}\) (\(k=1,2,3\)) take different values, the same result can be obtained by using a similar method, so we omit the details.

Example 3.5

Consider the following fractional differential system on a half-line:

$$ \textstyle\begin{cases} -D^{2.5}u(t)=\frac{2}{(10+t)^{2}}+ \frac{e^{-t} \vert u(t) \vert ^{0.1}}{(1+t^{3.6})^{0.1}} + \frac{e^{-2t} \vert v(t) \vert ^{0.3}}{(1+t^{3.6})^{0.3}} + \frac{ \vert D^{1.1}v(t) \vert ^{0.4}}{1+t^{2}}, \\ -D^{2.1}v(t)=\frac{1}{(20+t)^{3}}+ \frac{e^{-3t} \vert u(t) \vert ^{0.2}}{(1+t^{3.6})^{0.2}} + \frac{e^{-4t} \vert v(t) \vert ^{0.4}}{(1+t^{3.6})^{0.4}} + \frac{3t^{2} \vert D^{1.5}u(t) \vert ^{0.2}}{(3+t^{3})^{2}}, \\ u(0)=u'(0)=0,\qquad D^{1.5}u(+\infty )= \int _{0}^{+\infty }t^{-1.1}e^{-t}v(t)\,dt, \\ v(0)=v'(0)=0,\qquad D^{1.1}v(+\infty )= \int _{0}^{+\infty }t^{-1.5}e^{-2t}u(t)\,dt, \end{cases} $$
(3.14)

where \(\alpha =2.5\), \(\beta =2.1\), \(h(t)=t^{-1.1}e^{-t}\), \(g(t)=t^{-1.5}e^{-2t}\), \(\lambda _{1}=0.1\), \(\lambda _{2}=0.3\), \(\lambda _{3}=0.4\), \(\tau _{1}=0.2\), \(\tau _{2}=0.4\), \(\tau _{3}=0.2\), \(\varGamma (2.5)=1.329340\), \(\varGamma (2.1)=1.046486\), \(\varUpsilon _{1}=\int _{0}^{+\infty }h(t)t^{1.1}\,dt=1\), \(\varUpsilon _{2}=\int _{0}^{+ \infty }g(t)t^{1.5}\,dt=0.5\), and

$$\begin{aligned}& \varphi (t,u,v,w)=\frac{2}{(10+t)^{2}}+ \frac{e^{-t} \vert u \vert ^{0.1}}{(1+t^{3.6})^{0.1}} + \frac{e^{-2t} \vert v \vert ^{0.3}}{(1+t^{3.6})^{0.3}} + \frac{ \vert w \vert ^{0.4}}{1+t^{2}}, \\& \psi (t,u,v,z)=\frac{1}{(20+t)^{3}}+ \frac{e^{-3t} \vert u \vert ^{0.2}}{(1+t^{3.6})^{0.2}} + \frac{e^{-4t} \vert v \vert ^{0.4}}{(1+t^{3.6})^{0.4}} + \frac{3t^{2} \vert z \vert ^{0.2}}{(3+t^{3})^{2}}. \end{aligned}$$

It is easy to know that \(\varGamma (2.5)\varGamma (2.1)>\varUpsilon _{1}\varUpsilon _{2}\). So assumptions (C0) and (C1) are satisfied.

Noting that

$$\begin{aligned}& \begin{aligned} \bigl\vert \varphi (t,u,v,w) \bigr\vert &\leq \frac{2}{(10+t)^{2}}+ \frac{e^{-t} \vert u \vert ^{0.1}}{(1+t^{3.6})^{0.1}} + \frac{e^{-2t} \vert v \vert ^{0.3}}{(1+t^{3.6})^{0.3}} + \frac{ \vert w \vert ^{0.4}}{1+t^{2}} \\ &=a_{0}(t)+a_{1}(t) \vert u \vert ^{0.1}+a_{2}(t) \vert v \vert ^{0.3}+a_{3}(t) \vert w \vert ^{0.4}, \end{aligned} \\& \begin{aligned} \bigl\vert \psi (t,u,v,z) \bigr\vert &\leq \frac{1}{(20+t)^{3}}+ \frac{e^{-3t} \vert u \vert ^{0.2}}{(1+t^{3.6})^{0.2}} + \frac{e^{-4t} \vert v \vert ^{0.4}}{(1+t^{3.6})^{0.4}} + \frac{3t^{2} \vert z \vert ^{0.2}}{(3+t^{3})^{2}} \\ &=b_{0}(t)+b_{1}(t) \vert u \vert ^{0.2}+b_{2}(t) \vert v \vert ^{0.4}+b_{3}(t) \vert z \vert ^{0.2}, \end{aligned} \end{aligned}$$

and

$$\begin{aligned}& a_{0}^{\ast }= \int _{0}^{+\infty }a_{0}(t)\,dt= \frac{1}{5},\qquad a_{1}^{\ast }= \int _{0}^{+\infty }a_{1}(t) \bigl(1+t^{3.6}\bigr)^{0.1}\,dt=1, \\& a_{2}^{\ast }= \int _{0}^{+\infty }a_{2}(t) \bigl(1+t^{3.6}\bigr)^{0.3}\,dt= \frac{1}{2},\qquad a_{3}^{\ast }= \int _{0}^{+\infty }a_{3}(t)\,dt= \frac{\pi }{2}, \\& b_{0}^{\ast }= \int _{0}^{+\infty }b_{0}(t)\,dt= \frac{1}{800},\qquad b_{1}^{\ast }= \int _{0}^{+\infty }b_{1}(t) \bigl(1+t^{3.6}\bigr)^{0.2}\,dt= \frac{1}{3}, \\& b_{2}^{\ast }= \int _{0}^{+\infty }b_{2}(t) \bigl(1+t^{3.6}\bigr)^{0.4}\,dt= \frac{1}{4},\qquad b_{3}^{\ast }= \int _{0}^{+\infty }b_{3}(t)\,dt=\pi , \end{aligned}$$

which means that assumption (C2) is satisfied.

From the expression of functions φ, ψ, it is obvious that φ, ψ are increasing with respect to the variables u, v, w and u, v, z, and \(\varphi (t,0,0,0)\not \equiv 0\), \(\psi (t,0,0,0)\not \equiv 0\), \(\forall t \in J\). Thus assumption (C3) is satisfied. By Theorem 3.3, it follows that the fractional differential system (3.14) has two positive solutions, which can be established by the limit means of two explicit monotone iterative sequences in (3.8) and (3.9).

4 Conclusions

In this paper, we apply the monotone iterative technique to study a fractional differential system with coupled integral boundary conditions in a half-line. We first transform system (1.6) into an equivalent operator equation (3.1), and then we construct some norm inequalities related to nonlinear terms φ, ψ and a new Banach space. Finally, some explicit monotone iterative sequences for approximating the extreme positive solutions are obtained.