1 Introduction

Fractional calculus has recently evolved as an excellent tool for mathematical modeling owing to its widespread applications in the fields of engineering, physics, electrodynamics of complex medium, photoelasticity, etc; one can see [1,2,3,4,5,6,7,8,9,10,11,12] and the references cited therein. Meanwhile, relevant theory of fractional differential and integral equations has been established, and the research on fractional differential equations for boundary value problems is in a stage of rapid development.

Based on some kinds of analytical techniques, boundary value problems involving fractional differential equations attracted a considerable attention; see [13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33] and the references therein. It not only has promotional value and practical significance in medical image processing, seismic analysis, and large-scale climate research, but also has important research potential on numerical analysis.

Recently, the study of coupled systems involving fractional differential equations appeared in the literature [4, 9, 10, 17, 18, 32, 33]. Much of the work has been considered on finite intervals; however, a study of boundary value problems on unbounded domain is well under way. Wang, Ahmad, and Zhang [34] studied a coupled system of fractional differential equations with m-point fractional boundary conditions

$$ \textstyle\begin{cases} D^{p} u(t)+ f(t,v(t))=0, \quad p\in (2,3), \\ D^{q} v(t)+ g(t,u(t))=0, \quad q\in (2,3), \\ u(0)=u'(0)=0, \quad\quad D^{p-1}u(+\infty )=\sum_{i=1}^{m-2}\beta _{i}u( \xi _{i}), \\ v(0)=v'(0)=0, \quad\quad D^{q-1}v(+\infty )=\sum_{i=1}^{m-2}\gamma _{i}v( \xi _{i}), \end{cases} $$

where \(t\in J=[0,+\infty )\), \(f,g\in C(J \times \mathbb{R}, \mathbb{R})\), \(0<\xi _{1}<\xi _{2}<\cdots <\xi _{m-2}<+\infty \), \(\beta _{i},\gamma _{i}>0\) such that \(0<\sum_{i=1}^{m-2}\beta _{i}u(\xi _{i})< \varGamma (p)\) and \(0<\sum_{i=1}^{m-2}\gamma _{i}v(\xi _{i})<\varGamma (q)\). \(D^{p}\), \(D^{q}\) denote the standard Riemann–Liouville fractional derivatives. By virtue of standard fixed point theorems, the authors discussed the existence and uniqueness of solutions.

In [35], the authors investigated a class of fractional differential equations on an infinite interval

$$ D_{0^{+}}^{\alpha }u(t)+ f\bigl(t,u(t),D_{0^{+}}^{\alpha -1} u(t)\bigr)=0, \quad t\in (0,+\infty ), $$

with integral boundary conditions

$$ \textstyle\begin{cases} u(0)=0, \\ D_{0^{+}}^{\alpha -1}u(\infty )=\int _{0}^{\tau }g_{1}(s)u(s)\,ds+a, \\ D_{0^{+}}^{\alpha -2}u(0)=\int _{0}^{\tau }g_{2}(s)u(s)\,ds+b, \end{cases} $$

where \(2<\alpha \leq 3\), \(f:\mathbb{R}^{+}\times (\mathbb{R}^{+})^{2} \rightarrow \mathbb{R}^{+}\), \(f(t,u,v)\not \equiv 0\), and f satisfies \(L^{1}\)-Carathéodory conditions. Existence results for positive solutions to the boundary value problem were obtained in three cases by using Krasnoselskii’s fixed point theorem.

To our knowledge, some remarkable results on the existence and multiplicity of solutions for fractional differential equations have been discussed widely on finite intervals [14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33]. Instead, it is relatively rare for work to be done related to existence results on infinite intervals [34,35,36,37,38,39,40,41,42,43,44,45,46,47].

In [31], the authors discussed the existence and uniqueness of positive solutions for the fractional differential equation

$$ \textstyle\begin{cases} D^{\alpha }u(t)=f(t,u(t),D^{p} u(t)), \\ I^{3-\alpha }u(0)=D^{\alpha -2}u(0)=u(1)=0, \end{cases} $$

where \(0< p<1\), \(2<\alpha <3\), \(t\in (0,1)\), \(D^{\alpha }\) is the standard Riemann–Liouville fractional derivative of order α. By applying a nonlinear alternative of Leray–Schauder type and the Banach contraction theorem, the existence and uniqueness of solutions were obtained.

Motivated by the above papers, we are devoted to establishing some results on the existence and uniqueness of solutions for a new coupled system of nonlinear fractional differential equations

$$ \textstyle\begin{cases} D^{\alpha }u(t)=-\varphi (t,v(t),D^{\gamma _{1}}v(t)), \quad \gamma _{1} \in (0,1), \\ D^{\beta }v(t)=-\psi (t,u(t),D^{\gamma _{2}}u(t)), \quad \gamma _{2}\in (0,1), \\ I^{3-\alpha }u(0)=0, \quad\quad D^{\alpha -2}u(0)=\int _{0}^{h} g_{1}(s)u(s)\,ds, \quad\quad D^{\alpha -1}u(+\infty )=Mu(\xi )+a, \\ I^{3-\beta }v(0)=0, \quad\quad D^{\beta -2}v(0)=\int _{0}^{h} g_{2}(s)v(s)\,ds, \quad\quad D ^{\beta -1}v(+\infty )=Nv(\eta )+b, \end{cases} $$
(1.1)

where \(2<\alpha , \beta \leq 3\), \(0<\gamma _{i}<1\), \(i=1,2\), \(t\in J=[0,+ \infty )\), M, N are real numbers with \(0< M\xi ^{\alpha -1}<{\varGamma ( \alpha )}\), \(0< N\eta ^{\beta -1}<{\varGamma (\beta )}\), \(\xi , \eta , h>0\), parameters \(a,b\in \mathbb{R}^{+}\), \(g_{1}, g_{2} \in L^{1}[0,h]\) are nonnegative functions, \(\varphi ,\psi \in C(J\times \mathbb{R}\times \mathbb{R}, \mathbb{R})\) and \(D^{\alpha }\), \(D^{\beta }\) denote the fractional derivatives of Riemann–Liouville type of order α and β. Our conclusion is a natural expansion of the previous results in [31].

In this paper, the aim is to deal with the new coupled system of fractional differential equations on infinite intervals. Sufficient conditions for the existence and uniqueness of unbounded solutions for system (1.1) are obtained base upon Schauder’s fixed point theorem and the Banach contraction theorem. Unlike previous works, the main difficulty of this paper is that we have to construct an appropriate Banach space, because the functions φ, ψ contain the fractional derivatives.

2 Preliminaries and auxiliary results

For the convenience of the readers, we recall some useful definitions and lemmas.

Definition 2.1

([1])

The fractional integral of Riemann–Liouville type of order \(\alpha >0\) of a function f is defined as

$$ \bigl(I^{\alpha }f\bigr) (t)={\frac{1}{\varGamma (\alpha )}} \int _{0}^{t}(t-s)^{ \alpha -1}f(s)\,ds, \quad \alpha >0, $$

provided the integral exists.

Definition 2.2

([1])

The fractional derivative of Riemann–Liouville type of order \(\alpha >0\) of a function f is given by

$$ \bigl(D^{\alpha }f\bigr) (t)=\bigl(D^{\lceil \alpha \rceil }I^{\lceil \alpha \rceil - \alpha }f\bigr) (t)={\frac{1}{\varGamma (n-\alpha )}} { \biggl(\frac{d}{dt} \biggr)} ^{n} \int _{0}^{t} {\frac{f(s)}{(t-s)^{\alpha -n+1}}}\,ds, $$

where \(\lceil \alpha \rceil \) is the smallest integer greater than or equal to α, provided that the right-hand side is pointwise defined on \((0,+\infty )\).

For further analysis, let

$$\begin{aligned} &\sigma _{1}=\frac{1}{\varGamma (\alpha )-M\xi ^{\alpha -1}}, \quad\quad \sigma _{2}= \frac{1}{ \varGamma (\beta )-N\eta ^{\beta -1}}, \\ &\omega _{1}= \int ^{h}_{0}g_{1}(t)t^{\alpha -1}\,dt, \quad\quad \omega _{2}= \int ^{h}_{0}g_{2}(t)t^{\beta -1} \,dt, \\ &\delta _{1}= \int _{0}^{h}\bigl(1+t^{\alpha -1} \bigr)g_{1}(t)\,dt, \quad\quad \delta _{2}= \int _{0}^{h}\bigl(1+t^{\beta -1} \bigr)g_{2}(t)\,dt, \\ &T_{1}(t)=\frac{\sigma _{1}M\xi ^{\alpha -2}t^{\alpha -1}+t^{\alpha -2}}{ \varGamma (\alpha -1)}, \quad\quad T_{2}(t)= \frac{\sigma _{2}N\eta ^{\beta -2}t^{ \beta -1}+t^{\beta -2}}{\varGamma (\beta -1)}, \\ &l_{1}={\frac{ 1+\sigma _{1}M\xi ^{\alpha -2} }{(1-\mu _{1})\varGamma ( \alpha -1)}}, \quad\quad l_{2}={ \frac{ 1+\sigma _{2}N\eta ^{\beta -2} }{(1-\mu _{2})\varGamma (\beta -1)}}. \end{aligned}$$

In this paper, we always assume that \(g_{i}:[0,+\infty )\rightarrow [0,+ \infty )\) are continuous, and \(\mu _{i}=\int _{0}^{h}g_{i}(t)T_{i}(t)\,dt<1\), \(i=1,2\).

Lemma 2.1

Assume that \(f\in L^{1}(J)\) with \(0< M\xi ^{\alpha -1}< {\varGamma (\alpha )}\), \(\alpha \in (2,3]\). Then the fractional differential equation

$$ D^{\alpha }u(t)+f(t)=0, \quad t\in [0,+\infty ), $$

with

$$ \textstyle\begin{cases} I^{3-\alpha }u(0)=0, \\ D^{\alpha -2}u(0)=\int _{0}^{h} g_{1}(s)u(s)\,ds, \\ D^{\alpha -1}u(+\infty )=Mu(\xi )+a, \end{cases} $$

has the solution

$$ u(t)= a\sigma _{1}t^{\alpha -1}+{\frac{a\sigma _{1}\omega _{1}T_{1}(t)}{1- \mu _{1}}}+ \int ^{+\infty }_{0} H(t,s)f(s)\,ds, $$

where

$$\begin{aligned}& H(t,s)=H_{1}(t,s)+{\frac{T_{1}(t)}{1-\mu _{1}}} \int ^{h}_{0}H_{1}( \tau ,s)g_{1}(\tau )\,d\tau , \end{aligned}$$
(2.1)
$$\begin{aligned}& H_{1}(t,s)=\sigma _{1}Mt^{\alpha -1}G_{1}(\xi ,s)+G_{1}(t,s), \end{aligned}$$
(2.2)

and

$$ G_{1}(t,s)=\frac{1}{\varGamma (\alpha )}\textstyle\begin{cases} t^{\alpha -1}-(t-s)^{\alpha -1}, \quad 0\leq s\leq t< +\infty , \\ t^{\alpha -1}, \hfill 0\leq t\leq s< +\infty . \end{cases} $$
(2.3)

Proof

First, we can reduce the above problem to an equivalent integral equation

$$ u(t)=c_{1}t^{\alpha -1}+c_{2}t^{\alpha -2}+c_{3}t^{\alpha -3}-I^{ \alpha }f(t) $$
(2.4)

for some \(c_{i}\in \mathbb{R}\), \(i=1,2,3\). By the condition \(I^{3- \alpha }u(t)\vert _{t=0}=0\), we have

$$\begin{aligned} I^{3-\alpha }u(t) =&c_{1}I^{3-\alpha }t^{\alpha -1}+c_{2}I^{3-\alpha }t^{\alpha -2}+c_{3}I^{3-\alpha }t^{\alpha -3}-I^{3-\alpha }I^{\alpha }f(t) \\ =&c_{1}{\frac{\varGamma (\alpha )}{\varGamma (3)}}t^{2}+c_{2}{ \frac{\varGamma (\alpha -1)}{\varGamma (2)}}t+c_{3}{\frac{\varGamma (\alpha -2)}{\varGamma (1)}}-I ^{3}f(t), \end{aligned}$$

since \(I^{3}f(t)\rightarrow 0\) as \(t \rightarrow 0\), we must set \(c_{3}=0\). On application of \(D^{\alpha -2}u(0)=\int _{0}^{h} g_{1}(s)u(s)\,ds\) and \(D^{\alpha -1}u(+\infty )=Mu(\xi )+a\), we have

$$\begin{aligned}& \begin{aligned} D^{\alpha -2}u(t) &=c_{1}D^{\alpha -2}t^{\alpha -1}+c_{2}D^{\alpha -2}t ^{\alpha -2}-D^{\alpha -2}I^{\alpha }f(t) \\ &=c_{1}\varGamma (\alpha )t+c_{2}\varGamma (\alpha -1)- \int _{0}^{t}(t-s)f(s)\,ds, \end{aligned} \\ & \begin{aligned} D^{\alpha -1}u(t) &=c_{1}D^{\alpha -1}t^{\alpha -1}+c_{2}D^{\alpha -1}t ^{\alpha -2}-D^{\alpha -1}I^{\alpha }f(t) \\ &=c_{1}\varGamma (\alpha )- \int _{0}^{t}f(s)\,ds, \end{aligned} \end{aligned}$$

that is,

$$\begin{aligned}& \begin{aligned} c_{1}&={\frac{1}{\varGamma (\alpha )-M\xi ^{\alpha -1}}} \biggl(a+ \int ^{+\infty }_{0}f(s)\,ds+{\frac{M\xi ^{\alpha -2}}{\varGamma (\alpha -1)}} \int _{0}^{h} g_{1}(s)u(s)\,ds \\ &\quad{} -{ \frac{M}{\varGamma (\alpha )}} \int ^{\xi } _{0}{(\xi -s)}^{\alpha -1}f(s)\,ds \biggr), \end{aligned} \\ & c_{2}={\frac{1}{\varGamma (\alpha -1)}} \int _{0}^{h} g_{1}(s)u(s)\,ds. \end{aligned}$$

This implies

$$\begin{aligned} u(t) =&\sigma _{1}t^{\alpha -1} \biggl(a+ \int ^{+\infty }_{0} f(s)\,ds+ {\frac{M\xi ^{\alpha -2}}{\varGamma (\alpha -1)}} \int _{0}^{h} g_{1}(s)u(s)\,ds \\ & {}- { \frac{M}{\varGamma (\alpha )}} \int ^{\xi }_{0}{(\xi -s)}^{\alpha -1}f(s)\,ds \biggr) \\ & {} +{\frac{t^{\alpha -2}}{\varGamma (\alpha -1)}} \int _{0}^{h} g_{1}(s)u(s)\,ds- { \frac{1}{\varGamma (\alpha )}} \int ^{t}_{0}{(t-s)}^{\alpha -1}f(s)\,ds \\ =&a\sigma _{1}t^{\alpha -1}+\frac{\sigma _{1}M\xi ^{\alpha -2}t^{\alpha -1}+t^{\alpha -2}}{\varGamma (\alpha -1)} \int _{0}^{h} g_{1}(s)u(s)\,ds- { \frac{\sigma _{1}Mt^{\alpha -1}}{\varGamma (\alpha )}} \int ^{\xi }_{0} {(\xi -s)}^{\alpha -1}f(s)\,ds \\ & {} + \biggl({\frac{t^{\alpha -1}}{\varGamma (\alpha )}}+{\frac{\sigma _{1}M \xi ^{\alpha -1}t^{\alpha -1}}{\varGamma (\alpha )}} \biggr) \int ^{+ \infty }_{0} f(s)\,ds -{\frac{1}{\varGamma (\alpha )}} \int ^{t}_{0}{(t-s)} ^{\alpha -1}f(s)\,ds \\ =&a\sigma _{1}t^{\alpha -1}+{\frac{\sigma _{1}M\xi ^{\alpha -1}t^{ \alpha -1}}{\varGamma (\alpha )}} \int ^{+\infty }_{0} f(s)\,ds-{\frac{ \sigma _{1}Mt^{\alpha -1}}{\varGamma (\alpha )}} \int ^{\xi }_{0}{(\xi -s)} ^{\alpha -1}f(s)\,ds \\ & {} +{\frac{t^{\alpha -1}}{\varGamma (\alpha )}} \int ^{+\infty }_{0} f(s)\,ds- {\frac{1}{\varGamma (\alpha )}} \int ^{t}_{0}{(t-s)}^{\alpha -1}f(s)\,ds+T _{1}(t) \int _{0}^{h} g_{1}(s)u(s)\,ds \\ =& a\sigma _{1}t^{\alpha -1}+\sigma _{1}Mt^{\alpha -1} \int ^{+\infty } _{0} G_{1}(\xi ,s)f(s)\,ds+ \int ^{+\infty }_{0} G_{1}(t,s)f(s) \,ds \\ & {}+T_{1}(t) \int _{0}^{h} g_{1}(s)u(s)\,ds. \end{aligned}$$

Multiplying both sides of the above equality by \(g_{1}(t)\) and integrating from 0 to h, then

$$\begin{aligned} \begin{aligned} \int _{0}^{h} g_{1}(t)u(t)\,dt &=a\sigma _{1} \int _{0}^{h} g_{1}(t)t^{ \alpha -1} \,dt+M\sigma _{1} \int _{0}^{h} g_{1}(t)t^{\alpha -1}\,dt \int ^{+\infty }_{0} G_{1}(\xi ,s)f(s)\,ds \\ & \quad {} + \int _{0}^{h} g_{1}(t) \int ^{+\infty }_{0} G_{1}(t,s)f(s)\,ds\,dt+ \int _{0}^{h} g_{1}(t)T_{1}(t) \,dt \int _{0}^{h} g_{1}(s)u(s)\,ds. \end{aligned} \end{aligned}$$

Next we have

$$\begin{aligned} \int _{0}^{h} g_{1}(t)u(t)\,dt =&{ \frac{a\sigma _{1}}{1-\mu _{1}}} \int _{0} ^{h} g_{1}(t)t^{\alpha -1} \,dt+{\frac{M\sigma _{1}}{1-\mu _{1}}} \int _{0} ^{h} g_{1}(t)t^{\alpha -1} \,dt \int ^{+\infty }_{0} G_{1}(\xi ,s)f(s)\,ds \\ & {} +{\frac{1}{1-\mu _{1}}} \int _{0}^{h} g_{1}(t) \int ^{+\infty }_{0} G _{1}(t,s)f(s)\,ds\,dt. \end{aligned}$$

Finally, we can obtain

$$\begin{aligned} u(t) =& a\sigma _{1}t^{\alpha -1}+\sigma _{1}Mt^{\alpha -1} \int ^{+ \infty }_{0} G_{1}(\xi ,s)f(s)\,ds+ \int ^{+\infty }_{0} G_{1}(t,s)f(s)\,ds \\ & {} +{\frac{a\sigma _{1}T_{1}(t)}{1-\mu _{1}}} \int _{0}^{h} g_{1}(\tau ) \tau ^{\alpha -1}\,d\tau +{\frac{T_{1}(t)}{1-\mu _{1}}} \int _{0}^{h} g_{1}( \tau ) \int ^{+\infty }_{0} G_{1}(\tau ,s)f(s)\,ds\,d \tau \\ & {} +{\frac{M\sigma _{1}T_{1}(t)}{1-\mu _{1}}} \int _{0}^{h} g_{1}(\tau ) \tau ^{\alpha -1}\,d\tau \int ^{+\infty }_{0} G_{1}(\xi ,s)f(s)\,ds \\ =&a\sigma _{1}t^{\alpha -1}+{\frac{a\sigma _{1}T_{1}(t)}{1-\mu _{1}}} \int _{0}^{h} g_{1}(\tau )\tau ^{\alpha -1}\,d\tau + \int ^{+\infty }_{0} H _{1}(t,s)f(s)\,ds \\ & {} +{\frac{T_{1}(t)}{1-\mu _{1}}} \int _{0}^{h} g_{1}(\tau ) \int ^{+ \infty }_{0} H_{1}(\tau ,s)f(s)\,ds\,d \tau \\ =&a\sigma _{1}t^{\alpha -1}+{\frac{a\sigma _{1}\omega _{1}T_{1}(t)}{1- \mu _{1}}}+ \int ^{+\infty }_{0} H(t,s)f(s)\,ds. \end{aligned}$$

This completes the proof of the lemma. □

We can easily get the following result.

Lemma 2.2

The function \(G_{1}(t,s)\) defined by (2.3) satisfies:

  1. (i)

    \(G_{1}\) is continuous and \(G_{1}(t,s)\geq 0\), \(0\leq t, s< + \infty \);

  2. (ii)

    \(G_{1}(t,s)\) is increasing in t, \(0\leq t,s< +\infty \).

Remark 2.1

For \(0\leq t, s< +\infty \), we can easily obtain

$$ {\frac{G_{1}(t,s)}{1+t^{\alpha -1}}}\leq {\frac{1}{\varGamma (\alpha )}}, \qquad {\frac{G_{1}(\xi ,s)t^{\alpha -1}}{1+t^{\alpha -1}}}\leq { \frac{ \xi ^{\alpha -1}}{\varGamma (\alpha )}}, \quad \xi >0. $$

Lemma 2.3

The function \(H(t,s)\) satisfies the following inequality:

$$ {\frac{H(t,s)}{1+t^{\alpha -1}}}\leq \sigma _{1}+{\frac{\delta _{1}( \sigma _{1}+\sigma ^{2}_{1}M\xi ^{\alpha -2})}{(1-\mu _{1})\varGamma (\alpha -1)}}=\sigma _{1}(1+\delta _{1}l_{1}), \quad \forall t,s\in [0,+\infty ). $$

Proof

From Remark 2.1, we have

$$ {\frac{H_{1}(t,s)}{1+t^{\alpha -1}}}={\frac{\sigma _{1}Mt^{\alpha -1}G _{1}(\xi ,s)}{1+t^{\alpha -1}}}+{\frac{G_{1}(t,s)}{1+t^{\alpha -1}}} \leq { \frac{ 1+\sigma _{1} M\xi ^{\alpha -1}}{ \varGamma (\alpha )}}=\sigma _{1}, $$

thus, from (2.1), we get

$$\begin{aligned} {\frac{H(t,s)}{1+t^{\alpha -1}}} =& {\frac{H_{1}(t,s)}{1+t^{\alpha -1}}}+ {\frac{T_{1}(t)}{(1-\mu _{1})(1+t^{\alpha -1})}} \int ^{h}_{0}H_{1}( \tau ,s)g_{1}(\tau )\,d\tau \\ \leq &\sigma _{1}+\frac{(1+\sigma _{1}M\xi ^{\alpha -2})\sigma _{1}}{(1- \mu _{1})\varGamma (\alpha -1)} \int ^{h}_{0}\bigl(1+\tau ^{\alpha -1} \bigr)g_{1}( \tau )\,d\tau \\ =&\sigma _{1}+{\frac{\delta _{1}(\sigma _{1}+\sigma ^{2}_{1}M\xi ^{\alpha -2})}{(1-\mu _{1})\varGamma (\alpha -1)}}=\sigma _{1}(1+\delta _{1}l_{1}). \end{aligned}$$

The proof is completed. □

The general solution of

$$ \textstyle\begin{cases} D^{\beta }v(t)+g(t)=0, \quad \beta \in (2,3], t\in [0,+\infty ), \\ I^{3-\beta }v(t)\vert _{t=0}=0, \quad\quad D^{\beta -2}v(t)\vert _{t=0}=\int _{0}^{h} g _{2}(s)v(s)\,ds, \quad\quad D^{\beta -1}v(+\infty )=Nv(\eta )+b \end{cases} $$

can be written by

$$ v(t)= b\sigma _{2}t^{\beta -1}+{\frac{b\sigma _{2}\omega _{2}T_{2}(t)}{1- \mu _{2}}}+ \int ^{+\infty }_{0} K(t,s)g(s)\,ds, $$

where

$$\begin{aligned}& K(t,s)=K_{1}(t,s)+{\frac{T_{2}(t)}{1-\mu _{2}}} \int ^{h}_{0}K_{1}( \tau ,s)g_{2}(\tau )\,d\tau , \\& K_{1}(t,s)=\sigma _{2}Nt^{\beta -1}G_{2}( \xi ,s)+G_{2}(t,s), \end{aligned}$$

and \(G_{2}(t,s)\) can be obtained from \(G_{1}(t,s)\) by replacing α with β.

Hence, system (1.1) is equivalent to the following integral system:

$$ \textstyle\begin{cases} u(t)= a\sigma _{1}t^{\alpha -1}+{\frac{a\sigma _{1}\omega _{1}T_{1}(t)}{1- \mu _{1}}}+\int ^{+\infty }_{0} H(t,s)\varphi (s,v(s),D^{\gamma _{1}}v(s))\,ds, \\ v(t)= b\sigma _{2}t^{\beta -1}+{\frac{b\sigma _{2}\omega _{2}T_{2}(t)}{1- \mu _{2}}}+\int ^{+\infty }_{0} K(t,s)\psi (s,u(s),D^{\gamma _{2}}u(s))\,ds. \end{cases} $$

Define two spaces

$$\begin{aligned}& X= \biggl\{ u\in C(J), D^{\gamma _{1}}u\in C(J) \Bigm| \underset{t \in J}{\sup } { \frac{ \vert u(t) \vert }{1+t^{\alpha -1}}}< +\infty , \underset{t \in J}{\sup } {\frac{ \vert D^{\gamma _{1}}u(t) \vert }{1+t^{\alpha -1-\gamma _{1}}}}< + \infty \biggr\} , \\& Y= \biggl\{ v\in C(J), D^{\gamma _{2}}v\in C(J)\Bigm| \underset{t \in J}{\sup } { \frac{ \vert v(t) \vert }{1+t^{\beta -1}}}< +\infty , \underset{t \in J}{\sup } {\frac{ \vert D^{\gamma _{2}}v(t) \vert }{1+t^{\beta -1-\gamma _{2}}}}< + \infty \biggr\} , \end{aligned}$$

equipped with the norms

$$ \Vert u \Vert _{X}=\underset{t\in J}{\sup } { \frac{ \vert u(t) \vert }{1+t^{\alpha -1}}}+\underset{t \in J}{\sup } {\frac{ \vert D^{\gamma _{1}}u(t) \vert }{1+t^{\alpha -1-\gamma _{1}}}}, \qquad \Vert v \Vert _{Y}=\underset{t\in J}{\sup } {\frac{ \vert v(t) \vert }{1+t^{\beta -1}}}+ \underset{t \in J}{\sup } {\frac{ \vert D^{\gamma _{2}}v(t) \vert }{1+t^{\beta -1-\gamma _{2}}}}, $$

where \(0<\gamma _{i}<1\), \(i=1,2\). \(C(J)\) denotes the space of all continuous functions defined on \([0,+\infty )\).

Lemma 2.4

\((X, \Vert \cdot \Vert _{X})\) is a Banach space.

Proof

Let \(\{u_{n}\}^{\infty }_{n=1}\) be a Cauchy sequence in the space \((X, \Vert \cdot \Vert _{X})\); then \(\forall \varepsilon >0\), \(\exists N( \varepsilon )>0\) such that

$$ \biggl\vert {\frac{u_{n}(t)}{1+t^{\alpha -1}}}-{\frac{u_{m}(t)}{1+t^{ \alpha -1}}} \biggr\vert + \biggl\vert {\frac{D^{\gamma _{1}}u_{n}(t)}{1+t^{ \alpha -1-\gamma _{1}}}}-{\frac{D^{\gamma _{1}}u_{m}(t)}{1+t^{\alpha -1- \gamma _{1}}}} \biggr\vert < \varepsilon , $$

for any \(t\in J\) and \(n,m>N(\varepsilon )\). We have \(\lim_{n\rightarrow +\infty }\frac{u_{n}(t)}{1+t^{\alpha -1}}=\frac{u(t)}{1+t ^{\alpha -1}}\), \(u(t)\in C(J)\). Then, for \(\frac{\varLambda _{0}}{2}=\underset{t \in J}{\sup }{\frac{ \vert u(t) \vert }{1+t^{\alpha -1}}}>0\), there exists \(N>0\) such that \(\vert {\frac{u_{n}(t)}{1+t^{\alpha -1}}}-{\frac{u(t)}{1+t ^{\alpha -1}}} \vert <\frac{\varLambda _{0}}{2}\), \(n>N\). Further, set \(\varLambda _{i}=\underset{t\in J}{\sup }{\frac{ \vert u_{i}(t) \vert }{1+t^{\alpha -1}}}\), \(i=1,2,\ldots,N\), and \(\varLambda = \max \{\varLambda _{i}, i=0,1,2,\ldots,N\}\). Then \({\frac{ \vert u_{n}(t) \vert }{1+t^{\alpha -1}}}\leq \varLambda \). Clearly, \(\{{\frac{u_{n}(t)}{1+t^{\alpha -1}}}\}^{\infty }_{n=1}\) and \(\{{\frac{D^{\gamma _{1}}u_{n}(t)}{1+t^{\alpha -1-\gamma _{1}}}}\}^{ \infty }_{n=1}\) are Cauchy sequences in the space \(C(J)\). Therefore, \(\{{\frac{D^{\gamma _{1}}u_{n}(t)}{1+t^{\alpha -1-\gamma _{1}}}} \} ^{\infty }_{n=1}\) converges uniformly to some \(v\in C(J)\) and \(\underset{t\in J}{\sup } \vert v(t) \vert <+\infty \). We need to prove that \(v={\frac{D^{\gamma _{1}}u(t)}{1+t^{\alpha -1-\gamma _{1}}}}\). For any \(t\in J\), we have

$$\begin{aligned}& \int ^{t}_{0} {(t-s)}^{-\gamma _{1}-1} \bigl(1+s^{\alpha -1}\bigr){\frac{u_{n}(s)}{1+s ^{\alpha -1}}}\,ds \\& \quad \leq \varLambda \int ^{t}_{0} {(t-s)}^{-\gamma _{1}-1} \bigl(1+s^{\alpha -1}\bigr)\,ds \\& \quad = \varLambda t^{-\gamma _{1}} \int ^{1}_{0} {(1-\tau )}^{-\gamma _{1}-1}d \tau + \varLambda t^{\alpha -1-\gamma _{1}} \int ^{1}_{0}\tau ^{\alpha -1} {(1-\tau )}^{-\gamma _{1}-1}\,d\tau \\& \quad = \varLambda t^{-\gamma _{1}}B(1,-\gamma _{1})+\varLambda t^{\alpha -1-\gamma _{1}}B(\alpha ,-\gamma _{1}) \\& \quad = \varLambda t^{-\gamma _{1}}\frac{\varGamma (-\gamma _{1})}{\varGamma (1-\gamma _{1})}+\varLambda t^{\alpha -1-\gamma _{1}} \frac{\varGamma (\alpha )\varGamma (- \gamma _{1})}{\varGamma (\alpha -\gamma _{1})}. \end{aligned}$$

Furthermore, by Lebesgue’s dominated convergence theorem, and considering the uniform convergence of \(\{{\frac{D^{\gamma _{1}}u _{n}(t)}{1+t^{\alpha -1-\gamma _{1}}}} \}^{\infty }_{n=1}\), one has

$$\begin{aligned} v(t) =&\lim_{n\rightarrow +\infty }{\frac{D^{\gamma _{1}}u_{n}(t)}{1+t ^{\alpha -1-\gamma _{1}}}} \\ =&\lim_{n\rightarrow +\infty }{\frac{1}{(1+t^{\alpha -1-\gamma _{1}}){\varGamma (1-\gamma _{1})}}}\cdot {\frac{d}{dt}} \int ^{t}_{0} {(t-s)}^{-\gamma _{1}}u_{n}(s) \,ds \\ =&{\frac{1}{(1+t^{\alpha -1-\gamma _{1}}){\varGamma (1-\gamma _{1})}}} \int ^{t}_{0} {(t-s)}^{-\gamma _{1}-1} \bigl(1+s^{\alpha -1}\bigr){\frac{u(s)}{1+s^{ \alpha -1}}}\,ds \\ =& {\frac{D^{\gamma _{1}}u(t)}{1+t^{\alpha -1-\gamma _{1}}}}. \end{aligned}$$

Thus

$$ \lim_{n\rightarrow +\infty }{\frac{u_{n}(t)}{1+t^{\alpha -1}}}+ {\frac{D^{\gamma _{1}}u_{n}(t)}{1+t^{\alpha -1-\gamma _{1}}}} ={ \frac{u(t)}{1+t ^{\alpha -1}}}+{\frac{D^{\gamma _{1}}u(t)}{1+t^{\alpha -1-\gamma _{1}}}}. $$

Therefore, we conclude that \((X, \Vert \cdot \Vert _{X})\) is a Banach space. □

To prove the existence-uniqueness of solutions for system (1.1), we state the following compactness criterion.

Lemma 2.5

([33])

Let \(U\subseteq Y\) be a bounded set; then U is relatively compact in Y if:

  1. (i)

    for any \(u\in U\), \({\frac{u(t)}{1+t^{\alpha -1}}}\) and \(D^{\alpha -1}u(t)\) are equicontinuous on any compact interval of J;

  2. (ii)

    for any \(\varepsilon >0\), there exists a constant \(T=T(\varepsilon )>0\) such that

    $$ \biggl\vert {\frac{u(t_{1})}{1+t_{1}^{\alpha -1}}}-{\frac{u(t_{2})}{1+t _{2}^{\alpha -1}}} \biggr\vert < \varepsilon , \qquad \bigl\vert D^{\alpha -1}u(t_{1})-D^{\alpha -1}u(t_{2}) \bigr\vert < \varepsilon , $$

    for any \(t_{1}, t_{2}\geq T\) and \(u\in U\).

Remark 2.2

According to Lemmas 2.4 and 2.5, it is clear that Z is relatively compact in X if the following conditions hold:

  1. (i)

    for any \(v\in Z\), \({\frac{v(t)}{1+t^{\alpha -1}}}\) and \({\frac{D^{\gamma _{1}}v(t)}{1+t^{\alpha -1-\gamma _{1}}}}\) are equicontinuous on any compact interval of J;

  2. (ii)

    for any \(\varepsilon >0\), there exists a constant \(L=L(\varepsilon )>0\) such that

    $$ \biggl\vert {\frac{v(t_{1})}{1+t_{1}^{\alpha -1}}}-{\frac{v(t_{2})}{1+t _{2}^{\alpha -1}}} \biggr\vert + \biggl\vert {\frac{D^{\gamma _{1}}v(t_{1})}{1+t _{1}^{\alpha -1-\gamma _{1}}}}-{\frac{D^{\gamma _{1}}v(t_{2})}{1+t_{2} ^{\alpha -1-\gamma _{1}}}} \biggr\vert < \varepsilon $$

    for any \(t_{1}, t_{2}\geq L\) and \(v\in Z\).

Lemma 2.6

(Schauder’s fixed point theorem)

Let C be a nonempty, closed, bounded, and convex subset of a Banach space X. Suppose that \(T:C\rightarrow C\) is a continuous and compact mapping. Then T has at least one fixed point in C.

3 Main results

In our considerations, we work in the space \(Q=\{(u,v)\mid u\in X, v \in Y\}\) endowed with the norm

$$ \bigl\Vert (u,v) \bigr\Vert _{Q} = \max \bigl\{ \Vert u \Vert _{X}, \Vert v \Vert _{Y}\bigr\} , \quad (u,v)\in Q. $$

By Lemma 2.4, Q is a Banach space. Let \(T: Q\rightarrow Q\) be the operator defined as

$$ T(u,v) (t)=\bigl(T_{1}v(t), T_{2}u(t)\bigr), $$

where

$$\begin{aligned}& T_{1}v(t)=a\sigma _{1}t^{\alpha -1}+{ \frac{a\sigma _{1}\omega _{1}T_{1}(t)}{1- \mu _{1}}}+ \int ^{+\infty }_{0} H(t,s)\varphi \bigl(s,v(s),D^{\gamma _{1}}v(s) \bigr)\,ds, \\& T_{2}u(t)=b\sigma _{2}t^{\beta -1}+{ \frac{b\sigma _{2}\omega _{2}T_{2}(t)}{1- \mu _{2}}}+ \int ^{+\infty }_{0} K(t,s)\psi \bigl(s,u(s),D^{\gamma _{2}}u(s) \bigr)\,ds. \end{aligned}$$

Notice that system (1.1) has a solution if and only if the operator T has a fixed point. For the forthcoming analysis, denote

$$\begin{aligned} &L_{1}=\sigma _{1}(1+\delta _{1}l_{1}), \quad\quad L_{2}=\sigma _{2}(1+\delta _{2}l _{2}), \\ &\zeta _{1}={\frac{1+\sigma _{1}\varGamma (\alpha )+2l_{1}\delta _{1}+ \sigma _{1}M\xi ^{\alpha -1}(1+2\omega _{1}l_{1})}{\varGamma (\alpha -\gamma _{1})}}, \\ &\zeta _{2}={\frac{1+\sigma _{2}\varGamma (\beta )+2l_{2}\delta _{2}+\sigma _{2}N\eta ^{\beta -1}(1+2\omega _{2}l_{2})}{\varGamma (\beta -\gamma _{2})}}, \\ &\theta (s)=\max \bigl\{ 1+s^{\alpha -1},1+s^{\alpha -1-\gamma _{1}},1+s^{ \beta -1},1+s^{\beta -1-\gamma _{2}} \bigr\} , \quad s\in [0,+\infty ). \end{aligned}$$

We need the following assumptions:

\((H_{1})\) :

There exist nonnegative functions \(c_{i}(t), d_{i}(t) \in L^{1}(J)\cap C(J)\), \(i=1,2,3\), such that

$$\begin{aligned}& \bigl\vert \varphi (t,u,v) \bigr\vert \leq c_{1}(t)+c_{2}(t) \vert u \vert +c_{3}(t) \vert v \vert , \quad t\in [0,+ \infty ), \\& \int ^{+\infty }_{0}c_{1}(t)\,dt< +\infty , \qquad \int ^{+\infty }_{0}\bigl(c_{2}(t)+c_{3}(t) \bigr)\theta (t) \,dt< \max \biggl\{ {\frac{1}{2L _{1}}},{\frac{1}{2\zeta _{1}}} \biggr\} ; \end{aligned}$$

and

$$\begin{aligned}& \bigl\vert \psi (t,u,v) \bigr\vert \leq d_{1}(t)+d_{2}(t) \vert u \vert +d_{3}(t) \vert v \vert , \quad t\in [0,+\infty ), \\& \int ^{+\infty }_{0}d_{1}(t)\,dt< +\infty , \qquad \int ^{+\infty }_{0}\bigl(d_{2}(t)+d_{3}(t) \bigr)\theta (t) \,dt< \max \biggl\{ {\frac{1}{2L _{2}}},{\frac{1}{2\zeta _{2}}} \biggr\} . \end{aligned}$$
\((H_{2})\) :

For any \(u,v,x,y\in \mathbb{R}\), there exist \(\lambda _{i}(t) \in L^{1}(J)\cap C(J)\) with \(\lambda _{i}(t)>0\), \(i=1,2\), such that

$$\begin{aligned}& \bigl\vert \varphi (t,u,v)-\varphi (t,x,y) \bigr\vert \leq \lambda _{1}(t) \bigl( \vert u-x \vert + \vert v-y \vert \bigr), \quad t \in [0,+\infty ), \\& \bigl\vert \psi (t,u,v)-\psi (t,x,y) \bigr\vert \leq \lambda _{2}(t) \bigl( \vert u-x \vert + \vert v-y \vert \bigr), \quad t\in [0,+ \infty ). \end{aligned}$$

This section is devoted to some existence and uniqueness results of system (1.1). In order to do this, define

$$ B_{R}=\bigl\{ (u,v)\in Q\mid \bigl\Vert (u,v) \bigr\Vert _{Q}\leq R\bigr\} , $$

where

$$\begin{aligned} R >& \biggl\{ {\frac{a\sigma _{1}(1+\omega _{1}l_{1})+L_{1}\int ^{+\infty }_{0}c_{1}(s)\,ds }{\frac{1}{2}-L_{1}\int ^{+\infty }_{0}(c_{2}(s)+c_{3}(s)) \theta (s)\,ds}}, {\frac{{\frac{2a\sigma _{1}\varGamma (\alpha -1)}{\varGamma (\alpha -\gamma _{1})}} (1+\omega _{1}l_{1} )+\zeta _{1} \int ^{+\infty }_{0}c_{1}(s)\,ds }{\frac{1}{2}-\zeta _{1}\int ^{+\infty } _{0}(c_{2}(s)+c_{3}(s))\theta (s)\,ds }}, \\ &{\frac{b\sigma _{2}(1+\omega _{2}l_{2})+L_{2}\int ^{+\infty } _{0}d_{1}(s)\,ds }{\frac{1}{2}-L_{2}\int ^{+\infty }_{0}(d_{2}(s)+d_{3}(s)) \theta (s)\,ds}}, {\frac{{\frac{2b\sigma _{2}\varGamma (\beta -1)}{\varGamma ( \beta -\gamma _{2})}} (1+\omega _{2}l_{2} )+\zeta _{2} \int ^{+\infty }_{0}d_{1}(s)\,ds }{\frac{1}{2}-\zeta _{2}\int ^{+\infty } _{0}(d_{2}(s)+d_{3}(s))\theta (s)\,ds }} \biggr\} . \end{aligned}$$

We observe that \(B_{R}\) is a bounded closed ball in the Banach space Q.

Lemma 3.1

If \((H_{1})\) is satisfied, then \(T: B_{R}\rightarrow B_{R}\).

Proof

First, for any \((u,v)\in B_{R}\), we know that

$$ \Vert T_{1}v \Vert _{X}=\underset{t\in J}{\sup } { \frac{ \vert T_{1}v(t) \vert }{1+t^{ \alpha -1}}}+\underset{t\in J}{\sup } {\frac{ \vert D^{\gamma _{1}}T_{1}v(t) \vert }{1+t ^{\alpha -1-\gamma _{1}}}}, $$
(3.1)

and from condition \((H_{1})\), we have

$$\begin{aligned} {\frac{ \vert T_{1}v(t) \vert }{1+t^{\alpha -1}}} =& \biggl\vert {\frac{a\sigma _{1}t ^{\alpha -1}}{1+t^{\alpha -1}}}+{\frac{T_{1}(t)}{1+t^{\alpha -1}}} \cdot {\frac{a\sigma _{1}\omega _{1}}{1-\mu _{1}}}+ \int ^{+\infty }_{0}\frac{ H(t,s)}{1+t^{\alpha -1}}\varphi \bigl(s,v(s),D^{\gamma _{1}}v(s)\bigr)\,ds \biggr\vert \\ \le &a\sigma _{1}+\frac{a\sigma _{1}\omega _{1}(1+\sigma _{1}M\xi ^{ \alpha -2})}{(1-\mu _{1})\varGamma (\alpha -1)} + \int ^{+\infty }_{0}\sigma _{1}(1+\delta _{1}l_{1}) \bigl\vert \varphi \bigl(s,v(s),D^{\gamma _{1}}v(s) \bigr) \bigr\vert \,ds \\ \le &a\sigma _{1}(1+\omega _{1}l_{1})+L_{1} \int ^{+\infty }_{0}\bigl(c_{1}(s)+c _{2}(s) \bigl\vert v(s) \bigr\vert +c_{3}(s) \bigl\vert D^{\gamma _{1}}v(s) \bigr\vert \bigr)\,ds \\ \leq &a\sigma _{1}(1+\omega _{1}l_{1})+L_{1} \int ^{+\infty }_{0}c_{1}(s)\,ds+L _{1} \int ^{+\infty }_{0}\bigl(c_{2}(s)+c_{3}(s) \bigr)\theta (s)\,ds \bigl\Vert (v,v) \bigr\Vert _{Q} \\ < &{\frac{a\sigma _{1}(1+\omega _{1}l_{1})+L_{1}\int ^{+\infty }_{0}c _{1}(s)\,ds }{1-2L_{1}\int ^{+\infty }_{0}(c_{2}(s)+c_{3}(s))\theta (s)\,ds}}< \frac{R}{2}. \end{aligned}$$

In view of Lemma 2.1, one has

$$\begin{aligned} D^{\gamma _{1}}u(t) =&D^{\gamma _{1}} \biggl(\sigma _{1}t^{\alpha -1} \biggl(a+ \int ^{+\infty }_{0}f(s)\,ds+{\frac{M\xi ^{\alpha -2}}{\varGamma ( \alpha -1)}} \int _{0}^{h} g_{1}(s)u(s) \,ds-MI^{\alpha }f(\xi ) \biggr) \\ & {} +{\frac{t^{\alpha -2}}{\varGamma (\alpha -1)}} \int _{0}^{h} g _{1}(s)u(s) \,ds-I^{\alpha }f(t) \biggr) \\ =& \biggl(a+ \int ^{+\infty }_{0}f(s)\,ds+{\frac{M\xi ^{\alpha -2}}{\varGamma (\alpha -1)}} \int _{0}^{h} g_{1}(s)u(s) \,ds-MI^{\alpha }f(\xi ) \biggr) \cdot {\frac{\sigma _{1}t^{\alpha -1-\gamma _{1}}\varGamma (\alpha )}{ \varGamma (\alpha -\gamma _{1})}} \\ & {} +{\frac{t^{\alpha -2-\gamma _{1}}}{\varGamma (\alpha -1-\gamma _{1})}} \int _{0}^{h} g_{1}(s)u(s) \,ds-I^{\alpha -\gamma _{1}} f(t), \end{aligned}$$

and thus, we can easily show that

$$\begin{aligned} \biggl\vert \int _{0}^{h} g_{1}(t)T_{1}v(t) \,dt \biggr\vert =& \biggl\vert {\frac{a \sigma _{1}}{1-\mu _{1}}} \int _{0}^{h} g_{1}(t)t^{\alpha -1} \,dt \\ &{}+{\frac{M \sigma _{1}}{1-\mu _{1}}} \int _{0}^{h} g_{1}(t)t^{\alpha -1}\,dt \int ^{+ \infty }_{0} G_{1}(\xi ,s)\varphi \bigl(s,v(s),D^{\gamma _{1}}v(s)\bigr)\,ds \\ &{}+{\frac{1}{1-\mu _{1}}} \int _{0}^{h} g_{1}(t) \int ^{+\infty } _{0} G_{1}(t,s)\varphi \bigl(s,v(s),D^{\gamma _{1}}v(s)\bigr)\,ds\,dt \biggr\vert \\ \leq &{\frac{a\sigma _{1}\omega _{1}}{1-\mu _{1}}}+{\frac{\sigma _{1} \omega _{1}M\xi ^{\alpha -1}}{(1-\mu _{1})\varGamma (\alpha )}} \int ^{+ \infty }_{0} \bigl\vert \varphi \bigl(s,v(s),D^{\gamma _{1}}v(s)\bigr) \bigr\vert \,ds \\ & {} +{\frac{1}{(1-\mu _{1})\varGamma (\alpha )}} \int _{0}^{h} g_{1}(t) \bigl(1+t ^{\alpha -1}\bigr)\,dt \int ^{+\infty }_{0} \bigl\vert \varphi \bigl(s,v(s),D^{\gamma _{1}}v(s)\bigr) \bigr\vert \,ds \\ \leq &{\frac{a\sigma _{1}\omega _{1}}{1-\mu _{1}}}+{\frac{\sigma _{1} \omega _{1}M\xi ^{\alpha -1} +\delta _{1}}{(1-\mu _{1})\varGamma (\alpha )}} \int ^{+\infty }_{0}c_{1}(s)\,ds \\ & {} +{\frac{\sigma _{1}\omega _{1}M\xi ^{\alpha -1}+\delta _{1}}{(1-\mu _{1})\varGamma (\alpha )}} \int ^{+\infty }_{0}\bigl(c_{2}(s)+c_{3}(s) \bigr)\theta (s)\,ds \bigl\Vert (v,v) \bigr\Vert _{Q}. \end{aligned}$$

Further,

$$\begin{aligned} {\frac{ \vert D^{\gamma _{1}}T_{1}v(t) \vert }{1+t^{\alpha -1-\gamma _{1}}}} =& \biggl\vert {\frac{\sigma _{1}t^{\alpha -1-\gamma _{1}}\varGamma (\alpha )}{ (1+t ^{\alpha -1-\gamma _{1}})\varGamma (\alpha -\gamma _{1})}} \biggl(a+ \int ^{+\infty }_{0}\varphi \bigl(s,v(s),D^{\gamma _{1}}v(s) \bigr)\,ds \\ &{}+{\frac{M \xi ^{\alpha -2}}{\varGamma (\alpha -1)}} \int _{0}^{h} g_{1}(s)T_{1}v(s) \,ds \\ & {} -\frac{M}{\varGamma (\alpha )} \int ^{\xi }_{0}{(\xi -s)}^{ \alpha -1}\varphi \bigl(s,v(s),D^{\gamma _{1}}v(s)\bigr)\,ds \biggr) \\ &{}+{\frac{t^{ \alpha -2-\gamma _{1}}}{(1+t^{\alpha -1-\gamma _{1}})}} \int _{0}^{h} g _{1}(s) \frac{T_{1}v(s)}{\varGamma (\alpha -1-\gamma _{1})}\,ds \\ & {} -\frac{1}{\varGamma (\alpha -\gamma _{1})} \int ^{t}_{0}\frac{ {(t-s)}^{\alpha -1-\gamma _{1}}}{1+t^{\alpha -1-\gamma _{1}}} \varphi \bigl(s,v(s),D ^{\gamma _{1}}v(s)\bigr)\,ds \biggr\vert \\ \leq &{\frac{a\sigma _{1}\varGamma (\alpha )}{\varGamma (\alpha -\gamma _{1})}} +{\frac{\sigma _{1}\varGamma (\alpha )}{\varGamma (\alpha -\gamma _{1})}} \int ^{+\infty }_{0} \bigl\vert \varphi \bigl(s,v(s),D^{\gamma _{1}}v(s)\bigr) \bigr\vert \,ds \\ &{}+{\frac{ \sigma _{1}M\xi ^{\alpha -1}}{\varGamma (\alpha -\gamma _{1})}} \int ^{\xi } _{0} \bigl\vert \varphi \bigl(s,v(s),D^{\gamma _{1}}v(s)\bigr) \bigr\vert \,ds \\ & {} +{\frac{\sigma _{1}M\xi ^{\alpha -2}(\alpha -1)+\alpha -1-\gamma _{1}}{ \varGamma (\alpha -\gamma _{1})}} \biggl\vert \int _{0}^{h} g_{1}(s)T_{1}v(s) \,ds \biggr\vert \\ & {} +\frac{1}{\varGamma (\alpha -\gamma _{1})} \int ^{t}_{0}{ \biggl(1- \frac{s}{t} \biggr)}^{\alpha -1-\gamma _{1}} \bigl\vert \varphi \bigl(s,v(s),D^{\gamma _{1}}v(s) \bigr) \bigr\vert \,ds \\ \leq &{\frac{1+\sigma _{1}\varGamma (\alpha )+2l_{1}\delta _{1}+\sigma _{1}M\xi ^{\alpha -1}(1+2\omega _{1}l_{1})}{\varGamma (\alpha -\gamma _{1})}} \int ^{+\infty }_{0}c_{1}(s)\,ds \\ & {} +{\frac{1+\sigma _{1}\varGamma (\alpha )+2l_{1}\delta _{1}+\sigma _{1}M \xi ^{\alpha -1}(1+2\omega _{1}l_{1})}{\varGamma (\alpha -\gamma _{1})}} \\ &{}\times \int ^{+\infty }_{0}\bigl(c_{2}(s)+c_{3}(s) \bigr)\theta (s)\,ds \bigl\Vert (v,v) \bigr\Vert _{Q} \\ & {} +a\sigma _{1} \biggl({\frac{\varGamma (\alpha )}{\varGamma (\alpha -\gamma _{1})}}+{ \frac{2\omega _{1}l_{1}\varGamma (\alpha -1)}{\varGamma (\alpha - \gamma _{1})}} \biggr) \\ < &{\frac{{\frac{2a\sigma _{1}\varGamma (\alpha -1)}{\varGamma (\alpha - \gamma _{1})}}(1+\omega _{1}l_{1})+\zeta _{1}\int ^{+\infty }_{0}c_{1}(s)\,ds }{1-2\zeta _{1}\int ^{+\infty }_{0}(c_{2}(s)+c_{3}(s))\theta (s)\,ds}}< \frac{R}{2}, \end{aligned}$$

which implies that

$$ \Vert T_{1}v \Vert _{X}=\underset{t\in J}{\sup } { \frac{ \vert T_{1}v(t) \vert }{1+t^{ \alpha -1}}}+\underset{t\in J}{\sup } {\frac{ \vert D^{\gamma _{1}}T_{1}v(t) \vert }{1+t ^{\alpha -1-\gamma _{1}}}}< \frac{R}{2}+\frac{R}{2}=R. $$
(3.2)

Similarly, we can obtain

$$\begin{aligned} \Vert T_{2}u \Vert _{Y} =&\underset{t\in J}{\sup } {\frac{ \vert T_{2}u(t) \vert }{1+t^{ \beta -1}}}+\underset{t\in J}{\sup } {\frac{ \vert D^{\gamma _{2}}T_{2}u(t) \vert }{1+t ^{\beta -1-\gamma _{2}}}} \\ \leq &{\frac{b\sigma _{2}(1+\omega _{2}l_{2})+L_{2}\int ^{+\infty }_{0}d _{1}(s)\,ds }{\frac{1}{2}-L_{2}\int ^{+\infty }_{0}(d_{2}(s)+d_{3}(s)) \theta (s) \,ds}}+{\frac{{\frac{2b\sigma _{2}\varGamma (\beta -1)}{\varGamma ( \beta -\gamma _{2})}} (1+\omega _{2}l_{2} )+\zeta _{2} \int ^{+\infty }_{0}d_{1}(s)\,ds }{\frac{1}{2}-\zeta _{2}\int ^{+\infty } _{0}(d_{2}(s)+d_{3}(s))\theta (s) \,ds }} \\ < &\frac{R}{2}+\frac{R}{2}=R. \end{aligned}$$
(3.3)

It shows that \(\Vert T(u,v) \Vert _{Q}\leq R\), and \(T_{1}\), \(T_{2}\) are continuous on J. Thus \(T:B_{R}\rightarrow B_{R}\) is well defined. □

Theorem 3.1

If \((H_{1})\) holds, then system (1.1) has at least one solution.

Proof

First, the operator \(T:B_{R}\rightarrow B_{R}\) is continuous owing to the continuity of φ and ψ. We are going to show that T is a completely continuous operator. By Lemma 3.1, T is bounded. We need to show that T is relatively compact by means of Remark 2.2. This part consists of two steps as follows.

Step 1 We show that T is equicontinuous on any compact interval of J.

Let ω be a bounded subset of \(B_{R}\), \(J_{1} \subseteq [0,+ \infty )\) be a compact interval. Then, for any \(t_{1},t_{2}\in J_{1}\) with \(t_{1}< t_{2}\), \(v\in \omega \), we have

$$\begin{aligned}& \biggl\vert {\frac{T_{1}v(t_{2})}{1+t_{2}^{\alpha -1}}}-{\frac{T_{1}v(t _{1})}{1+t_{1}^{\alpha -1}}} \biggr\vert \\& \quad = \biggl\vert {\frac{a\sigma _{1}t_{2}^{\alpha -1}}{1+t_{2}^{\alpha -1}}} - {\frac{a\sigma _{1}t_{1}^{\alpha -1}}{1+t_{1}^{\alpha -1}}}+ \int ^{+ \infty }_{0} \biggl( {\frac{H(t_{2},s)}{1+t_{2}^{\alpha -1}}}-{ \frac{H(t _{1},s)}{1+t_{1}^{\alpha -1}}} \biggr)\varphi \bigl(s,v(s),D^{\gamma _{1}}v(s)\bigr)\,ds \\& \quad \quad {} + \biggl({\frac{T_{1}(t_{2})}{(1-\mu _{1})(1+t_{2}^{\alpha -1})}}- {\frac{T_{1}(t_{1})}{(1-\mu _{1})(1+t_{1}^{\alpha -1})}} \biggr) \cdot a\sigma _{1}\omega _{1} \biggr\vert \\& \quad \leq {\frac{ \vert t_{2}^{\alpha -1}-t_{1}^{\alpha -1} \vert a\sigma _{1}}{(1+t _{2}^{\alpha -1})(1+t_{1}^{\alpha -1})}}+ \biggl\vert {\frac{\sigma _{1}Mt _{2}^{\alpha -1}}{1+t_{2}^{\alpha -1}}}-{ \frac{\sigma _{1}Mt_{1}^{ \alpha -1}}{1+t_{1}^{\alpha -1}}} \biggr\vert \int ^{+\infty }_{0}G_{1}( \xi ,s) \bigl\vert \varphi \bigl(s,v(s),D^{\gamma _{1}}v(s)\bigr) \bigr\vert \,ds \\& \quad \quad {} + \biggl\vert {\frac{T_{1}(t_{2})}{(1-\mu _{1})(1+t_{2}^{\alpha -1})}}- {\frac{T_{1}(t_{1})}{(1-\mu _{1})(1+t_{1}^{\alpha -1})}} \biggr\vert \\& \quad \quad {}\times \int ^{+\infty }_{0} \int ^{h}_{0}H_{1}(\tau ,s)g_{1} (\tau ) \bigl\vert \varphi \bigl(s,v(s),D ^{\gamma _{1}}v(s)\bigr) \bigr\vert \,d\tau \,ds \\& \quad \quad {} + \int ^{+\infty }_{0} {\frac{G_{1}(t_{2},s)-G_{1}(t_{1},s)}{1+t_{2} ^{\alpha -1}}} \bigl\vert \varphi \bigl(s,v(s),D^{\gamma _{1}}v(s)\bigr) \bigr\vert \,ds \\& \quad \quad {}+ \int ^{+\infty }_{0} \biggl\vert {\frac{G_{1}(t_{1},s)}{1+t_{2}^{\alpha -1}}}-{ \frac{G _{1}(t_{1},s)}{1+t_{1}^{\alpha -1}}} \biggr\vert \bigl\vert \varphi \bigl(s,v(s),D^{ \gamma _{1}}v(s) \bigr) \bigr\vert \,ds \\& \quad \le {\frac{\sigma _{1}M\xi ^{\alpha -2} \vert t_{2}^{\alpha -1}-t_{1}^{ \alpha -1} \vert + \vert t_{2}^{\alpha -2}-t_{1}^{\alpha -2} \vert +(t_{1}t_{2})^{ \alpha -2} \vert t_{1}-t_{2} \vert }{(1+t_{2}^{\alpha -1})(1+t_{1}^{\alpha -1})(1- \mu _{1})\varGamma (\alpha -1)}} \\& \quad \quad {}\times \biggl(a\sigma _{1}\omega _{1}+\sigma _{1}\delta _{1} \int ^{+\infty }_{0} \bigl\vert \varphi \bigl(s,v(s),D^{\gamma _{1}}v(s)\bigr) \bigr\vert \,ds \biggr) \\& \quad \quad {} +{\frac{ \vert t_{2}^{\alpha -1}-t_{1}^{\alpha -1} \vert a\sigma _{1}}{(1+t_{2} ^{\alpha -1})(1+t_{1}^{\alpha -1})}}+{\frac{ \vert t_{2}^{\alpha -1}-t_{1} ^{\alpha -1} \vert \sigma _{1}M\xi ^{\alpha -1}}{(1+t_{2}^{\alpha -1})(1+t _{1}^{\alpha -1})\varGamma (\alpha )}} \int ^{+\infty }_{0} \bigl\vert \varphi \bigl(s,v(s),D ^{\gamma _{1}}v(s)\bigr) \bigr\vert \,ds \\& \quad \quad {} + \int ^{t_{1}}_{0} \biggl\vert {\frac{(t_{2}^{\alpha -1}-t_{1}^{\alpha -1})+ {(t_{1}-s)}^{\alpha -1}-{(t_{2}-s)}^{\alpha -1}}{\varGamma (\alpha )(1+t _{2}^{\alpha -1})}} \biggr\vert \bigl\vert \varphi \bigl(s,v(s),D^{\gamma _{1}}v(s)\bigr) \bigr\vert \,ds \\& \quad \quad {} + \int ^{t_{2}}_{t_{1}} \biggl\vert {\frac{(t_{2}^{\alpha -1}-t_{1}^{ \alpha -1})-{(t_{2}-s)}^{\alpha -1}}{\varGamma (\alpha )(1+t_{2}^{\alpha -1})}} \biggr\vert \bigl\vert \varphi \bigl(s,v(s),D^{\gamma _{1}}v(s)\bigr) \bigr\vert \,ds \\& \quad \quad {} + \int ^{+\infty }_{t_{2}} \biggl\vert {\frac{t_{2}^{\alpha -1}-t_{1}^{ \alpha -1}}{\varGamma (\alpha )(1+t_{2}^{\alpha -1})}} \biggr\vert \bigl\vert \varphi \bigl(s,v(s),D^{\gamma _{1}}v(s)\bigr) \bigr\vert \,ds \\& \quad \quad {} + \int ^{+\infty }_{0} \biggl\vert {\frac{t_{1}^{\alpha -1}-t_{2}^{\alpha -1}}{\varGamma (\alpha )(1+t_{2}^{\alpha -1})}} \biggr\vert \bigl\vert \varphi \bigl(s,v(s),D ^{\gamma _{1}}v(s)\bigr) \bigr\vert \,ds. \end{aligned}$$

Then we have \(\vert {\frac{T_{1}v(t_{2})}{1+t_{2}^{\alpha -1}}}- {\frac{T_{1}v(t_{1})}{1+t_{1}^{\alpha -1}}} \vert \rightarrow 0\) as \(t_{1}\rightarrow t_{2}\). Further, we know that

$$\begin{aligned}& \biggl\vert {\frac{D^{\gamma _{1}}T_{1}v(t_{2})}{1+t_{2}^{\alpha -1- \gamma _{1}}}}-{\frac{D^{\gamma _{1}}T_{1}v(t_{1})}{1+t_{1}^{\alpha -1- \gamma _{1}}}} \biggr\vert \\& \quad = \biggl\vert {\frac{ t_{1}^{\alpha -1-\gamma _{1}}-t_{2}^{\alpha -1-\gamma _{1}}}{(1+t_{1}^{\alpha -1-\gamma _{1}})(1+t_{2}^{\alpha -1-\gamma _{1}})}} \biggl(\frac{a\sigma _{1}\varGamma (\alpha )}{\varGamma (\alpha -\gamma _{1})} +{ \frac{\sigma _{1}\varGamma (\alpha )}{\varGamma (\alpha -\gamma _{1})}} \int ^{+\infty }_{0}\varphi \bigl(s,v(s),D^{\gamma _{1}}v(s) \bigr)\,ds \\& \quad \quad {} -{\frac{\sigma _{1}M}{\varGamma (\alpha -\gamma _{1})}} \int ^{ \xi }_{0}{(\xi -s)}^{\alpha -1}\varphi \bigl(s,v(s),D^{\gamma _{1}}v(s)\bigr)\,ds \\& \quad \quad {}+ {\frac{\sigma _{1}M\xi ^{\alpha -2}(\alpha -1)}{\varGamma (\alpha -\gamma _{1})}} \int _{0}^{h} g_{1}(s)T_{1}v(s) \,ds \biggr) \\& \quad \quad {} + \biggl({\frac{t_{2}^{\alpha -1-\gamma _{1}}}{1+t_{2}^{\alpha -1- \gamma _{1}}}} -{\frac{t_{1}^{\alpha -1-\gamma _{1}}}{1+t_{1}^{\alpha -1- \gamma _{1}}}} \biggr) \int _{0}^{h} g_{1}(s)\frac{T_{1}v(s)}{\varGamma ( \alpha -1-\gamma _{1})} \,ds \\& \quad \quad {} +\frac{1}{\varGamma (\alpha -\gamma _{1})} \int ^{t_{1}}_{0}{\frac{ {(t_{1}-s)}^{\alpha -1-\gamma _{1}}}{1+t_{1}^{\alpha -1-\gamma _{1}}}} \varphi \bigl(s,v(s),D^{\gamma _{1}}v(s)\bigr)\,ds \\& \quad \quad {} -\frac{1}{\varGamma (\alpha -\gamma _{1})} \int ^{t_{2}}_{0}{\frac{ {(t_{2}-s)}^{\alpha -1-\gamma _{1}}}{1+t_{2}^{\alpha -1-\gamma _{1}}}} \varphi \bigl(s,v(s),D^{\gamma _{1}}v(s)\bigr)\,ds \biggr\vert \\& \quad \le {\frac{ \vert t_{2}^{\alpha -1-\gamma _{1}}-t_{1}^{\alpha -1- \gamma _{1}} \vert }{(1+t_{1}^{\alpha -1-\gamma _{1}})(1+t_{2}^{\alpha -1-\gamma _{1}})}} \biggl(\frac{a\sigma _{1}\varGamma (\alpha )}{\varGamma ( \alpha -\gamma _{1})} +{\frac{\sigma _{1}\varGamma (\alpha )}{\varGamma ( \alpha -\gamma _{1})}} \int ^{+\infty }_{0} \bigl\vert \varphi \bigl(s,v(s),D^{\gamma _{1}}v(s)\bigr) \bigr\vert \,ds \\& \quad \quad {} +{\frac{\sigma _{1}M\xi ^{\alpha -1}}{\varGamma (\alpha -\gamma _{1})}} \int ^{\xi }_{0} \bigl\vert \varphi \bigl(s,v(s),D^{\gamma _{1}}v(s)\bigr) \bigr\vert \,ds +{\frac{2 \sigma _{1}M\xi ^{\alpha -2}}{\varGamma (\alpha -\gamma _{1})}} \int _{0}^{h} g_{1}(s)T_{1}v(s) \,ds \biggr) \\& \quad \quad {} +{\frac{ \vert t_{2}^{\alpha -1-\gamma _{1}}-t_{1}^{\alpha -1-\gamma _{1}} \vert }{(1+t_{1}^{\alpha -1-\gamma _{1}})(1+t_{2}^{\alpha -1- \gamma _{1}})\varGamma (\alpha -1-\gamma _{1})}} \int _{0}^{h} g_{1}(s)T _{1}v(s)\,ds \\& \quad \quad {} +\frac{1}{\varGamma (\alpha -\gamma _{1})} \int ^{t_{1}}_{0} \biggl\vert {\frac{ {(t_{1}-s)}^{\alpha -1-\gamma _{1}}}{1+t_{1}^{\alpha -1-\gamma _{1}}}} - {\frac{{(t_{2}-s)}^{\alpha -1-\gamma _{1}}}{1+t_{2}^{\alpha -1-\gamma _{1}}}} \biggr\vert \bigl\vert \varphi \bigl(s,v(s),D^{\gamma _{1}}v(s) \bigr) \bigr\vert \,ds \\& \quad \quad {} +\frac{1}{\varGamma (\alpha -\gamma _{1})} \int ^{t_{2}}_{t_{1}}{\frac{ {(t_{2}-s)}^{\alpha -1-\gamma _{1}}}{1+t_{2}^{\alpha -1-\gamma _{1}}}} \bigl\vert \varphi \bigl(s,v(s),D^{\gamma _{1}}v(s)\bigr) \bigr\vert \,ds, \end{aligned}$$

so \(\vert {\frac{D^{\gamma _{1}}T_{1}v(t_{2})}{1+t_{2}^{\alpha -1- \gamma _{1}}}} -{\frac{D^{\gamma _{1}}T_{1}v(t_{1})}{1+t_{1}^{\alpha -1- \gamma _{1}}}} \vert \rightarrow 0\) as \(t_{1}\rightarrow t_{2}\). Moreover, notice that \(\varphi (t,v(t),D^{\gamma _{1}}v(t))\) is bounded on \(J_{1}\). For any \(v\in \omega \), \({\frac{T_{1}v(t)}{1+t^{\alpha -1}}}\) and \({\frac{D^{\gamma _{1}}T_{1}v(t)}{1+t ^{\alpha -1-\gamma _{1}}}}\) are equicontinuous on \(J_{1}\), that is, \(T_{1}\) is equicontinuous. Similarly, we know that \(T_{2}\) is also equicontinuous. Thus T is equicontinuous on \(J_{1}\).

Step 2 We show that T is equiconvergent at ∞.

Since \(\lim_{t\rightarrow +\infty }\frac{t^{\lambda -1}}{1+t ^{\lambda -1}}=1\), for any \(\varepsilon >0\), there exists a constant \(\mu _{1}>0\), for each \(t>\mu _{1}\), one has \(\vert \frac{t^{\lambda -1}}{1+t ^{\lambda -1}}-1 \vert <\frac{\varepsilon }{2}\). Thus, for each \(t_{1}, t_{2}>\mu _{1}\), we have

$$ \biggl\vert \frac{t_{2}^{\lambda -1}}{1+t_{2}^{\lambda -1}}-\frac{t_{1} ^{\lambda -1}}{1+t_{1}^{\lambda -1}} \biggr\vert \leq \biggl\vert \frac{t_{2} ^{\lambda -1}}{1+t_{2}^{\lambda -1}}-1 \biggr\vert + \biggl\vert \frac{t_{1} ^{\lambda -1}}{1+t_{1}^{\lambda -1}}-1 \biggr\vert < \frac{\varepsilon }{2}+\frac{ \varepsilon }{2}=\varepsilon . $$

Further, there exists \(\varsigma \geq s\) such that \(\lim_{t\rightarrow +\infty }\frac{{(t-\varsigma )}^{\lambda -1}}{1+t ^{\lambda -1}}=1\). Then, for any \(\varepsilon >0\), there exists \(\mu _{2}>\varsigma >0\) such that, for each \(t_{1}, t_{2}>\mu _{2}\), we have

$$\begin{aligned} \biggl\vert \frac{{(t_{2}-s)}^{\lambda -1}}{1+t_{2}^{\lambda -1}}-\frac{ {(t_{1}-s)}^{\lambda -1}}{1+t_{1}^{\lambda -1}} \biggr\vert \leq & \biggl\vert \frac{ {(t_{2}-\varsigma )}^{\lambda -1}}{1+t_{2}^{\lambda -1}}-1 \biggr\vert + \biggl\vert \frac{{(t_{1}-\varsigma )}^{\lambda -1}}{1+t_{1}^{\lambda -1}}-1 \biggr\vert \\ < &\frac{\varepsilon }{2}+\frac{\varepsilon }{2}=\varepsilon . \end{aligned}$$

Therefore, for any \(\varepsilon >0\), choose \(\mu \geq \max \{\mu _{1}, \mu _{2}\}\); then, for each \(t_{1}, t_{2}>\mu \), one has

$$\begin{aligned}& \biggl\vert {\frac{T_{1}v(t_{2})}{1+t_{2}^{\alpha -1}}}-{\frac{T_{1}v(t _{1})}{1+t_{1}^{\alpha -1}}} \biggr\vert \\ & \quad \leq \biggl\vert \frac{t_{2}^{\alpha -1}}{1+t_{2}^{\alpha -1}}-\frac{t _{1}^{\alpha -1}}{1+t_{1}^{\alpha -1}} \biggr\vert a \sigma _{1}+ \biggl( {\frac{\sigma _{1}M\xi ^{\alpha -2}}{\varGamma (\alpha -1)(1-\mu _{1})}} \biggl\vert \frac{t_{2}^{\alpha -1}}{1+t_{2}^{\alpha -1}}-\frac{t_{1}^{ \alpha -1}}{1+t_{1}^{\alpha -1}} \biggr\vert \\ & \quad \quad {} +{\frac{1}{\varGamma (\alpha -1)(1-\mu _{1})}} \biggl\vert \frac{t _{2}^{\alpha -2}}{1+t_{2}^{\alpha -1}}- \frac{t_{1}^{\alpha -2}}{1+t _{1}^{\alpha -1}} \biggr\vert \biggr) \\ & \quad \quad {}\times\biggl(a\sigma _{1}\omega _{1}+ \int ^{+\infty }_{0} \int ^{h}_{0}H_{1}(\tau ,s)g_{1}(\tau ) \bigl\vert \varphi \bigl(s,v(s),D ^{\gamma _{1}}v(s) \bigr) \bigr\vert \,d\tau \,ds \biggr) \\ & \quad \quad {} + \biggl\vert \frac{t_{2}^{\alpha -1}}{1+t_{2}^{\alpha -1}}-\frac{t_{1} ^{\alpha -1}}{1+t_{1}^{\alpha -1}} \biggr\vert \cdot \frac{\sigma _{1}M \xi ^{\alpha -1}}{\varGamma (\alpha )} \int ^{+\infty }_{0} \bigl\vert \varphi \bigl(s,v(s),D ^{\gamma _{1}}v(s)\bigr) \bigr\vert \,ds \\ & \quad \quad {} +\frac{1}{\varGamma (\alpha )} \int ^{t_{1}}_{0} \biggl( \biggl\vert \frac{t _{2}^{\alpha -1}}{1+t_{2}^{\alpha -1}}-\frac{t_{1}^{\alpha -1}}{1+t _{1}^{\alpha -1}} \biggr\vert + \biggl\vert { \frac{{(t_{1}-s)}^{\alpha -1}}{1+t _{1}^{\alpha -1}}}-{\frac{{(t_{2}-s)}^{\alpha -1}}{1+t_{2}^{\alpha -1}}} \biggr\vert \biggr) \\ & \quad \quad {}\times\bigl\vert \varphi \bigl(s,v(s),D^{\gamma _{1}}v(s)\bigr) \bigr\vert \,ds \\ & \quad \quad {} +\frac{1}{\varGamma (\alpha )} \int ^{+\infty }_{t_{1}} \biggl\vert \frac{t _{2}^{\alpha -1}}{1+t_{2}^{\alpha -1}}- \frac{t_{1}^{\alpha -1}}{1+t _{1}^{\alpha -1}} \biggr\vert \bigl\vert \varphi \bigl(s,v(s),D^{\gamma _{1}}v(s) \bigr) \bigr\vert \,ds \\ & \quad \leq \varepsilon a\sigma _{1}+{\frac{\varepsilon (1+\sigma _{1}M \xi ^{\alpha -2})}{\varGamma (\alpha -1)(1-\mu _{1})}} \biggl(a\sigma _{1} \omega _{1}+\sigma _{1}\delta _{1} \int ^{+\infty }_{0} \bigl\vert \varphi \bigl(s,v(s),D ^{\gamma _{1}}v(s)\bigr) \bigr\vert \,ds \biggr) \\ & \quad \quad {} +\frac{\varepsilon \sigma _{1}M\xi ^{\alpha -1}}{\varGamma (\alpha )} \int ^{+\infty }_{0} \bigl\vert \varphi \bigl(s,v(s),D^{\gamma _{1}}v(s)\bigr) \bigr\vert \,ds+\frac{2 \varepsilon }{\varGamma (\alpha )} \int ^{t_{1}}_{0} \bigl\vert \varphi \bigl(s,v(s),D^{ \gamma _{1}}v(s)\bigr) \bigr\vert \,ds \\ & \quad \quad {} +\frac{\varepsilon }{\varGamma (\alpha )} \int ^{+\infty }_{t_{1}} \bigl\vert \varphi \bigl(s,v(s),D^{\gamma _{1}}v(s)\bigr) \bigr\vert \,ds \\ & \quad \leq \varepsilon a\sigma _{1}(1+\omega _{1}l_{1})+ \varepsilon \biggl(L _{1}+\frac{2}{\varGamma (\alpha )} \biggr) \int ^{+\infty }_{0}\bigl(c_{1}(s)+c _{2}(s) \bigl\vert v(s) \bigr\vert +c_{3}(s) \bigl\vert D^{\gamma _{1}}v(s) \bigr\vert \bigr)\,ds. \end{aligned}$$

In addition, we can obtain

$$\begin{aligned}& \biggl\vert {\frac{D^{\gamma _{1}}T_{1}v(t_{2})}{1+t_{2}^{\alpha -1- \gamma _{1}}}}-{\frac{D^{\gamma _{1}}T_{1}v(t_{1})}{1+t_{1}^{\alpha -1- \gamma _{1}}}} \biggr\vert \\ & \quad \le \biggl\vert \frac{t_{2}^{\alpha -1-\gamma _{1}}}{1+t_{2}^{\alpha -1- \gamma _{1}}}-\frac{t_{1}^{\alpha -1-\gamma _{1}}}{1+t_{1}^{\alpha -1- \gamma _{1}}} \biggr\vert \biggl( \frac{a\sigma _{1}\varGamma (\alpha )}{\varGamma (\alpha -\gamma _{1})} +{\frac{\sigma _{1}\varGamma (\alpha )}{\varGamma ( \alpha -\gamma _{1})}} \int ^{+\infty }_{0} \bigl\vert \varphi \bigl(s,v(s),D^{\gamma _{1}}v(s)\bigr) \bigr\vert \,ds \\ & \quad \quad {} +{\frac{M\sigma _{1}}{\varGamma (\alpha -\gamma _{1})}} \int ^{ \xi }_{0}{(\xi -s)}^{\alpha -1} \bigl\vert \varphi \bigl(s,v(s),D^{\gamma _{1}}v(s)\bigr) \bigr\vert \,ds \\ & \quad \quad {}+{ \frac{\sigma _{1}M\xi ^{\alpha -2}(\alpha -1)+\alpha -1-\gamma _{1}}{ \varGamma (\alpha -\gamma _{1})}} \int ^{h}_{0}g_{1}(s)T_{1}v(s) \,ds \biggr) \\ & \quad \quad {} +\frac{1}{\varGamma (\alpha -\gamma _{1})} \int ^{t_{1}}_{0} \biggl\vert {\frac{ {(t_{1}-s)}^{\alpha -1-\gamma _{1}}}{1+t_{1}^{\alpha -1-\gamma _{1}}}} - {\frac{{(t_{2}-s)}^{\alpha -1-\gamma _{1}}}{1+t_{2}^{\alpha -1-\gamma _{1}}}} \biggr\vert \bigl\vert \varphi \bigl(s,v(s),D^{\gamma _{1}}v(s) \bigr) \bigr\vert \,ds \\ & \quad \quad {} +\frac{1}{\varGamma (\alpha -\gamma _{1})} \int ^{t_{2}}_{t_{1}}{\frac{ {(t_{2}-s)}^{\alpha -1-\gamma _{1}}}{1+t_{2}^{\alpha -1-\gamma _{1}}}} \bigl\vert \varphi \bigl(s,v(s),D^{\gamma _{1}}v(s)\bigr) \bigr\vert \,ds \\ & \quad \le \varepsilon a\sigma _{1} \biggl({\frac{\varGamma (\alpha )}{\varGamma ( \alpha -\gamma _{1})}}+{ \frac{2\omega _{1}l_{1}\varGamma (\alpha -1)}{ \varGamma (\alpha -\gamma _{1})}} \biggr) +{\frac{\varepsilon \sigma _{1} \varGamma (\alpha )}{\varGamma (\alpha -\gamma _{1})}} \int ^{+\infty }_{0} \bigl\vert \varphi \bigl(s,v(s),D^{\gamma _{1}}v(s)\bigr) \bigr\vert \,ds \\ & \quad \quad {} +{\frac{\varepsilon \sigma _{1}M\xi ^{\alpha -1}}{\varGamma (\alpha - \gamma _{1})}} \int ^{\xi }_{0} \bigl\vert \varphi \bigl(s,v(s),D^{\gamma _{1}}v(s)\bigr) \bigr\vert \,ds \\ & \quad \quad {}+ {\frac{2\varepsilon l_{1}(\sigma _{1}\omega _{1}M\xi ^{\alpha -1}+\delta _{1})}{\varGamma (\alpha -\gamma _{1})}} \int ^{+\infty }_{0} \bigl\vert \varphi \bigl(s,v(s),D ^{\gamma _{1}}v(s)\bigr) \bigr\vert \,ds \\ & \quad \quad {} +\frac{\varepsilon }{\varGamma (\alpha -\gamma _{1})} \int ^{t_{1}}_{0} \bigl\vert \varphi \bigl(s,v(s),D^{\gamma _{1}}v(s)\bigr) \bigr\vert \,ds \\ & \quad \quad {} +\frac{1}{\varGamma (\alpha -\gamma _{1})} \int ^{t_{2}}_{t_{1}} \bigl\vert \varphi \bigl(s,v(s),D^{\gamma _{1}}v(s)\bigr) \bigr\vert \,ds \\ & \quad \le {\frac{2\varepsilon a\sigma _{1}\varGamma (\alpha -1)}{\varGamma ( \alpha -\gamma _{1})}}(1+\omega _{1}l_{1}) \\ & \quad \quad {}+ \biggl( \varepsilon \zeta _{1}+{\frac{1}{\varGamma (\alpha -\gamma _{1})}} \biggr) \int ^{+\infty } _{0}\bigl(c_{1}(s)+c_{2}(s) \bigl\vert v(s) \bigr\vert +c_{3}(s) \bigl\vert D^{\gamma _{1}}v(s) \bigr\vert \bigr)\,ds. \end{aligned}$$

Thus we have

$$\begin{aligned}& \biggl\vert {\frac{T_{1}v(t_{2})}{1+t_{2}^{\alpha -1}}}-{\frac{T_{1}v(t _{1})}{1+t_{1}^{\alpha -1}}} \biggr\vert + \biggl\vert {\frac{D^{\gamma _{1}}T _{1}v(t_{2})}{1+t_{2}^{\alpha -1-\gamma _{1}}}}-{\frac{D^{\gamma _{1}}T _{1}v(t_{1})}{1+t_{1}^{\alpha -1-\gamma _{1}}}} \biggr\vert \\& \quad \le \biggl(\varepsilon (L_{1}+\zeta _{1})+{ \frac{1}{\varGamma (\alpha - \gamma _{1})}}+\frac{2\varepsilon }{\varGamma (\alpha )} \biggr) \int ^{+\infty }_{0}\bigl(c_{1}(s)+c_{2}(s) \bigl\vert v(s) \bigr\vert +c_{3}(s) \bigl\vert D^{\gamma _{1}}v(s) \bigr\vert \bigr)\,ds \\& \quad\quad {} +\varepsilon a\sigma _{1}(1+\omega _{1}l_{1}) \biggl(1+\frac{2\varGamma (\alpha -1)}{\varGamma (\alpha -\gamma _{1})} \biggr). \end{aligned}$$

Then, for all \(\varepsilon >0\), there exists \(\mu >0\) such that, for \(t_{1}, t_{2}>\mu \), \(T_{1}:\omega \rightarrow \omega \) is equiconvergent at infinity. Using the same argument, \(T_{2}:\omega \rightarrow \omega \) is also equiconvergent at infinity. Thus \(T:\omega \rightarrow \omega \) is equiconvergent at infinity. By means of Remark 2.2, we know \(T:B_{R}\rightarrow B_{R}\) is completely continuous.

According to Schauder’s fixed point theorem, we conclude that T has at least one fixed point, that is, system (1.1) has at least one solution in \(B_{R}\). □

Corollary 3.1

Assume that

\((H_{3})\) :

there exist nonnegative functions \(a(t), b(t),a_{i}(t) \in L^{1}(J)\cap C(J)\), \(i=1,2\), such that

$$\begin{aligned}& \bigl\vert \varphi (t,u,v) \bigr\vert \leq a(t)+a_{1}(t) \bigl({ \vert u \vert }^{p_{1}}+{ \vert v \vert }^{p_{2}}\bigr), \quad 0< p _{i}< 1, i=1,2, t\in J, \\& \int ^{+\infty }_{0}a(t)\,dt< +\infty , \qquad \int ^{+\infty }_{0}a_{1}(t)\theta (t) \,dt< \max \biggl\{ {\frac{1}{4L _{1}}},{\frac{1}{4\zeta _{1}}} \biggr\} ; \end{aligned}$$

and

$$\begin{aligned}& \bigl\vert \psi (t,u,v) \bigr\vert \leq b(t)+a_{2}(t) \bigl({ \vert u \vert }^{q_{1}}+{ \vert v \vert }^{q_{2}}\bigr), \quad 0< q _{i}< 1, i=1,2, t\in J, \\& \int ^{+\infty }_{0}b(t)\,dt< +\infty , \qquad \int ^{+\infty }_{0}a_{2}(t)\theta (t) \,dt< \max \biggl\{ {\frac{1}{4L _{2}}},{\frac{1}{4\zeta _{2}}} \biggr\} . \end{aligned}$$

Here, \(a_{i},b_{i}, i=1,2\), are nonnegative constants, then system (1.1) has at least one solution.

Proof

In this case, let \(p*=\max \{p_{1},p_{2}\}\), \(q*=\max \{q _{1},q_{2}\}\), we take

$$\begin{aligned} R >& \biggl\{ \biggl({\frac{a\sigma _{1}(1+\omega _{1}l_{1})+L_{1} \int ^{+\infty }_{0}a(s)\,ds }{\frac{1}{2}-2L_{1}\int ^{+\infty }_{0}a _{1}(s)\theta (s) \,ds}} \biggr)^{1/p*}, \biggl({ \frac{{\frac{2a\sigma _{1}\varGamma (\alpha -1)}{\varGamma (\alpha -\gamma _{1})}} (1+\omega _{1}l_{1} )+\zeta _{1}\int ^{+\infty }_{0}a(s)\,ds }{\frac{1}{2}-2 \zeta _{1}\int ^{+\infty }_{0}a_{1}(s)\theta (s) \,ds }} \biggr)^{1/p*} \\ & \biggl({\frac{b\sigma _{2}(1+\omega _{2}l_{2})+L_{2} \int ^{+\infty }_{0}b(s)\,ds }{\frac{1}{2}-2L_{2}\int ^{+\infty }_{0}a _{2}(s)\theta (s) \,ds}} \biggr)^{1/q*}, \\ &\biggl({ \frac{{\frac{2b\sigma _{2}\varGamma (\beta -1)}{\varGamma (\beta -\gamma _{2})}} (1+\omega _{2}l_{2} )+\zeta _{2}\int ^{+\infty }_{0}b(s)\,ds }{\frac{1}{2}-2 \zeta _{2}\int ^{+\infty }_{0}a_{2}(s)\theta (s) \,ds }} \biggr)^{1/q*} \biggr\} . \end{aligned}$$

The rest of the proof is similar to Theorem 3.1, so we omit the details. □

Remark 3.1

For the sake of simplicity, if \(a(t)=b(t)=0 \) in condition \((H_{3})\), that is,

$$\begin{aligned}& \bigl\vert \varphi (t,u,v) \bigr\vert \leq a_{1}(t) \bigl({ \vert u \vert }^{p_{1}}+{ \vert v \vert }^{p_{2}}\bigr), \quad p_{i}>1, i=1,2, t\in J, \\& \int ^{+\infty }_{0}a_{1}(t)\theta (t) \,dt< \max \biggl\{ {\frac{1}{4L _{1}}},{\frac{1}{4\zeta _{1}}} \biggr\} ; \end{aligned}$$

and

$$\begin{aligned}& \bigl\vert \psi (t,u,v) \bigr\vert \leq a_{2}(t) \bigl({ \vert u \vert }^{q_{1}}+{ \vert v \vert }^{q_{2}}\bigr), \quad q_{i}>1, i=1,2, t\in J, \\& \int ^{+\infty }_{0}a_{2}(t)\theta (t) \,dt< \max \biggl\{ {\frac{1}{4L _{2}}},{\frac{1}{4\zeta _{2}}} \biggr\} . \end{aligned}$$

Due to the different values of R, the conclusion of Theorem 3.1 is also true for the nonstrict inequalities \(p_{i}, q_{i}>1\). It should be replaced by a weak form which can be derived easily from (3.2) and (3.3).

When \(h=0\), the boundary conditions of system (1.1) are changed to the form:

$$ \textstyle\begin{cases} I^{3-\alpha }u(0)=D^{\alpha -2}u(0)=0, \quad\quad D^{\alpha -1}u(+\infty )=Mu( \xi )+a, \\ I^{3-\beta }v(0)=D^{\beta -2}v(0)=0, \quad\quad D^{\beta -1}v(+\infty )=Nv( \eta )+b. \end{cases} $$
(3.4)

Similar to Theorem 3.1, we can obtain the following result.

Theorem 3.2

Assume that

\((H_{1}')\) :

there exist nonnegative functions \(c_{i}(t), d_{i}(t) \in L^{1}(J)\), \(i=1,2,3\), such that

$$\begin{aligned}& \bigl\vert \varphi (t,u,v) \bigr\vert \leq c_{1}(t)+c_{2}(t) \vert u \vert +c_{3}(t) \vert v \vert , \quad t\in [0,+ \infty ), \\& \int ^{+\infty }_{0}c_{1}(t)\,dt< +\infty , \qquad \int ^{+\infty }_{0}\bigl(c_{2}(t)+c_{3}(t) \bigr)\theta (t)\,dt< \max \biggl\{ {\frac{1}{2L _{1}'}},{\frac{1}{2\zeta _{1}'}} \biggr\} , \end{aligned}$$

where

$$ L_{1}'=\sigma _{1}, \quad\quad \zeta _{1}'={\frac{1+\sigma _{1}(1+M\xi ^{\alpha -1})}{ \varGamma (\alpha -\gamma _{1})}}, $$

and

$$\begin{aligned}& \bigl\vert \psi (t,u,v) \bigr\vert \leq d_{1}(t)+d_{2}(t) \vert u \vert +d_{3}(t) \vert v \vert , \quad t\in [0,+\infty ), \\& \int ^{+\infty }_{0}d_{1}(t)\,dt< +\infty , \qquad \int ^{+\infty }_{0}\bigl(d_{2}(t)+d_{3}(t) \bigr)\theta (t) \,dt< \max \biggl\{ {\frac{1}{2L _{2}'}},{\frac{1}{2\zeta _{2}'}} \biggr\} , \end{aligned}$$

where

$$ L_{2}'=\sigma _{2}, \quad\quad \zeta _{2}'={\frac{1+\sigma _{2}(1+N\eta ^{\beta -1})}{ \varGamma (\beta -\gamma _{2})}}. $$

Then system (1.1) with boundary condition (3.4) has at least one solution.

Theorem 3.3

Assume that \((H_{1})\), \((H_{2})\) hold, then system (1.1) has a unique solution if

$$ m_{1}+m_{2}< 1, \quad\quad n_{1}+n_{2}< 1, $$

where

$$\begin{aligned} &m_{1}=L_{1} \int ^{+\infty }_{0}\lambda _{1}(s)\theta (s) \,ds, \qquad n_{1}=L_{2} \int ^{+\infty }_{0}\lambda _{2}(s)\theta (s) \,ds, \\ &m_{2}=\zeta _{1} \int ^{+\infty }_{0}\lambda _{1}(s)\theta (s) \,ds, \qquad n_{2}=\zeta _{2} \int ^{+\infty }_{0}\lambda _{2}(s)\theta (s) \,ds. \end{aligned}$$

Proof

Let \(u_{i}(t), v_{i}(t)\in C^{1}(J)\), \(i=1,2\); then we have

$$\begin{aligned}& \biggl\vert {\frac{T_{1}v_{2}(t)}{1+t^{\alpha -1}}}-{\frac{T_{1}v_{1}(t)}{1+t ^{\alpha -1}}} \biggr\vert \\& \quad \leq \int ^{+\infty }_{0}{\frac{H(t,s)}{1+t^{\alpha -1}}} \bigl\vert \varphi \bigl(s,v _{2}(s),D^{\gamma _{1}}v_{2}(s)\bigr)- \varphi \bigl(s,v_{1}(s),D^{\gamma _{1}}v _{1}(s)\bigr) \bigr\vert \,ds \\& \quad \leq L_{1} \int ^{+\infty }_{0}\lambda _{1}(s)\theta (s) \,ds \Vert v_{2}-v _{1} \Vert _{X}=m_{1} \Vert v_{2}-v_{1} \Vert _{X} \end{aligned}$$

and

$$\begin{aligned}& \biggl\vert {\frac{D^{\gamma _{1}}T_{1}v_{2}(t)}{1+t^{\alpha -1-\gamma _{1}}}}-{\frac{D^{\gamma _{1}}T_{1}v_{1}(t)}{1+t^{\alpha -1-\gamma _{1}}}} \biggr\vert \\& \quad \le {\frac{\sigma _{1}\varGamma (\alpha )}{\varGamma (\alpha -\gamma _{1})}} \int ^{+\infty }_{0} \bigl\vert \varphi \bigl(s,v_{2}(s),D^{\gamma _{1}}v_{2}(s)\bigr)- \varphi \bigl(s,v_{1}(s),D^{\gamma _{1}}v_{1}(s)\bigr) \bigr\vert \,ds \\& \quad\quad {} +{\frac{M\sigma _{1}\xi ^{\alpha -1}}{\varGamma (\alpha -\gamma _{1})}} \int ^{\xi }_{0} \bigl\vert \varphi \bigl(s,v_{2}(s),D^{\gamma _{1}}v_{2}(s)\bigr)- \varphi \bigl(s,v_{1}(s),D^{\gamma _{1}}v_{1}(s)\bigr) \bigr\vert \,ds \\& \quad\quad {} +{\frac{1}{\varGamma (\alpha -\gamma _{1})}} \int ^{t}_{0} \bigl\vert \varphi \bigl(s,v_{2}(s),D^{\gamma _{1}}v_{2}(s)\bigr)-\varphi \bigl(s,v_{1}(s),D^{\gamma _{1}}v _{1}(s)\bigr) \bigr\vert \,ds \\& \quad\quad {} + \frac{2(1+\sigma _{1}M\xi ^{\alpha -2})}{\varGamma (\alpha -\gamma _{1})} \int ^{h}_{0}g_{1}(s) \bigl(T_{1}v_{2}(s)-T_{1}v_{1}(s) \bigr)\,ds \\& \quad \leq \zeta _{1} \int ^{+\infty }_{0}\lambda _{1}(s)\theta (s) \,ds \Vert v_{2}-v _{1} \Vert _{X}=m_{2} \Vert v_{2}-v_{1} \Vert _{X}. \end{aligned}$$

We can see that

$$ \Vert T_{1}v_{2}-T_{1}v_{1} \Vert _{X}\leq (m_{1}+m_{2}) \Vert v_{2}-v_{1} \Vert _{X}. $$

Analogously, it can be proved that

$$\begin{aligned} \biggl\vert {\frac{T_{2}u_{2}(t)}{1+t^{\beta -1}}}-{\frac{T_{2}u_{1}(t)}{1+t ^{\beta -1}}} \biggr\vert \leq & \sigma _{2}(1+\delta _{2}l_{2}) \int ^{+ \infty }_{0}\lambda _{2}(s)\theta (s) \,ds \Vert u_{2}-u_{1} \Vert _{Y} \\ =&n_{1} \Vert u_{2}-u_{1} \Vert _{Y} \end{aligned}$$

and

$$\begin{aligned}& \biggl\vert {\frac{D^{\gamma _{2}}T_{2}u_{2}(t)}{1+t^{\beta -1-\gamma _{2}}}}-{\frac{D^{\gamma _{2}}T_{2}u_{1}(t)}{1+t^{\beta -1-\gamma _{2}}}} \biggr\vert \\& \quad \leq \biggl({\frac{\sigma _{1}(\varGamma (\alpha )+M\xi ^{\alpha -1})+1}{ \varGamma (\alpha -\gamma _{1})}} +{\frac{(\sigma _{1}\omega _{1}M \xi ^{\alpha -1}+\delta _{1})}{(1-\mu _{1})\varGamma (\alpha )}}\cdot \frac{2(1+ \sigma _{1}M\xi ^{\alpha -2})}{\varGamma (\alpha -\gamma _{1})} \biggr) \\& \quad\quad{}\times \int ^{+\infty }_{0}\lambda _{2}(s)\theta (s) \,ds \Vert u_{2}-u_{1} \Vert _{Y} \\& \quad = n_{2} \Vert u_{2}-u_{1} \Vert _{Y}. \end{aligned}$$

Thus we know that

$$ \Vert T_{2}u_{2}-T_{2}u_{1} \Vert _{Y}\leq (n_{1}+n_{2}) \Vert u_{2}-u_{1} \Vert _{Y}. $$

In conclusion, we have

$$ \bigl\Vert T(u_{2},v_{2})-T(u_{1},v_{1}) \bigr\Vert _{Q}\leq \max \{m_{1}+m_{2}, n_{1}+n _{2}\} \bigl\Vert (u_{2},v_{2})-(u_{1},v_{1}) \bigr\Vert _{Q}. $$

Obviously, T is a contraction. By means of the Banach contraction theorem, T has a unique fixed point which is the unique solution of system (1.1). □

Corollary 3.2

Assume that \((H_{2})\), \((H_{3})\) hold, then system (1.1) has a unique solution if \(m_{1}+m_{2}<1\), \(n_{1}+n_{2}<1\).

Corollary 3.3

On the basis of Remark 3.1, if condition \((H_{2})\) holds, then system (1.1) has a unique solution if \(m_{1}+m_{2}<1\), \(n_{1}+n_{2}<1\). In short, if φ, ψ are bounded and continuous on \(J\times R\times R\), then there exists a solution for system (1.1).

Remark 3.2

If \(\varphi ,\psi \in C(J\times \mathbb{R}^{+} \times \mathbb{R}^{+}, \mathbb{R}^{+})\), \(\varphi (t,u,v),\psi (t,u,v) \not \equiv 0\), under condition \((H_{1})\) or \((H_{3})\), then system (1.1) has at least one positive solution. Further, the positive solution is unique if \((H_{1})\), \((H_{2})\) or \((H_{2})\), \((H_{3})\) are satisfied with \(m_{1}+m_{2}<1\), \(n_{1}+n_{2}<1\).

4 An example

Example 4.1

Consider the system

$$ \textstyle\begin{cases} D^{\frac{5}{2}}u(t)+\frac{\sin t}{\sqrt{2+t^{2}}}+\frac{\sqrt{ \vert v(t)D ^{\frac{1}{2}}v(t) \vert }}{72e^{t}(1+t^{\frac{3}{2}})}=0, \quad t\in [0,+\infty ), \\ D^{\frac{5}{2}}v(t)+ (2+\cos t)e^{-t^{2}}+\frac{\sin \vert u(t) \vert }{48e^{t}(1+t ^{\frac{3}{2}})}+\frac{\ln (1+ \vert D^{\frac{1}{2}}u(t) \vert )}{192e^{ \sqrt{t}}(1+t^{\frac{3}{2}})}=0, \\ I^{\frac{1}{2}}u(0)=0, \quad\quad D^{\frac{1}{2}}u(0)=( \frac{3\pi ^{\frac{3}{2}}}{16}-\frac{\pi }{4})\int ^{1}_{0}s^{4}u(s)\,ds, \quad\quad D^{\frac{3}{2}}u(+\infty )=u(1)+2, \\ I^{\frac{1}{2}}v(0)=0, \quad\quad D^{\frac{1}{2}}v(0)=( \frac{\pi ^{\frac{3}{2}}}{8}-\frac{\pi }{6})\int ^{1}_{0}s^{2}v(s)\,ds, \quad\quad D ^{\frac{3}{2}}v(+\infty )=v(1)+2, \end{cases} $$
(4.1)

where \(\alpha =\beta ={\frac{5}{2}}\), \(\gamma _{1}=\gamma _{2}={\frac{1}{2}}\), \(h=1\), \(M,N,\xi ,\eta =1\), \(a=b=2\), \(g_{1}(t)=( \frac{3\pi ^{\frac{3}{2}}}{16}-\frac{\pi }{4})t^{4}\), \(g_{2}(t)=(\frac{ \pi ^{\frac{3}{2}}}{8}-\frac{\pi }{6})t^{2}\), and

$$\begin{aligned}& \varphi (t,u,v)=\frac{\sin t}{\sqrt{2+t^{2}}}+\frac{\sqrt{ \vert uv \vert }}{72e ^{t}(1+t^{\frac{3}{2}})}, \\& \psi (t,u,v)=(2+\cos t)e^{-t^{2}}+\frac{ \sin \vert u \vert }{48e^{t}(1+t^{\frac{3}{2}})}+\frac{\ln (1+ \vert v \vert )}{192e^{ \sqrt{t}}(1+t^{\frac{3}{2}})}. \end{aligned}$$

Choose

$$\begin{aligned} &c_{1}(t)=\frac{1}{\sqrt{2+t^{2}}}, \qquad c_{2}(t)=c_{3}(t)= \frac{1}{144e^{t}(1+t^{\frac{3}{2}})}, \\ &d_{1}(t)=3e^{-t^{2}}, \qquad d_{2}(t)= \frac{1}{48e^{t}(1+t^{\frac{3}{2}})}, \quad\quad d_{3}(t)=\frac{1}{192e ^{\sqrt{t}}(1+t^{\frac{3}{2}})}. \end{aligned}$$

Obviously, \(|\varphi (t,u,v)\vert \leq c_{1}(t)+c_{2}(t) \vert u \vert +c_{3}(t) \vert v \vert \), \(\vert \psi (t,u,v) \vert \leq d_{1}(t)+d_{2}(t) \vert u \vert +d_{3}(t) \vert v \vert \), and by simple computations, we find that \(0< M\xi ^{\alpha -1}\), \(N\eta ^{\beta -1}<{\varGamma ( \frac{5}{2})}\approx 1.329\), \(\sigma _{1}=\sigma _{2}=\frac{4}{3\sqrt{ \pi }-4}\), \(T_{1}(t)=T_{2}(t)=\frac{8}{3\pi -4\sqrt{\pi }}t^{ \frac{3}{2}}+\frac{2}{\sqrt{\pi }}t^{\frac{1}{2}}\), \(\mu _{1}=\int _{0}^{1}g_{1}(t)T_{1}(t)\,dt=\frac{\sqrt{\pi }}{14}+\frac{3\pi -4\sqrt{ \pi }}{24}<1\), \(\mu _{2}=\int _{0}^{1}g_{2}(t)T_{2}(t)\,dt=\frac{\pi }{8}-\frac{\sqrt{ \pi }}{12}<1\), \(\delta _{1}=\frac{9\pi ^{\frac{3}{2}}-12\pi }{140}\), \(\delta _{2}=\frac{7\pi ^{\frac{3}{2}}}{96}-\frac{7\pi }{72}\), \(\omega _{1}=\frac{3\pi ^{\frac{3}{2}}-4\pi }{112}\), \(\omega _{2}=\frac{ \pi ^{\frac{3}{2}}}{32}-\frac{\pi }{24}\), \(l_{1}=\frac{1008}{(3\sqrt{ \pi }-4)(168+16\sqrt{\pi }-21\pi )}\) and \(l_{2}=\frac{144}{(3\sqrt{ \pi }-4)(24+2\sqrt{\pi }-3\pi )}\). Further, we can obtain

$$\begin{aligned}& \int _{0}^{+\infty }c_{1}(t)\,dt= \frac{\pi }{4}< +\infty , \\& \begin{aligned} \int _{0}^{+\infty }\bigl(c_{2}(t)+c_{3}(t) \bigr)\theta (t)\,dt &=\frac{1}{72} \int _{0}^{1}\frac{1+t}{e^{t}(1+t^{\frac{3}{2}})}\,dt+ \frac{1}{72} \int _{1} ^{+\infty }e^{-t}\,dt \\ &\leq \frac{1}{72} \biggl(2+\frac{1}{e}-\frac{4\ln 2}{3} \biggr)< \max \biggl\{ { \frac{1}{2L_{1}}},{\frac{1}{2\zeta _{1}}} \biggr\} . \end{aligned} \end{aligned}$$

Here,

$$\begin{aligned} &L_{1}=\sigma _{1}(1+\delta _{1}l_{1})= \frac{4}{3\sqrt{\pi }-4}+\frac{3024 \pi }{35(3\sqrt{\pi }-4)(168+16\sqrt{\pi }-21\pi )}\approx 4.617, \\ &\zeta _{1}=\frac{6\sqrt{\pi }}{3\sqrt{\pi }-4}+\frac{4536 \pi ^{\frac{3}{2}}-1008\pi }{35(3\sqrt{\pi }-4)(168+16\sqrt{\pi }-21 \pi )}\approx 11.747, \\ &\int _{0}^{+\infty }d_{1}(t)\,dt= \frac{3\sqrt{\pi }}{2}< +\infty , \\ &\begin{aligned} \int _{0}^{+\infty }\bigl(d_{2}(t)+d_{3}(t) \bigr)\theta (t)\,dt &= \int _{0}^{1} \biggl(\frac{1+t}{48e^{t}(1+t^{\frac{3}{2}})}+ \frac{1+t}{192e^{ \sqrt{t}}(1+t^{\frac{3}{2}})} \biggr)\,dt \\ &\quad{} + \int _{1}^{+\infty } \biggl(\frac{1}{48e ^{t}}+ \frac{1}{192e^{\sqrt{t}}} \biggr)\,dt \\ &\leq \frac{5}{96}-\frac{5\ln 2}{144}+\frac{1}{32e}< \max \biggl\{ {\frac{1}{2L _{2}}},{\frac{1}{2\zeta _{2}}} \biggr\} , \end{aligned} \\ & L_{2}=\sigma _{2}(1+\delta _{2}l_{2})= \frac{4}{3\sqrt{\pi }-4}+\frac{14 \pi }{(3\sqrt{\pi }-4)(24+2\sqrt{\pi }-3\pi )}\approx 4.879, \end{aligned}$$

and

$$ \zeta _{2}=\frac{6\sqrt{\pi }}{3\sqrt{\pi }-4}+\frac{21 \pi ^{\frac{3}{2}}-7\pi }{(3\sqrt{\pi }-4)(24+2\sqrt{\pi }-3\pi )} \approx 12.050. $$

Then the conditions of Theorem 3.1 are satisfied, so system (4.1) has at least one solution.