1 Introduction

In this article, we study a new coupled system of fractional differential equations and consider the existence and uniqueness of solutions for the system with integral boundary conditions. Namely, we discuss the following problem:

$$\begin{aligned} \textstyle\begin{cases} D^{\alpha}u(t)+f(t,v(t))=a,\quad 0< t< 1,\\ D^{\beta}v(t)+g(t,u(t))=b,\quad 0< t< 1,\\ u(0)=0,\qquad u(1)=\int_{0}^{1} \phi(t)u(t)\,dt,\\ v(0)=0,\qquad v(1)=\int_{0}^{1} \psi (t)v(t)\,dt, \end{cases}\displaystyle \end{aligned}$$
(1.1)

where \(1< \alpha,\beta\le2, f,g \in C([0,1]\times(-\infty,+\infty ),(-\infty,+\infty)), \phi,\psi\in L^{1}[0,1]\), \(a,b\) are constants and D denotes the usual Riemann-Liouville fractional derivative (see [1, 2]). A pair of functions \((u,v)\in C([0,1])\times C([0,1])\) is called a solution of system (1.1) if it satisfies (1.1). We will consider system (1.1) under the case \(a>0, b>0\). When \(a=b=0\), we can study problem (1.1) by using the usual methods and give some results about the existence and uniqueness of positive solutions (see [3]). However, when \(a,b>0\), we cannot get similar results by using the previous methods. So system (1.1) is new, and we need to seek new methods to discuss problem (1.1). Fortunately, we can use a recent fixed point theorem for φ-\((h,e)\)-concave operators to resolve problem (1.1). First, we list the following hypotheses on the functions \(\phi(t),\psi (t)\):

\((Q)\) \(\phi,\psi:[0,1]\rightarrow[0,+\infty)\) with \(\phi,\psi\in L^{1}[0,1]\) and

$$\begin{aligned} &\sigma_{1}:= \int^{1}_{0} \phi(t)t^{\alpha-1}\,dt,\qquad \sigma_{2}:= \int^{1}_{0} \psi (t)t^{\beta-1}\,dt \in(0,1); \\ &\sigma_{3}:= \int^{1}_{0} t^{\alpha-1}(1-t)\phi(t)\,dt,\qquad \sigma_{4}:= \int^{1}_{0} t^{\beta-1}(1-t)\psi(t)\,dt >0. \end{aligned}$$

A lot of boundary value problems of coupled systems involving fractional differential equations have been investigated extensively, see the works [4–25] and the references therein. Different boundary conditions of coupled systems can be found in the discussions of some problems such as Sturm-Liouville problems and some reaction-diffusion equations (see [26, 27]), and they have some applications in many fields such as mathematical biology (see [28, 29]), natural sciences and engineering; for example, we can see beam deformation and steady-state heat flow [30, 31] and heat equations [14, 32, 33]. So nonlinear coupled systems subject to different boundary conditions have been paid much attention to, and the existence or multiplicity of solutions for the systems has been given in literature, see [4–14, 16–25] for example. The usual methods used are Schauder’s fixed point theorem, Banach’s fixed point theorem, Guo-Krasnosel’skii’s fixed point theorem on cone, nonlinear differentiation of Leray-Schauder type and so on. Recently, several papers [4, 8–10] considered some new coupled systems of fractional differential equations and obtained some new results about the existence and uniqueness of solutions by using general methods.

From literature, no papers have considered system (1.1). Inspired by the works of coupled systems and recent papers [16, 34], we study the coupled system (1.1) and give the existence and uniqueness of solutions. By using a fixed point theorem of increasing φ-\((h,e)\)-concave operators, we establish the existence and uniqueness of solutions for the coupled system dependent on two constants. Our result shows that the unique solution exists in a product set and can be approximated by making an iterative sequence for any initial point in the product set. So our result is an extension and improvement of the previous works.

2 Preliminaries

Lemma 2.1

(see [11])

Let \(\int_{0}^{1} \phi(t)t^{\alpha-1}\,dt \neq 1\) and \(\sigma\in C[0,1]\). Then the problem

$$\textstyle\begin{cases} D^{\alpha}u(t)+\sigma(t)=0, \quad 0< t< 1,\\ u(0)=0, \qquad u(1)=\int^{1}_{0} \phi(t)u(t)\,dt \end{cases} $$

has the unique solution \(u(t)=\int_{0}^{1} G_{1\alpha}(t,s)\sigma(s)\,ds\), where \(G_{1\alpha}(t,s)=G_{2\alpha}(t,s)+G_{3\alpha}(t,s)\) with

$$\begin{aligned} &G_{2\alpha}(t,s)= \textstyle\begin{cases} \frac{t^{\alpha-1}(1-s)^{\alpha-1}-(t-s)^{\alpha-1}}{\Gamma(\alpha)},& 0\le s\le t\le1,\\ \frac{t^{\alpha-1}(1-s)^{\alpha-1}}{\Gamma(\alpha)},& 0\le t\le s \le 1, \end{cases}\displaystyle \\ &G_{3\alpha}(t,s)=\frac{t^{\alpha-1}}{1-\int_{0}^{1} \phi(t)t^{\alpha -1}\,dt} \int_{0}^{1} \phi(t)G_{2\alpha}(t,s)\,dt. \end{aligned}$$

The function \(G(t,s)=(G_{1\alpha}(t,s),G_{1\beta}(t,s))\) is the Green’s function of problem (1.1).

Lemma 2.2

(see [11])

Let \(\int_{0}^{1} \phi(t)t^{\alpha-1}\,dt\in [0,1)\). Then \(G_{1\alpha}(t,s)\ge0 \) is continuous for all \(t,s\in[0,1]\), and \(G_{1\alpha}(t,s)>0\) for all \(t,s \in(0,1)\).

Lemma 2.3

(see [3])

The function \(G_{2\alpha}(t,s)\) satisfies

$$\frac{\alpha-1}{\Gamma(\alpha)}t^{\alpha-1}(1-t) (1-s)^{\alpha-1}s\le G_{2\alpha}(t,s)\le\frac{t^{\alpha-1}(1-s)^{\alpha-1}}{\Gamma(\alpha )},\quad t,s\in[0,1]. $$

Lemma 2.4

(see [3])

Suppose \(\alpha,\beta\in(1,2]\) and \((Q)\) is satisfied. Then the functions \(G_{1\alpha}(t,s)\), \(G_{1\beta}(t,s)\) satisfy

$$\begin{aligned} &\frac{(\alpha-1)\sigma_{3} s(1-s)^{\alpha-1}t^{\alpha-1}}{(1-\sigma _{1})\Gamma(\alpha)}\le G_{1\alpha}\le\frac{(1-s)^{\alpha-1}t^{\alpha -1}}{\Gamma(\alpha)(1-\sigma_{1})},\quad t,s\in[0,1]; \\ &\frac{(\beta-1)\sigma_{4} s(1-s)^{\beta-1}t^{\beta-1}}{(1-\sigma_{2})\Gamma (\beta)}\le G_{1\beta}\le\frac{(1-s)^{\beta-1}t^{\beta-1}}{\Gamma (\beta)(1-\sigma_{2})},\quad t,s\in[0,1]. \end{aligned}$$

Next we present a fixed point theorem which can be easily used to study some systems of differential equations.

Suppose \((E,\Vert \cdot\Vert )\) is a real Banach space and it is partially ordered by a cone \(P \subset E\). For any \(x,y \in E\), \(x\sim y\) denotes that there are \(\mu> 0\) and \(\nu> 0\) such that \(\mu x \leq y \leq\nu x\). Take \(h > \theta \) (i.e., \(h \geq\theta\) and \(h \neq\theta\)), we consider the set \(P_{h}=\{ x \in E \mid x \sim h \}\). Clearly, \(P_{h} \subset P\). Take another element \(e\in P\) with \(\theta\le e\le h\), we define \(P_{h,e}=\{x\in E| x+e\in P_{h}\}\).

Definition 2.1

(see [34])

Assume that \(A:P_{h,e}\rightarrow E\) is an operator which satisfies: for any \(x\in P_{h,e}\) and \(\lambda\in(0,1)\), there exists \(\varphi(\lambda )>\lambda\) such that \(A(\lambda x+(\lambda-1)e)\ge\varphi(\lambda)Ax+(\varphi(\lambda )-1)e\). Then we call A a φ-\((h,e)\)-concave operator.

Lemma 2.5

(see [34])

Suppose that P is normal and A is an increasing φ-\((h,e)\)-concave operator satisfying \(Ah \in P_{h,e}\). Then A has a unique fixed point \(x^{*}\) in \(P_{h,e}\). In addition, for any \(w_{0} \in P_{h,e}\), constructing the sequence \(w_{n} =A w_{n-1}, n=1,2,\ldots\) , then \(\Vert w_{n}-x^{*} \Vert \rightarrow0\) as \(n\rightarrow\infty\).

Given \(h_{1},h_{2}\in P\) with \(h_{1},h_{2}\neq\theta\). Let \(h=(h_{1},h_{2})\), then \(h\in\overline{P}:=P\times P\). Take \(\theta\leq e_{1}\leq h_{1}\), \(\theta \leq e_{2}\leq h_{2}\), and let \(\bar{\theta}=(\theta,\theta)\), \(e=(e_{1},e_{2})\). Then \(\bar{\theta}=(\theta,\theta)\leq(e_{1},e_{2})\leq (h_{1},h_{2})=h\). That is, \(\bar{\theta}\leq e\leq h\). If P is normal, then \(\overline{P}=P\times P\) is normal (see [35]).

Lemma 2.6

(see [3])

\(\overline{P_{h}}=P_{h_{1}}\times P_{h_{2}}\).

Lemma 2.7

\(\overline{P}_{h,e}=P_{h_{1},e_{1}}\times P_{h_{2},e_{2}}\).

Proof

By Lemma 2.6, we obtain

$$\begin{aligned} \overline{P}_{h,e}&=\bigl\{ (x,y)\in E\times E\mid(x,y)+e\in \bar{P}_{h}\bigr\} \\ & =\bigl\{ (x,y)\in E\times E\mid(x+e_{1},y+e_{2}) \in P_{h_{1}}\times P_{h_{2}}\bigr\} \\ &=\bigl\{ (x,y)\in E\times E\mid x+e_{1}\in P_{h_{1}}, y+e_{2}\in P_{h_{2}} \bigr\} =\bigl\{ (x,y)\in E\times E\mid x\in P_{h_{1},e_{1}}, y\in P_{h_{2},e_{2}} \bigr\} \\ &=P_{h_{1},e_{1}}\times P_{h_{2},e_{2}}. \end{aligned}$$

 □

3 Existence and uniqueness of solutions

In this section, let \(E=\{u|u\in C[0,1]\}\) and the norm of E is \(\Vert u \Vert =\max_{t\in[0,1]} \vert u(t) \vert \). We will consider (1.1) in \(E\times E\). For \((u,v)\in E \times E\), let \(\Vert (u,v) \Vert =\max\{ \Vert u \Vert , \Vert v \Vert \}\). It is clear that \((E\times E, \Vert (u,v) \Vert )\) is a Banach space. Let \(\overline{P} = \{(u,v)\in E\times E| u(t)\ge0,v(t)\ge0\}, P=\{u\in E|u(t)\ge0, t\in[0,1]\}\), then the cone \(\overline{P}\subset E\times E\) and \(\overline{P}=P\times P\) is normal, and the space \(E\times E\) has a partial order: \((u_{1},v_{1})\le(u_{2},v_{2})\Leftrightarrow u_{1}(t)\le u_{2}(t),v_{1}(t)\le v_{2}(t),t\in[0,1]\). By Lemma 2.1 and the result of [21], we can easily get the following result.

Lemma 3.1

Suppose that \((Q)\) is satisfied and \(f(t,x), g(t,x)\) are continuous, then \((u,v)\in E\times E\) is a solution of (1.1) if and only if \((u,v)\in E\times E\) is a solution of the following equations:

$$\textstyle\begin{cases} u(t)=\int_{0}^{1} G_{1\alpha}(t,s)f(s,v(s))\,ds-a\int^{1}_{0}G_{1\alpha}(t,s)\,ds,\\ v(t)=\int_{0}^{1} G_{1\beta}(t,s)g(s,u(s))\,ds-b\int^{1}_{0}G_{1\beta}(t,s)\,ds.\end{cases} $$

For \((u,v)\in E\times E\), we define three operators \(A_{1},A_{2}\) and T by

$$\begin{aligned} &A_{1} u(t)= \int_{0}^{1} G_{1\alpha} (t,s)f\bigl(s,v(s) \bigr)\,ds-a \int^{1}_{0}G_{1\alpha}(t,s)\,ds, \\ &A_{2} v(t)= \int_{0}^{1} G_{1\beta}(t,s)g\bigl(s,u(s) \bigr)\,ds-b \int^{1}_{0}G_{1\beta}(t,s)\,ds, \end{aligned}$$

and \(T(u,v)(t)=(A_{1} u(t),A_{2} v(t))\). Then \(A_{1},A_{2}:E\rightarrow E\) and \(T:E\times E\rightarrow E\times E\). From Lemma 3.1, \((u,v)\) is the solution of system (1.1) if and only if \((u,v)\) is the fixed point of operator T. Let

$$\begin{aligned} \begin{aligned} &e_{1}(t)=a \int^{1}_{0}G_{1\alpha}(t,s)\,ds,\qquad e_{2}(t)=b \int^{1}_{0}G_{1\beta }(t,s)\,ds,\\ & h_{1}(t)=M_{1}t^{\alpha-1}, \qquad h_{2}(t)=M_{2}t^{\beta-1}, \end{aligned} \end{aligned}$$
(3.1)

where \(M_{1}\geq\frac{a}{\Gamma(\alpha+1)(1-\sigma_{1})}\), \(M_{2}\geq\frac {b}{\Gamma(\beta+1)(1-\sigma_{2})}\).

Theorem 3.1

Let \(1< \alpha,\beta\le2,a>0,b>0\) and \(e_{1},e_{2},h_{1},h_{2}\) be given as in (3.1). Assume that \(f,g \in C([0,1]\times(-\infty,+\infty),(-\infty,+\infty))\) and \((Q)\) holds. Moreover,

\((H_{1})\) \(f:[0,1]\times[-e_{2}^{*},+\infty)\rightarrow(-\infty,+\infty)\) is increasing with respect to the second variable, where \(e_{2}^{*}=\max\{ e_{2}(t):t\in[0,1]\}\); \(g:[0,1]\times[-e_{1}^{*},+\infty)\rightarrow(-\infty ,+\infty)\) is increasing with respect to the second variable, where \(e_{1}^{*}=\max\{e_{1}(t):t\in[0,1]\}\);

\((H_{2})\) for \(\lambda\in(0,1)\), there exists \(\varphi(\lambda)>\lambda\) such that

$$\begin{aligned} &f\bigl(t,\lambda x+(\lambda-1)y\bigr)\geq\varphi(\lambda)f(t,x),\quad t\in[0,1],x\in (-\infty,+\infty),y\in\bigl[0,e_{2}^{*}\bigr]; \\ &g\bigl(t,\lambda x+(\lambda-1)y\bigr)\geq\varphi(\lambda)g(t,x),\quad t\in[0,1],x\in (-\infty,+\infty),y\in\bigl[0,e_{1}^{*}\bigr]; \end{aligned}$$

\((H_{3})\) \(f(t,0)\geq0\), \(g(t,0)\geq0\) with \(f(t,0)\not\equiv0, g(t,0)\not \equiv0\) for \(t\in[0,1]\).

Then:

(1) system (1.1) has a unique solution \((u^{*},v^{*})\) in \(\overline {P}_{h,e}\), where

$$e(t)=\bigl(e_{1}(t),e_{2}(t)\bigr), \qquad h(t)= \bigl(h_{1}(t),h_{2}(t)\bigr),\quad t\in[0,1]; $$

(2) taking any point \((u_{0},v_{0})\in\overline{P}_{h,e}\), construct the following sequences:

$$\begin{aligned} &u_{n+1}(t)= \int_{0}^{1} G_{1\alpha}(t,s)f \bigl(s,v_{n}(s)\bigr)\,ds-a \int^{1}_{0}G_{1\alpha}(t,s)\,ds, \\ &v_{n+1}(t)= \int_{0}^{1} G_{1\beta}(t,s)g \bigl(s,u_{n}(s)\bigr)\,ds-b \int^{1}_{0}G_{1\beta}(t,s)\,ds, \end{aligned}$$

\(n=0,1,2,\ldots\) , we have \(u_{n+1}(t)\rightarrow u^{*}(t),v_{n+1}(t)\rightarrow v^{*}(t)\) as \(n\rightarrow\infty\).

Proof

By Lemma 2.2, for \(t\in[0,1]\),

$$e_{1}(t)=a \int^{1}_{0}G_{1\alpha}(t,s)\,ds \geq0,\qquad e_{2}(t)=b \int^{1}_{0}G_{1\beta }(t,s)\,ds\geq0. $$

From Lemma 2.4, for \(t\in[0,1]\),

$$\begin{aligned} e_{1}(t)={}&a \int^{1}_{0}G_{1\alpha}(t,s)\,ds\leq a \int^{1}_{0}\frac{(1-s)^{\alpha -1}t^{\alpha-1}}{\Gamma(\alpha)(1-\sigma_{1})}=\frac{at^{\alpha-1}}{\Gamma (\alpha) (1-\sigma_{1})} \int^{1}_{0}(1-s)^{\alpha-1}\,ds \\ ={}&\frac{at^{\alpha-1}}{\alpha\Gamma(\alpha)(1-\sigma_{1})}=\frac {a}{\Gamma(\alpha+1)(1-\sigma_{1})}t^{\alpha-1}\leq M_{1}t^{\alpha-1}=h_{1}(t); \\ e_{2}(t)={}&b \int^{1}_{0}G_{1\beta}(t,s)\,ds\leq b \int^{1}_{0}\frac{(1-s)^{\beta -1}t^{\beta-1}}{\Gamma(\beta)(1-\sigma_{2})}=\frac{bt^{\beta-1}}{\Gamma (\beta) (1-\sigma_{2})} \int^{1}_{0}(1-s)^{\beta-1}\,ds \\ ={}&\frac{bt^{\beta-1}}{\beta\Gamma(\beta)(1-\sigma_{2})}=\frac{b}{\Gamma (\beta+1)(1-\sigma_{2})}t^{\beta-1}\leq M_{2}t^{\beta-1}=h_{2}(t). \end{aligned}$$

That is, \(0\leq e_{1}\leq h_{1}\), \(0\leq e_{2}\leq h_{2}\).

In the following, we prove that \(T:\overline{P}_{h,e}\rightarrow E\times E\) is a φ-\((h,e)\)-concave operator. For \((u,v)\in \overline{P}_{h,e}\), \(\lambda\in(0,1)\), we obtain

$$\begin{aligned} &T\bigl(\lambda(u,v)+(\lambda-1)e\bigr) (t)\\ &\quad =T\bigl(\lambda(u,v)+(\lambda -1) (e_{1},e_{2})\bigr) (t) \\ &\quad =T\bigl(\lambda u+(\lambda-1)e_{1},\lambda v+(\lambda-1)e_{2} \bigr) (t) \\ &\quad =\bigl(A_{1}\bigl(\lambda u+(\lambda-1)e_{1}\bigr), A_{2}\bigl(\lambda v+(\lambda-1)e_{2}\bigr)\bigr) (t). \end{aligned}$$

We discuss \(A_{1}(\lambda u+(\lambda-1)e_{1})(t)\) and \(A_{2}(\lambda v+(\lambda-1)e_{2})(t)\) respectively. From \((H_{2})\),

$$\begin{aligned} &A_{1}\bigl(\lambda u+(\lambda-1)e_{1}\bigr) (t)\\ &\quad= \int^{1}_{0}G_{1\alpha}(t,s)f\bigl(s,\lambda v(s)+(\lambda-1)e_{2}(s)\bigr)\,ds-e_{1}(t) \\ &\quad\geq\varphi(\lambda) \int^{1}_{0}G_{1\alpha}(t,s)f\bigl(s,v(s) \bigr)\,ds-e_{1}(t) \\ &\quad=\varphi(\lambda) \biggl[ \int^{1}_{0}G_{1\alpha }(t,s)f\bigl(s,v(s) \bigr)\,ds-e_{1}(t) \biggr]+\bigl[\varphi(\lambda)-1\bigr]e_{1}(t) \\ &\quad =\varphi(\lambda)A_{1}u(t)+\bigl[\varphi(\lambda)-1 \bigr]e_{1}(t), \\ &A_{2}\bigl(\lambda v+(\lambda-1)e_{2}\bigr) (t)\\ &\quad = \int^{1}_{0}G_{1\beta}(t,s)g\bigl(s,\lambda u(s)+(\lambda-1)e_{1}(s)\bigr)\,ds-e_{2}(t) \\ &\quad\geq\varphi(\lambda) \int^{1}_{0}G_{1\beta}(t,s)g\bigl(s,u(s) \bigr)\,ds-e_{2}(t) \\ &\quad =\varphi(\lambda) \biggl[ \int^{1}_{0}G_{1\beta}(t,s)g\bigl(s,u(s) \bigr)\,ds-e_{2}(t) \biggr]+\bigl[\varphi(\lambda)-1\bigr]e_{2}(t) \\ &\quad =\varphi(\lambda)A_{2}v(t)+\bigl[\varphi(\lambda)-1 \bigr]e_{2}(t). \end{aligned}$$

So we have

$$\begin{aligned} &T\bigl(\lambda(u,v)+(\lambda-1)e\bigr) (t)\\ &\quad \geq \bigl(\varphi(\lambda )A_{1}u(t)+\bigl[\varphi(\lambda)-1\bigr]e_{1}(t),\varphi( \lambda)A_{2}v(t)+\bigl[\varphi (\lambda)-1\bigr]e_{2}(t) \bigr) \\ &\quad =\bigl(\varphi(\lambda)A_{1}u(t),\varphi(\lambda)A_{2}v(t) \bigr)+\bigl(\bigl(\varphi(\lambda )-1\bigr)e_{1}(t),\bigl(\varphi( \lambda)-1\bigr)e_{2}(t)\bigr) \\ &\quad =\varphi(\lambda) \bigl(A_{1}u(t),A_{2}v(t)\bigr)+\bigl( \varphi(\lambda )-1\bigr) \bigl(e_{1}(t),e_{2}(t)\bigr) \\ &\quad =\varphi(\lambda)T(u,v) (t)+\bigl(\varphi(\lambda)-1\bigr)e(t). \end{aligned}$$

That is,

$$\begin{aligned} &T\bigl(\lambda(u,v)+(\lambda-1)e\bigr) \\ &\quad\geq\varphi(\lambda)T(u,v)+\bigl[\varphi( \lambda )-1\bigr]e,\quad (u,v)\in\overline{P}_{h,e},\lambda\in(0,1). \end{aligned}$$

Hence, T is a φ-\((h,e)\)-concave operator.

Next we show that \(T:\overline{P}_{h,e}\rightarrow E\times E\) is increasing. For \((u,v)\in\overline{P}_{h,e}\), we have \((u,v)+e\in \overline{P}_{h}\). From Lemma 2.6, \((u+e_{1},v+e_{2})\in P_{h_{1}}\times P_{h_{2}}\). So there are \(\lambda_{1},\lambda_{2}>0\) such that

$$u(t)+e_{1}(t)\geq\lambda_{1}h_{1}(t),\qquad v(t)+e_{2}(t) \geq\lambda_{2}h_{2}(t),\quad t\in[0,1]. $$

Therefore, \(u(t)\geq\lambda_{1}h_{1}(t)-e_{1}(t)\geq-e_{1}(t)\geq-e_{1}^{*}\), \(v(t)\geq\lambda_{2}h_{2}(t)-e_{2}(t)\geq-e_{2}(t)\geq-e_{2}^{*}\). By \((H_{1})\) and the definitions of \(A_{1}\), \(A_{2}\), we obtain \(T:\overline {P}_{h,e}\rightarrow E\times E\) is increasing.

Now we prove that \(Th\in\overline{P}_{h,e}\), so we need to prove \(Th+e\in\overline{P}_{h}\). For \(t\in[0,1]\),

$$\begin{aligned} Th(t)+e(t)&=T(h_{1},h_{2}) (t)+e(t)=\bigl(A_{1}h_{1}(t),A_{2}h_{2}(t) \bigr)+\bigl(e_{1}(t),e_{2}(t)\bigr) \\ &=\bigl(A_{1}h_{1}(t)+e_{1}(t),A_{2}h_{2}(t)+e_{2}(t) \bigr). \end{aligned}$$

We discuss \(A_{1}h_{1}(t)+e_{1}(t)\), \(A_{2}h_{2}(t)+e_{2}(t)\), respectively. By Lemma 2.4 and \((H_{1})\), \((H_{3})\),

$$\begin{aligned} A_{1}h_{1}(t)+e_{1}(t)&= \int^{1}_{0}G_{1\alpha}(t,s)f \bigl(s,h_{2}(s)\bigr)\,ds \\ &\geq \int^{1}_{0}\frac{(\alpha-1)\sigma_{3}s(1-s)^{\alpha-1}}{(1-\sigma _{1})\Gamma(\alpha)}t^{\alpha-1}f \bigl(s,M_{2}s^{\alpha-1}\bigr)\,ds \\ &\geq \frac{(\alpha-1)\sigma_{3}}{(1-\sigma_{1})\Gamma(\alpha)}t^{\alpha -1} \int^{1}_{0}s(1-s)^{\alpha-1}f(s,0)\,ds \\ &=\frac{(\alpha-1)\sigma_{3}}{(1-\sigma_{1})M_{1}\Gamma(\alpha)}h_{1}(t) \int ^{1}_{0}s(1-s)^{\alpha-1}f(s,0)\,ds, \\ A_{1}h_{1}(t)+e_{1}(t) &\leq \int^{1}_{0}\frac{(1-s)^{\alpha-1}t^{\alpha-1}}{(1-\sigma_{1})\Gamma (\alpha)}f(s,M_{2})\,ds \\ &=\frac{1}{M_{1}(1-\sigma_{1})\Gamma(\alpha)}h_{1}(t) \int^{1}_{0}(1-s)^{\alpha -1}f(s,M_{2})\,ds. \end{aligned}$$

From \((H_{1})\), \((H_{3})\), one has \(\int^{1}_{0}(1-s)^{\alpha-1}f(s,M_{2})\,ds\geq \int^{1}_{0}s(1-s)^{\alpha-1}f(s,0)\,ds>0\). Note that \(\sigma_{3}\leq\sigma _{1}<1\) and \(\alpha-1\leq1\), we obtain

$$\begin{aligned} l_{1}&:=\frac{(\alpha-1)\sigma_{3}}{(1-\sigma_{1})H_{1}\Gamma(\alpha)} \int ^{1}_{0}s(1-s)^{\alpha-1}f(s,0)\,ds\\ &\leq l_{2}:=\frac{1}{H_{1}(1-\sigma_{1})\Gamma (\alpha)} \int^{1}_{0}(1-s)^{\alpha-1}f(s,M_{2})\,ds, \end{aligned}$$

and thus \(l_{1}h_{1}(t)\leq A_{1}h_{1}(t)+e_{1}(t)\leq l_{2}h_{1}(t)\). This shows \(A_{1}h_{1}+e_{1}\in P_{h_{1}}\). Similarly, by using Lemma 2.4 and \((H_{1})\), \((H_{3})\), we also can get \(A_{2}h_{2}+e_{2}\in P_{h_{2}}\). Consequently, by Lemma 2.7,

$$Th+e=(A_{1}h_{1}+e_{1},A_{2}h_{2}+e_{2}) \in P_{h_{1}}\times P_{h_{2}}=\overline{P}_{h}. $$

Finally, by using Lemma 2.5, T has a unique fixed point \((u^{*},v^{*})\in \overline{P}_{h,e}\). In addition, for any given \((u_{0},v_{0})\in\overline {P}_{h,e}\), the sequence

$$(u_{n},v_{n})=(A_{1}u_{n-1},A_{2}v_{n-1}),\quad n=1,2, \ldots $$

converges to \((u^{*},v^{*})\) as \(n\rightarrow\infty\). Therefore, system (1.1) has a unique solution \((u^{*},v^{*})\) in \(\overline{P}_{h,e}\); taking any point \((u_{0},v_{0})\in\overline{P}_{h,e}\), construct the following sequences:

$$\begin{aligned} &u_{n+1}(t)= \int_{0}^{1} G_{1\alpha}(t,s)f \bigl(s,v_{n}(s)\bigr)\,ds-a \int^{1}_{0}G_{1\alpha}(t,s)\,ds, \\ &v_{n+1}(t)= \int_{0}^{1} G_{1\beta}(t,s)g \bigl(s,u_{n}(s)\bigr)\,ds-b \int^{1}_{0}G_{1\beta}(t,s)\,ds, \end{aligned}$$

\(n=0,1,2,\ldots\) , we have \(u_{n+1}(t)\rightarrow u^{*}(t),v_{n+1}(t)\rightarrow v^{*}(t)\) as \(n\rightarrow\infty\). □

Example 3.1

Consider the coupled system:

$$\begin{aligned} \textstyle\begin{cases} D^{\frac{3}{2}}v(t)+ (\frac{1280\sqrt [3]{140}}{3087}u(t)+\frac{1}{\Gamma(\frac{4}{3})} )^{\frac{1}{5}} (\frac{147}{160}-\frac{3}{4}t )^{\frac{1}{5}}t^{\frac {1}{15}}=1,\quad 0< t< 1,\\ D^{\frac{4}{3}}u(t)+ (\frac{1215\sqrt{15}}{1372}v(t)+\frac {1}{\Gamma(\frac{3}{2})} )^{\frac{1}{5}} (\frac{98}{135}-\frac{2}{3}t )^{\frac{1}{5}}t^{\frac {1}{10}}=1,\quad 0< t< 1,\\ u(0)=0,\qquad u(1)= \int^{1}_{0}t^{2}u(t)\,dt,\qquad v(0)=0,\qquad v(1)= \int^{1}_{0}tv(t)\,dt. \end{cases}\displaystyle \end{aligned}$$
(3.2)

In this example, \(\alpha=\frac{3}{2}\), \(\beta=\frac{4}{3}\), \(a=b=1\), \(\varphi(t)=t^{2}\), \(\psi(t)=t\) and

$$\begin{aligned} &f(t,x)= \biggl(\frac{1280\sqrt[3]{140}}{3087}x+\frac{1}{\Gamma(\frac {4}{3})} \biggr)^{\frac{1}{5}} \biggl(\frac{147}{160}-\frac{3}{4}t \biggr)^{\frac{1}{5}}t^{\frac{1}{15}}, \\ &g(t,x)= \biggl(\frac{1215\sqrt{15}}{1372}x+\frac{1}{\Gamma(\frac {3}{2})} \biggr)^{\frac{1}{5}} \biggl(\frac{98}{135}-\frac{2}{3}t \biggr)^{\frac{1}{5}}t^{\frac{1}{10}}. \end{aligned}$$

\(f, g:[0,1]\times(-\infty,+\infty)\rightarrow(-\infty,+\infty)\) are continuous and increasing with respect to the second variable. After a simple computation, we have

$$\sigma_{1}=\frac{2}{7},\qquad \sigma_{2}= \frac{3}{7},\qquad \sigma_{3}=\frac{4}{63},\qquad \sigma _{4}=\frac{9}{70}. $$

Evidently, \(\sigma_{1},\sigma_{2}\in(0,1),\sigma_{3},\sigma_{4}>0\). Moreover,

$$\begin{aligned} &G_{1\alpha}(t,s)=G_{2\alpha}(t,s)+G_{3\alpha}(t,s),\qquad G_{1\beta }(t,s)=G_{2\beta}(t,s)+G_{3\beta}(t,s), \\ &G_{2\alpha}(t,s)= \textstyle\begin{cases} \frac{t^{\frac{1}{2}}(1-s)^{\frac{1}{2}}-(t-s)^{\frac{1}{2}}}{\Gamma(\frac{3}{2})},& s\le t,\\ \frac{t^{\frac{1}{2}}(1-s)^{\frac{1}{2}}}{\Gamma(\frac{3}{2})},& t\le s,\end{cases}\displaystyle \\ &G_{2\beta}(t,s)= \textstyle\begin{cases} \frac{t^{\frac{1}{3}}(1-s)^{\frac{1}{3}}-(t-s)^{\frac{1}{3}}}{\Gamma(\frac{4}{3})},& s\le t,\\ \frac{t^{\frac{1}{3}}(1-s)^{\frac{1}{3}}}{\Gamma(\frac{4}{3})},& t\le s,\end{cases}\displaystyle \\ &G_{3\alpha}(t,s)= \frac{t^{\frac{1}{2}}}{1-\sigma_{1}} \int_{0}^{1} t^{2} G_{2\alpha}(t,s)\,dt \\ &\phantom{G_{3\alpha}(t,s)}= \frac{t^{\frac{1}{2}}}{\frac{5}{7} \Gamma(\frac{3}{2})} \biggl\{ \int_{0}^{s} t^{2}\cdot t^{\frac{1}{2}}(1-s)^{\frac{1}{2}}\,dt+ \int_{s}^{1} t^{2} \cdot \bigl[t^{\frac{1}{2}}(1-s)^{\frac{1}{2}}-(t-s)^{\frac{1}{2}} \bigr]\,dt \biggr\} \\ &\phantom{G_{3\alpha}(t,s)}= \frac{2t^{\frac{1}{2}}}{75\Gamma(\frac{3}{2})} \bigl[15(1-s)^{\frac{1}{2}}-35(1-s)^{\frac{3}{2}}+28(1-s)^{\frac{5}{2}}-8(1-s)^{\frac{7}{2}} \bigr] \end{aligned}$$

and

$$\begin{aligned} G_{3\beta}(t,s)&= \frac{t^{\frac{1}{3}}}{1-\sigma_{2}} \int_{0}^{1} tG_{2\beta }(t,s)\,dt \\ &= \frac{t^{\frac{1}{3}}}{\frac{4}{7} \Gamma(\frac{4}{3})} \biggl\{ \int_{0}^{s} t\cdot t^{\frac{1}{3}}(1-s)^{\frac{1}{3}}\,dt+ \int_{s}^{1} t \bigl[t^{\frac{1}{3} } (1-s)^{\frac{1}{3}}-(t-s)^{\frac{1}{3}} \bigr]\,dt \biggr\} \\ &= \frac{3t^{\frac{1}{3}}}{16 \Gamma(\frac{4}{3})} \bigl[4(1-s)^{\frac{1}{3}}-7(1-s)^{\frac{4}{3}}+3(1-s)^{\frac{7}{3}} \bigr]. \end{aligned}$$

Further,

$$\begin{aligned} &e_{1}(t)= \int^{1}_{0} G_{1\alpha}(t,s)\,ds= \frac{1}{\Gamma(\frac{3}{2})} \biggl(\frac{98}{135}t^{\frac{1}{2}}-\frac{2}{3}t^{\frac{3}{2}} \biggr),\\ & e_{2}(t)= \int^{1}_{0} G_{1\beta}(t,s)\,ds= \frac{1}{\Gamma(\frac{4}{3})} \biggl(\frac{147}{160}t^{\frac{1}{3}}-\frac{3}{4}t^{\frac{4}{3}} \biggr), \\ &e_{1}^{*}=\max\bigl\{ e_{1}(t):t\in[0,1]\bigr\} = \frac{1372}{1215\sqrt{15}\Gamma(\frac{3}{2})},\\ & e_{2}^{*}=\max\bigl\{ e_{2}(t):t\in[0,1] \bigr\} =\frac{3087}{1280\sqrt[3]{140}\Gamma(\frac {4}{3})}. \end{aligned}$$

Take \(h_{1}(t)=M_{1}t^{\frac{1}{2}}\), \(h_{2}(t)=M_{2}t^{\frac{1}{3}}\), where

$$\begin{aligned} &M_{1}\geq\frac{a}{\Gamma(\alpha+1)(1-\sigma_{1})}=\frac{7}{5\Gamma(\frac {5}{2})},\\ & M_{2}\ge \frac{b}{\Gamma(\beta+1)(1-\sigma_{2})}=\frac{7}{4\Gamma (\frac{7}{3})}. \end{aligned}$$

Then

$$\begin{aligned} e_{1}(t)&=\frac{t^{\frac{1}{2}}}{\Gamma(\frac{3}{2})} \biggl(\frac {98}{135}- \frac{2}{3}t \biggr) \le\frac{98t^{\frac{1}{2}}}{135\Gamma(\frac{3}{2})} < \frac{49}{45\Gamma(\frac{5}{2})}t^{\frac{1}{2}}< \frac{7}{5\Gamma(\frac {5}{2})}t^{\frac{1}{2}} \le M_{1}t^{\frac{1}{2}}=h_{1}(t), \\ e_{2}(t)&=\frac{t^{\frac{1}{3}}}{\Gamma(\frac{4}{3})} \biggl(\frac {147}{160}- \frac{3}{4}t \biggr) \le\frac{147t^{\frac{1}{3}}}{160\Gamma(\frac{4}{3})} < \frac{49}{40\Gamma(\frac{4}{3})}t^{\frac{1}{3}}< \frac{7}{4\Gamma(\frac {7}{3})}t^{\frac{1}{3}} \le M_{2}t^{\frac{1}{3}}=h_{2}(t). \end{aligned}$$

\(g(t,0)=(\frac{1}{\Gamma(\frac{3}{2})})^{\frac{1}{5}} (\frac{98}{135}-\frac{2}{3}t)^{\frac{1}{5}}t^{\frac{1}{10}}\geq0\) with \(g(t,0)\not\equiv0\), \(f(t,0)=(\frac{1}{\Gamma(\frac{4}{3})})^{\frac{1}{5}} (\frac{147}{160}-\frac{3}{4}t)^{\frac{1}{5}}t^{\frac{1}{15}}\geq0\) with \(f(t,0)\not\equiv0\). In addition,

$$\begin{aligned} f(t,x)&= \biggl(\frac{1280\sqrt[3]{140}}{3087}x+ \frac{1}{\Gamma(\frac{4}{3})} \biggr)^{\frac{1}{5}} \biggl(\frac {147}{160}-\frac{3}{4}t \biggr)^{\frac{1}{5}}t^{\frac{1}{15}} \\ &= \biggl(\frac{1}{\Gamma(\frac{4}{3})} \biggr)^{\frac{1}{5}} \biggl( \frac {1280\sqrt[3]{140} \Gamma(\frac{4}{3})}{3087}x+1 \biggr)^{\frac{1}{5}} \biggl(\frac {147}{160}t^{\frac{1}{3}}- \frac{3}{4}t^{\frac{4}{3}} \biggr)^{\frac {1}{5}} \\ &= \biggl(\frac{1280\sqrt[3]{140} \Gamma(\frac{4}{3})}{3087}x+1 \biggr)^{\frac{1}{5}}\bigl[e_{2}(t) \bigr]^{\frac {1}{5}} \\ &= \biggl(\frac{1280\sqrt[3]{140} \Gamma(\frac{4}{3})}{3087}xe_{2}(t)+e_{2}(t) \biggr)^{\frac{1}{5}}, \\ g(t,x)&= \biggl(\frac{1}{\Gamma(\frac{3}{2})} \biggr)^{\frac{1}{5}} \biggl( \frac{1215\sqrt{15} \Gamma(\frac{3}{2})}{1372}x+1 \biggr)^{\frac{1}{5}} \biggl(\frac {98}{135}t^{\frac{1}{2}}- \frac{2}{3}t^{\frac{3}{2}} \biggr)^{\frac {1}{5}} \\ &= \biggl(\frac{1215\sqrt{15} \Gamma(\frac{3}{2})}{1372}x+1 \biggr)^{\frac{1}{5}}\bigl[e_{1}(t) \bigr]^{\frac {1}{5}} \\ &= \biggl(\frac{1215\sqrt{15}\Gamma(\frac {3}{2})}{1372}xe_{1}(t)+e_{1}(t) \biggr)^{\frac{1}{5}}. \end{aligned}$$

For \(\lambda\in(0,1)\), \(x\in(-\infty,+\infty)\), \(y\in[0,e_{2}^{*}]\),

$$\begin{aligned} &f\bigl(t,\lambda x+(\lambda-1)y\bigr)\\ &\quad = \biggl\{ \frac{1280\sqrt[3]{140} \Gamma(\frac{4}{3})}{3087}e_{2}(t) \bigl[\lambda x+(\lambda-1)y\bigr]+e_{2}(t) \biggr\} ^{\frac{1}{5}} \\ &\quad=\lambda^{\frac{1}{5}} \biggl\{ \frac{1280\sqrt[3]{140} \Gamma(\frac{4}{3})}{3087}e_{2}(t)\biggl[x+ \biggl(1-\frac{1}{\lambda}\biggr)y\biggr]+\frac {1}{\lambda}e_{2}(t) \biggr\} ^{\frac{1}{5}} \\ &\quad=\lambda^{\frac{1}{5}} \biggl\{ \frac{1280\sqrt[3]{140} \Gamma(\frac{4}{3})}{3087}e_{2}(t)x+\biggl(1- \frac{1}{\lambda}\biggr)\frac{1280\sqrt[3]{140} \Gamma(\frac{4}{3})}{3087}e_{2}(t)y+ \frac{1}{\lambda}e_{2}(t) \biggr\} ^{\frac {1}{5}} \\ &\quad\geq\lambda^{\frac{1}{5}} \biggl\{ \frac{1280\sqrt[3]{140} \Gamma(\frac{4}{3})}{3087}e_{2}(t)x+ \biggl(1-\frac{1}{\lambda}\biggr)e_{2}(t)+\frac {1}{\lambda}e_{2}(t) \biggr\} ^{\frac{1}{5}} \\ &\quad=\lambda^{\frac{1}{5}} \biggl\{ \frac{1280\sqrt[3]{140} \Gamma(\frac{4}{3})}{3087}e_{2}(t)x+e_{2}(t) \biggr\} ^{\frac{1}{5}} \\ &\quad=\lambda^{\frac{1}{5}}f(t,x)=\varphi(\lambda)f(t,x), \end{aligned}$$

here \(\varphi(\lambda)=\lambda^{\frac{1}{5}}\). By Theorem 3.1, system (3.2) has a unique nontrivial solution \((u^{*},v^{*})\) in \(\overline {P}_{h,e}\), where

$$\begin{aligned} &e(t)=\bigl(e_{1}(t),e_{2}(t)\bigr)= \biggl( \frac{1}{\Gamma(\frac{3}{2})} \biggl(\frac {98}{135}t^{\frac{1}{2}}-\frac{2}{3}t^{\frac{3}{2}} \biggr), \frac{1}{\Gamma(\frac{4}{3})} \biggl(\frac{147}{160}t^{\frac{1}{3}}- \frac {3}{4}t^{\frac{4}{3}} \biggr) \biggr), \\ &h(t)=\bigl(h_{1}(t),h_{2}(t)\bigr)=\bigl(M_{1}t^{\frac{1}{2}}, M_{2}t^{\frac{1}{3}}\bigr), \quad t\in[0,1]. \end{aligned}$$

Taking any point \((u_{0},v_{0})\in\overline{P}_{h,e}\), construct the following sequences:

$$\begin{aligned} &u_{n+1}(t)= \int_{0}^{1} G_{1\alpha}(t,s)f \bigl(s,v_{n}(s)\bigr)\,ds-\frac{1}{\Gamma(\frac {3}{2})} \biggl( \frac{98}{135}t^{\frac{1}{2}}-\frac{2}{3}t^{\frac {3}{2}} \biggr), \\ &v_{n+1}(t)= \int_{0}^{1} G_{1\beta}(t,s)g \bigl(s,u_{n}(s)\bigr)\,ds-\frac{1}{\Gamma(\frac {4}{3})} \biggl( \frac{147}{160}t^{\frac{1}{3}}-\frac{3}{4}t^{\frac {4}{3}} \biggr), \end{aligned}$$

\(n=0,1,2,\ldots\) , we have \(u_{n+1}(t)\rightarrow u^{*}(t),v_{n+1}(t)\rightarrow v^{*}(t)\) as \(n\rightarrow\infty\).

4 Conclusions

Recently, fractional coupled systems of differential equations have gained more attention in different fields of science and engineering such as physics, control systems and dynamical systems. So, for nonlinear coupled systems subject to different boundary conditions, there are many articles studying the existence or multiplicity of solutions or positive solutions. But the unique results are very rare. In this paper, we study the new coupled system of fractional differential equations (1.1). Our method is a new fixed point theorem of increasing φ-\((h,e)\)-concave operators. We present the existence and uniqueness of solutions for (1.1) dependent on two constants. Our result shows that the unique solution exists in a product set \(\overline{P}_{h,e}=P_{h_{1},e_{1}}\times P_{h_{2},e_{2}}\) and can be approximated by making an iterative sequence for any initial point in \(\overline{P}_{h,e}\). Finally, an interesting example is presented to demonstrate the main result.