1 Introduction

Consider the following fractional differential system with the nonlocal coupled integral boundary conditions:

$$ \textstyle\begin{cases} -D^{\beta }(\phi _{p}(-D^{\alpha }x(t)))= f(t,x(t),y(t),D^{\alpha }x(t),D^{ \alpha }y(t)), \quad t\in (0,1], \\ -D^{\beta }(\phi _{p}(-D^{\alpha }y(t)))= g(t,y(t),x(t),D^{\alpha }y(t)),D^{ \alpha }x(t)), \quad t\in (0,1], \\ D^{\alpha }x(0)=0, \\ D^{\beta -1}(\phi _{p}(-D^{\alpha }x(1)))=I^{\sigma }h(\eta ,\phi _{p}(-D^{ \alpha }x(\eta )))+a_{1}\\ \hphantom{D^{\beta -1}(\phi _{p}(-D^{\alpha }x(1)))}=\frac{1}{\Gamma (\sigma )} \int _{0}^{\eta }( \eta -s)^{\sigma -1}h(s,\phi _{p}(-D^{\alpha }x(s)))\,ds+a_{1}, \\ x(0)=0, \qquad D^{\alpha -1}x(1)=I^{\omega }x(\xi )+d_{1}= \frac{1}{\Gamma (\omega )}\int _{0}^{\xi }(\xi -s)^{\omega -1}x(s)\,ds+d_{1}, \\ D^{\alpha }y(0)=0, \\ D^{\beta -1}(\phi _{p}(-D^{\alpha }y(1)))=I^{\sigma }k(\eta ,\phi _{p}(-D^{ \alpha }y(\eta )))+a_{2} \\ \hphantom{D^{\beta -1}(\phi _{p}(-D^{\alpha }y(1)))}=\frac{1}{\Gamma (\sigma )} \int _{0}^{\eta }( \eta -s)^{\sigma -1}k(s,\phi _{p}(-D^{\alpha }y(s)))\,ds+a_{2}, \\ y(0)=0, \qquad D^{\alpha -1}y(1)=I^{\omega }y(\xi )+d_{2}= \frac{1}{\Gamma (\omega )}\int _{0}^{\xi }(\xi -s)^{\omega -1}y(s)\,ds+d_{2}, \end{cases} $$
(1.1)

where \(D^{\alpha }\) and \(D^{\beta }\) are the standard Riemann–Liouville fractional derivatives, \(I^{\sigma }\) and \(I^{\omega }\) are the Riemann–Liouville fractional integrals, and \(1<\alpha , \beta <2\), \(\sigma ,\omega >1\), \(0<\eta ,\xi <1\), \(a_{1},a_{2},d_{1},d_{2}\in \mathbb{R}\), \(a_{2}\geq a_{1}\), \(d_{2}\geq d_{1}\), \(f,g\in C([0,1]\times \mathbb{R}\times \mathbb{R},\mathbb{R})\), \(h,k\in C([0,1]\times \mathbb{R},\mathbb{R})\). The p-Laplacian operator is defined as \(\phi _{p}(t)=|t|^{p-2}t\), \(p>1\), and \((\phi _{p})^{-1}=\phi _{q}\), \(\frac{1}{p}+\frac{1}{q}=1\).

Fractional differential equations have recently gained much attention. In particular, much effort has been made toward the study of the existence of solutions for fractional differential equations with p-Laplacian operator [18]. The monotone iterative technique, combined with the method of upper and lower solutions, is a powerful tool for proving the existence of solutions of nonlinear differential equations; see [9, 10] and the references therein. However, only a few papers considered the upper an lower solutions method and the monotone iteration technique for p-Laplacian boundary value problems with fractional coupled systems. The purpose of this paper is developing a monotone iterative technique to show the existence of an extremal solution for the nonlinear system (1.1) with nonlocal integral boundary conditions.

The paper is organized as follows. In Sect. 2, we give sufficient conditions guaranteeing that (1.1) has an extremal solution and discuss some comparison results, which play a key role in establishing the proposed work. In Sect. 3, we give the main result. Finally, we present an example illustrating our results.

2 Preliminaries

In this section, we introduce definitions and some useful lemmas which play an important role in obtaining the main results of this paper.

Denote

$$ C_{\alpha }[0,1]=\bigl\{ u :u \in C[0,1], D^{\alpha }u(t)\in C[0,1]\bigr\} . $$

It is a Banach spaces with the norm \(\|u\|_{\alpha }=\|u\|+\|D^{\alpha }u\|\), where \(\|u\|=\max_{0\leq t\leq 1}|u(t)|\) and \(\|D^{\alpha }u\|=\max_{0\leq t\leq 1}|D^{\alpha }u(t)|\).

We need the following assumptions.

\((H_{1})\) There exist \(x_{0},y_{0}\in C_{\alpha }[0,1]\) satisfying \(D^{\beta }(\phi _{p}(-D^{\alpha }x_{0}(t))), D^{\beta }(\phi _{p}(-D^{ \alpha }y_{0}(t)))\in C[0,1]\), \(x_{0}(t)\leq y_{0}(t)\), and \(D^{\alpha }y_{0}(t)\leq D^{\alpha }x_{0}(t)\) such that

$$ \textstyle\begin{cases} -D^{\beta }(\phi _{p}(-D^{\alpha }x_{0}(t)))\leq f(t,x_{0}(t),y_{0}(t),D^{ \alpha }x_{0}(t),D^{\alpha }y_{0}(t)), \quad t\in (0,1], \\ D^{\alpha }x_{0}(0)=0, \qquad D^{\beta -1}(\phi _{p}(-D^{\alpha }x_{0}(1))) \leq I^{\sigma }h(\eta ,\phi _{p}(-D^{\alpha }x_{0}(\eta )))+a_{1}, \\ x_{0}(0)=0, \qquad D^{\alpha -1}x_{0}(1)\leq I^{\omega }x_{0}(\xi )+d_{1}, \\ -D^{\beta }(\phi _{p}(-D^{\alpha }y_{0}(t)))\geq g(t,y_{0}(t),x_{0}(t),D^{ \alpha }y_{0}(t)),D^{\alpha }x_{0}(t)), \quad t\in (0,1], \\ D^{\alpha }y_{0}(0)=0, \qquad D^{\beta -1}(\phi _{p}(-D^{\alpha }y_{0}(1))) \geq I^{\sigma }k(\eta ,\phi _{p}(-D^{\alpha }y_{0}(\eta )))+a_{2}, \\ y_{0}(0)=0, \qquad D^{\alpha -1}y_{0}(1)\geq I^{\omega }y_{0}(\xi )+d_{2}. \end{cases} $$

\((H_{2})\) There exist two constants \(M, N\in \mathbb{R}\), \(M\geq N\), such that

$$\begin{aligned} &f\bigl(t,\overline{x(t)},\overline{y(t)},D^{\alpha }\overline{x(t)},D^{ \alpha } \overline{y(t)}\bigr) -f\bigl(t,x(t),y(t),D^{\alpha }x(t),D^{\alpha }y(t) \bigr) \\ &\quad \leq M\bigl[\phi _{p}\bigl(-D^{\alpha }x(t)\bigr)-\phi _{p}\bigl(-D^{\alpha }\overline{x(t)}\bigr)\bigr]+ N\bigl[\phi _{p}\bigl(-D^{\alpha }y(t)\bigr)-\phi _{p} \bigl(-D^{\alpha }\overline{y(t)}\bigr)\bigr] \\ &g\bigl(t,\overline{x(t)},\overline{y(t)},D^{\alpha }\overline{x(t)},D^{ \alpha } \overline{y(t)}\bigr) -g\bigl(t,x(t),y(t),D^{\alpha }x(t),D^{\alpha }y(t) \bigr) \\ & \quad \leq M\bigl[\phi _{p}\bigl(-D^{\alpha }x(t)\bigr)-\phi _{p}\bigl(-D^{\alpha }\overline{x(t)}\bigr)\bigr]+ N\bigl[\phi _{p}\bigl(-D^{\alpha }y(t)\bigr)-\phi _{p} \bigl(-D^{\alpha }\overline{y(t)}\bigr)\bigr], \end{aligned}$$

where \(x_{0}(t)\leq \overline{x(t)}\leq x(t)\leq y_{0}(t)\), \(x_{0}(t)\leq y(t)\leq \overline{y(t)}\leq y_{0}(t)\), and

$$\begin{aligned} &f\bigl(t,x(t),y(t),D^{\alpha }x(t),D^{\alpha }y(t)\bigr)-g \bigl(t,y(t),x(t),D^{ \alpha }y(t),D^{\alpha }x(t)\bigr) \\ &\quad \leq M\bigl[\phi _{p}\bigl(-D^{\alpha }y(t)\bigr)-\phi _{p}\bigl(-D^{\alpha }x(t)\bigr)\bigr]+ N\bigl[ \phi _{p}\bigl(-D^{\alpha }x(t)\bigr)-\phi _{p} \bigl(-D^{\alpha }y(t)\bigr)\bigr] \end{aligned}$$

for \(x_{0}(t)\leq x(t)\leq y(t)\leq y_{0}(t)\).

\((H_{3})\) There exists a constant \(\lambda \geq 0\) such that

$$\begin{aligned}& h\bigl(t,\phi _{p}\bigl(-D^{\alpha }y(t)\bigr)\bigr)-h\bigl(t, \phi _{p}\bigl(-D^{\alpha }x(t)\bigr)\bigr)\geq \lambda \bigl[\phi _{p}\bigl(-D^{\alpha }y(t)\bigr)-\phi _{p} \bigl(-D^{\alpha }x(t)\bigr)\bigr], \\& k\bigl(t,\phi _{p}\bigl(-D^{\alpha }y(t)\bigr)\bigr)-k\bigl(t, \phi _{p}\bigl(-D^{\alpha }x(t)\bigr)\bigr)\geq \lambda \bigl[\phi _{p}\bigl(-D^{\alpha }y(t)\bigr)-\phi _{p} \bigl(-D^{\alpha }x(t)\bigr)\bigr], \end{aligned}$$

where \(x_{0}(t)\leq x(t)\leq y(t) \leq y_{0}(t)\), \(D^{\alpha } y_{0}(t)\leq D^{ \alpha }y(t)\leq D^{\alpha } x(t)\leq D^{\alpha } x_{0}(t)\), \(t\in [0,1]\), and

$$ k\bigl(t,\phi _{p}\bigl(-D^{\alpha }y(t)\bigr)\bigr)-h\bigl(t, \phi _{p}\bigl(-D^{\alpha }x(t)\bigr)\bigr)\geq \lambda \bigl[\phi _{p}\bigl(-D^{\alpha }y(t)\bigr)-\phi _{p} \bigl(-D^{\alpha }x(t)\bigr)\bigr] $$

for \(x_{0}(t)\leq x(t)\leq y(t) \leq y_{0}(t)\), \(D^{\alpha } y_{0}(t)\leq D^{ \alpha }y(t)\leq D^{\alpha } x(t)\leq D^{\alpha } x_{0}(t)\), \(t\in [0,1]\).

\((H_{4})\) \(\Gamma (\beta +\sigma )>\lambda \eta ^{\beta +\sigma -1}\).

\((H_{5})\) \(2\Gamma (\beta +\sigma ) (M+N)<\Gamma (\beta )[\Gamma (\beta + \sigma )-\lambda \eta ^{\beta +\sigma -1}]\).

\((H_{6})\) For any \(t\in (0,1)\), we have

$$ \Gamma (2-\beta )\lambda \eta ^{\sigma }< \Gamma (\sigma ). $$

Lemma 2.1

([11])

Let \(h\in C[0,1]\), \(b\in \mathbb{R}\), and \(\Gamma (\beta +\sigma )\neq \lambda \eta ^{\beta +\sigma -1} \). Then the fractional boundary value problem

$$ \textstyle\begin{cases} -D^{\beta }w(t)= h(t), \quad t\in [0,1], \\ w(0)=0, \\ D^{\beta -1}w(1)=\lambda I^{\sigma }w(\eta )+b= \frac{\lambda }{\Gamma (\sigma )}\int _{0}^{\eta }(\eta -s)^{\sigma -1}w(s)\,ds+b, \end{cases} $$
(2.1)

has the following integral representation of the solution:

$$ w(t)= \int _{0}^{1}G(t,s)h(s)\,ds+ \frac{b\Gamma (\beta +\sigma )t^{\beta -1}}{\Gamma (\beta )[\Gamma (\beta +\sigma )- \lambda \eta ^{\beta +\sigma -1}]}, $$

where

$$ G(t,s)=\frac{1}{\Delta } \textstyle\begin{cases} [\Gamma (\beta +\sigma )-\lambda (\eta -s)^{\beta +\sigma -1}]t^{ \beta -1} \\ \quad {}- [\Gamma (\beta +\sigma )-\lambda \eta ^{\beta +\sigma -1}](t-s)^{ \beta -1}, &s\leq t,s\leq \eta , \\ \Gamma (\beta +\sigma )t^{\beta -1}-\lambda (\eta -s)^{\beta +\sigma -1}t^{ \beta -1}, &t\leq s\leq \eta , \\ \Gamma (\beta +\sigma )[t^{\beta -1}-(t-s)^{\beta -1}]+\lambda \eta ^{ \beta +\sigma -1}(t-s)^{\beta -1}, &\eta \leq s\leq t, \\ \Gamma (\beta +\sigma )t^{\beta -1}, &s\geq t,s\geq \eta , \end{cases} $$

and \(\Delta =\Gamma (\beta )[\Gamma (\beta +\sigma )-\lambda \eta ^{ \beta +\sigma -1}]\).

Lemma 2.2

([11])

Let \(M, b\in \mathbb{R}\), \(h(t)\in C[0,1]\), \(2\Gamma (\beta +\sigma ) |M|<\Gamma (\beta )[\Gamma (\beta +\sigma )- \lambda \eta ^{\beta +\sigma -1}]\), and \((H_{4})\) hold. Then

$$ \textstyle\begin{cases} -D^{\beta }w(t)+Mw(t)= h(t), \quad t\in [0,1], \\ w(0)=0, \\ D^{\beta -1}w(1)=\lambda I^{\sigma }w(\eta )+b, \end{cases} $$

has a unique solution \(w(t)\in C[0,1]\).

Lemma 2.3

([10, Lemma 2.4])

Let \(z(t)\in C[0,1]\) and \(l\in \mathbb{R}\). Then the fractional value boundary problem

$$ \textstyle\begin{cases} -D^{\alpha }u(t)=z(t), \quad 0< t< 1, \\ u(0)=0, \qquad D^{\alpha -1}u(1)=l, \end{cases} $$
(2.2)

is equivalent to

$$ u(t)= \int _{0}^{1}H(t,s)z(s)\,ds+ \frac{lt^{\alpha -1}}{\Gamma (\alpha )}, $$

where

$$ H(t,s)= \textstyle\begin{cases} t^{\alpha -1}-(t-s)^{\alpha -1}, &0\leq s\leq t\leq 1, \\ t^{\alpha -1}, &0\leq t\leq s\leq 1. \end{cases} $$

Lemma 2.4

Assume that \(1<\alpha , \beta <2\), \(\sigma _{1}, \sigma _{2}\in C[0,1]\), M, N are nonnegative constants satisfying \(M\geq N\), and \((H_{4})\) and \((H_{5})\) hold. Then the fractional differential system

$$ \textstyle\begin{cases} -D^{\beta }(\phi _{p}(-D^{\alpha }x(t)))= \sigma _{1}(t)-M(\phi _{p}(-D^{ \alpha }x(t)))-N(\phi _{p}(-D^{\alpha }y(t))), \quad t\in (0,1], \\ -D^{\beta }(\phi _{p}(-D^{\alpha }y(t)))= \sigma _{2}(t)-M(\phi _{p}(-D^{ \alpha }y(t)))-N(\phi _{p}(-D^{\alpha }x(t))), \quad t\in (0,1], \\ D^{\alpha }x(0)=0,\qquad D^{\beta -1}(\phi _{p}(-D^{\alpha }x(1)))=\lambda I^{ \sigma }\phi _{p}(-D^{\alpha }x(\eta ))+b_{1}, \\ x(0)=0, \qquad D^{\alpha -1}x(1)=l_{1}, \\ D^{\alpha }y(0)=0,\qquad D^{\beta -1}(\phi _{p}(-D^{\alpha }y(1)))=\lambda I^{ \sigma }\phi _{p}(-D^{\alpha }y(\eta ))+b_{2}, \\ y(0)=0, \qquad D^{\alpha -1}y(1)=l_{2}, \end{cases} $$
(2.3)

has a unique solution in \(C_{\alpha }[0,T]\times C_{\alpha }[0,T]\).

Proof

Let

$$ \phi _{p}\bigl(-D^{\alpha }x(t)\bigr)=\frac{u(t)+v(t)}{2} \quad \text{and} \quad \phi _{p}\bigl(-D^{ \alpha }y(t)\bigr)= \frac{u(t)-v(t)}{2},\quad \forall t\in [0,1]. $$

Using (2.3), we have that

$$ \textstyle\begin{cases} -D^{\beta }u(t)=\sigma _{1}(t)+\sigma _{2}(t)-(M+N)u(t), \\ u(0)=0, \\ D^{\beta -1}u(1)=\lambda I^{\sigma }u(\eta )+b_{1}+b_{2}, \end{cases} $$
(2.4)

and

$$ \textstyle\begin{cases} -D^{\beta }v(t)=\sigma _{1}(t)-\sigma _{2}(t)-(M-N)v(t), \\ u(0)=0, \\ D^{\beta -1}v(1)=\lambda I^{\sigma }v(\eta )+b_{1}-b_{2}. \end{cases} $$
(2.5)

Since M, N are nonnegative constants and \(M\geq N\), by assumption \((H_{5})\) we see that

$$ 2\Gamma (\beta +\sigma ) (M-N)\leq 2\Gamma (\beta +\sigma ) (M+N)< \Gamma (\beta )\bigl[\Gamma (\beta +\sigma )-\lambda \eta ^{\beta +\sigma -1}\bigr]. $$
(2.6)

By (2.6) and Lemma 2.2 we know that (2.4) and (2.5) have a unique solution. In consequence, \(\phi _{p}(-D^{\alpha }x(t))\) and \(\phi _{p}(-D^{\alpha }y(t)))\) are also unique, that is,

$$ \phi _{p}\bigl(-D^{\alpha }x(t)\bigr)=\omega _{1}(t) \in C[0,1], \qquad \phi _{p}\bigl(-D^{ \alpha }y(t)\bigr)=\omega _{2}(t)\in C[0,1]. $$

Then

$$ -D^{\alpha }x(t)=\phi _{q}\bigl(\omega _{1}(t)\bigr), \qquad -D^{\alpha }y(t))= \phi _{q}\bigl(\omega _{2}(t) \bigr). $$

In view of the boundary value condition (2.3), we obtain

$$ \textstyle\begin{cases} -D^{\alpha }x(t)=\phi _{q}(\omega _{1}(t)), \\ -D^{\alpha }y(t))=\phi _{q}(\omega _{2}(t)), \\ x(0)=0,\qquad D^{\alpha -1}x(1)=l_{1}, \\ y(0)=0,\qquad D^{\alpha -1}y(1)=l_{2}. \end{cases} $$
(2.7)

Let

$$ x(t)=\frac{p(t)+q(t)}{2} \quad \text{and} \quad y(t)=\frac{p(t)-q(t)}{2}. $$

Using (2.7), we have

$$ \textstyle\begin{cases} -D^{\alpha }p(t)=\phi _{q}(\omega _{1}(t))+\phi _{q}(\omega _{2}(t)), \\ p(0)=0, \\ D^{\alpha -1}p(1)=l_{1}+l_{2}, \end{cases} $$
(2.8)

and

$$ \textstyle\begin{cases} -D^{\alpha }q(t)=\phi _{q}(\omega _{1}(t))-\phi _{q}(\omega _{2}(t)), \\ q(0)=0, \\ D^{\alpha -1}q(1)=l_{1}-l_{2}, \end{cases} $$
(2.9)

By Lemma 2.3 we know that both (2.8) and (2.9) have a unique solution. In consequence, x and y are also unique. □

Lemma 2.5

([9, Lemma 2.6])

Let M be nonnegative constant, and let \((H_{6})\) hold. If \(w(t)\in C[0,1]\) satisfies \(D^{\beta }w(t)\in C[0,1]\) and

$$ \textstyle\begin{cases} -D^{\beta }w(t)\geq -Mw(t), \quad t\in [0,1], \\ w(0)=0, \\ D^{\beta -1}w(1)\geq \lambda I^{\sigma }w(\eta ), \end{cases} $$

then \(w(t)\geq 0\) for all \(t\in [0,1]\).

Lemma 2.6

([10, Lemma 2.7])

If \(x(t)\in C[0,1]\) satisfies

$$ \textstyle\begin{cases} -D^{\alpha }x(t)\geq 0, \quad 0< t< 1, \\ x(0)=0, \\ D^{\alpha -1}x(1)\geq 0, \end{cases} $$

then \(x(t)\geq 0\) for all \(t \in [0,1]\).

Lemma 2.7

Let M, N be nonnegative constants and \(M\geq N\). If \(u,v\in C[0,1]\) satisfy \(D^{\beta }u(t), D^{\beta }v(t)\in C[0,1]\), and

$$ \textstyle\begin{cases} -D^{\beta }u(t)\geq -Mu(t)+Nv(t), \quad t\in [0,T), \\ -D^{\beta }v(t)\geq -Mv(t)+Nu(t), \quad t\in [0,T), \\ u(0)=0,\qquad D^{\beta -1}u(1)\geq \lambda I^{\sigma }u(\eta ), \\ v(0)=0,\qquad D^{\beta -1}v(1)\geq \lambda I^{\sigma }v(\eta ), \end{cases} $$
(2.10)

then \(u(t)\geq 0\) and \(v(t)\geq 0\) for all \(t\in [0,1]\).

Proof

Let \(p(t)=u(t)+v(t)\), \(t\in [0,1]\). Then by (2.10) we have

$$ \textstyle\begin{cases} -D^{\beta }p(t)\geq -(M-N)p(t), \quad t\in [0,1], \\ p(0)=0, \\ D^{\beta -1}p(1)\geq \lambda I^{\sigma }p(\eta ). \end{cases} $$
(2.11)

Thus by (2.11) and Lemma 2.5 we have that

$$ p(t)\geq 0, \quad \forall t\in [0,1],\quad \text{i.e.},\quad u(t)+v(t)\geq 0, \quad \forall t\in [0,1]. $$
(2.12)

Next, we show that \(u(t)\geq 0\) and \(v(t)\geq 0\) for all \(t\in [0,1]\). Using (2.10) and (2.12), we find that

$$ \textstyle\begin{cases} -D^{\beta }u(t)\geq -(M+N)u(t), \quad t\in [0,1], \\ u(0)=0, \\ D^{\beta -1}u(1)\geq \lambda I^{\sigma }u(\eta ), \end{cases} $$
(2.13)

which, in view of (2.13) and Lemma 2.5, yield \(u(t)\geq 0\) for all \(t\in [0,1]\). In a similar way, we can show that \(v(t)\geq 0\) for all \(t\in [0,1]\). □

3 Main results

Theorem 3.1

Suppose that conditions \((H_{1})\)\((H_{6})\) hold. Then there is an extremal solution \((x^{*},y^{*})\in [x_{0},y_{0}]\times [x_{0},y_{0}]\) of the nonlinear problem (1.1). Moreover, there exist monotone iterative sequences \(\{x_{n}\},\{y_{n}\}\subset [x_{0},y_{0}]\) such that \(x_{n}\rightarrow x^{*}\), \(y_{n}\rightarrow y^{*}\) (\(n\rightarrow \infty \)) uniformly for \(t\in [0,1]\) and

$$ x_{0}\leq x_{1}\leq \cdots \leq x_{n}\leq \cdots \leq x^{*}\leq y^{*} \leq \cdots \leq y_{n}\leq \cdots \leq y_{1}\leq y_{0}. $$

Moreover, we have

$$ D^{\alpha } y_{0}\leq D^{\alpha } y_{1}\leq \cdots \leq D^{\alpha } y_{n} \leq \cdots \leq D^{\alpha } y^{*}\leq D^{\alpha } x^{*}\leq \cdots \leq D^{\alpha } x_{n}\leq \cdots \leq D^{\alpha } x_{1} \leq D^{\alpha } x_{0}, $$

where

$$ [x_{0},y_{0}]=\bigl\{ x\in C_{\alpha }[0,1]: x_{0}(t)\leq x(t)\leq y_{0}(t),t \in [0,1]\bigr\} . $$

Proof

For any \(x_{n-1},y_{n-1}\in C_{\alpha }[0,1]\), \(n\geq 1\), we define

$$ \textstyle\begin{cases} \sigma _{n}^{1}(t)=f(t,x_{n-1}(t),y_{n-1}(t),D^{\alpha }x_{n-1}(t),D^{ \alpha }y_{n-1}(t))\\ \hphantom{\sigma _{n}^{1}(t)=}{}+M\phi _{p}(-D^{\alpha }x_{n-1}(t)) +N\phi _{p}(-D^{ \alpha }y_{n-1}(t)), \\ \sigma _{n}^{2}(t)=g(t,y_{n-1}(t),x_{n-1}(t),D^{\alpha }y_{n-1}(t),D^{ \alpha }x_{n-1}(t))\\ \hphantom{\sigma _{n}^{2}(t)=}{}+M\phi _{p}(-D^{\alpha }y_{n-1}(t)) +N\phi _{p}(-D^{ \alpha }x_{n-1}(t)). \end{cases} $$

Consider (2.3) as follows:

$$ \textstyle\begin{cases} -D^{\beta }(\phi _{p}(-D^{\alpha }x_{n}(t)))= \sigma ^{1}_{n}(t)-M( \phi _{p}(-D^{\alpha }x_{n}(t)))-N(\phi _{p}(-D^{\alpha }y_{n}(t))), \\ -D^{\beta }(\phi _{p}(-D^{\alpha }y_{n}(t)))= \sigma ^{2}_{n}(t)-M( \phi _{p}(-D^{\alpha }y_{n}(t)))-N(\phi _{p}(-D^{\alpha }x_{n}(t))), \\ D^{\alpha }x_{n}(0)=0, \\ D^{\beta -1}(\phi _{p}(-D^{\alpha }x_{n}(1)))\\ \quad =I^{\sigma } \{h( \eta ,\phi _{p}(-D^{\alpha }x_{n-1}(\eta )))+\lambda [\phi _{p}(-D^{ \alpha }x_{n}(\eta )) -\phi _{p}(-D^{\alpha }x_{n-1}(\eta ))] \}+a_{1}, \\ x_{n}(0)=0, \qquad D^{\alpha -1}x_{n}(1)=I^{\omega }x_{n-1}(\xi )+d_{1}, \\ D^{\alpha }y_{n}(0)=0, \\ D^{\beta -1}(\phi _{p}(-D^{\alpha }y_{n}(1)))\\ \quad =I^{\sigma } \{k( \eta ,\phi _{p}(-D^{\alpha }y_{n-1}(\eta )))+\lambda [\phi _{p}(-D^{ \alpha }y_{n}(\eta )) -\phi _{p}(-D^{\alpha }y_{n-1}(\eta ))] \}+a_{2}, \\ y_{n}(0)=0, \qquad D^{\alpha -1}y_{n}(1)=I^{\omega }y_{n-1}(\xi )+d_{2}. \end{cases} $$
(3.1)

In view of Lemma 2.4, problem (3.1) has a unique solution in \(C_{\alpha }[0,1]\times C_{\alpha }[0,1]\).

Now we show that \(\{x_{n}(t)\}\) and \(\{y_{n}(t)\}\) satisfy the relations

$$ x_{n-1}\leq x_{n}\leq y_{n}\leq y_{n-1} \quad \text{and}\quad D^{\alpha }y_{n-1} \leq D^{\alpha }y_{n} \leq D^{\alpha }x_{n}\leq D^{\alpha }x_{n-1}, \quad n=1,2, \ldots . $$
(3.2)

Let \(u(t)=\phi _{p}(-D^{\alpha }x_{1}(t))-\phi _{p}(-D^{\alpha }x_{0}(t))\), \(v(t)= \phi _{p}(-D^{\alpha }y_{0}(t)) -\phi _{p}(-D^{\alpha }y_{1}(t))\). By condition (3.1) and \((H_{1})\) we have

$$\begin{aligned} -D^{\beta }u(t) =&-D^{\beta }\bigl(\phi _{p} \bigl(-D^{\alpha }x_{1}(t)\bigr)\bigr)+D^{\beta }\bigl( \phi _{p}\bigl(-D^{\alpha }x_{0}(t)\bigr)\bigr) \\ \geq &f\bigl(t,x_{0}(t),y_{0}(t),D^{\alpha }x_{0}(t),D^{\alpha }y_{0}(t) \bigr) +M \phi _{p}\bigl(-D^{\alpha }x_{0}(t)\bigr) +N \phi _{p}\bigl(-D^{\alpha }y_{0}(t)\bigr) \\ &{}-M\phi _{p}\bigl(-D^{\alpha }x_{1}(t)\bigr) -N\phi _{p}\bigl(D^{\alpha }y_{1}(t)\bigr)-f \bigl(t,x_{0}(t),y_{0}(t),D^{ \alpha }x_{0}(t),D^{\alpha }y_{0}(t) \bigr) \\ =&-Mu(t)+Nv(t). \end{aligned}$$

Also, \(u(0)=0\), and

$$\begin{aligned} D^{\beta -1}u(1) =&D^{\beta -1}\bigl(\phi _{p} \bigl(-D^{\alpha }x_{1}(1)\bigr)\bigr) -D^{ \beta -1}\bigl(\phi _{p}\bigl(-D^{\alpha }x_{0}(1)\bigr)\bigr) \\ \geq & I^{\sigma } \bigl\{ h\bigl(\eta ,\phi _{p} \bigl(-D^{\alpha }x_{0}(\eta )\bigr)\bigr)+ \lambda \bigl[\phi _{p}\bigl(-D^{\alpha }x_{1}(\eta )\bigr) -\phi _{p}\bigl(-D^{\alpha }x_{0}( \eta )\bigr)\bigr] \bigr\} +a_{1} \\ &{}-I^{\sigma }h\bigl(\eta ,\phi _{p}\bigl(-D^{\alpha }x_{0}( \eta )\bigr)\bigr)-a_{1} \\ =& \lambda I^{\sigma } u(\eta ). \end{aligned}$$

In a similar way, we can prove that

$$ -D^{\beta }v(t)\geq -Mv(t)+Nu(t), \qquad v(0)=0,\qquad D^{\beta -1}v(1)\geq \lambda I^{\sigma }v(\eta ). $$

So, from the above inequality we have

$$ \textstyle\begin{cases} -D^{\beta }u(t)\geq -Mu(t)+Nv(t), \\ -D^{\beta }v(t)\geq -Mv(t)+Nu(t), \\ u(0)=0,\qquad D^{\beta -1}u(1)\geq \lambda I^{\sigma }u(\eta ), \\ v(0)=0,\qquad D^{\beta -1}v(1)\geq \lambda I^{\sigma }v(\eta ). \end{cases} $$

Thus, in view of Lemma 2.7, we get \(\phi _{p}(-D^{\alpha }x_{1}(t))\geq \phi _{p}(-D^{\alpha }x_{0}(t))\), \(\phi _{p}(-D^{\alpha }y_{0}(t)) \geq [4] \phi _{p}(-D^{\alpha }y_{1}(t))\) for all \(t\in [0,1]\). Since \(\Phi _{p}(x)\) is nondecreasing, we have \(D^{\alpha }x_{1}(t)\leq D^{\alpha }x_{0}(t)\) and \(D^{\alpha }y_{0}(t)\leq D^{\alpha }y_{1}(t)\) for all \(t\in [0,1]\).

Let \(\epsilon (t)=x_{1}(t)-x_{0}(t)\), \(\theta (t)=y_{0}(t)-y_{1}(t)\). From (3.1) and \((H_{1})\) we have

$$ \textstyle\begin{cases} -D^{\alpha } \epsilon (t)\geq 0, \quad t\in (0,1], \\ \epsilon (0)=0, \\ D^{\alpha -1} \epsilon (1)\geq I^{\omega } x_{0}(\xi )+d_{1}- I^{ \omega } x_{0}(\xi )-d_{1}=0, \end{cases} $$
(3.3)

and

$$ \textstyle\begin{cases} -D^{\alpha } \theta (t)\geq 0, \quad t\in (0,1], \\ \theta (0)=0, \\ D^{\alpha -1} \theta (1)\geq 0. \end{cases} $$
(3.4)

By Lemma 2.6 we have \(x_{1}(t)\geq x_{0}(t)\) and \(y_{0}(t)\geq y_{1}(t)\) for all \(t\in [0,1]\).

Now we put \(w(t)=\phi _{p}(-D^{\alpha }y_{1}(t))-\phi _{p}(-D^{\alpha }x_{1}(t))\). Applying \((H_{2})\), \((H_{3})\), and (3.1), we obtain

$$\begin{aligned} -D^{\beta }w(t) =&g\bigl(t,y_{0}(t),x_{0}(t),D^{\alpha }y_{0}(t),D^{\alpha }x_{0}(t) \bigr)+M \phi _{p}\bigl(-D^{\alpha }y_{0}(t)\bigr) +N \phi _{p}\bigl(-D^{\alpha }x_{0}(t)\bigr) \\ &{}-M\phi _{p}\bigl(-D^{\alpha }y_{1}(t)\bigr)-N\phi _{p}\bigl(-D^{\alpha }x_{1}(t)\bigr)-f \bigl(t,x_{0}(t),y_{0}(t),D^{ \alpha }x_{0}(t),D^{\alpha }y_{0}(t) \bigr) \\ &{}-M\phi _{p}\bigl(-D^{\alpha }x_{0}(t)\bigr) -N\phi _{p}\bigl(-D^{\alpha }y_{0}(t)\bigr)+M \phi _{p}\bigl(-D^{\alpha }x_{1}(t)\bigr) +N\phi _{p}\bigl(-D^{\alpha }y_{1}(t)\bigr) \\ \geq &-M \bigl[\phi _{p}\bigl(-D^{\alpha }y_{0}(t) \bigr)-\phi _{p}\bigl(-D^{\alpha }x_{0}(t)\bigr) \bigr]- N \bigl[\phi _{p}\bigl(-D^{\alpha }x_{0}(t)\bigr)-\phi _{p}\bigl(-D^{\alpha }y_{0}(t)\bigr) \bigr] \\ &{}+M\phi _{p}\bigl(-D^{\alpha }y_{0}(t)\bigr)+N\phi _{p}\bigl(-D^{\alpha }x_{0}(t)\bigr)-M \phi _{p}\bigl(-D^{\alpha }y_{1}(t)\bigr)-N\phi _{p}\bigl(-D^{\alpha }x_{1}(t)\bigr) \\ &{}-M\phi _{p}\bigl(-D^{\alpha }x_{0}(t)\bigr) -N\phi _{p}\bigl(-D^{\alpha }y_{0}(t)\bigr)+M \phi _{p}\bigl(-D^{\alpha }x_{1}(t)\bigr) +N\phi _{p}\bigl(-D^{\alpha }y_{1}(t)\bigr) \\ =&-(M-N)w(t). \end{aligned}$$

Also, \(w(0)=\phi _{p}(-D^{\alpha }y_{1}(0))-\phi _{p}(-D^{\alpha }x_{1}(0))=0\), and

$$\begin{aligned} D^{\beta -1}w(1) =& I^{\sigma } \bigl\{ k\bigl(\eta ,\phi _{p} \bigl(-D^{\alpha }y_{0}( \eta )\bigr)\bigr)+\lambda \bigl[\phi _{p}\bigl(-D^{\alpha }y_{1}(\eta )\bigr) -\phi _{p}\bigl(-D^{ \alpha }y_{0}(\eta )\bigr)\bigr] \bigr\} +a_{2} \\ &{}-I^{\sigma } \bigl\{ h\bigl(\eta ,\phi _{p} \bigl(-D^{\alpha }x_{0}(\eta )\bigr)\bigr)+ \lambda \bigl[\phi _{p}\bigl(-D^{\alpha }x_{1}(\eta )\bigr) -\phi _{p}\bigl(-D^{\alpha }x_{0}( \eta )\bigr)\bigr] \bigr\} -a_{1} \\ \geq &I^{\sigma } \bigl\{ \lambda \bigl[\phi _{p} \bigl(-D^{\alpha }y_{0}(\eta )\bigr) - \phi _{p} \bigl(-D^{\alpha }x_{0}(\eta )\bigr)\bigr]+\lambda \bigl[\phi _{p}\bigl(-D^{\alpha }y_{1}( \eta )\bigr) -\phi _{p}\bigl(-D^{\alpha }y_{0}(\eta )\bigr)\bigr] \\ &{}-\lambda \bigl[\phi _{p}\bigl(-D^{\alpha }x_{1}( \eta )\bigr) -\phi _{p}\bigl(-D^{ \alpha }x_{0}(\eta ) \bigr)\bigr] \bigr\} +(a_{2}-a_{1}) \\ \geq &\lambda I^{\sigma }w(\eta ). \end{aligned}$$

In view of Lemma 2.5, we have that \(w(t)\geq 0\) for all \(t\in [0,1]\). Thus we have the relation \(\phi _{p}(-D^{\alpha }x_{1}(t))\leq \phi _{p}(-D^{\alpha }y_{1}(t))\), that is, \(D^{\alpha }x_{1}(t)\geq D^{\alpha }y_{1}(t)\), since \(\Phi _{p}(x)\) is nondecreasing. Therefore \(D^{\alpha }y_{0}(t)\leq D^{\alpha }y_{1}(t)\leq D^{\alpha }x_{1}(t) \leq D^{\alpha }x_{0}(t)\) for all \(t\in [0,1]\).

Let \(\delta (t)=y_{1}(t)-x_{1}(t)\). It follows from (3.1) that

$$ \textstyle\begin{cases} -D^{\alpha } \delta (t)=-D^{\alpha }y_{1}(t)+D^{\alpha }x_{1}(t)\geq 0, \\ \delta (0)= 0, \\ D^{\alpha -1}\delta (1)=I^{\omega }y_{0}(\xi )+d_{2}-I^{\omega }x_{0}( \xi )-d_{1}\geq 0. \end{cases} $$

By Lemma 2.6 we obtain \(y_{1}(t)\geq x_{1}(t)\) for all \(t\in [0,1]\). Hence we have the relation \(x_{0}(t)\leq x_{1}(t)\leq y_{1}(t)\leq y_{0}(t)\).

Now we assume that

$$ x_{{k-1}}\leq x_{{k}}\leq y_{{k}}\leq y_{{k-1}} \quad \text{and}\quad D^{ \alpha }y_{{k-1}}\leq D^{\alpha }y_{{k}} \leq D^{\alpha }x_{{k}}\leq D^{ \alpha }x_{{k-1}} \quad \text{for some }k\geq 1. $$

We will prove that (3.2) is also true for \(k+1\). Let

$$\begin{aligned} &u(t)=\phi _{p}\bigl(-D^{\alpha }x_{{k+1}}(t)\bigr)-\phi _{p}\bigl(-D^{\alpha }x_{{k}}(t)\bigr),\qquad v(t)=\phi _{p}\bigl(-D^{\alpha }y_{{k}}(t)\bigr)-\phi _{p}\bigl(-D^{\alpha }y_{{k+1}}(t)\bigr), \\ &w(t)=\phi _{p}\bigl(-D^{\alpha }y_{{k+1}}(t)\bigr)- \phi _{p}\bigl(-D^{\alpha }x_{{k+1}}(t)\bigr),\qquad \epsilon (t)=x_{{k+1}}(t)-x_{{k}}(t),\\ & \theta (t)=y_{{k}}(t)-y_{{k+1}}(t),\qquad \delta (t)=y_{{k+1}}(t)-x_{{k+1}}(t). \end{aligned}$$

By \((H_{2})\), \((H_{3})\), and (3.1) we have that

$$\begin{aligned}& \textstyle\begin{cases} -D^{\beta }u(t)\geq -Mu(t)+Nv(t), \\ -D^{\beta }v(t)\geq -Mv(t)+Nu(t), \\ u(0)=0,\qquad D^{\beta -1}u(1)\geq \lambda I^{\sigma }u(\eta ), \\ v(0)=0,\qquad D^{\beta -1}v(1)\geq \lambda I^{\sigma }v(\eta ), \end{cases}\displaystyle \\& \textstyle\begin{cases} -D^{\alpha } \epsilon (t)\geq 0, \\ \epsilon (0)=0, \\ D^{\alpha -1}\epsilon (1)\geq 0, \end{cases}\displaystyle \\& \textstyle\begin{cases} -D^{\alpha } \theta (t)\geq 0, \\ \theta (0)=0, \\ D^{\alpha -1}\theta (1)\geq 0, \end{cases}\displaystyle \end{aligned}$$

and

$$\begin{aligned}& \textstyle\begin{cases} -D^{\beta }w(t)\geq -(M-N)w(t), \\ w(0)=0, \\ D^{\beta -1}w(1)\geq \lambda I^{\sigma }w(\eta ), \end{cases}\displaystyle \\& \textstyle\begin{cases} -D^{\alpha } \delta (t)\geq 0, \\ \delta (0)=0, \\ D^{\alpha -1}\delta (1)\geq 0. \end{cases}\displaystyle \end{aligned}$$

In view of Lemmas 2.52.7, we obtain

$$ x_{{k}}\leq x_{{k+1}}\leq y_{{k+1}}\leq y_{{k}} \quad \text{and}\quad D^{ \alpha }y_{{k}}\leq D^{\alpha }y_{{k+1}} \leq D^{\alpha }x_{{k+1}} \leq D^{\alpha }x_{{k}}, \quad \forall t\in [0,1] . $$

From the above, by induction, it is not difficult to prove that \(x_{0}\leq x_{1}\leq \cdots \leq x_{n}\leq \cdots \leq y_{n}\leq \cdots \leq y_{1}\leq y_{0}\) and \(D^{\alpha } y_{0}\leq D^{\alpha } y_{1}\leq \cdots \leq D^{\alpha } y_{n} \leq \cdots \leq D^{\alpha } x_{n}\leq \cdots \leq D^{\alpha } x_{1} \leq D^{\alpha } x_{0}\).

Since the solution space is \(C_{\alpha }[0,1]\), the sequences \(\{x_{n}\}\) and \(\{y_{n}\}\) are uniformly bounded and equicontinuous. The Arzelà–Ascoli theorem guarantees that they are relatively compact sets in the space \(C_{\alpha }[0,1]\). Therefore \(\{x_{n}\}\) and \(\{y_{n}\}\) converge to \(x^{*}(t)\) and \(y^{*}(t)\) uniformly on \([0,1]\), respectively, that is,

$$ \lim_{n\rightarrow \infty }x_{n}(t)=x^{*}(t),\qquad \lim _{n \rightarrow \infty }y_{n}(t)=y^{*}(t), \quad \forall t \in [0,1], $$

and

$$ \lim_{n\rightarrow \infty }D^{\alpha }x_{n}(t)=D^{\alpha }x^{*}(t), \qquad \lim_{n\rightarrow \infty }D^{\alpha }y_{n}(t)=D^{\alpha }y^{*}(t), \quad \forall t\in [0,1], $$

uniformly in \(t\in [0,1]\). Moreover, from (3.1) and (3.2) we obtain that \(x^{*}(t)\) and \(y^{*}(t)\) are solutions of problem (1.1).

Finally, we show that \((x^{*},y^{*})\) is an extremal solution of system (1.1). Let \((x,y)\in [x_{0},y_{0}]\times [x_{0},y_{0}]\) be any solution of problem (1.1), that is,

$$ \textstyle\begin{cases} -D^{\beta }(\phi _{p}(-D^{\alpha }x(t)))= f(t,x(t),y(t),D^{\alpha }x(t),D^{ \alpha }y(t)), \quad t\in (0,1], \\ -D^{\beta }(\phi _{p}(-D^{\alpha }y(t)))= g(t,y(t),x(t),D^{\alpha }y(t)),D^{ \alpha }x(t)), \quad t\in (0,1], \\ D^{\alpha }x(0)=0, \\ D^{\beta -1}(\phi _{p}(-D^{\alpha }x(1)))=I^{\sigma }h(\eta ,\phi _{p}(-D^{ \alpha }x(\eta )))+a_{1}\\ \hphantom{D^{\beta -1}(\phi _{p}(-D^{\alpha }x(1)))}=\frac{1}{\Gamma (\sigma )} \int _{0}^{\eta }( \eta -s)^{\sigma -1}h(s,\phi _{p}(-D^{\alpha }x(s)))\,ds+a_{1}, \\ x(0)=0, \qquad D^{\alpha -1}x(1)=I^{\omega }x(\xi )+d_{1}= \frac{1}{\Gamma (\omega )}\int _{0}^{\xi }(\xi -s)^{\omega -1}x(s)\,ds+d_{1}, \\ D^{\alpha }y(0)=0, \\ D^{\beta -1}(\phi _{p}(-D^{\alpha }y(1)))=I^{\sigma }k(\eta ,\phi _{p}(-D^{ \alpha }y(\eta )))+a_{2} \\ \hphantom{D^{\beta -1}(\phi _{p}(-D^{\alpha }y(1)))}=\frac{1}{\Gamma (\sigma )} \int _{0}^{\eta }( \eta -s)^{\sigma -1}k(s,\phi _{p}(-D^{\alpha }y(s)))\,ds+a_{2}, \\ y(0)=0, \qquad D^{\alpha -1}y(1)=I^{\omega }y(\xi )+d_{2}= \frac{1}{\Gamma (\omega )}\int _{0}^{\xi }(\xi -s)^{\omega -1}y(s)\,ds+d_{2}. \end{cases} $$
(3.5)

Applying (3.1), (3.5), \((H_{2})\), \((H_{3})\), Lemma 2.6, and Lemma 2.7, we have

$$ x_{n}\leq x, \qquad y\leq y_{n}, \qquad D^{\alpha }x \leq D^{\alpha }x_{n}, \qquad D^{ \alpha }y_{n}\leq D^{\alpha }y, \quad n=1,2,\ldots . $$
(3.6)

Taking the limit as \(n\rightarrow \infty \) in (3.6), we have \(x^{*}\leq x\), \(y\leq y^{*}\), that is, \((x^{*}, y^{*})\) is an extremal solution of system (1.1) in \([x_{0},y_{0}]\times [x_{0},y_{0}]\). This completes the proof. □

4 Iteration procedure and a numerical example

In this section, we introduce a numerical procedure to obtain an appropriate solution of (1.1). Define

$$ E(n)= \bigl\Vert x_{n}(t)-y_{n}(t) \bigr\Vert _{1}= \int _{0}^{1} \bigl\vert x_{n}(t)-y_{n}(t) \bigr\vert \,dt. $$

For the iteration Eq. (3.1), let \(\phi _{p}(-D^{\alpha }x_{n}(t))=u_{n}\). Then \(-D^{\alpha }x_{n}(t)=\phi _{q}(u_{n})\), and with the boundary conditions \(x_{n}(0)=0\) and \(D^{\alpha -1}x_{n}(1)=D^{\frac{2}{3}}x_{n}(1)=I^{\omega }x_{n-1}(\xi )+d_{1}=l_{1}\), by Lemma 2.3 we have

$$ x_{n}(t)=\frac{l_{1}}{\Gamma (\alpha )}t^{\alpha -1}+ \int _{0}^{1}H(t,s) \phi _{q} \bigl(u_{n}(s)\bigr)\,ds, $$
(4.1)

where \(l_{1}=\frac{1}{\Gamma (\omega )}\int _{0}^{\xi }(\xi -s)^{\omega -1}x_{n-1}(s)\,ds+d_{1}\) and

$$ H(t,s)= \textstyle\begin{cases} t^{\alpha -1}-(t-s)^{\alpha -1}, &0\leq s\leq t\leq 1, \\ t^{\alpha -1}, &0\leq t\leq s\leq 1. \end{cases} $$

We can also put \(\phi _{p}(-D^{\alpha }y_{n}(t))=v_{n}\). Then \(-D^{\alpha }y_{n}(t)=\phi _{q}(v_{n})\). In a similar way, we can prove that

$$ y_{n}(t)=\frac{l_{2}}{\Gamma (\alpha )}t^{\alpha -1}+ \int _{0}^{1}H(t,s) \phi _{q} \bigl(v_{n}(s)\bigr)\,ds, $$
(4.2)

where \(l_{2}=\frac{1}{\Gamma (\omega )}\int _{0}^{\xi }(\xi -s)^{\omega -1}y_{n-1}(s)\,ds+d_{2}=1.1284 \int _{0}^{\frac{1}{2}}(\frac{1}{2}-s)^{\frac{1}{2}}y_{n-1}(s)\,ds+0.004\). Thus the iteration Eq. (3.1) can be rewritten as

$$ \textstyle\begin{cases} -D^{\beta }u_{n}=-Mu_{n}-Nv_{n}+f(t,x_{n-1},y_{n-1},-\phi _{q}(u_{n-1}),- \phi _{q}(v_{n-1}))+Mu_{n-1}+Nv_{n-1} , \\ -D^{\beta }v_{n}=-Mv_{n}-Nu_{n}+g(t,y_{n-1},x_{n-1},-\phi _{q}(v_{n-1}),- \phi _{q}(u_{n-1}))+Mv_{n-1}+Nu_{n-1} , \\ u_{n}(0)=0, \qquad D^{\beta -1}u_{n}(1)=\lambda I^{\sigma }u_{n}(\eta )+b_{1}, \\ v_{n}(0)=0, \qquad D^{\beta -1}v_{n}(1)=\lambda I^{\sigma }v_{n}(\eta )+b_{2}. \end{cases} $$
(4.3)

Applying Lemma 2.1 to (4.3), we obtain

$$ \textstyle\begin{cases} u_{n}(t)= \frac{b_{1}\Gamma (\beta +\sigma )}{\Gamma (\beta )[\Gamma (\beta +\sigma )- \lambda \eta ^{\beta +\sigma -1}]}t^{ \beta -1}\\ \hphantom{u_{n}(t)=}{}+\int _{0}^{1}G(t,s)[-Mu_{n}(s)-Nv_{n}(s) +f(s,x_{n-1}(s),y_{n-1}(s),- \phi _{q}(u_{n-1}(s)), \\ \hphantom{u_{n}(t)=}{}-\phi _{q}(v_{n-1}(s))) +Mu_{n-1}(s)+Nv_{n-1}(s)]\,ds, \\ v_{n}(t)= \frac{b_{2}\Gamma (\beta +\sigma )}{\Gamma (\beta )[\Gamma (\beta +\sigma )- \lambda \eta ^{\beta +\sigma -1}]}t^{ \beta -1}\\ \hphantom{v_{n}(t)=}{}+\int _{0}^{1}G(t,s)[-Mv_{n}(s)-Nu_{n}(s) +g(s,y_{n-1}(s),x_{n-1}(s),- \phi _{q}(v_{n-1}(s)), \\ \hphantom{v_{n}(t)=}{}-\phi _{q}(u_{n-1}(s)))+Mv_{n-1}(s)+Nu_{n-1}(s)]\,ds, \end{cases} $$
(4.4)

where \(b_{1}= I^{\sigma }h(\eta ,u_{n-1}(\eta )) -\lambda I^{\sigma }u_{n-1}( \eta )+a_{1}\), \(b_{2}= I^{\sigma }k(\eta ,v_{n-1}(\eta )) -\lambda I^{\sigma }v_{n-1}( \eta )+a_{2}\), and

$$ G(t,s)=\frac{1}{\Delta } \textstyle\begin{cases} [\Gamma (\beta +\sigma )-\lambda (\eta -s)^{\beta +\sigma -1}]t^{ \beta -1} \\ \quad {}- [\Gamma (\beta +\sigma )-\lambda \eta ^{\beta +\sigma -1}](t-s)^{ \beta -1}, &s\leq t,s\leq \eta , \\ \Gamma (\beta +\sigma )t^{\beta -1}-\lambda (\eta -s)^{\beta +\sigma -1}t^{ \beta -1}, &t\leq s\leq \eta , \\ \Gamma (\beta +\sigma )[t^{\beta -1}-(t-s)^{\beta -1}]+\lambda \eta ^{ \beta +\sigma -1}(t-s)^{\beta -1}, &\eta \leq s\leq t, \\ \Gamma (\beta +\sigma )t^{\beta -1}, &s\geq t,s\geq \eta , \end{cases} $$

\(\Delta =\Gamma (\beta )[\Gamma (\beta +\sigma )-\lambda \eta ^{ \beta +\sigma -1}]\).

Discretize the interval \([0,1]\) with the nodes \(t_{i}=ih\), \(h=\frac{1}{K}\), \(K\in \mathbb{N}\). Let \(x_{n}^{(i)}\approx x_{n}(t_{i})\), \(u_{n}^{(i)}\approx u_{n}(t_{i})\), \(H(i,j)=H(t_{i},s_{j})\), \(G(i,j)=G(t_{i},s_{j})\), and

$$ \textstyle\begin{cases} f_{n-1}^{(j)}=f(s_{j},x_{n-1}(s_{j}),y_{n-1}(s_{j}), -\phi _{q}(u_{n-1}(s_{j})), -\phi _{q}(v_{n-1}(s_{j})))+Mu_{n-1}(s_{j})+Nv_{n-1}(s_{j}), \\ g_{n-1}^{(j)}= g(s_{j},y_{n-1}(s_{j}),x_{n-1}(s_{j}),-\phi _{q}(v_{n-1}(s_{j})), -\phi _{q}(u_{n-1}(s_{j})))+Mv_{n-1}(s_{j})+Nu_{n-1}(s_{j}). \end{cases} $$

Using the trapezoidal quadrature rule to approximate the integrals in the right-hand sides of (4.4), (4.2), and (4.1), we obtain the following linear systems of equations:

$$ \textstyle\begin{cases} u_{n}^{(i)}= \frac{b_{1}\Gamma (\beta +\sigma )}{\Gamma (\beta )[\Gamma (\beta +\sigma )- \lambda \eta ^{\beta +\sigma -1}]}t_{i}^{ \beta -1} -\frac{h}{2}\sum_{j=0}^{K}MG(i,j)d_{j}u_{n}^{(j)}\\ \hphantom{u_{n}^{(i)}=}{}- \frac{h}{2}\sum_{j=0}^{K}NG(i,j)d_{j}v_{n}^{(j)} + \frac{h}{2}\sum_{j=0}^{K}G(i,j)d_{j}f_{n-1}^{(j)}, \\ v_{n}^{(i)}= \frac{b_{2}\Gamma (\beta +\sigma )}{\Gamma (\beta )[\Gamma (\beta +\sigma )- \lambda \eta ^{\beta +\sigma -1}]}t_{i}^{ \beta -1} -\frac{h}{2}\sum_{j=0}^{K}MG(i,j)d_{j}v_{n}^{(j)}\\ \hphantom{v_{n}^{(i)}=}{}- \frac{h}{2}\sum_{j=0}^{K}NG(i,j)d_{j}u_{n}^{(j)} + \frac{h}{2}\sum_{j=0}^{K}G(i,j)d_{j}g_{n-1}^{(j)}, \end{cases} $$
(4.5)

and

$$ \textstyle\begin{cases} x_{n}^{(i)}=\frac{l_{1}}{\Gamma (\alpha )}t_{i}^{\alpha -1} + \frac{h}{2}\sum_{j=0}^{K}H(i,j)d_{j}\phi _{q}(u_{n}^{(j)}), \\ y_{n}^{(i)}=\frac{l_{2}}{\Gamma (\alpha )}t_{i}^{\alpha -1} + \frac{h}{2}\sum_{j=0}^{K}H(i,j)d_{j}\phi _{q}(v_{n}^{(j)}), \end{cases} $$
(4.6)

for the unknown \(u_{n}^{(i)}\), \(x_{n}^{(i)}\), \(0\leq i\leq K\), where \(\{d_{j}\}\) are the coefficients in the rule, \(d_{0}=d_{K}=1\), and \(d_{j}=2\) for \(1\leq j\leq K-1\).

Setting \(G_{ij}=\frac{h}{2}\sum_{j=0}^{K}G(i,j)d_{j}\), \(H_{ij}=\frac{h}{2}\sum_{j=0}^{K}H(i,j)d_{j}\), the matrix \(\Phi =(G_{ij})\), and \(B=(H_{ij})\) with the identity matrix \(\mathbb{I}\). Systems (4.5) and (4.6) can be written as a system of matrix–vector equations

$$ \textstyle\begin{cases} (I+M\Phi )\overrightarrow{U}_{n}+N\Phi \overrightarrow{v}_{n}= \frac{b_{1}\Gamma (\beta +\sigma )}{\Gamma (\beta )[\Gamma (\beta +\sigma )- \lambda \eta ^{\beta +\sigma -1}]}S^{ \beta -1}+\overrightarrow{F}_{n-1}, \\ (I+M\Phi )\overrightarrow{V}_{n}+N\Phi \overrightarrow{u}_{n}= \frac{b_{2}\Gamma (\beta +\sigma )}{\Gamma (\beta )[\Gamma (\beta +\sigma )- \lambda \eta ^{\beta +\sigma -1}]}S^{ \beta -1}+\overrightarrow{G}_{n-1}, \\ \overrightarrow{X}_{n}=\frac{l_{1}}{\Gamma (\alpha )}S^{\alpha -1}+B \phi _{q}(\overrightarrow{U}_{n}), \\ \overrightarrow{Y}_{n}=\frac{l_{2}}{\Gamma (\alpha )}S^{\alpha -1}+B \phi _{q}(\overrightarrow{V}_{n}), \end{cases} $$
(4.7)

where \(\overrightarrow{X}_{n}=[{x}_{n}^{(0)},{x}_{n}^{(1)},\ldots ,{x}_{n}^{(K)}] \), \(\overrightarrow{Y}_{n}=[{y}_{n}^{(0)},{y}_{n}^{(1)},\ldots ,{y}_{n}^{(K)}]\), \(\overrightarrow{U}_{n}=[{u}_{n}^{(0)},{u}_{n}^{(1)},\ldots ,{u}_{n}^{(K)}]\), \(\overrightarrow{V}_{n}=[{v}_{n}^{(0)},{v}_{n}^{(1)},\ldots ,{v}_{n}^{(K)}]\), \(S=[t_{0},t_{1},\ldots ,t_{K}]^{T}\), and \(\overrightarrow{F}_{n-1}\), \(\overrightarrow{G}_{n-1}\) are column vectors of their components \(F_{n-1}^{(i)}=\frac{h}{2}\sum_{j=0}^{K}G(i,j)d_{j}f_{n-1}^{(j)}\), \(G_{n-1}^{(i)}=\frac{h}{2}\sum_{j=0}^{K}G(i,j)d_{j}g_{n-1}^{(j)}\).

Example 4.1

Consider the following problem:

$$ \textstyle\begin{cases} -D^{\frac{7}{4}}(\phi _{4}((-D^{\frac{5}{3}}x(t)))\\ \quad = \frac{1}{6}x^{ \frac{1}{3}}(t) [(-D^{\frac{5}{3}}x(t))^{\frac{1}{3}}-18 -t^{ \frac{2}{9}} ]-y(t) [(-D^{\frac{5}{3}}y(t))-2t^{\frac{2}{3}} ], \quad t\in (0,1], \\ -D^{\frac{7}{4}}(\phi _{4}(-D^{\frac{5}{3}}y(t)))\\ \quad = \frac{1}{6}y^{ \frac{1}{3}}(t) [(-D^{\frac{5}{3}}y(t))^{\frac{1}{3}}-18 -t^{ \frac{2}{9}} ]-x(t) [(-D^{\frac{5}{3}}x(t))-2t^{\frac{2}{3}} ], \quad t\in (0,1], \\ D^{\frac{5}{3}}x(0)=0, \\ D^{\frac{3}{4}}(\phi _{4}(-D^{\frac{5}{3}}x(1)))=I^{\frac{5}{4}}h( \frac{1}{4},\phi _{4}(-D^{\frac{5}{3}}x(\frac{1}{4})))+0.1 \\ \hphantom{D^{\frac{3}{4}}(\phi _{4}(-D^{\frac{5}{3}}x(1)))}= \frac{1}{\Gamma (\frac{5}{4})} \int _{0}^{\frac{1}{4}}(\frac{1}{4}-s)^{ \frac{1}{4}}(s+1)(\phi _{4}(-D^{\frac{5}{3}}x(s)))\,ds+0.1, \\ x(0)=0, \qquad D^{\frac{2}{3}}x(1)= \frac{1}{\Gamma (\frac{3}{2})}\int _{0}^{ \frac{1}{2}}(\frac{1}{2}-s)^{\frac{1}{2}}x(s)\,ds+0.3, \\ D^{\frac{5}{3}}y(0)=0, \\ D^{\frac{3}{4}}(\phi _{4}(-D^{\frac{5}{3}}y(1)))=I^{\frac{5}{4}}k( \frac{1}{4},\phi _{4}(-D^{\frac{5}{3}}y(\frac{1}{4})))+0.2 \\ \hphantom{D^{\frac{3}{4}}(\phi _{4}(-D^{\frac{5}{3}}y(1)))}= \frac{1}{\Gamma (\frac{5}{4})} \int _{0}^{\frac{1}{4}}(\frac{1}{4}-s)^{ \frac{1}{4}}(e^{s}+1)(\phi _{4}(-D^{\frac{5}{3}}y(s)))\,ds+0.2, \\ y(0)=0, \qquad D^{\frac{2}{3}}y(1)= \frac{1}{\Gamma (\frac{3}{2})}\int _{0}^{ \frac{1}{2}}(\frac{1}{2}-s)^{\frac{1}{2}}y(s)\,ds+0.4, \end{cases} $$
(4.8)

where \(\beta =\frac{7}{4}\), \(\alpha =\frac{5}{3}\), \(\sigma =\frac{5}{4}\), \(\omega =\frac{3}{2}\), \(\eta =\frac{1}{4} \), \(\xi =\frac{1}{2}\), \(a_{1}=0.1\), \(a_{2}=0.2\), \(d_{1}=0.003\), \(d_{2}=0.004\), \(p=4\), and

$$ \textstyle\begin{cases} f(t,x(t),y(t),D^{\frac{5}{3}}x(t),D^{\frac{5}{3}}y(t))\\ \quad =\frac{1}{6}x^{ \frac{1}{3}}(t) [(-D^{\frac{5}{3}}x(t))^{\frac{1}{3}}-18 -t^{ \frac{2}{9}} ]-y(t) [(-D^{\frac{5}{3}}y(t))-2t^{\frac{2}{3}} ], \\ g(t,y(t),x(t),D^{\frac{5}{3}}y(t),D^{\frac{5}{3}}x(t))\\ \quad =\frac{1}{6}y^{ \frac{1}{3}}(t) [(-D^{\frac{5}{3}}y(t))^{\frac{1}{3}}-18 -t^{ \frac{2}{9}} ]-x(t) [(-D^{\frac{5}{3}}x(t))-2t^{\frac{2}{3}} ], \\ h(t,\phi _{4}(-D^{\frac{5}{3}}x))=(t+1)(\phi _{4}(-D^{\frac{5}{3}}x)), \\ k(t,\phi _{4}(-D^{\frac{5}{3}}y))=(e^{t}+1)(\phi _{4}(-D^{\frac{5}{3}}y)). \end{cases} $$

Take \(x_{0}(t)=0\) and \(y_{0}(t)=3t^{\frac{2}{3}}- \frac{9\Gamma (\frac{2}{3})}{14\Gamma (\frac{1}{3})}t^{\frac{7}{3}}\). Then \(-1\leq -t^{\frac{2}{3}}=D^{\frac{5}{3}}y_{0}(t)\leq D^{\frac{5}{3}}x_{0}(t)=0\). It is not difficult to verify that \((H_{1})\) holds.

Since the function \(\sqrt[3]{x}+x^{3}\) is increasing for \(x\in R\), we obtain

$$\begin{aligned}& \begin{aligned}[b] &f\bigl(t,\overline{x(t)},\overline{y(t)},D^{\frac{5}{3}} \overline{x(t)},D^{ \frac{5}{3}}\overline{y(t)}\bigr)-f\bigl(t,x(t),y(t),D^{\frac{5}{3}}x(t),D^{ \frac{5}{3}}y(t) \bigr) \\ &\quad =\frac{1}{6}\overline{ x(t)}^{\frac{1}{3}} \bigl[ \bigl(-D^{\frac{5}{3}} \overline{x(t)}\bigr)^{\frac{1}{3}}-600t^{{\frac{1}{100}}} -t^{ \frac{1}{9}} \bigr]-\overline{y(t)} \bigl[\bigl(-D^{\frac{5}{3}} \overline{y(t)}\bigr)-2t^{ \frac{1}{3}} \bigr] \\ &\qquad {}-\frac{1}{6}x^{\frac{1}{3}}(t) \bigl[\bigl(-D^{\frac{5}{3}}x(t) \bigr)^{ \frac{1}{3}}-600t^{{\frac{1}{100}}} -t^{\frac{1}{9}} \bigr]+y(t) \bigl[ \bigl(-D^{\frac{5}{3}}y(t)\bigr)-2t^{\frac{1}{3}} \bigr], \\ &\quad \leq \frac{1}{6}x^{\frac{1}{3}}(t) \bigl[\bigl(-D^{\frac{5}{3}} \overline{x(t)}\bigr)^{ \frac{1}{3}} -\bigl(-D^{\frac{5}{3}}x(t) \bigr)^{\frac{1}{3}} \bigr] \\ &\quad \leq \frac{1}{6}\sqrt[3]{3} \bigl[\bigl(-D^{\frac{5}{3}}x(t) \bigr)^{3} -\bigl(-D^{ \frac{5}{3}}\overline{x(t)}\bigr)^{3} \bigr] \\ &\quad =\frac{1}{6}\sqrt[3]{3} \bigl[\Phi _{4} \bigl(-D^{\frac{5}{3}}x(t)\bigr) -\Phi _{4}\bigl(-D^{ \frac{5}{3}} \overline{x(t)}\bigr) \bigr], \end{aligned} \end{aligned}$$
(4.9)
$$\begin{aligned}& \begin{aligned}[b] &k\bigl(t,\phi _{4}\bigl(-D^{\frac{5}{3}}y\bigr) \bigr)-k\bigl(t,\phi _{4}\bigl(-D^{\frac{5}{3}}x\bigr)\bigr) \\ &\quad = \bigl(e^{t}+1\bigr) \bigl( \phi _{4}\bigl(-D^{\frac{5}{3}}y \bigr)\bigr)-\bigl(e^{t}+1\bigr) \bigl(\phi _{4} \bigl(-D^{\frac{5}{3}}x\bigr)\bigr) \\ &\quad =\bigl(e^{t}+1\bigr)\bigl[\phi _{4}\bigl(-D^{\frac{5}{3}}y \bigr)-\bigl(\phi _{4}\bigl(-D^{\frac{5}{3}}x\bigr)\bigr)\bigr] \\ &\quad \geq (t+1)\bigl[\phi _{4}\bigl(-D^{\frac{5}{3}}y\bigr)-\bigl(\phi _{4}\bigl(-D^{\frac{5}{3}}x\bigr)\bigr)\bigr] \\ &\quad \geq \phi _{4}\bigl(-D^{\frac{5}{3}}y\bigr)-\bigl(\phi _{4}\bigl(-D^{ \frac{5}{3}}x\bigr)\bigr), \end{aligned} \end{aligned}$$
(4.10)

where \(x_{0}(t)\leq \overline{x(t)}\leq x(t)\leq y_{0}(t)\), \(x_{0}(t)\leq y(t)\leq \overline{y(t)}\leq y_{0}(t)\), and \(x_{0}(t)\leq x(t)\leq y(t)\leq y_{0}(t)\). Thus \((H_{2})\) and \((H_{3})\) hold. From (4.9) and (4.10) we have \(M=\frac{1}{6}\sqrt[3]{3}\), \(N=0\), and \(\lambda =1\). Then

$$\begin{aligned} &\Gamma (\beta +\sigma )=\Gamma \biggl(\frac{7}{4}+\frac{5}{4} \biggr)=\Gamma (3)=2> \lambda \eta ^{\beta +\sigma -1}=1\cdot \biggl( \frac{1}{4}\biggr)^{2}=0.0625, \\ &2\Gamma (\beta +\sigma ) (M+N) \\ &\quad =2\cdot \Gamma (3)\cdot \frac{\sqrt[3]{3}}{6} \approx 0.9614 < \Gamma (\beta )\bigl[\Gamma (\beta + \sigma )-\lambda \eta ^{\beta +\sigma -1}\bigr] \\ &\quad= \Gamma \biggl(\frac{7}{4}\biggr)\biggl[ \Gamma (3)-1\cdot \biggl(\frac{1}{4}\biggr)^{2}\biggr] \approx 1.7808, \\ &\Gamma (2-\beta )\lambda \eta ^{\sigma } =\Gamma \biggl(\frac{1}{4} \biggr)\cdot 1 \cdot \biggl(\frac{1}{4}\biggr)^{\frac{5}{4}}\approx 0.6410 < \Gamma (\sigma )= \Gamma \biggl(\frac{5}{4}\biggr)\approx 0.9064, \end{aligned}$$

which show that \((H_{4})\), \((H_{5})\), and \((H_{6})\) hold. Thus all conditions of Theorem 3.1 are satisfied. In consequence, the nonlinear system (4.8) has an extremal solution \((x^{*},y^{*})\in [x_{0}(t),y_{0}(t)]\times [x_{0}(t),y_{0}(t)]\). Moreover, for this example, we found that for \(\delta =10^{-10}\), which took \(N=16\) iterations for \(E(N)<\delta \). The graphs of \({x_{n}}\) and \({y_{n}}\) for some values of n are shown in Table 1 and Fig. 1.

Figure 1
figure 1

Graphs of \(X_{n}\) and \(Y_{n}\)

Table 1 \(E(n)=3n+1\), \(n=0,1,2,3,4,5\)