1 Introduction

The work on dynamics of predator-prey systems is one of the dominant topics in mathematical ecology. Among the relationships between the species living in the same environment, the predator-prey theory plays important role. The spatial content of the environment has often been ignored in traditional predator-prey systems. These systems have been formulated and investigated to reveal the time evolution of uniform population distributions in their habitats. However, the spatial distribution of the species is usually inhomogeneous, and ecologists and mathematical ecologists employ the reaction-diffusion predator-prey systems to model the interaction and the tendency of movement between predator and prey which imply that the species diffuse to areas of smaller population concentration during the process of evolution, mainly due to resource limitation. In the past two decades, reaction-diffusion predator-prey systems have been extensively discussed [1–9], but what these models reveal is that the future state of the models is determined only by the present, that is, it is independent of the past. For these systems, the predators must take time to digest their food (preys) before further responses and activities take place. Therefore, the models of species without delays are approximate at best. A realistic model must incorporate the past history of the system, that is, it must include the delay. Recently, some work has studied a delayed diffusive predator-prey system [10–13].

The theory of traveling fronts for the reaction-diffusion equations is one of the fastest developing areas of modern mathematics and has already attracted much attention due to its significance in physics, chemistry, biology and epidemiology. The traveling wave problem for reaction-diffusion systems has been studied by many authors [14–16] in the past ten years. The food chain is a common phenomenon in population ecology. It is also central to understanding of the community structure in ecology. We will explore a basic example of a food chain, namely, a three species food chain in which a resource species is preyed upon by an intermediate predator which in turn is preyed upon by a dominant predator, and this model includes nonlocal delays.

Motivated by the above work, we mainly take into account the following food chain reaction-diffusion predator-prey system with nonlocal delays:

$$ \begin{aligned} &\frac{\partial u_{1}}{\partial t}-d_{1}\Delta u_{1} =u_{1}(r_{1}-a_{1}u_{2}-b_{1}u_{1}), \\ &\frac{\partial u_{2}}{\partial t}-d_{2}\Delta u_{2} =u_{2} \biggl(r_{2}-b_{2}u_{2}-a_{2}u_{3} +a_{3} \int_{\Omega} \int_{-\infty }^{t}K_{1}(x,y,t-s)u_{1}(s,y) \,ds\,dy \biggr), \\ &\frac{\partial u_{3}}{\partial t}-d_{3}\Delta u_{3}=u_{3} \biggl(-\alpha-b_{3}u_{3}+a_{4} \int_{\Omega} \int_{-\infty}^{t}K_{2}(x,y,t-s)u_{2}(s,y) \,ds\,dy \biggr) \\ &\quad \text{in } (0, \infty)\times\Omega, \\ &\frac{\partial u_{1}}{\partial\nu}=\frac{\partial u_{2}}{\partial \nu}=\frac{\partial u_{3}}{\partial\nu}=0 \quad \text{on } (0, \infty)\times\partial\Omega, \\ &u_{i0}(\theta,x)=\phi_{i}(\theta,x)\geq0\quad (i=1,2,3) \text{ in } [-\infty, 0]\times\Omega, \end{aligned} $$
(1.1)

where Ω is bounded domain in \(R^{N}\) (\(N\geq1 \) is an integer) with a smooth boundary ∂Ω; \(u_{1}\) represents the densities of the prey; \(u_{2}\) is for the density of the prey, and the same for the predator; \(u_{3}\) represents the densities of predator; the positive constants \(d_{1}\), \(d_{2}\), and \(d_{3}\) are the diffusion coefficients of the corresponding species; the positive constants \(r_{1}\), \(r_{2}\), and α represent the prey intrinsic growth rates and the death rate of predator, respectively. \(a_{1}\) and \(a_{2}\) represent interaction rates, respectively; \(b_{1}\), \(b_{2}\), and \(b_{3}\) represent self-limitation rates, respectively; the initial functions \(u_{i0}(t, x)\) (\(i=1,2,3\)) are Hölder continuous on \([-\infty, 0]\times\overline{\Omega}\).

The terms \(\int_{\Omega}\int_{-\infty}^{t}K_{i}(x,y,t-s)u_{i}(s,y)\,ds\,dy\) (\(i=1,2\)) represent a time delay because of gestation, that is, predator contributes to the reproduction of predator biomass. In system (1.1), we suppose that the kernels \(K_{i}(x,y,t)\) depend on both the temporal and the spatial variables. The delays in these formulations are nonlocal delays. These formulations reveal that the species drift to their present position (at time t) from all possible positions at all previous times (see [17]). Here, we suppose this drift cannot be viewed as being sufficiently small so as to be purely a local phenomenon.

In this article, we assume that

$$\begin{aligned}& K_{i}(x,y,t)=G_{i}(x,y,t)k_{i}(t), \quad x, y\in\Omega, k_{i}(t)\geq0, \\& \int_{\Omega}G_{i}(x,y,t)\,dx= \int_{\Omega}G_{i}(x,y,t)\,dy=1, \quad t \geq0, \\& \int_{0}^{\infty}k_{i}(t)\,dt=1, \quad t k_{i}(t)\in L^{1}\bigl((0,\infty);R\bigr), i=1,2, \end{aligned}$$
(1.2)

where \(G_{i}(x,y,t)\) are nonnegative functions which are continuous in \((x,y)\in\overline{\Omega}\times\overline{\Omega}\) for each \(t\in[0,\infty)\) and measurable in \(t\in[0,\infty)\) for each pair \((x,y)\in\overline{\Omega}\times\overline{\Omega}\).

In this article, by employing the method of eigenvalue and Lyapunov function, we investigate the stability of the unique positive constant solutions and find the sufficient conditions of stability which indicate that nonlocal delays often make impact on the stability of positive constant solution, but they do not impact that of a trivial solution. This result generalizes partially the one proved in [18]. By constructing upper-lower solutions, we establish the existence of the traveling wavefronts when the wave speed is large enough. The novelty of this article is that the system (1.1) incorporates two nonlocal delay terms with which are difficult to deal.

This paper is organized into four sections. In Section 2, the stability of the positive constant solution and the instability of trivial solution of system (1.1) are studied. In Section 3, the existence of traveling waves is established by constructing the upper and lower solution. In the final section, we give a short comment and conclusion.

2 Stability of positive equilibrium

It is easy to check that \((0, 0, 0)\) and \((M_{1}, M_{2}, M_{3})\) are a pair of coupled lower-upper solutions of system (1.1), where

$$\begin{aligned}& M_{1}=\max \biggl\{ \frac{r_{1}}{b_{1}}, \sup _{\theta\leq0}\bigl\Vert \phi_{1}(\theta,\cdot) \bigr\Vert _{C(\overline{\Omega},R)} \biggr\} , \\& M_{2}=\max \biggl\{ \frac{r_{1}+a_{3}M_{1}}{b_{2}}, \sup_{\theta \leq0} \bigl\Vert \phi_{2}(\theta,\cdot)\bigr\Vert _{C(\overline{\Omega},R)} \biggr\} , \\& M_{3}=\max \biggl\{ \frac{a_{4}M_{2}-\alpha}{b_{3}}, \sup_{\theta\leq0} \bigl\Vert \phi_{3}(\theta,\cdot)\bigr\Vert _{C(\overline{\Omega},R)} \biggr\} . \end{aligned}$$
(2.1)

Hence, there exists a unique global solution \((u_{1}(t, x), u_{2}(t, x), u_{3}(t, x))\) satisfying \(0\leq u_{1}\leq M_{1}\), \(0\leq u_{2}\leq M_{2}\), \(0\leq u_{3}\leq M_{3}\) to system (1.1) (see [12]).

Note that (1.1) admits the following six equilibria: \((0, 0, 0)\), \((r_{1}/b_{1}, 0, 0)\), \((0, r_{2}/b_{2}, 0)\), \((\frac{b_{2}r_{1}-a_{2}r_{2}}{a_{1}a_{3}+b_{1}b_{2}},\frac{a_{3}r_{1} +b_{1}r_{2}}{a_{1}a_{3}+b_{1}b_{2}},0)\), \((0,\frac{r_{2}b_{3}+\alpha a_{2}}{a_{2}a_{4}+b_{2}b_{3}}, \frac{r_{2}a_{4}-\alpha b_{2}}{a_{2}a_{4}+b_{2}b_{3}})\), and \((k_{1}^{\ast}, k_{2}^{\ast}, k_{3}^{\ast})\), where

$$\begin{aligned}& k_{1}^{\ast}=\frac{(r_{1}a_{4}-\alpha a_{1})a_{2}+(r_{1}b_{3} -r_{2}a_{1})b_{3}}{a_{2}a_{4}b_{1}+(a_{1}a_{3}+b_{1}b_{3})b_{3}}, \qquad k_{2}^{\ast}= \frac{(r_{2}b_{3}+\alpha a_{2})b_{1}+r_{1}a_{3}b_{3}}{a_{2}a_{4}b_{1} +(a_{1}a_{3}+b_{1}b_{3})b_{3}}, \\& k_{3}^{\ast}=\frac{(r_{1}a_{4}-\alpha a_{1})a_{3}+(r_{2}a_{4} -\alpha b_{3})b_{1}}{a_{2}a_{4}b_{1}+(a_{1}a_{3}+b_{1}b_{3})b_{3}}. \end{aligned}$$

For the existence of a positive constant solution \((k_{1}^{\ast}, k_{2}^{\ast}, k_{3}^{\ast})\), it is necessary to assume that

$$r_{1}a_{4}>\alpha a_{1}, \qquad r_{1}b_{3} > r_{2}a_{1}, \qquad r_{2}a_{4} > \alpha b_{3}. $$

Let \(0=\mu_{1}<\mu_{2}<\cdots\rightarrow+\infty\) denote the eigenvalues of −Δ in Ω under a homogeneous Neuman boundary condition and φ be the set of eigenfunctions corresponding to μ.

Notation 2.1

  1. (i)

    \(X_{ij} := \{C\varphi_{ij} : C\in R^{3}\}\), where \(\varphi_{ij}\) are orthonormal basis of \(S(\mu_{i})\) for \(j=1,\ldots, \operatorname{dim}[S(\mu_{i})]\).

  2. (ii)

    \(X:=\{(u, v, w)\in C^{1}(\overline{\Omega})\times C^{1}(\overline{\Omega}): \frac{\partial u}{\partial\nu}=\frac{\partial u}{\partial \nu}=\frac{\partial w}{\partial\nu}=0 \mbox{ on } \partial \Omega\}\), so that

    $$X=\bigoplus_{i=0}^{\infty}\bigoplus _{j=1}^{\operatorname{dim}[S(\mu_{i})]}X_{ij}. $$

Lemma 2.1

[19, 20]

Let a and b be positive constants. Suppose that \(\xi, \eta\in C^{1}[a,+\infty)\), \(\eta\geq0\), and ξ is bounded from below. If \(\xi'(t)\leq-b\eta(t)\) and \(\eta'(t)\) is bounded from above in \([a,+\infty)\), then \(\lim t_{t\rightarrow\infty}\eta(t) =0\).

The following theorem is the global stability result of the positive constant solution \((k_{1}^{\ast}, k_{2}^{\ast}, k_{3}^{\ast})\) of (1.1). This result extends partially the one proved in [18].

Theorem 2.1

Suppose that \(b_{2}b_{3} > 2a_{2}a_{4}\), \(b_{1}b_{2}> 2a_{1}a_{3}\). Then the positive constant solution \((k_{1}^{\ast}, k_{2}^{\ast}, k_{3}^{\ast})\) is globally stable for the system (1.1).

Proof

We use the approach developed by [18] to find the proof. Let \((u_{1}(t, x), u_{2}(t, x), u_{3}(t,x))\) be positive solution of (1.1) and define the following Lyapunov function:

$$V_{1}(t)=\sum_{i=1}^{3} \beta_{i} \int_{\Omega} \biggl(u_{i}-k_{i}^{\ast}-k_{i}^{\ast} \ln \frac{u_{i}}{k_{i}^{\ast}} \biggr)\,dx, $$

where \(\beta_{3} = 1\), \(\beta_{1}\) and \(\beta_{2}\) are positive constant to be determined. By calculating the derivative of \(V_{1}(t)\) along positive solutions of system (1.1), we obtain

$$\begin{aligned} \frac{dV_{1}(t)}{dt} =&\sum_{i=1}^{2} \beta_{i} \int_{\Omega}\frac {\partial u_{i}}{\partial t} \biggl(1-\frac{k_{i}^{\ast}}{u_{i}} \biggr) \,dx+ \int_{\Omega}\frac {\partial u_{3}}{\partial t} \biggl(1-\frac{k_{3}^{\ast}}{u_{3}} \biggr) \,dx \\ =&-\sum_{i=1}^{3}\beta_{i}d_{i}k_{i}^{\ast} \int_{\Omega}\frac {|\nabla u_{i}|}{u_{i}^{2}}\,dx+ \int_{\Omega}\beta_{1}\bigl(u_{1}-k_{1}^{\ast } \bigr) (r_{1}-a_{1}u_{2}-b_{1}u_{1}) \,dx \\ &{}+ \int_{\Omega}\beta_{2}\bigl(u_{2}-k_{2}^{\ast } \bigr) \biggl(r_{2}-b_{2}u_{2}-a_{2}u_{3}+a_{3} \int_{\Omega} \int_{-\infty}^{t} K_{1}(x,y,t-s)u_{1}(s,y) \,ds\,dy \biggr)\,dx \\ &{}+ \int_{\Omega}\bigl(u_{3}-k_{3}^{\ast} \bigr) \biggl(-\alpha-b_{3}u_{3}+a_{4} \int _{\Omega} \int_{-\infty}^{t} K_{2}(x,y,t-s)u_{2}(s,y) \,ds\,dy\biggr)\,dx. \end{aligned}$$
(2.2)

By using the inequality \(ab\leq\frac{1}{2}\lambda a^{2}+\frac{1}{2\lambda}b^{2}\), we obtain

$$\begin{aligned} \frac{dV_{1}(t)}{dt} \leq &-\sum_{i=1}^{3} \beta_{i}d_{i}k_{i}^{\ast} \int_{\Omega}\frac {|\nabla u_{i}|}{u_{i}^{2}}\,dx-\beta_{1}b_{1} \int_{\Omega}\bigl(u_{1}-k_{1}^{\ast } \bigr)^{2}\,dx \\ &{}+\beta_{1}a_{1} \int_{\Omega} \biggl[\frac{1}{2}\lambda _{1} \bigl(u_{1}-k_{1}^{\ast}\bigr)^{2} + \frac{1}{2\lambda_{1}}\bigl(u_{2}-k_{2}^{\ast} \bigr)^{2} \biggr]\,dx-\beta _{2}b_{2} \int_{\Omega} \bigl(u_{2}-k_{2}^{\ast} \bigr)^{2}\,dx \\ &{}+\beta_{2}a_{2} \int_{\Omega} \biggl[\frac{1}{2}\lambda _{2} \bigl(u_{2}-k_{2}^{\ast}\bigr)^{2} + \frac{1}{2\lambda_{2}}\bigl(u_{3}-k_{3}^{\ast} \bigr)^{2} \biggr]\,dx-b_{3} \int _{\Omega}\bigl(u_{3}-k_{3}^{\ast} \bigr)^{2}\,dx \\ &{}+\frac{1}{2}\lambda_{3}a_{3}\beta_{2} \int_{\Omega} \int_{\Omega } \int_{-\infty}^{t} K_{1}(x,y,t-s) \bigl(u_{1}(s,y)-k_{1}^{\ast}\bigr)^{2}\,ds \,dy\,dx \\ &{}+\frac{1}{2\lambda_{3}}a_{3}\beta_{2} \int_{\Omega} \int_{\Omega } \int_{-\infty}^{t} K_{1}(x,y,t-s) \bigl(u_{2}(t,y)-k_{2}^{\ast}\bigr)^{2}\,ds \,dy\,dx \\ &{}+\frac{1}{2}\lambda_{4}a_{4} \int_{\Omega} \int_{\Omega} \int _{-\infty}^{t} K_{2}(x,y,t-s) \bigl(u_{2}(s,y)-k_{2}^{\ast}\bigr)^{2}\,ds \,dy\,dx \\ &{}+\frac{1}{2\lambda_{4}}a_{4} \int_{\Omega} \int_{\Omega} \int _{-\infty}^{t} K_{2}(x,y,t-s) \bigl(u_{3}(t,y)-k_{3}^{\ast}\bigr)^{2}\,ds \,dy\,dx. \end{aligned}$$
(2.3)

Employing the property of \(K_{i}(x, y, t)\) (\(i = 1, 2\)) as described in (1.2), we obtain

$$\begin{aligned} \frac{dV_{1}(t)}{dt} \leq &-\sum_{i=1}^{3} \beta_{i}d_{i}k_{i}^{\ast} \int_{\Omega}\frac {|\nabla u_{i}|}{u_{i}^{2}}\,dx-\beta_{1}(b_{1}-a_{1} \lambda_{1}/2) \int_{\Omega }\bigl(u_{1}-k_{1}^{\ast} \bigr)^{2}\,dx \\ &{}-(\beta_{2}b_{2}-\beta_{1}a_{1}/2 \lambda_{1}-\lambda_{2}\beta_{2}a_{2}/2 -a_{3}\beta_{2}/2\lambda_{3}) \int_{\Omega}\bigl(u_{2}-k_{2}^{\ast } \bigr)^{2}\,dx \\ &{}-(b_{3}-\beta_{2}a_{2}/2\lambda_{2}-a_{4}/2 \lambda_{4}) \int _{\Omega}\bigl(u_{3}-k_{3}^{\ast} \bigr)^{2}\,dx \\ &{}+\frac{1}{2}\lambda_{3}a_{3}\beta_{2} \int_{\Omega} \int_{\Omega } \int_{0}^{\infty} K_{1}(x,y,r) \bigl(u_{1}(t-r,y)-k_{1}^{\ast}\bigr)^{2}\,dr \,dy\,dx \\ &{}+\frac{1}{2}\lambda_{4}a_{4} \int_{\Omega} \int_{\Omega} \int _{0}^{\infty} K_{2}(x,y,r) \bigl(u_{2}(t-r,y)-k_{2}^{\ast}\bigr)^{2}\,dr \,dy\,dx. \end{aligned}$$
(2.4)

Define a new Lyapunov function

$$\begin{aligned} V(t) =& V_{1}(t)+\frac{1}{2} \lambda_{3}a_{3}\beta_{2} \int_{\Omega } \int_{\Omega} \int_{0}^{\infty} \int_{t-r}^{t}K_{1}(x,y,r) \bigl(u_{1}(l,y)-k_{1}^{\ast}\bigr)^{2}\,dl \,dr\,dy\,dx \\ &{}+\frac{1}{2}\lambda_{4}a_{4} \int_{\Omega} \int_{\Omega} \int _{0}^{\infty} \int_{t-r}^{t} K_{2}(x,y,r) \bigl(u_{2}(l,y)-k_{2}^{\ast}\bigr)^{2}\,dl \,dr\,dy\,dx. \end{aligned}$$
(2.5)

Then, combining (2.4) and (2.5), we get

$$\begin{aligned} \frac{dV_{1}(t)}{dt} \leq &-\sum_{i=1}^{3} \beta_{i}d_{i}k_{i}^{\ast} \int_{\Omega}\frac {|\nabla u_{i}|}{u_{i}^{2}}\,dx-\beta_{1}(b_{1}-a_{1} \lambda_{1}/2) \int_{\Omega }\bigl(u_{1}-k_{1}^{\ast} \bigr)^{2}\,dx \\ &{}-(\beta_{2}b_{2}-\beta_{1}a_{1}/2 \lambda_{1}-\lambda_{2}\beta_{2}a_{2}/2 -a_{3}\beta_{2}/2\lambda_{3}) \int_{\Omega}\bigl(u_{2}-k_{2}^{\ast } \bigr)^{2}\,dx \\ &{}-(b_{3}-\beta_{2}a_{2}/2\lambda_{2}-a_{4}/2 \lambda_{4}) \int _{\Omega}\bigl(u_{3}-k_{3}^{\ast} \bigr)^{2}\,dx \\ &{}+\frac{1}{2}\lambda_{3}a_{3}\beta_{2} \int_{\Omega} \int_{\Omega } \int_{0}^{\infty} K_{1}(x,y,r) \bigl(u_{1}(t,y)-k_{1}^{\ast}\bigr)^{2}\,dr \,dy\,dx \\ &{}+\frac{1}{2}\lambda_{4}a_{4} \int_{\Omega} \int_{\Omega} \int _{0}^{\infty} K_{2}(x,y,r) \bigl(u_{2}(t,y)-k_{2}^{\ast}\bigr)^{2}\,dr \,dy\,dx. \end{aligned}$$
(2.6)

Since

$$\begin{aligned}& \int_{\Omega} \int_{\Omega} \int_{0}^{\infty} K_{i}(x,y,r) \bigl(u_{i}(t,y)-k_{i}^{\ast}\bigr)^{2}\,dr \,dy\,dx \\& \quad = \int_{\Omega}\bigl(u_{i}(t,y)-k_{i}^{\ast} \bigr)^{2}\,dy\quad (i=1,2), \end{aligned}$$

it follows from (2.6) that

$$\begin{aligned} \frac{dV_{1}(t)}{dt} \leq &-\sum_{i=1}^{3} \beta_{i}d_{i}k_{i}^{\ast} \int_{\Omega}\frac {|\nabla u_{i}|}{u_{i}^{2}}\,dx \\ &{}-\bigl(\beta_{1}(b_{1}-a_{1} \lambda_{1}/2)-\lambda_{3}a_{3} \beta_{2}/2\bigr) \int_{\Omega}\bigl(u_{1}-k_{1}^{\ast} \bigr)^{2}\,dx \\ &{}-(\beta_{2}b_{2}-\beta_{1}a_{1}/2 \lambda_{1}-\lambda_{2}\beta_{2}a_{2}/2 -a_{3}\beta_{2}/2\lambda_{3}-\lambda_{4}a_{4}/2) \int_{\Omega }\bigl(u_{2}-k_{2}^{\ast} \bigr)^{2}\, dx \\ &{}-(b_{3}-\beta_{2}a_{2}/2\lambda_{2}-a_{4}/2 \lambda_{4}) \int _{\Omega}\bigl(u_{3}-k_{3}^{\ast} \bigr)^{2}\,dx. \end{aligned}$$
(2.7)

Integrating (2.7) over \([0, T]\) (\(T > 0\)), we obtain

$$\begin{aligned}& \sum_{i=1}^{3} \beta_{i}d_{i}k_{i}^{\ast}\biggl\Vert \frac{|\nabla u_{i}|}{u_{i}}\biggr\Vert _{L^{2}_{(\Omega_{T})}}^{2} + \beta_{1}b_{1}\bigl\Vert u_{1}-k_{1}^{\ast} \bigr\Vert _{L^{2}_{(\Omega _{T})}}^{2}+\beta_{2}b_{2}\bigl\Vert u_{2}-k_{2}^{\ast}\bigr\Vert _{L^{2}_{(\Omega_{T})}}^{2}+b_{3}\bigl\Vert u_{3}-k_{3}^{\ast} \bigr\Vert _{L^{2}_{(\Omega_{T})}}^{2} \\& \quad \leq V(0)+\frac{1}{2}(\beta_{1}a_{1} \lambda_{1}+\lambda_{3}\beta _{2}a_{3}) \bigl\Vert u_{1}-k_{1}^{\ast}\bigr\Vert _{L^{2}_{(\Omega_{T})}}^{2} +\frac{1}{2}(\beta_{1}a_{1}/ \lambda_{1}+\lambda_{2}\beta_{2}a_{2} +a_{3}\beta_{2}/\lambda_{3} \\& \qquad {}+\lambda_{4}a_{4})\bigl\Vert u_{2}-k_{2}^{\ast}\bigr\Vert _{L^{2}_{(\Omega_{T})}}^{2} +\frac{1}{2}(\beta_{2}a_{2}/\lambda_{2}+a_{4}/ \lambda_{4})\bigl\Vert u_{3}-k_{3}^{\ast} \bigr\Vert _{L^{2}_{(\Omega_{T})}}^{2}. \end{aligned}$$
(2.8)

Taking

$$\lambda_{1}=\lambda_{3}=\frac{2\beta_{1}b_{1}}{a_{1}\beta _{1}+a_{3}} \quad \text{and} \quad \lambda_{2}=\lambda_{4}=\frac{\beta_{2}a_{2}+a_{4}}{2b_{3}}, $$

it is derived from (2.8) that

$$\begin{aligned}& \sum_{i=1}^{3} \frac{\beta_{i}d_{i}k_{i}^{\ast}}{M_{i}}\bigl\Vert \vert \nabla u_{i}\vert \bigr\Vert _{L^{2}_{(\Omega_{T})}}^{2} +\beta_{2}b_{2}\bigl\Vert u_{2}-k_{2}^{\ast}\bigr\Vert _{L^{2}_{(\Omega_{T})}}^{2} \\& \quad \leq V(0)+\bigl((a_{1}\beta_{1}+a_{3} \beta_{2})^{2}/4\beta _{1}b_{1}+(a_{2} \beta_{2} +a_{4})^{2}/4b_{3}\bigr)\bigl\Vert u_{2}-k_{2}^{\ast}\bigr\Vert _{L^{2}_{(\Omega _{T})}}^{2}. \end{aligned}$$
(2.9)

Using the conditions \(b_{2}b_{3} > 2a_{2}a_{4}\), \(b_{1}b_{2} > 2a_{1}a_{3}\), one can choose \(\beta_{1},\beta_{2}>0\) such that

$$\beta_{2}b_{2}>\frac{1}{4}\bigl[(a_{1} \beta_{1}+a_{3}\beta _{2})^{2}/ \beta_{1}b_{1} +(a_{2}\beta_{2}+a_{4})^{2}/b_{3} \bigr], $$

because the inequalities \(2b_{1}b_{2}\beta_{1}\beta_{2}>a_{1}^{2}\beta_{1}^{2} +2a_{1}a_{3}\beta_{1}\beta_{2}+a_{2}^{2}\beta_{2}^{2}\) and \(2b_{2}b_{3}\beta_{2}>a_{2}^{2}\beta_{2}^{2} +2a_{2}a_{4}\beta_{2}+a_{4}^{2}\) hold for certain \(\beta_{1}\) and \(\beta_{2}\).

Therefore, we obtain

$$ \bigl\Vert \vert \nabla u_{i}\vert \bigr\Vert _{L^{2}_{(\Omega_{T})}}\leq C_{1}, \qquad \bigl\Vert u_{2}-k_{2}^{\ast} \bigr\Vert _{L^{2}_{(\Omega_{T})}}\leq C_{1}, $$
(2.10)

where \(C_{1}\) is constant independent of T. In a similar way, by taking

$$\lambda_{1}=\lambda_{3}=\frac{\beta_{1}a_{1}+a_{3}\beta_{2}}{\beta _{2}b_{2}} \quad \text{and} \quad \lambda_{2}=\lambda_{4}=\frac{\beta_{2}b_{2}}{a_{2}\beta_{2}+a_{4}}, $$

it is derived from (2.8) that

$$\begin{aligned}& \sum_{i=1}^{3} \frac{\beta_{i}d_{i}k_{i}^{\ast}}{M_{i}}\bigl\Vert \vert \nabla u_{i}\vert \bigr\Vert _{L^{2}_{(\Omega_{T})}}^{2} +\beta_{1}b_{1}\bigl\Vert u_{1}-k_{1}^{\ast}\bigr\Vert _{L^{2}_{(\Omega_{T})}}^{2} +b_{3}\bigl\Vert u_{3}-k_{3}^{\ast}\bigr\Vert _{L^{2}_{(\Omega_{T})}}^{2} \\& \quad \leq V(0)+\bigl((\beta_{1}a_{1}+a_{3} \beta_{2})^{2}/2\beta_{2}b_{2}\bigr)\bigl\Vert u_{1}-k_{1}^{\ast}\bigr\Vert _{L^{2}_{(\Omega_{T})}}^{2} \\& \qquad {}+\bigl((\beta_{2}a_{2}+a_{4})^{2}/2 \beta_{2}b_{2}\bigr)\bigl\Vert u_{3}-k_{3}^{\ast} \bigr\Vert _{L^{2}_{(\Omega_{T})}}^{2}. \end{aligned}$$
(2.11)

Using the conditions \(b_{2}b_{3} > 2a_{2}a_{4}\), \(b_{1}b_{2} > 2a_{1}a_{2}\), one can choose \(\beta_{1}, \beta_{2}>0 \) again such that

$$\beta_{1}b_{1}>\frac{(a_{1}\beta_{1}+a_{3}\beta_{2})^{2}}{2\beta _{2}b_{2}} \quad \text{and} \quad b_{3}>\frac{(a_{2}\beta_{2}+a_{4})^{2}}{2\beta_{2}b_{2}}. $$

Therefore, we see that

$$ \bigl\Vert u_{1}-k_{1}^{\ast}\bigr\Vert _{L^{2}_{(\Omega_{T})}}^{2}\leq C_{2}\quad \text{and} \quad \bigl\Vert u_{3}-k_{3}^{\ast}\bigr\Vert _{L^{2}_{(\Omega_{T})}}^{2}\leq C_{2}, $$
(2.12)

where \(C_{2}\) is constant independent of T.

Choosing

$$\lambda_{1}=\lambda_{3}=\frac{\beta_{1}a_{1}+a_{3}\beta_{2}}{\beta _{2}b_{2}}+\varepsilon, \qquad \lambda_{2}=\lambda_{4}=\frac{\beta_{2}b_{2}}{a_{2}\beta_{2}+a_{4}}, $$

where ε is sufficiently small positive constant. Using the conditions of Theorem 2.1 and (2.7), one can easily verify that there exists a positive constant δ (\(\delta>0\)) such that

$$ \begin{aligned} &\frac{dV}{dt}\leq -\delta \int_{\Omega}\bigl[\bigl(u_{1}-k_{1}^{\ast} \bigr)^{2}+\bigl(u_{2}-k_{2}^{\ast } \bigr)^{2}+\bigl(u_{3}-k_{3}^{\ast} \bigr)^{2}\bigr]\,dx, \\ & \frac{dV}{dt}\leq0, \qquad (u_{1}, u_{2}, u_{3})\neq \bigl(k_{1}^{\ast}, k_{2}^{\ast}, k_{3}^{\ast}\bigr). \end{aligned} $$
(2.13)

Using integration by parts, the Hölder inequality, (2.1), and (2.10), one can easily check that \(\frac{d}{dt}\int_{\Omega}[(u_{1}-k_{1}^{\ast})^{2} + (u_{2}-k_{2}^{\ast})^{2} + (u_{3}-k_{3}^{\ast})^{2}]\,dx \) is bounded from above. Then, using Lemma 2.1, (2.10), and (2.13), we see that

$$\begin{aligned}& \bigl\Vert u_{1}(t,\cdot)-k_{1}^{\ast} \bigr\Vert _{L_{(\Omega)}^{2}}\rightarrow0, \\& \bigl\Vert u_{2}(t, \cdot)-k_{2}^{\ast}\bigr\Vert _{L_{(\Omega)}^{2}}\rightarrow0, \\& \bigl\Vert u_{3}(t,\cdot)-k_{3}^{\ast}\bigr\Vert _{L_{(\Omega)}^{2}}\rightarrow0. \end{aligned}$$
(2.14)

Obviously,

$$ \bigl\Vert u(t,x)\bigr\Vert _{L_{(\Omega)}^{\infty}}\leq C_{3}\|u\|_{W_{2}^{1}}^{\frac{1}{2}}\|u\|_{L^{2}(\Omega)}. $$
(2.15)

It follows from (2.10), (2.12), (2.14), (2.15), and (2.1) that

$$\begin{aligned}& \bigl\Vert u_{1}(t,\cdot)-k_{1}^{\ast}\bigr\Vert _{L_{(\Omega)}^{\infty}}\rightarrow 0, \\& \bigl\Vert u_{2}(t, \cdot)-k_{2}^{\ast}\bigr\Vert _{L_{(\Omega)}^{\infty}}\rightarrow 0, \\& \bigl\Vert u_{3}(t,\cdot)-k_{3}^{\ast}\bigr\Vert _{L_{(\Omega)}^{\infty}}\rightarrow 0. \end{aligned}$$

Namely, \((u_{1}, u_{2}, u_{3})\) converges uniformly to \((k_{1}^{\ast}, k_{2}^{\ast}, k_{3}^{\ast})\). Using the fact that \(V(u_{1}, u_{2}, u_{3})\) is decreasing for t, one can derive that \((k_{1}^{\ast}, k_{2}^{\ast}, k_{3}^{\ast})\) is globally stable. This completes the proof. □

Theorem 2.2

The trivial equilibrium \((0, 0, 0)\) is unstable for the system (1.1).

Proof

The linearized problem of (1.1) at \((0, 0, 0)\) can be expressed by

$$\mathbf{w}_{t}=\bigl(D\Delta+F_{\mathbf{w}}(0,0,0)\bigr)\mathbf{w}, $$

where \(\mathbf{w}=(u_{1}(t,x), u_{2}(t,x), u_{3}(t,x))^{T}\), and \(F=(u_{1}(r_{1}-a_{1}u_{2}-b_{1}u_{1}), u_{2}(r_{2}-b_{2}u_{2}-a_{2}u_{3} +a_{3}\int_{\Omega}\int_{-\infty}^{t}K_{1}(x,y,t-s)u_{1}(s,y)\,ds\,dy), u_{3}(-\alpha-b_{3}u_{3}+a_{4}\int_{\Omega}\int_{-\infty }^{t}K_{2}(x,y,t-s)u_{2}(s,y)\,ds\,dy))\). By direct calculations, we obtain

$$D=\left ( \textstyle\begin{array}{@{}c@{\quad}c@{\quad}c@{}} d_{1} & 0 & 0 \\ 0 & d_{2} & 0 \\ 0 & 0 & d_{3} \end{array}\displaystyle \right ), \qquad F_{\mathbf{w}}(0,0,0)=\left ( \textstyle\begin{array}{@{}c@{\quad}c@{\quad}c@{}} r_{1} & 0 & 0 \\ 0 & r_{2} & 0 \\ 0 & 0 & \alpha \end{array}\displaystyle \right ). $$

Consider the following eigenvalue problem:

$$\bigl(D\Delta+F_{\mathbf{w}}(\mathbf{w}_{\ast})\bigr) \left ( \textstyle\begin{array}{@{}c@{}} \phi\\ \varphi\\ \psi \end{array}\displaystyle \right )=\tilde{\lambda} \left ( \textstyle\begin{array}{@{}c@{}} \phi\\ \varphi\\ \psi \end{array}\displaystyle \right ), $$

where \(\mathbf{w}_{\ast}\) is constant solution of (1.1). Using the eigenfunction expansions (2.6) in [3] for ϕ, φ and ψ, λ̃ is an eigenvalue of \(D\Delta+F_{\mathbf{w}}(\mathbf{w}_{\ast})\) if and only if λ̃ is an eigenvalue of the matrix \(-\mu_{k}D+F_{\mathbf{w}}(0, 0, 0)\) for each \(k\geq1\). Therefore, to study the local stability at \((0, 0, 0)\), it is necessary to investigate the characteristic equation

$$\operatorname{det}\bigl(\tilde{\lambda}I+\mu_{k}D-F_{\mathbf{w}}(0,0,0) \bigr) =(\tilde{\lambda}+\mu_{k}d_{1}-r_{1}) ( \tilde{\lambda}+\mu _{k}d_{2}-r_{2}) (\tilde{ \lambda}+\mu_{k}d_{2}+\alpha)=0. $$

If \(i = 1\), then \(\mu_{k} = 0\). Therefore, there exist two positive characteristic roots, which, in view of Theorem 5.13 in [21], yields the desired result. □

3 The existence of traveling waves

In this section, we assume that \(\Omega\subset R^{1}\). Denote \(f_{1}(u_{1},u_{2},u_{3})=u_{1}(r_{1}-a_{1}u_{2}-b_{1}u_{1})\), \(f_{2}(u_{1},u_{2},u_{3})=u_{2}(r_{2}-b_{2}u_{2}-a_{2}u_{3} +a_{3}\int_{\Omega}\int_{-\infty }^{t}K_{1}(x,y,t-s)u_{1}(s,y)\,ds\,dy)\), \(f_{3}(u_{1},u_{2},u_{3})=u_{3}(-\alpha-b_{3}u_{3} +a_{4}\int_{\Omega}\int_{-\infty}^{t}K_{2}(x,y,t-s)u_{2}(s,y)\,ds\,dy)\). Let \((u_{1}(t, x), u_{2}(t, x), u_{3}(t, x)) = (\phi(x + c t), \varphi(x +c t), \psi(x + c t))\) be a traveling wave solution of (1.1), where \(\phi,\varphi,\psi\in C^{2}(R,R^{2})\) and \(c > 0\) is a constant accounting for the wave speed, and denote the traveling wave coordinate \(x+ct \) still by t. Then the system (1.1) can be rewritten in the form

$$\begin{aligned}& d_{1}\phi''(t)-c \phi'(t)+f_{c1}(\phi_{t}, \varphi_{t}, \psi_{t})=0, \\& d_{2}\varphi''(t)-c\varphi'(t)+f_{c2}( \phi_{t}, \varphi_{t}, \psi_{t})=0, \\& d_{3}\psi''(t)-c\psi'(t)+f_{c3}( \phi_{t}, \varphi_{t}, \psi_{t})=0, \end{aligned}$$
(3.1)

where \(f_{ci}\) (\(i=1, 2, 3\)) are defined by

$$\begin{aligned}& f_{ci}(\phi,\varphi,\psi)=f_{i}\bigl(\phi^{c}, \varphi^{c},\psi^{c}\bigr),\qquad \phi^{c}(s)=\phi(cs),\qquad \varphi^{c}(s)=\varphi(cs), \\& \psi^{c}(s)=\psi(cs),\quad s\in(-\infty, 0], i=1,2,3. \end{aligned}$$

If (3.1) has a solution satisfying the following asymptotic boundary conditions:

$$\begin{aligned}& \lim_{t\rightarrow-\infty}\phi(t)=\phi_{-},\qquad \lim _{t\rightarrow-\infty}\varphi(t)=\varphi_{-},\qquad \lim _{t\rightarrow-\infty}\psi(t)=\psi_{-}, \\& \lim_{t\rightarrow+\infty}\phi(t)=\phi_{+},\qquad \lim _{t\rightarrow+\infty}\varphi(t)=\varphi_{+},\qquad \lim _{t\rightarrow+\infty}\psi(t)=\psi_{+}, \end{aligned}$$

then system (1.1) has a traveling wave solution (see [15, 16]). Without loss of generality, we assume that \((\phi_{-}, \varphi_{-},\psi_{-} ) = (0, 0, 0)\) and \((\phi_{+}, \varphi_{+},\psi_{+} )= (k_{1}^{\ast}, k_{2}^{\ast}, k_{3}^{\ast})\).

According to basic theory of the existence of traveling wave solutions (see [15, 16]), we mainly need to check that the system (1.1) satisfies partial quasi-monotonicity conditions, that is, there exist three positive constants \(\rho_{1}, \rho_{2}, \rho_{3}> 0\) such that

$$\begin{aligned}& f_{1}(\phi_{1},\varphi_{1}, \psi_{1})-f_{1}(\phi_{2},\varphi _{1}, \psi_{2}) +\rho_{1}\bigl[\phi_{1}(0)- \phi_{2}(0)\bigr]\geq0, \\& f_{1}(\phi_{1},\varphi_{1},\psi_{1})-f_{1}( \phi_{1},\varphi _{2},\psi_{1}) \leq0, \\& f_{2}(\phi_{1},\varphi_{1},\psi_{1})-f_{2}( \phi_{2},\varphi _{2},\psi_{1}) +\rho_{2} \bigl[\varphi_{1}(0)-\varphi_{2}(0)\bigr]\geq0, \\& f_{2}(\phi_{1},\varphi_{1},\psi_{1})-f_{2}( \phi_{1},\varphi _{1},\psi_{2}) \leq0, \\& f_{3}(\phi_{1},\varphi_{1},\psi_{1})-f_{3}( \phi_{2},\varphi _{2},\psi_{2}) +\rho_{3} \bigl[\psi_{1}(0)-\psi_{2}(0)\bigr]\geq0, \end{aligned}$$
(3.2)

with \(0\leq\phi_{2}(s)\leq\phi_{1}(s)\leq M_{1}\), \(0\leq \varphi_{2}(s)\leq\varphi_{1}(s)\leq M_{2}\), \(0\leq\psi_{2}(s)\leq \psi_{1}(s)\leq M_{3}\), and we also need to check that a pair of continuous functions \((\overline{\phi}, \overline{\varphi}, \overline{\psi})\) and \((\underline{\phi},\underline{\varphi},\underline{\psi})\) is a pair of upper-lower solution of system (3.1), that is,

$$\begin{aligned}& d_{1}\overline{\phi}''(t)-c \overline{\phi}'(t)+f_{c1}(\overline {\phi}_{t}, \underline{\varphi}_{t},\overline{\psi}_{t})\leq0, \\& d_{2}\overline{\varphi}''(t)-c\overline{ \varphi }'(t)+f_{c2}(\overline{\varphi}_{t}, \overline{\varphi}_{t},\underline{\psi}_{t})\leq0, \\& d_{3}\overline{\psi}''(t)-c\overline{ \psi}'(t)+f_{c3}(\overline {\psi}_{t}, \overline{ \varphi}_{t},\overline{\psi}_{t})\leq0 \end{aligned}$$
(3.3)

and

$$\begin{aligned}& d_{1}\underline{\phi}''(t)-c \underline{\phi}'(t)+f_{c1}(\underline { \phi}_{t}, \overline{\varphi}_{t},\underline{ \psi}_{t})\geq0, \\& d_{2}\underline{\varphi}''(t)-c\underline{ \varphi }'(t)+f_{c2}(\underline{\varphi}_{t}, \underline{\varphi}_{t},\overline{\psi}_{t})\geq0, \\& d_{3}\underline{\psi}''(t)-c\underline{ \psi}'(t)+f_{c3}(\underline {\psi}_{t}, \underline{\varphi}_{t},\underline{\psi}_{t})\geq0, \end{aligned}$$
(3.4)

where \((0, 0, 0)\leq (\underline{\phi},\underline{\varphi},\underline{\psi})\leq (\overline{\phi},\overline{\varphi},\overline{\psi})\leq (M_{1},M_{2},M_{3})\), \(t\in R\).

Lemma 3.1

\(f_{c1}(\phi_{t},\varphi_{t},\psi_{t})\), \(f_{c2}(\phi_{t},\varphi_{t},\psi_{t})\), and \(f_{c3}(\phi_{t},\varphi_{t},\psi_{t})\) of system (1.1) satisfy (3.2).

Proof

Let \(\phi_{1}(s)\), \(\phi_{2}(s)\), \(\varphi_{1}(s)\), \(\varphi_{2}(s)\), \(\psi_{1}(s)\), \(\psi_{2}(s)\) satisfy \(0\leq\phi_{2}(s)\leq\phi_{1}(s)\leq M_{1}\), \(0\leq \varphi_{2}(s)\leq\varphi_{1}(s)\leq M_{2}\), \(0\leq \psi_{2}(s)\leq\psi_{1}(s)\leq M_{3}\), \(s\in(-\infty, 0]\).

For any \(\phi_{i},\varphi_{i},\psi_{i}\in((-\infty, 0], R)\), \(i = 1, 2\), we have

$$\begin{aligned}& f_{c1}(\phi_{1t},\varphi_{1t}, \psi_{1t})-f_{c1}(\phi_{2t},\varphi _{1t}, \psi_{2t}) \\& \quad =\phi_{1}(0) \bigl(r_{1}-a_{1} \varphi_{1}(0)-b_{1}\phi_{1}(0)\bigr)- \phi_{2}(0) \bigl(r_{1} -a_{1}\varphi_{1}(0)-b_{1} \phi_{2}(0)\bigr) \\& \quad \geq r_{1}\bigl(\phi_{1}(0)-\phi_{2}(0) \bigr)-a_{1}\varphi_{1}(0) \bigl(\phi_{1}(0)- \phi_{2}(0)\bigr) -2b_{1}M_{1}\bigl( \phi_{1}(0)-\phi_{2}(0)\bigr) \\& \quad \geq(-a_{1}M_{2}-2b_{1}M_{1}) \bigl(\phi_{1}(0)-\phi_{2}(0)\bigr). \end{aligned}$$
(3.5)

Let \(\rho_{1}=a_{1}M_{2}+2b_{1}M_{1}>0\), then it is easy to see that

$$ \begin{aligned} &f_{c1}(\phi_{1t}, \varphi_{1t},\psi_{1t})-f_{c1}(\phi_{2t}, \varphi _{1t},\psi_{2t}) +\rho_{1}\bigl( \phi_{1}(0)-\phi_{2}(0)\bigr)\geq0, \\ &f_{c1}(\phi_{1t},\varphi_{1t}, \psi_{1t})-f_{c1}(\phi_{1t},\varphi _{2t}, \psi_{1t}) \\ &\quad =\phi_{1}(0) \bigl(r_{1}-a_{1} \varphi_{1}(0)-b_{1}\phi_{1}(0)\bigr)- \phi_{1}(0) \bigl(r_{1} -a_{1}\varphi_{2}(0)-b_{1} \phi_{1}(0)\bigr) \\ &\quad =-a_{1}\phi_{1}(0) \bigl(\varphi_{1}(0)- \varphi_{2}(0)\bigr)\leq0. \end{aligned} $$
(3.6)

For \(f_{c2}(\phi_{t},\varphi_{t},\psi_{t})\), we have

$$\begin{aligned}& f_{c2}(\phi_{1t},\varphi_{1t}, \psi_{1t})-f_{c2}(\phi_{2t},\varphi _{2t}, \psi_{1t}) \\& \quad =\varphi_{1}(0) \bigl(r_{2}-b_{2} \varphi_{1}(0)-a_{2}\psi _{1}(0)+a_{3} \phi_{1}(0)\bigr)-\varphi_{2}(0) \bigl(r_{2} -b_{2}\varphi_{2}(0) \\& \qquad {}-a_{2}\psi_{1}(0)+a_{3} \phi_{2}(0)\bigr) \\& \quad \geq r_{2}\bigl(\varphi_{1}(0)-\varphi_{2}(0) \bigr)-2b_{2}M_{2}\bigl(\varphi _{1}(0)- \varphi_{2}(0)\bigr) -a_{2}M_{3}\bigl( \varphi_{1}(0)-\varphi_{2}(0)\bigr) \\& \qquad {}+a_{3}\phi_{1}(0) \bigl(\varphi_{1}(0)- \varphi_{2}(0)\bigr) \\& \quad \geq(-a_{2}M_{3}-2b_{2}M_{2}) \bigl(\varphi_{1}(0)-\varphi_{2}(0)\bigr). \end{aligned}$$
(3.7)

Let \(\rho_{2}=a_{2}M_{3}+2b_{2}M_{2}>0\), then it is easy to see that

$$\begin{aligned}& f_{c2}(\phi_{1t},\varphi_{1t}, \psi_{1t})-f_{c2}(\phi_{2t},\varphi _{2t}, \psi_{1t}) +\rho_{2}\bigl(\varphi_{1}(0)- \varphi_{2}(0)\bigr)\geq0, \\& f_{c2}(\phi_{1t},\varphi_{1t}, \psi_{1t})-f_{c2}(\phi_{1t},\varphi _{1t}, \psi_{2t}) \\& \quad =\varphi_{1}(0) \bigl(r_{2}-b_{1} \varphi_{1}(0)-a_{2}\psi _{1}(0)+a_{3} \phi_{1}(0)\bigr)-\varphi_{1}(0) \bigl(r_{2} -b_{1}\varphi_{1}(0) \\& \qquad {}-a_{2}\psi_{2}(0)+a_{3} \phi_{1}(0)\bigr) \\& \quad =-a_{2}\varphi_{1}(0) \bigl(\psi_{1}(0)- \psi_{2}(0)\bigr). \end{aligned}$$
(3.8)

For \(f_{c3}(\phi_{t},\varphi_{t},\psi_{t})\), we have

$$\begin{aligned}& f_{c3}(\phi_{1t},\varphi_{1t}, \psi_{1t})-f_{c3}(\phi_{2t},\varphi _{2t}, \psi_{2t}) \\& \quad =\psi_{1}(0) \bigl(-\alpha-b_{3}\psi_{1}(0)+a_{4} \varphi_{1}(0)\bigr) \\& \qquad {}-\psi_{2}(0) \bigl(-\alpha-b_{3} \psi_{2}(0)+a_{4}\varphi_{2}(0)\bigr) \\& \quad \geq-\alpha \bigl(\psi_{1}(0)-\psi_{2}(0) \bigr)-2b_{3}M_{3}\bigl(\psi_{1}(0)- \psi_{2}(0)\bigr) \\& \quad =(-\alpha-2b_{3}M_{3}) \bigl(\psi_{1}(0)- \psi_{2}(0)\bigr). \end{aligned}$$
(3.9)

Let \(\rho_{3}=2b_{3}M_{3}+\alpha>0 \), then it is easy to see that

$$f_{c3}(\phi_{1t},\varphi_{1t}, \psi_{1t})-f_{c3}(\phi_{2t},\varphi _{2t}, \psi_{2t}) +\rho_{3}\bigl(\psi_{1}(0)- \psi_{2}(0)\bigr)\geq0. $$

This completes the proof of Lemma 3.1. □

We assume that \(c^{2} > c^{2}_{\ast}\triangleq\max\{ 4d_{1}r_{1}, 4d_{2}(r_{2}+a_{3}M_{1}), 4d_{3}(\alpha+a_{4}M_{2})\} \), \(c_{\ast}> 0\). Using this assumption, one can see that there exist \(\eta_{i}>0\) (\(i = 1, 2, 3\)) such that

$$\begin{aligned}& d_{1}\eta_{1}^{2}-c \eta_{1}+r_{1}=0, \\& d_{2}\eta_{2}^{2}-c \eta_{2}+r_{2}+a_{3}M_{1}=0, \\& d_{3}\eta_{3}^{2}-c\eta_{3}+ \alpha+a_{4}M_{2}=0. \end{aligned}$$
(3.10)

Assume that \(r_{1}>\max\{a_{1}k_{2}^{\ast}, a_{1}k_{1}^{\ast}\}\) and \(\alpha>\frac{b_{2}b_{3}k_{2}^{\ast}}{a_{2}}\), then one can choose positive constants \(\varepsilon_{5}\) and \(\varepsilon_{6}\) such that

$$ \begin{aligned} & a_{1}\bigl(\varepsilon_{5}-k_{2}^{\ast} \bigr)>b_{1}k_{1}^{\ast}, \qquad a_{2}\bigl( \varepsilon_{6}-k_{3}^{\ast}\bigr)>b_{2}k_{2}^{\ast}, \qquad \alpha >b_{3}\bigl(\varepsilon_{6}-k_{3}^{\ast} \bigr), \\ & r_{1}b_{2}>a_{1}a_{2}\bigl( \varepsilon_{6}-k_{3}^{\ast}\bigr), \qquad r_{2}+b_{2}\bigl(\varepsilon_{5}-k_{2}^{\ast} \bigr)>a_{2}k_{3}^{\ast}, \end{aligned} $$
(3.11)

and

$$ \begin{aligned} &\bigl(k_{1}^{\ast}+ \varepsilon_{1}\bigr) \bigl(r_{1}-a_{1} \bigl(k_{2}^{\ast}-\varepsilon _{5} \bigr)-b_{1}\bigl(k_{1}^{\ast}+\varepsilon_{1} \bigr)\bigr)< 0, \\ &\bigl(k_{2}^{\ast}+\varepsilon_{2}\bigr) \bigl(r_{2}-b_{2}\bigl(k_{2}^{\ast}+ \varepsilon _{2}\bigr)-a_{2}\bigl(k_{3}^{\ast}- \varepsilon_{6}\bigr)+a_{3}M_{1}\bigr)< 0, \\ &\bigl(k_{3}^{\ast}+\varepsilon_{3}\bigr) \bigl(- \alpha-b_{3}\bigl(k_{3}^{\ast }+\varepsilon_{3} \bigr)+a_{4}M_{2}\bigr)< 0, \\ &\bigl(k_{1}^{\ast}-\varepsilon_{4}\bigr) \bigl[r_{1}-a_{1}\bigl(k_{2}^{\ast}+ \varepsilon _{2}\bigr)-b_{1}\bigl(k_{1}^{\ast}- \varepsilon_{4}\bigr)\bigr]> 0, \\ &\bigl(k_{2}^{\ast}-\varepsilon_{5}\bigr) \bigl[r_{2}-b_{2}\bigl(k_{2}^{\ast}- \varepsilon _{5}\bigr)-a_{2}\bigl(k_{3}^{\ast}+ \varepsilon_{3}\bigr)\bigr]> 0, \\ &\bigl(k_{3}^{\ast}-\varepsilon_{6}\bigr) \bigl(- \alpha-b_{3}\bigl(k_{3}^{\ast }-\varepsilon_{6} \bigr)\bigr)> 0, \end{aligned} $$
(3.12)

for \(\varepsilon_{i}\) (\(i=1,2,3\)) being relatively big.

For the above constants and suitable constants \(\tilde{t}_{i}> 0\) (\(i = 1, 2, 3, 4, 5, 6\)) satisfying \(\tilde{t}_{5}<\min\{\tilde{t}_{1},\tilde{t}_{3}\}\), \(\tilde{t}_{2}>\max\{\tilde{t}_{4}, \tilde{t}_{6}\}\), we define the continuous functions \(\overline{\Phi}(t) = (\overline{\phi}(t), \overline{\varphi}(t), \overline{\psi}(t))\) and \(\underline{\Phi}(t) =(\underline{\phi}(t), \underline{\varphi}(t), \underline{\psi}(t))\) as follows:

$$\begin{aligned}& \overline{\phi}(t)=\left \{ \textstyle\begin{array}{l@{\quad}l} k_{1}^{\ast}e^{\eta_{1}t}, & t\leq\tilde{t}_{1}, \\ k_{1}^{\ast}+\varepsilon_{1}e^{-\eta t},& t>\tilde{t}_{1}, \end{array}\displaystyle \right . \qquad \overline{\varphi}(t)= \left \{ \textstyle\begin{array}{l@{\quad}l} k_{2}^{\ast}e^{\eta_{2}t}, & t\leq\tilde{t}_{2}, \\ k_{2}^{\ast}+\varepsilon_{2}e^{-\eta t},& t>\tilde{t}_{2}, \end{array}\displaystyle \right . \\& \overline{\psi}(t)=\left \{ \textstyle\begin{array}{l@{\quad}l} k_{3}^{\ast}e^{\eta_{3}t}, & t\leq\tilde{t}_{3}, \\ k_{3}^{\ast}+\varepsilon_{3}e^{-\eta t},& t>\tilde{t}_{3}, \end{array}\displaystyle \right .\qquad \underline{\phi}(t)= \left \{ \textstyle\begin{array}{l@{\quad}l} 0, & t\leq\tilde{t}_{4}, \\ k_{1}^{\ast}-\varepsilon_{4}e^{-\eta t},& t>\tilde{t}_{4}, \end{array}\displaystyle \right . \\& \underline{\varphi}(t)=\left \{ \textstyle\begin{array}{l@{\quad}l} 0, & t\leq\tilde{t}_{5}, \\ k_{2}^{\ast}-\varepsilon_{5}e^{-\eta t},& t>\tilde{t}_{5}, \end{array}\displaystyle \right . \qquad \underline{\psi}(t)= \left \{ \textstyle\begin{array}{l@{\quad}l} 0, & t\leq\tilde{t}_{6}, \\ k_{3}^{\ast}-\varepsilon_{6}e^{-\eta t},& t>\tilde{t}_{6}, \end{array}\displaystyle \right . \end{aligned}$$

where

$$ \begin{aligned} &\tilde{t}_{2}=\frac{1}{\eta_{2}}\ln \frac {r_{1}}{a_{1}k_{2}^{\ast}},\qquad \tilde{t}_{6}=\frac{1}{\eta_{2}}\ln \frac{a_{2}(\varepsilon _{6}-k_{3}^{\ast})}{b_{2}k_{2}^{\ast}}, \\ &\tilde{t}_{5}=\max \biggl\{ \frac{1}{\eta_{1}}\ln\frac {a_{1}(\varepsilon_{5}-k_{2}^{\ast})}{b_{1}k_{1}^{\ast}}, \frac{1}{\eta_{3}}\ln\frac{r_{2}+b_{2}(\varepsilon_{5}-k_{2}^{\ast })}{a_{2}k_{3}^{\ast}} \biggr\} . \end{aligned} $$
(3.13)

Lemma 3.2

Assume that \(r_{1}>\max\{a_{1}k_{2}^{\ast}, a_{1}k_{1}^{\ast}\}\) and \(\alpha>\frac{b_{2}b_{3}k_{2}^{\ast}}{a_{2}}\), then \((\overline{\phi},\overline{\varphi},\overline{\psi})\) is an upper solution of system (3.1).

Proof

If \(t\leq\tilde{t}_{5}<\tilde{t}_{1} \), then \(\overline{\phi}(t)=k_{1}^{\ast}e^{\eta_{1}t} \), and \(\underline{\varphi}(t)=0 \). Therefore, we have

$$\begin{aligned}& d_{1}\overline{\phi}''(t)-c \overline{\phi}'(t)+\overline{\phi}\bigl(r_{1}-a_{1} \underline{\varphi }(t)-b_{1}\overline{\phi}(t)\bigr) \\& \quad \leq \bigl(d_{1}\eta_{1}^{2}-c \eta_{1}+r_{1}\bigr)k_{1}^{\ast}e^{\eta_{1}t}=0. \end{aligned}$$
(3.14)

If \(t>\tilde{t}_{5}\) and \(t\leq\tilde{t}_{1}\), then \(\overline{\phi}(t)=k_{1}^{\ast}e^{\eta_{1} t}\) and \(\underline{\varphi}(t)=k_{2}^{\ast}-\varepsilon_{5}e^{-\eta t}\). Therefore, we obtain

$$ d_{1}\overline{\phi}''(t)-c \overline{\phi}'(t)+\overline{\phi}(t) \bigl(r_{1} -a_{1}\underline{\varphi}(t)-b_{1}\overline{\phi}(t) \bigr)=I_{1}(\eta), $$
(3.15)

where \(I_{1}(\eta)=k_{1}^{\ast}e^{\eta_{1} t}[a_{1}(\varepsilon_{5}e^{-\eta t}-k_{2}^{\ast})-b_{1}k_{1}^{\ast}e^{\eta_{1} t}]\), \(I_{1}(0)=k_{1}^{\ast}e^{\eta_{1} t}[a_{1}(\varepsilon_{5}-k_{2}^{\ast})-b_{1}k_{1}^{\ast}e^{\eta_{1} t}]< 0\) when \(t>\tilde{t}_{5}\). Therefore, there exists \(\eta_{1}^{\ast}>0\) such that \(d_{1}\overline{\phi}''(t)-c \overline{\phi}'(t)+\overline{\phi}(t)(r_{1} -a_{1}\underline{\varphi}(t)-b_{1}\overline{\phi}(t))\leq0\) for all \(\eta\in(0,\eta_{1}^{\ast})\).

If \(t > \tilde{t}_{1}> \tilde{t}_{5}\), then \(\overline{\phi}(t)=k_{1}^{\ast}+\varepsilon_{1}e^{-\eta t}\) and \(\underline{\varphi}(t)=k_{2}^{\ast}-\varepsilon_{5}e^{-\eta t}\). Therefore, we have

$$ d_{1}\overline{\phi}''(t)-c \overline{\phi}'(t)+\overline{\phi}(t) \bigl(r_{1} -a_{1}\underline{\varphi}(t)-b_{1}\overline{\phi}(t) \bigr)=I_{2}(\eta), $$
(3.16)

where \(I_{2}(\eta)=(d_{1}\varepsilon_{1}\eta^{2}+c\varepsilon_{1}\eta )e^{-\eta t}+(k_{1}^{\ast}+\varepsilon_{1}e^{-\eta t})(r_{1}-a_{1}(k_{2}^{\ast}-\varepsilon_{5}e^{-\eta t})-b_{1}(k_{1}^{\ast}+\varepsilon_{1}e^{-\eta t}))\). It follows from (3.12) that \(I_{2}(0)=(k_{1}^{\ast}+\varepsilon_{1})(r_{1}-a_{1}(k_{2}^{\ast }-\varepsilon_{5})-b_{1}(k_{1}^{\ast}+\varepsilon_{1}))< 0\). Therefore, there exists \(\eta_{2}^{\ast}>0\) such that \(d_{1}\overline{\phi}''(t)-c \overline{\phi}'(t)+\overline{\phi}(t)(r_{1} -a_{1}\underline{\varphi}(t)-b_{1}\overline{\phi}(t))\leq0\) for all \(\eta\in(0,\eta_{2}^{\ast})\).

If \(t\leq\tilde{t}_{6}< \tilde{t}_{2} \), then \(\overline{\varphi}(t)=k_{2}^{\ast}e^{\eta_{2}t} \), and \(\underline{\psi}(t)=0 \). It follows that

$$\begin{aligned}& d_{2}\overline{\varphi}''(t)-c \overline{\varphi}'(t)+f_{c2}\bigl(\overline{\phi}(t), \overline {\varphi}(t),\underline{\psi}(t)\bigr) \\& \quad \leq\bigl(d_{2}\eta_{2}^{2}-c \eta_{2}+r_{2}\bigr)k_{2}^{\ast}e^{\eta_{2}t} +k_{2}^{\ast}e^{\eta_{2}t}\bigl(-b_{2}k_{2}^{\ast}e^{\eta _{2}t}+a_{3}M_{1} \bigr) \\& \quad \leq \bigl(d_{2}\eta_{2}^{2}-c \eta_{2}+r_{2}+a_{3}M_{1} \bigr)k_{2}^{\ast}e^{\eta_{2}t}=0. \end{aligned}$$
(3.17)

If \(t>\tilde{t}_{6}\) and \(t\leq\tilde{t}_{2}\), then \(\overline{\varphi}(t)=k_{2}^{\ast}e^{\eta_{2} t}\) and \(\underline{\psi}=k_{3}^{\ast}-\varepsilon_{6}e^{-\eta t}\). By calculating, we have

$$ d_{2}\overline{\varphi}''(t)-c \overline{\varphi}'(t)+f_{c2}\bigl(\overline{\phi}(t), \overline{\varphi}(t),\underline{\psi}(t)\bigr)\leq I_{3}(\eta), $$
(3.18)

where \(I_{3}(\eta)=k_{2}^{\ast}e^{\eta_{2} t}(a_{2}(\varepsilon_{6}e^{-\eta t}-k_{3}^{\ast})-b_{2}k_{2}^{\ast}e^{\eta_{2} t})\). Thus, \(I_{3}(0)=k_{2}^{\ast}e^{\eta_{2} t}(a_{2}(\varepsilon_{6} -k_{3}^{\ast})-b_{2}k_{2}^{\ast}e^{\eta_{2} t})< 0\) when \(t>\tilde{t}_{6}\). Hence, there exists \(\eta_{3}^{\ast}>0\) such that \(d_{2}\overline{\varphi}''(t)-c \overline{\varphi}'(t)+f_{c2}(\overline{\phi}(t), \overline{\varphi}(t),\underline{\psi}(t))\leq0\) for all \(\eta\in(0,\eta_{3}^{\ast})\).

If \(t>\tilde{t}_{2} \), then \(\overline{\varphi}(t)=k_{2}^{\ast}+\varepsilon_{2}e^{-\eta t} \) and \(\underline{\psi}(t)=k_{3}^{\ast}-\varepsilon_{6}e^{-\eta t} \). By calculating, we have

$$ d_{2} \overline{\varphi}''(t)-c \overline{\varphi}'(t)+f_{c2}\bigl(\overline{\phi}(t), \overline{\varphi}(t),\underline{\psi}(t)\bigr)\leq I_{4}(\eta), $$
(3.19)

where \(I_{4}(\eta)=(d_{2}\varepsilon_{2}\eta^{2}+c\varepsilon_{2}\eta )e^{-\eta t}+(k_{2}^{\ast}+\varepsilon_{2}e^{-\eta t})(r_{2}+a_{3}M_{1}-b_{2}(k_{2}^{\ast}+\varepsilon_{2}e^{-\eta t})-a_{2}(k_{3}^{\ast}-\varepsilon_{6}e^{-\eta t}))\). It follows from (3.12) that \(I_{4}(0)=(k_{2}^{\ast}+\varepsilon _{2})(r_{2}+a_{3}M_{1}-b_{2}(k_{2}^{\ast}+\varepsilon_{2}) -a_{2}(k_{3}^{\ast}-\varepsilon_{6}))< 0\). Hence, there exists \(\eta_{4}^{\ast}>0\) such that \(d_{2}\overline{\varphi}''(t)-c \overline{\varphi}'(t)+f_{c2}(\overline{\phi}(t), \overline{\varphi}(t),\underline{\psi}(t))\leq0\) for all \(\eta\in(0,\eta_{4}^{\ast})\).

If \(t\leq\tilde{t}_{3}\), then \(\overline{\psi}(t)=k_{3}^{\ast}e^{\eta_{3}t}\). By calculating, we have

$$\begin{aligned}& d_{3}\overline{\psi}''(t)-c \overline{\psi}'(t)+f_{c3}\bigl(\overline{\phi}(t), \overline{\varphi }(t),\overline{\psi}(t)\bigr) \\& \quad \leq\bigl(d_{3}\eta_{3}^{2}-c \eta_{3}\bigr)k_{3}^{\ast}e^{\eta_{3}t} +k_{3}^{\ast}e^{\eta_{3}t}\bigl(-\alpha-b_{3}k_{3}^{\ast}e^{\eta _{3}t}+a_{4}M_{2} \bigr) \\& \quad \leq \bigl(d_{3}\eta_{3}^{2}-c \eta_{3}-\alpha+a_{4}M_{2}\bigr)k_{3}^{\ast}e^{\eta_{3}t}=0. \end{aligned}$$
(3.20)

If \(t>\tilde{t}_{3}\), then \(\overline{\psi}(t)=k_{3}^{\ast}+\varepsilon_{3}e^{-\eta_{3}t}\). By calculating, we get

$$ d_{3}\overline{\psi}''(t)-c \overline{\psi}'(t)+f_{c3}\bigl(\overline{\phi}(t), \overline{\varphi}(t),\overline{\psi}(t)\bigr)\leq I_{5}(\eta), $$
(3.21)

where \(I_{5} (\eta)=(d_{3}\varepsilon_{3}\eta^{2}+c\varepsilon_{3}\eta)e^{-\eta t}+(k_{3}^{\ast}+\varepsilon_{3}e^{-\eta t})(-\alpha-b_{3}(k_{3}^{\ast}+\varepsilon_{2}e^{-\eta t})+a_{4}M_{2})\). It follows from (3.12) that \(I_{5}(0)=(k_{3}^{\ast}+\varepsilon_{3})(-\alpha-b_{3}(k_{3}^{\ast }+\varepsilon_{3}) +a_{4}M_{2})< 0\). Hence, there exists \(\eta_{5}^{\ast}>0\) such that \(d_{3}\overline{\psi}''(t)-c \overline{\psi}'(t)+f_{c3}(\overline{\phi}(t), \overline{\varphi}(t),\underline{\psi}(t))\leq0\) for all \(\eta\in(0,\eta_{5}^{\ast})\).

Finally, for any \(\eta\in(0,\min\{\eta_{1}^{\ast}, \eta_{2}^{\ast}, \eta_{3}^{\ast}, \eta_{4}^{\ast}, \eta_{5}^{\ast} \})\), we see that (3.3) holds. This completes the proof. □

Lemma 3.3

Assume that \(r_{1}>\max\{a_{1}k_{2}^{\ast}, a_{1}k_{1}^{\ast}\}\) and \(\alpha>\frac{b_{2}b_{3}k_{2}^{\ast}}{a_{2}}\), then \((\underline{\phi}(t), \underline{\varphi}(t), \underline{\psi}(t))\) is a pair of lower solution of system (3.1).

Proof

If \(t\leq\tilde{t}_{4}\), then \(\underline{\phi}(t)=0\). We have \(d_{1}\underline{\phi}''(t)-c\underline{\phi}'(t)+\underline{\phi}(t) (r_{1}-a_{1}\overline{\varphi}(t)-b_{1}\underline{\phi}(t))=0\).

If \(\tilde{t}_{4}< t\leq\tilde{t}_{2}\), then \(\underline{\phi}(t)=k_{1}^{\ast}-\varepsilon_{4}e^{-\eta t}\), \(\overline{\varphi}=k_{2}^{\ast}e^{\eta_{2} t}\). We have

$$\begin{aligned}& d_{1}\underline{\phi}''(t)-c \underline{\phi}'(t)+\underline{\phi}(t) \bigl(r_{1}-a_{1} \overline{\varphi}(t)-b_{1}\underline{\phi}(t)\bigr) \\& \quad = (-d_{1}\eta-c)\varepsilon_{4}\eta e^{-\eta t}+ \bigl(k_{1}^{\ast}-\varepsilon_{4}e^{-\eta t} \bigr) \bigl(r_{1}-a_{1}k_{2}^{\ast}e^{\eta_{2} t}-b_{1} \bigl(k_{1}^{\ast}-\varepsilon_{4}e^{-\eta t} \bigr)\bigr) \\& \quad \triangleq I_{6}(\eta). \end{aligned}$$
(3.22)

Using (3.13), we see that \(I_{6}(0)=(k_{1}^{\ast}-\varepsilon_{4})(r_{1}-a_{1}k_{2}^{\ast }e^{\eta_{2}t} - b_{1}(k_{1}^{\ast}-\varepsilon_{4}))> 0\) when \(\tilde{t}_{4}< t\leq\tilde{t}_{2}\). Therefore, there exists \(\eta_{6}^{\ast}\) such that \(d_{1}\underline{\phi}''(t)-c\underline{\phi}'(t)+\underline{\phi}(t) (r_{1}-a_{1}\underline{\varphi}(t)-b_{1}\underline{\phi}(t))\geq0\) for \(\eta\in(0,\eta_{6}^{\ast})\).

If \(t>\tilde{t}_{2}\), then \(\underline{\phi}(t)=k_{1}^{\ast}-\varepsilon_{4}e^{-\eta t}\), \(\overline{\varphi}=k_{2}^{\ast}+\varepsilon_{2} e^{-\eta t}\). We have

$$\begin{aligned}& d_{1}\underline{\phi}''(t)-c \underline{\phi}'(t)+\underline{\phi}(t) \bigl(r_{1}-a_{1} \overline{\varphi}(t)-b_{1}\underline{\phi}(t)\bigr) \\& \quad \geq(-d_{1}\eta-c)\varepsilon_{4}\eta e^{-\eta t}+\bigl(k_{1}^{\ast}-\varepsilon_{4}e^{-\eta t} \bigr) \bigl(r_{1}-a_{1}\bigl(k_{2}^{\ast}+ \varepsilon_{2}e^{-\eta t}\bigr)-b_{1} \bigl(k_{1}^{\ast}-\varepsilon_{4}e^{-\eta t} \bigr)\bigr) \\& \quad \triangleq I_{7}(\eta). \end{aligned}$$
(3.23)

Using (3.12), we see that \(I_{7}(0)=(k_{1}^{\ast}-\varepsilon_{4})(r_{1}-a_{1}(k_{2}^{\ast} + \varepsilon_{2})- b_{1}(k_{1}^{\ast}-\varepsilon_{4}))> 0\). Therefore, there exists \(\eta_{7}^{\ast}\) such that \(d_{1}\underline{\phi}''(t)-c\underline{\phi}'(t)+\underline{\phi}(t) (r_{1}-a_{1}\underline{\varphi}(t)-b_{1}\underline{\phi}(t))\geq0\) for \(\eta\in(0,\eta_{7}^{\ast})\).

If \(t\leq\tilde{t}_{5}\), then \(\underline{\varphi}=0\). Therefore we have \(d_{2}\underline{\varphi}''(t)-c\underline{\varphi }'+f_{c2}(\underline{\phi}(t), \underline{\varphi}(t),\overline{\psi}(t))=0\).

If \(\tilde{t}_{5}< t\leq\tilde{t}_{3}\), then \(\underline{\varphi}(t) =k_{2}^{\ast}-\varepsilon_{5}e^{-\eta t}\), \(\overline{\psi}=k_{3}^{\ast}e^{\eta_{3}t}\). We get

$$\begin{aligned}& d_{2}\underline{\varphi}''(t)-c \underline{\varphi }'+f_{c2}\bigl(\underline{\phi}(t), \underline{\varphi}(t),\overline{\psi}(t)\bigr) \\& \quad = (-d_{2}\eta-c)\varepsilon_{5}\eta e^{-\eta t}+ \bigl(k_{2}^{\ast}-\varepsilon_{5}e^{-\eta t} \bigr) \bigl(r_{2}-b_{2}\bigl(k_{2}^{\ast}- \varepsilon_{5}e^{-\eta t}\bigr)-a_{2}k_{3}^{\ast}e^{\eta_{3} t} \bigr) \\& \quad \triangleq I_{8}(\eta). \end{aligned}$$
(3.24)

Using (3.13), we see that \(I_{8}(0)=(k_{2}^{\ast}-\varepsilon_{5})(r_{2}-b_{2}(k_{2}^{\ast} - \varepsilon_{5})- a_{2}k_{3}^{\ast}e^{\eta_{3}t})> 0\) when \(t>\tilde{t}_{5}\). Hence, there exists \(\eta_{8}^{\ast}\) such that \(d_{2}\underline{\varphi}''(t)-c\underline{\varphi }'(t)+f_{c2}(\underline{\phi}(t), \underline{\varphi}(t),\overline{\psi}(t))\geq0\) for all \(\eta\in (0,\eta_{8}^{\ast})\).

If \(t>\tilde{t}_{3}\), then \(\underline{\varphi}(t) =k_{2}^{\ast}-\varepsilon_{5}e^{-\eta t}\). We get

$$\begin{aligned} \begin{aligned}[b] &d_{2}\underline{\varphi}''(t)-c \underline{\varphi }'+f_{c2}\bigl(\underline{\phi}(t), \underline{\varphi}(t),\overline{\psi}(t)\bigr) \\ &\quad \geq(-d_{2}\eta-c)\varepsilon_{5}\eta e^{-\eta t}+\bigl(k_{2}^{\ast}-\varepsilon_{5}e^{-\eta t} \bigr) \bigl(r_{2}-b_{2}\bigl(k_{2}^{\ast}- \varepsilon_{5}e^{-\eta t}\bigr)-a_{2} \bigl(k_{3}^{\ast}+\varepsilon_{3}e^{-\eta t} \bigr)\bigr) \\ &\quad \triangleq I_{9}(\eta). \end{aligned} \end{aligned}$$
(3.25)

Using (3.12), we see that \(I_{9}(0)=(k_{2}^{\ast}-\varepsilon_{5})(r_{2}-b_{2}(k_{2}^{\ast} - \varepsilon_{5})- a_{2}(k_{3}^{\ast}+\varepsilon_{3}))> 0\). Hence, there exists \(\eta_{9}^{\ast}\) such that \(d_{2}\underline{\varphi}''(t)-c\underline{\varphi }'(t)+f_{c2}(\underline{\phi}(t), \underline{\varphi}(t),\overline{\psi}(t))\geq0\) for all \(\eta\in (0,\eta_{9}^{\ast})\).

If \(t\leq\tilde{t}_{6}\), then \(\underline{\psi}=0\). Therefore, we have \(d_{3}\underline{\psi}''(t)-c\underline{\psi}'+f_{c3}(\underline {\phi}(t), \underline{\varphi}(t),\underline{\psi}(t))=0\).

If \(t>\tilde{t}_{6}\), then \(\underline{\psi}(t) =k_{3}^{\ast}-\varepsilon_{6}e^{-\eta t}\). Hence, we get

$$\begin{aligned}& d_{3}\underline{\psi}''(t)-c \underline{\psi}'+f_{c3}\bigl(\underline {\phi}(t), \underline{\varphi}(t),\overline{\psi}(t)\bigr) \\& \quad \geq(-d_{3}\eta-c)\varepsilon_{6}\eta e^{-\eta t}+\bigl(k_{3}^{\ast}-\varepsilon_{6}e^{-\eta t} \bigr) \bigl(-\alpha-b_{3}\bigl(k_{3}^{\ast}- \varepsilon_{6}e^{-\eta t}\bigr)\bigr) \\& \quad \triangleq I_{10}(\eta). \end{aligned}$$
(3.26)

Using (3.12), we see that \(I_{10}(0)=(k_{3}^{\ast}-\varepsilon_{6})(-\alpha-b_{3}(k_{3}^{\ast }-\varepsilon_{6}))> 0\). Hence, there exists \(\eta_{10}^{\ast}\) such that \(d_{3}\underline{\psi}''(t)-c\underline{\psi}'(t)+f_{c3}(\underline {\phi}(t), \underline{\varphi}(t),\overline{\psi}(t))\geq0 \) for all \(\eta \in(0,\eta_{10}^{\ast})\).

Finally, for any \(\eta\in(0, \min\{\eta_{6}^{\ast}, \eta_{7}^{\ast}, \eta_{8}^{\ast}, \eta_{9}^{\ast}, \eta_{10}^{\ast}\}) \), we see that (3.4) holds. This completes the proof. □

By using Lemmas 3.1-3.3, we have the following conclusion.

Theorem 3.1

Assume that \(r_{1}>\max\{a_{1}k_{2}^{\ast}, a_{1}k_{1}^{\ast}\}\) and \(\alpha>\frac{b_{2}b_{3}k_{2}^{\ast}}{a_{2}}\), then, for any \(c > c^{\ast} > 0\), system (1.1) always has a traveling wave solution with speed c connecting the trivial steady state \((0, 0)\) and the positive steady state \((k_{1}^{\ast}, k_{2}^{\ast}, k_{3}^{\ast})\).

4 Results and discussion

In this article, we investigate a food chain reaction-diffusion predator-prey systems with nonlocal delay in a bounded domain with no flux boundary condition incorporating delay respecting gestation of the predators. By using the methods of the Lyapunov function, we prove global stability of positive equilibrium of the system (1.1). If the nonlocal delay terms are replaced by the general terms without delay, one can easily verify that the positive constant solution of (1.1) is globally stable without any condition when a positive constant equilibrium exists.

The above result shows that if the intra-specific competitions of the predators and preys dominate their inter-specific interaction, then the unique positive equilibrium is globally stable if it exists, which implies that predators and preys are permanent from biologic view. If the nonlocal delay terms are replaced by a general term without delay, without any condition the positive equilibrium is globally stable when it exists. This implies that nonlocal delays often impact the global stability of a positive constant solution. By using the method of the upper-lower solutions, we also see that there exists a traveling wavefront connecting the zero solution to the positive equilibrium of the system when the wave speed is large enough and the prey intrinsic growth rate and the death rate of predator are relatively big.

In modern mathematics, the theory and methods of traveling waves solutions develop quickly and they have attracted much attention due to their significance in the real word. We would like to extend this theory and these methods to the integral equation, and we refer to [22–26], but this question is still an open problem.