1 Introduction

The predator–prey model is one of the basic models between different species in nature which has been widely researched [16]. In the natural world, the population generally has a saturation effect. Based on experiments, in order to explain the phenomenon of predation for three different kinds of species, Holling [7] proposed different kinds of functional responses. One of the most important functional responses is Holling type II. For example, Xu [8] discussed the global dynamics of a delayed predator–prey model with stage structure and Holling type II functional response for the predator. By constructing a suitable Lyapunov functional, the permanence and global asymptotic stability of the model were derived. Scholars have obtained a large number of interesting results about predator–prey models associated with Holling type II functional response [911].

Allee [12] discussed both positive and negative interactions among species, and proposed the concept of the Allee effect. This effect can be caused by difficulties in mate finding [13], inbreeding depression [14], defense to avoid predators, social dysfunction at a low-densities [15]. Moreover, the effect is called to be a weak Allee effect if the density at zero per capita growth is positive, a strong Allee effect if the low-density per capita growth is negative. In the past decades, the Allee effect has always been introduced to explain some important biological phenomena by many mathematicians and ecologists [1625]. It was shown that the low density population can affect the birth rate of the species [26], but the coefficients of the growth rate are irrelevant to the Allee-type function. Hence, let \(F(x)\) be the fertility rate of a species x [27]:

$$ F(x)=\frac{ax}{A+x}, $$

where a denotes the per capita maximum fertility rate of species x; A denotes the Allee effect constant of the species. If \(A>0\), the fertility rate of the species is zero if \(x=0\) and approaches to a if x is large enough. The value of the parameter A determines the growth rate of \(F(x)\). When \(A=0\), the fertility rate \(F(x)=a\) is density independent. Therefore, when considering Allee effect, the logistic equation can be rewritten in the following form:

$$ \dot{x} = x \biggl( \frac{ax}{A+x}-d-bx \biggr) , $$
(1)

where d represents the death rate of the species; the intra-specific competition intensity of the species is represented by b; \(ax/(A+x)\) is a Michaelis–Menten type function. Obviously, when \(A=0\), system (1) is reduced to the traditional logistic equation.

Zu [28] studied a predator–prey system with Allee effect as follows:

$$ \begin{aligned} &\dot{x} = x \biggl( \frac{ax}{A+x}-d-bx \biggr) -mxy, \\ &\dot{y} = nmxy-{\delta }y, \end{aligned} $$
(2)

and investigated the local asymptotic stability of its equilibria. Meanwhile the author found that the Allee effect of the prey population can bring about unstable or stable periodic fluctuations. Further, Zu and Mimura [29] introduced the Holling type II functional response into the above system and proposed a system as follows:

$$ \begin{aligned} &\dot{x} = x \biggl( \frac{ax}{A+x}-d-bx \biggr) - \frac{mxy}{1+hx}, \\ &\dot{y} = \frac{nmxy}{1+hx}-{\delta }y. \end{aligned} $$
(3)

The local asymptotic stability of the equilibria was investigated. Also an explicit algorithm was obtained to determine the direction of Hopf bifurcations as well as the stability of the periodic solutions. The phenomenon of periodicity in [29] is similar to that in [28].

In most of ecosystems, maturation, pregnancy, and hunting occur all the time. Hence, time delay due to gestation has been greatly researched as a focus issue in a predator–prey system [25, 3032], since the current birth rate of the predator is related to its consumption of prey throughout the past history. Chen and Zhang [33] proposed a predator–prey model incorporating delay and predator migration, which can be used to describe biological control. By the study of the existence of stable equilibria, they showed that delay can change the stability of the positive equilibrium. But when the positive equilibrium is unstable, a Hopf bifurcation occurs. Yuan et al. [34] investigated the dynamics of a delayed logistic model with both impulsive and stochastic perturbations. They obtained sufficient conditions for the extinction and global attractivity of the system. Song et al. [35] studied a ratio-dependent predator–prey model with stage-structure and discussed the stability of the positive equilibrium. Chen and Chen [36], Wang and Chen [37] considered the bifurcation phenomenon of a ratio-dependent system incorporating a prey refuge.

However, still seldom scholars consider the dynamic behaviors of a delayed predator–prey system with Allee effect in prey and Holling type II function response. Motivated by the above papers, the main purpose of this paper is to study the stability and Hopf bifurcation of system (3) with delay. More precisely, we study the following model:

$$ \begin{aligned} &\dot{x} = x \biggl( \frac{ax}{A+x}-d-bx \biggr) - \frac{mxy}{1+hx}, \\ & \dot{y} = \frac{nmx{(t-\tau )}y{(t-\tau )}}{1+hx{(t-\tau )}}-{\delta }y, \end{aligned} $$
(4)

where x and y respectively denote the population densities of prey and predators; A, a, b, d, n, m, h, δ are positive constants; τ demonstrates the time delay because of the gestation of the predator; m and h measure the effects of capture rate and handling time, respectively; n is the food conversion rate of the predator; δ is the death rate of the predator; \((mxy)/(1+hx)\) is the Holling type II functional response. System (4) is restricted to the following initial conditions:

$$ x(\theta ) = \phi {(\theta )}\geq 0,\qquad y(\theta ) = \psi {(\theta )}\geq 0, \quad {\theta }\in [-\tau ,0],\phi (0)>0,\psi (0)>0, $$
(5)

where \(\phi {(\theta )}\), \(\psi {(\theta )}\) are continuous bounded functions in the interval \([-\tau ,0]\).

The rest of this paper is organized as follows: The dynamic behaviors of model (1) and the existence of equilibria of system (4) are derived in the next section. In Sect. 3, we study the local asymptotic stability and Hopf bifurcation of system (4). We end this paper with some examples and a brief discussion.

2 Boundedness and existence of equilibria

In this section, we shall present some preliminary results. Firstly, we study the dynamical behavior of system (1).

Define

$$ a_{1} = Ab+d+2\sqrt{Abd}. $$

Theorem 2.1

Let \(x(t)\) be a positive solution of system (1) with the initial value \(x(0)>0\).

  1. (1)

    If \(a< a_{1}\), then \(\lim_{t\to +\infty }x(t)=0\).

  2. (2)

    If \(a> a_{1}\), then we obtain the following results:

    1. (i)

      For \(0< x(0)< x_{2}\), \(\lim_{t\to +\infty }x(t)=0\);

    2. (ii)

      For \(x_{2}< x(0)\), \(\lim_{t\to +\infty }x(t)=x_{1}\), where \(x_{1}\) and \(x_{2}\) are defined below.

Proof

Note that the equation of model (1) can be rewritten as

$$\begin{aligned} \dot{x} &= x \biggl( \frac{ax}{A+x}-d-bx \biggr) \\ &= \frac{x}{A+x} \bigl[ -bx^{2}+ ( a-Ab-d ) x-Ad \bigr] . \end{aligned}$$
(6)

Denote

$$ f(x) = -bx^{2}+ ( a-Ab-d ) x-Ad. $$
(7)

Its discriminant is of the form

$$ \Delta = ( a-Ab-d ) ^{2}-4Abd = a^{2}-2 ( Ab+d ) a+ ( Ab-d ) ^{2} \overset{\mathrm{def}}{=}Q_{1}. $$
(8)

Clearly, when \(Q_{1}=0\), we have

$$ a_{0} = Ab+d-2\sqrt{Abd}\quad \mbox{and} \quad a_{1} = Ab+d+2\sqrt{Abd}. $$

It is easily seen from Eqs. (7), (8) that if \(a \leq Ab+d\) and \(x > 0\), then \(f(x) < 0\); if \(Ab+d < a < a_{1}\) and \(x > 0\), then \(f(x) < 0\). Further, if \(a > a_{1}\), then \(Q_{1} > 0\), hence, \(f(x) = 0\) has two positive roots

$$ x_{1}= \frac{k+\sqrt{Q_{1}}}{2b}, \qquad x_{2}= \frac{k-\sqrt{Q_{1}}}{2b}, $$
(9)

where \(k = a- ( Ab+d ) \). If \(0 < x < x_{2}\), then \(f(x) < 0\); if \(x_{2} < x < x_{1}\), then \(f(x) > 0\); if \(x_{1} < x\), then \(f(x) < 0\). The proof is complete. □

Remark 2.1

If \(A=0\), then system (1) reduces to the traditional logistic equation \(\dot{x} = x ( a-d-bx ) \). For this logistic equation, if \(a>d\), then every positive solution will tend to \(\frac{a-d}{b}\) monotonically; if \(a\leq d\), then every positive solution will tend to 0. Note that \(a\leq d\) implies that \(a < a_{1}\), then every positive solution of system (1) will tend to 0. Therefore, for system (1), the species is extinct when the birth rate is less than the death rate. However, we find that when the death rate is smaller than the birth rate, the species is also extinct when \(d< a < a_{1}\) due to the influence of Allee effect. Hence only when the birth rate a is larger than \(a_{1}\), the species maybe not extinct. This shows that the Allee effect may deduce the instability of system (1).

In (1), assume that \(A=0.5\), \(d=2\), \(b=5\), then \(a_{1}=8.9721\).

  1. (i)

    If \(a=6\), that is, \(a< a_{1}\), then \(\lim_{t\to +\infty }x(t)=0\) (see Fig. 1).

    Figure 1
    figure 1

    Dynamic behaviors of system (1) with \(A=0.5\), \(d=2\), \(b=5\), and \(a=6\)

  2. (ii)

    If \(a=10\), that is, \(a>a_{1}\), further if \(0< x(0)<0.2298\), then \(\lim_{t\to +\infty }x(t)=0\); if \(0.2298< x(0)\), then \(\lim_{t\to +\infty }x(t)=0.8702\) (see Fig. 2).

    Figure 2
    figure 2

    Dynamic behaviors of system (1) with \(A=0.5\), \(d=2\), \(b=5\), and \(a=10\)

In the following, we give the positivity and boundedness of a solution of system (4).

Lemma 2.2

Every solution of system (4) with the initial condition (5) is positive and ultimately bounded for all \(t\geq 0\).

Proof

It can be easily proved that every solution of system (4) with (5) remains positive for all \(t\geq 0\). Let \(V(t)=nx(t)+y(t-\tau )\), calculating the derivative of \(V(t)\) with respect to t along the positive solution of system (4), we then have

$$\begin{aligned} \dot{V}(t) &= n\dot{x}(t)+\dot{y}(t-\tau ) \\ &= -\delta y(t+\tau )+nx(t) \biggl(\frac{ax(t)}{A+x(t)}-d-bx(t) \biggr) \\ &= -\delta V(t)+n{\delta }x(t)+nx(t) \biggl(\frac{ax(t)}{A+x(t)}-d-bx(t) \biggr). \end{aligned}$$

By Theorem 2.1, there exist some positive constants B and T such that \(\dot{V}(t) \leq B-{\delta }V(t)\) for all \(t \geq T\). Thus \(\overline{\lim_{t\to +\infty }}V(t) \leq \frac{\delta }{B}\); consequently, \(x(t)\) and \(y(t)\) are ultimately bounded. The proof is complete. □

Let \(\dot{x}=\dot{y}=0\) in system (4), then

$$ \textstyle\begin{cases} x ( \frac{ax}{A+x}-d-bx ) - \frac{mxy}{1+hx}=0, \\ \frac{nmxy}{1+hx}-{\delta }y=0. \end{cases} $$
(10)

There always exists a trivial equilibrium \(E_{0}(0,0)\).

If \(a>a_{1}\), then system (4) has two boundary equilibria \(E_{1}(x_{1},0)\) and \(E_{2}(x_{2},0)\), where

$$ x_{1}= \frac{k+\sqrt{Q_{1}}}{2b}\quad \mbox{and}\quad x_{2}= \frac{k-\sqrt{Q_{1}}}{2b} $$

with \(k=a- ( Ab+d )\), \(Q_{1} = a^{2}-2 ( Ab+d ) a+ ( Ab-d ) ^{2}\).

Further, if

$$ Q_{0}>0\quad \mbox{and}\quad a>a_{2}, $$
(11)

then system (4) has a positive equilibrium

$$ E^{*} = \bigl( x^{*},y^{*} \bigr) = \biggl( \frac{\delta }{Q_{0}}, \frac{1+hx^{*}}{m ( A+x^{*} ) } \bigl[ax^{*}- \bigl( A+x^{*} \bigr) \bigl( d+bx^{*} \bigr) \bigr] \biggr), $$

where

$$\begin{aligned} \begin{aligned} &Q_{0} = nm-{\delta }h, \\ &a_{2} = Ab+d+ \frac{Ad}{x^{*}}+bx^{*}. \end{aligned} \end{aligned}$$
(12)

Obviously,

$$ a_{2} = Ab+d+ \frac{Ad}{x^{*}}+bx^{*} \geq Ab+d+2 \sqrt{Abd} = a_{1}, $$
(13)

where \(a_{2}=a_{1}\) if and only if \(A= \frac{b(x^{*})^{2}}{d}\).

The above analysis can be summarized in Table 1.

Table 1 Equilibria of system (4)

Define

$$ A_{1}=\frac{a+d-2\sqrt{ad}}{b},\qquad A_{2}= \frac{(a-d-bx^{*})x^{*}}{bx^{*}+d} \quad \mbox{and} \quad x_{0}^{*}= \frac{\sqrt{d}(\sqrt{a}-\sqrt{d})}{b}. $$

If \(a\leq d\), it follows from Table 1 that there exists only one trivial equilibrium \(E_{0}\) of system (4) for all A. Hence, in the following, we only consider \(a>d\). When \(Q_{0}\leq 0\), there does not exist a positive equilibrium \(E^{*}\) of system (4), so we assume that \(Q_{0}>0\), that is, \(x^{*}>0\). Note that \(a>a_{1}\) is equivalent to \(a>d\) and \(A< A_{1}\), and \(a>a_{2}\) is equivalent to \(a>d\) and \(A< A_{2}\). When \(x^{*}\geq \frac{a-d}{b}\), \(A_{2}\leq 0\), which means that system (4) does not admit a positive equilibrium \(E^{*}\). Note that \(A_{1}>0\). Hence, to ensure \(A_{2}>0\), we must assume that \(0< x^{*}< \frac{a-d}{b}\). By simple computation, we have

$$ A_{1}-A_{2}=\frac{[bx^{*}-\sqrt{d}(\sqrt{a}-\sqrt{d})]^{2}}{b(bx^{*}+d)} \geq 0, \quad \mbox{for all } 0< x^{*}< \frac{a-d}{b}, $$

particularly \(A_{1}=A_{2}\) if and only if \(x^{*}=x_{0}^{*}\). We summarize the above results in Table 2.

Table 2 Equilibria of system (4)

Tables 1 and 2 show that if the birth rate a is relatively small or the Allee effect A is relatively large, then system (4) only has one trivial equilibrium \(E_{0}\), that is, the predator and prey will be extinct. If the birth rate a is relatively large or the Allee effect A is relatively small, then system (4) has a positive equilibrium \(E^{*}\), which guarantees the coexistence of system (4).

3 Local stability and Hopf bifurcation

3.1 \(E_{0}=(0,0)\)

Firstly, we discuss the stability of \(E_{0}\). We obtain the following theorems.

Theorem 3.1

\(E_{0}\) of system (4) is locally asymptotically stable.

Proof

The variational matrix of system (4) at \(E_{0}\) is

$$ V(E_{0})=\left [\textstyle\begin{array}{@{}c@{\quad}c@{}} \lambda +d&0\\ 0&\lambda +\delta \end{array}\displaystyle \right ] . $$

Clearly, the characteristic equation of the equilibrium point \(E_{0}\) always has two negative real roots: \(\lambda _{1} = -d\), \(\lambda _{2} = -\delta \). The proof is complete. □

Therefore, \(E_{0}\) is always a locally stable node, which implies that both predators and prey will become extinct when their population densities lie in the attraction region of \(E_{0}\). In particular, if the population density of prey becomes low, then both prey and predators will become extinct.

According to Theorem 2.1 and the comparison theorem, it is easy to prove the following theorem.

Theorem 3.2

Assume that \(a < a_{1}\), then \(E_{0}\) of system (4) is globally asymptotically stable.

Remark 3.1

When \(a\leq d\), it follows from Theorem 3.2 that \(E_{0}\) of system (4) is globally asymptotically stable for any value of the Allee effect. It is easy to show that the condition \(a< a_{1}\) of Theorem 3.2 is equivalent to \(a>d\) and \(A> A_{1}\), which implies that both prey and predators will go extinct if the Allee effect is strong. Also, we can see that the Allee effect of prey species increases the extinction risk of both predators and prey. Only when the Allee effect is small, the species may not be extinct.

3.2 \(E_{1}=(x_{1},0)\)

We study the local stability of the boundary equilibrium \(E_{1}(x_{1},0)\) and have the following theorem.

Theorem 3.3

  1. (1)

    Assume that \(Q_{0}>0\) and \(A<\frac{b(x^{*})^{2}}{d}\) hold.

    1. (i)

      If \(a_{1}< a< a_{2}\), then \(E_{1}\) of system (4) is a locally asymptotically stable equilibrium point.

    2. (ii)

      If \(a>a_{2}\), then \(E_{1}\) of system (4) is unstable.

  2. (2)

    Assume that \(Q_{0}>0\) and \(A \geq \frac{b(x^{*})^{2}}{d}\) hold. If \(a>a_{1}\), then \(E_{1}\) of system (4) is unstable.

  3. (3)

    Assume that \(Q_{0}\leq 0\) holds. If \(a>a_{1}\), then \(E_{1}\) of system (4) is locally asymptotically stable.

Proof

The variational matrix of system (4) at the equilibrium point \(E_{1}\) is

$$ V(E_{1})= \left [ \textstyle\begin{array}{@{}c@{\quad}c@{}} \lambda -\frac{2Aax_{1}+ax_{1}^{2}}{ ( A+x_{1} ) ^{2}}+d+2bx_{1}& \frac{mx_{1}}{1+hx_{1}}\\ 0& \lambda +\delta -\frac{nmx_{1}e^{-\lambda {\tau }}}{1+hx_{1}} \end{array}\displaystyle \right ] , $$

and its associated characteristic equation is

$$ \biggl( \lambda -\frac{2Aax_{1}+ax_{1}^{2}}{ ( A+x_{1} ) ^{2}}+d+2bx_{1} \biggr) \biggl( \lambda +\delta -\frac{nmx_{1}e^{-\lambda {\tau }}}{1+hx_{1}} \biggr) =0. $$
(14)

First, we consider the following equation:

$$ F_{1}(\lambda ) = \lambda -\frac{2Aax_{1}+ax_{1}^{2}}{ ( A+x_{1} ) ^{2}}+d+2bx_{1} =0. $$
(15)

Then we obtain

$$ \begin{aligned} {\lambda _{1}} &= \frac{2Aax_{1}+ax_{1}^{2}}{ ( A+x_{1} ) ^{2}}-d-2bx_{1} \\ &= \frac{1}{ ( A+x_{1} ) ^{2}} \bigl[-2bx_{1}^{3}+ ( a-d-4Ab ) x_{1}^{2}+2A ( a-d-Ab ) x_{1}-A^{2}d \bigr] \\ &= -\frac{1}{2b^{2} ( A+x_{1} ) ^{2}} \bigl[ ( a-d ) Q_{1}+ \bigl(a^{2}-(Ab+2d)a-Abd+d^{2} \bigr)\sqrt{Q_{1}} \bigr]. \end{aligned} $$
(16)

Obviously, \(( a-d ) Q_{1} > 0\). Further, we consider an equation in a as follows:

$$ f_{1}(a) = a^{2}- ( Ab+2d ) a-Abd+d^{2}. $$
(17)

Since

$$\begin{aligned} f_{1}(a_{1}) &= f_{1}(Ab+d+2\sqrt{Abd}) \\ &= 2Abd+2Ab\sqrt{Abd} > 0, \end{aligned}$$

and the symmetry axis of \(f_{1}(a)\) is \(\overline{a} = \frac{Ab}{2}+d < a_{1}\), the inequality \(f_{1}(a) > 0\) holds for all \(a > a_{1}\). Therefore, we have \({\lambda _{1}} < 0\), then Eq. (15) has only one negative real root, which implies that all other roots of Eq. (14) are determined by the following equation:

$$ F_{2}(\lambda ) = \lambda +\delta -\frac{nmx_{1}e^{-\lambda {\tau }}}{1+hx_{1}} =0. $$
(18)

Denote \(\mathfrak{R}_{0} = \frac{N}{\delta }\), \(N=\frac{knm+nm\sqrt{Q_{1}}}{2b+hk+h\sqrt{Q_{1}}}\), and \(a^{0}_{1}=Ab+d+2bx^{*}\). We have the following discussion.

  1. (i)

    Assume that \(\mathfrak{R}_{0} > 1\), that is, \(\delta < N\), then \(F_{2}(0) < 0\) and \(F_{2}(+\infty ) = +\infty \). Hence \(F_{2}(\lambda )\) has at least one positive root and \(E_{1}\) is unstable.

  2. (ii)

    If \(\mathfrak{R}_{0} <1\), then \(N<\delta \). Suppose that there exists an eigenvalue λ with \(\operatorname{Re} \lambda \geq 0\), then we have

    $$\begin{aligned} \operatorname{Re}\lambda &= N{e^{-\operatorname{Re}(\lambda ){\tau }}}\cos (\operatorname{Im}\lambda ){ \tau }-\delta \\ & \leq N{e^{-\operatorname{Re}(\lambda ){\tau }}}-N < 0. \end{aligned}$$
    (19)

    It is a contradiction, so \(\operatorname{Re}\lambda < 0\). This implies that all the real parts of roots of \(F_{2}(\lambda ) =0\) are negative. Hence \(E_{1}\) is a locally asymptotically stable equilibrium.

On the one hand, when \(Q_{0}>0\), we know \(a_{1}=a^{0}_{1}=a_{2}\) if and only if \(A=\frac{b(x^{*})^{2}}{d}\). Thus \(a^{0}_{1}< a_{1}< a_{2}\) if \(A>\frac{b(x^{*})^{2}}{d}\); if \(A<\frac{b(x^{*})^{2}}{d}\), then \(a_{1}< a_{2}< a^{0}_{1}\). By calculation, for any \(a>a_{1}\), we have

$$\begin{aligned} \mathfrak{R}_{0} -1 &= \frac{1}{Q_{0}\delta (2b+hk+h\sqrt{Q_{1}})} \biggl[k-\frac{2b\delta }{Q_{0}}+ \sqrt{Q_{1}} \biggr] \\ &= \frac{1}{Q_{0}\delta (2b+hk+h\sqrt{Q_{1}})}H(a), \end{aligned}$$
(20)

where \(H(a)=a-a^{0}_{1}+\sqrt{a^{2}-2(Ab+d)a+(Ab-d)^{2}}\).

Hence, we obtain the following conclusions:

  1. (1)

    Assume that \(A<\frac{b(x^{*})^{2}}{d}\), that is, \(a_{1}< a_{2}< a^{0}_{1}\), then:

    1. (i)

      Let \(a_{1}< a< a^{0}_{1}\). When \(a_{1}< a< a_{2}\), we have \(H(a)<0\), that is, \(\mathfrak{R}_{0}<1\). Thus \(E_{1}\) of system (4) is locally asymptotically stable; if \(a_{2}< a< a^{0}_{1}\), then \(H(a)>0\), that is, \(\mathfrak{R}_{0}>1\), which means \(E_{1}\) of system (4) is unstable.

    2. (ii)

      If \(a^{0}_{1} \leq a\), then \(H(a)>0\), that is, \(\mathfrak{R}_{0}>1\). Therefore, \(E_{1}\) of system (4) is unstable.

  2. (2)

    If \(A \geq \frac{b(x^{*})^{2}}{d}\), then \(a_{1}\geq a^{0}_{1}\). Therefore, we have \(H(a)>0\) for any \(a>a_{1}\), that is, \(\mathfrak{R}_{0}>1\). Hence \(E_{1}\) of system (4) is unstable.

On the other hand, when \(Q_{0}\leq 0\), it follows from (20) that

$$ \mathfrak{R}_{0} -1 = \frac{1}{\delta (2b+hk+h\sqrt{Q_{1}})} [kQ_{0}-2b\delta +Q_{0}\sqrt{Q_{1}} ]< 0, $$

that is, \(\mathfrak{R}_{0}<1\). Hence, if \(a>a_{1}\), then \(E_{1}\) of system (4) is locally asymptotically stable.

Summarizing the above discussion, we prove Theorem 3.3. □

3.3 \(E_{2}=(x_{2},0)\)

Theorem 3.4

Let \(a>a_{1}\), then \(E_{2}\) of system (4) is unstable.

Proof

The variational matrix of system (4) at the equilibrium point \(E_{2}\) is

$$ V(E_{2})= \left [ \textstyle\begin{array}{@{}c@{\quad}c@{}} \lambda -\frac{2Aax_{2}+ax_{2}^{2}}{ ( A+x_{2} ) ^{2}}+d+2bx_{2}& \frac{mx_{2}}{1+hx_{2}}\\ 0& \lambda +\delta -\frac{nmx_{2}e^{-\lambda {\tau }}}{1+hx_{2}} \end{array}\displaystyle \right ] , $$

and its associated characteristic equation is

$$ \biggl( \lambda -\frac{2Aax_{2}+ax_{2}^{2}}{ ( A+x_{2} ) ^{2}}+d+2bx_{2} \biggr) \biggl( \lambda +\delta -\frac{nmx_{2}e^{-\lambda {\tau }}}{1+hx_{2}} \biggr) =0. $$
(21)

We consider the following equation:

$$ G_{1}(\lambda ) = \lambda -\frac{2Aax_{2}+ax_{2}^{2}}{ ( A+x_{2} ) ^{2}}+d+2bx_{2} =0. $$
(22)

Then we obtain

$$\begin{aligned} \lambda &= \frac{2Aax_{2}+ax_{2}^{2}}{ ( A+x_{2} ) ^{2}}-d-2bx_{2} \\ &= \frac{1}{ ( A+x_{2} ) ^{2}} \bigl[-2bx_{2}^{3}+ ( a-d-4Ab ) x_{2}^{2}+2A ( a-d-Ab ) x_{2}-A^{2}d \bigr] \\ &= -\frac{1}{2b^{2} ( A+x_{2} ) ^{2}} \bigl[ ( a-d ) Q_{1}+ \bigl( -a^{2}+Aba+2da+Abd-d^{2} \bigr) \sqrt{Q_{1}} \bigr] \\ &= -G_{2}(Q_{1})\times \frac{1}{2b^{2} ( A+x_{2} ) ^{2}} , \end{aligned}$$
(23)

where

$$ G_{2}(Q_{1}) = ( a-d ) Q_{1}+ \bigl( -a^{2}+Aba+2da+Abd-d^{2} \bigr) \sqrt{Q_{1}}. $$
(24)

Let \(\sqrt{Q_{1}} =t\), then \(t>0\). Hence, Eq. (24) can be rewritten as

$$\begin{aligned} g(t) = ( a-d ) t^{2}+ \bigl( -a^{2}+Aba+2da+Abd-d^{2} \bigr) t. \end{aligned}$$
(25)

Solving the equation \(g(t)=0\), we can get

$$ t_{1} = 0 \quad \mbox{and} \quad t_{2} =\frac{a^{2}- ( Ab+2d ) a-Abd+d^{2}}{a-d}= \frac{f_{1}(a)}{a-d}, $$
(26)

where \(f_{1}(a)\) is defined by (17). Similar to the analysis of Theorem 3.3, we obtain \(f_{1}(a) >0\), thus \(t_{2} =f_{1}(a)/(a-d) >0\).

Hence, if \(0 < t < t_{2}\), then \(g(t) < 0\); if \(t_{2} \leq t\), then \(g(t) \geq 0\). That is, if \(0 < Q_{1} < t_{2}^{2}\), then \(G_{2}(Q_{1}) < 0\); if \(t_{2}^{2} \leq Q_{1}\), then \(G_{2}(Q_{1}) \geq 0\). Clearly, the inequality \(t_{2}^{2} \leq Q_{1}\) does not hold, due to \(Q_{1}-t_{2}^{2} =-\frac{4A^{2}b^{2}ad}{(a-d)^{2}} <0\), which is a contradiction. Thus \(\lambda >0\), which implies Eq. (21) has at least one positive root and \(E_{2}\) is unstable. The proof of the theorem is complete. □

We summarize the results of Theorems 3.3 and 3.4 in Table 3. On the other hand, by the analysis of Sect. 2 and Theorems 3.3 and 3.4, we also obtain the local asymptotic stability of equilibria \(E_{i}, i=0,1,2\) (see Table 4).

Table 3 Equilibria \(E_{i}, i=0,1,2\), of system (4)
Table 4 Equilibria \(E_{i}, i=0,1,2\), of system (4) with \(a>d\)

By the definition of \(\mathfrak{R}_{0}\) and simple computation, from the proofs of Theorems 3.3 and 3.4, we can obtain the following corollaries.

Corollary 3.1

Let \(a>d\) and \(0< A< A_{1}\), then \(E_{1}\) of system (4) is:

  1. (i)

    unstable if \(0< h< \frac{nm}{\delta }-\frac{1}{x_{1}}\),

  2. (ii)

    locally asymptotically stable if \(h> \frac{nm}{\delta }-\frac{1}{x_{1}}\).

Corollary 3.2

Let \(a>d\) and \(0< A< A_{1}\), then \(E_{2}\) of system (4) is unstable.

When \(\tau =0\), by computation, the conditions of Corollaries 3.1 and 3.2 are the same as those of Theorems 1 and 2 [29]. Hence, we extend the local asymptotic stability results for system (3) to the time delay system (4). It shows that both \(E_{1}\) and \(E_{2}\) of system (4) are still locally asymptotically stable under the same condition as those for the nondelay system (3). Therefore, the time delay is harmless for the local stability of \(E_{1}\) and \(E_{2}\). Also, when considering the birth rate a or the Allee effect A as a parameter, we get the threshold condition for the stability of \(E_{0}, i=0,1,2\), of system (4). Hence, we obtain some more precise conditions than those of [29].

3.4 \(E^{*}=(x^{*},y^{*})\)

The variational matrix of system (4) at \(E^{*}\) is

$$ V\bigl(E^{*}\bigr)= \left [ \textstyle\begin{array}{@{}c@{\quad}c@{}} \lambda -\frac{2Aax^{*}+ax^{*}{^{2}}}{ ( A+x^{*} ) ^{2}}+d+2bx^{*}+\frac{my^{*}}{(1+hx^{*})^{2}}& \frac{mx^{*}}{1+hx^{*}}\\ -\frac{nmy^{*}e^{-\lambda {\tau }}}{(1+hx^{*})^{2}}& \lambda +\delta -\frac{nmx^{*}e^{-\lambda {\tau }}}{1+hx^{*}} \end{array}\displaystyle \right ] . $$

Now, to determine the local stability of the interior equilibrium \(E^{*}\), we use the method introduced by Beretta and Kuang [38]. Then the characteristic equation at \(E^{*}\) is

$$ \begin{aligned} D(\lambda ,\tau )&= \biggl(\lambda -\frac{2Aax^{*}+ax^{*}{^{2}}}{ ( A+x^{*} ) ^{2}}+d+2bx^{*}+ \frac{my^{*}}{(1+hx^{*})^{2}} \biggr) \\ & \quad {} \times \biggl(\lambda +\delta -\frac{nmx^{*}e^{-\lambda {\tau }}}{1+hx^{*}} \biggr)+ \frac{nm^{2}x^{*}y^{*}e^{-\lambda {\tau }}}{(1+hx^{*})^{3}} \\ &=\lambda ^{2}+p_{1}\lambda +p_{0}+(q_{1} \lambda +q_{0})e^{-\lambda \tau }=0, \end{aligned} $$
(27)

where

$$\begin{aligned}& p_{1}=\delta -\frac{2Aax^{*}}{ ( A+x^{*} ) ^{2}}-\frac{ax^{*}{^{2}}}{ ( A+x^{*} ) ^{2}}+d+2bx^{*}+ \frac{my^{*}}{(1+hx^{*})^{2}}, \\& p_{0}=-\frac{2Aax^{*}{\delta }}{ ( A+x^{*} ) ^{2}}-\frac{ax^{*}{^{2}}{\delta }}{ ( A+x^{*} ) ^{2}}+d{\delta }+2bx^{*}{\delta }+\frac{my^{*}{\delta }}{(1+hx^{*})^{2}}, \\& q_{1}=-\frac{nmx^{*}}{1+hx^{*}}, \\& q_{0}=\frac{2Aax^{*}{^{2}}nm}{ ( A+x^{*} ) ^{2}(1+hx^{*})}+\frac{ax^{*}{^{3}}nm}{ ( A+x^{*} ) ^{2}(1+hx^{*})}-\frac{dnmx^{*}}{1+hx^{*}}- \frac{2bx^{*}{^{2}}nm}{1+hx^{*}}. \end{aligned}$$

For the interior equilibrium \(E^{*}\), we have

$$ \textstyle\begin{cases} \frac{my^{*}}{1+hx^{*}} = \frac{ax^{*}}{(A+x^{*})}-d-bx^{*}, \\ \frac{nmx^{*}}{1+hx^{*}} = \delta . \end{cases} $$
(28)

When \(\tau = 0\), Eq. (27) reduces to

$$ D(\lambda ,0) = \lambda ^{2}+(p_{1}+q_{1}) \lambda +p_{0}+q_{0}. $$
(29)

From (28), we can obtain

$$ p_{0}+q_{0} = \frac{my^{*}{\delta }}{(1+hx^{*})^{2}} > 0. $$
(30)

Next, we consider \(p_{1}+q_{1}\). Using Eq. (28), we have

$$\begin{aligned} p_{1}+q_{1} &= -\frac{2Aax^{*}}{ ( A+x^{*} ) ^{2}}-\frac{ax^{*}{^{2}}}{ ( A+x^{*} ) ^{2}}+d+2bx^{*}+ \frac{my^{*}}{(1+hx^{*})^{2}} \\ &= \frac{ax^{*}{^{2}}}{ ( A+x^{*} ) ^{2}}-d-\frac{my^{*}}{1+hx^{*}}-\frac{mhx^{*}y^{*}}{(1+hx^{*})^{2}} \\ &= -B_{1}a+B_{2}, \end{aligned}$$

where

$$ B_{1} = \frac{hx^{*}{^{3}}+2hAx^{*}{^{2}}+Ax^{*}}{ ( A+x^{*} ) ^{2}(1+hx^{*})}\quad \mbox{and} \quad B_{2}= \frac{2bhx^{*}{^{2}}+bx^{*}+dhx^{*}}{1+hx^{*}}. $$

Define

$$ a_{3} = \frac{B_{2}}{B_{1}} = \frac{(2bhx^{*}+dh+b)(A+x^{*})^{2}}{2Ahx^{*}+h{x^{*}}^{2}+A}, $$

then

$$ p_{1}+q_{1}=B_{1}(a_{3}-a). $$

Note that

$$ \begin{aligned} a_{3}-a_{2} &= \frac{(2bhx^{*}+dh+b)(A+x^{*})^{2}}{2Ahx^{*}+h{x^{*}}^{2}+A}- \biggl(Ab+d+\frac{Ad}{x^{*}}+bx^{*} \biggr) \\ &= \frac{(1+hx^{*})(A+x^{*}) (b(x^{*})^{2}-Ad )}{x^{*}(2Ahx^{*}+h{x^{*}}^{2}+A)}. \end{aligned} $$
(31)

Let \(A<\frac{b(x^{*})^{2}}{d}\), then \(a_{3}>a_{2}\). If \(a< a_{3}\), then \(p_{1}+q_{1}=B_{1}(a_{3}-a)>0\); if \(a>a_{3}\), then \(p_{1}+q_{1}<0\). If \(A \geq \frac{b(x^{*})^{2}}{d}\) and \(a>a_{2}\), then \(p_{1}+q_{1}<0\). Accordingly, by the Routh–Hurwitz criterion, if \(p_{0}+q_{0}>0\) and \(p_{1}+q_{1}>0\), then \(E^{*}\) is locally asymptotically stable when \(\tau =0\). We have the following theorem.

Theorem 3.5

Let \(\tau =0\) and \(Q_{0}>0\) hold.

  1. (1)

    Assume that \(A<\frac{b(x^{*})^{2}}{d}\) holds.

    1. (i)

      If \(a_{2}< a< a_{3}\), then \(E^{*}\) of system (4) is locally asymptotically stable.

    2. (ii)

      If \(a>a_{3}\), then \(E^{*}\) of system (4) is unstable.

  2. (2)

    If \(A \geq \frac{b(x^{*})^{2}}{d}\) and \(a>a_{2}\), then \(E^{*}\) of system (4) is unstable.

Next, under the conditions \(Q_{0}>0\) and \(A<\frac{b(x^{*})^{2}}{d}\), we discuss system (4) with \(\tau >0\). Then, we discuss (27) has a pair of purely imaginary roots. (\(\omega >0\)) is a solution of (27) if and only if ω satisfies

$$ -\omega ^{2}+p_{1}\omega i+p_{0}+(q_{1} \omega i+q_{0}) (\cos \tau \omega -i\sin \tau \omega )=0, $$

that is,

$$ -\omega ^{2}+p_{0}=-q_{1}\omega \sin (\tau \omega )-q_{0}\cos (\tau \omega ),\qquad p_{1}\omega =q_{0}\sin (\tau \omega )-q_{1}\omega \cos (\tau \omega ), $$
(32)

which implies

$$ \omega ^{4}+\bigl(p_{1}^{2}-2p_{0}-q_{1}^{2} \bigr)\omega ^{2}+p_{0}^{2}-q_{0}^{2}=0. $$
(33)

By calculation, we derive that

$$\begin{aligned} p_{1}^{2}-2p_{0}-q_{1}^{2} &= \biggl(-\frac{2Aax^{*}}{ ( A+x^{*} ) ^{2}}-\frac{ax^{*}{^{2}}}{ ( A+x^{*} ) ^{2}}+d+2bx^{*}+ \frac{my^{*}}{(1+hx^{*})^{2}} \biggr)^{2} \\ &= (p_{1}+q_{1})^{2} > 0, \end{aligned}$$
(34)

and from (30) we have

$$ p_{0}+q_{0} = \frac{my^{*}{\delta }}{(1+hx^{*})^{2}} > 0. $$
(35)

Further, we consider \(p_{0}-q_{0}\). From (28) we have

$$\begin{aligned} p_{0}-q_{0} &= \delta \biggl(-\frac{4Aax^{*}}{ ( A+x^{*} ) ^{2}}- \frac{2ax^{*}{^{2}}}{ ( A+x^{*} ) ^{2}}+4bx^{*}+\frac{my^{*}}{(1+hx^{*})^{2}}+2d \biggr) \\ &= \delta \biggl(-\frac{4Aax^{*}}{ ( A+x^{*} ) ^{2}}-\frac{2ax^{*}{^{2}}}{ ( A+x^{*} ) ^{2}}+4bx^{*}+ \frac{ax^{*}-(A+x^{*})(d+bx^{*})}{(1+hx^{*})(A+x^{*})}+2d \biggr) \\ &= \frac{\delta }{(1+hx^{*})(A+x^{*})^{2}} \bigl(-aD_{1}\bigl(x^{*}\bigr)+ \bigl(A+x^{*}\bigr)^{2}D_{2}\bigl(x^{*} \bigr) \bigr), \end{aligned}$$
(36)

where

$$ \begin{aligned} &D_{1}\bigl(x^{*}\bigr) = 2hx^{*}{^{3}}+4Ahx^{*}{^{2}}+x^{*}{^{2}}+3Ax^{*} > 0, \\ &D_{2}\bigl(x^{*}\bigr) = 4bhx^{*}{^{2}}+3bx^{*}+d+2dhx^{*}>0. \end{aligned} $$
(37)

On the other hand, we denote

$$ a_{4} = \frac{(A+x^{*})^{2}D_{2}(x^{*})}{D_{1}(x^{*})}. $$

Then we have

$$ p_{0}-q_{0}=\frac{\delta }{(1+hx^{*})(A+x^{*})^{2}}D_{1} \bigl(x^{*}\bigr) (a_{4}-a). $$

Note that, due to \(A<\frac{b(x^{*})^{2}}{d}\), we derive that

$$\begin{aligned} a_{3}-a_{4} &= \frac{(2bhx^{*}+dh+b)(A+x^{*})^{2}}{2Ahx^{*}+h{x^{*}}^{2}+A}-\frac{(A+x^{*})^{2}D_{2}(x^{*})}{D_{1}(x^{*})} \\ &= \frac{(1+hx^{*})(A+x^{*})^{2} (b(x^{*})^{2}-Ad )}{x^{*}(4Ahx^{*}+2h{x^{*}}^{2}+3A+x^{*})(2Ahx^{*}+h{x^{*}}^{2}+A)} \\ &> 0, \end{aligned}$$
(38)

and

$$\begin{aligned} a_{4}-a_{2} &= \frac{(A+x^{*})^{2}D_{2}(x^{*})}{D_{1}(x^{*})}- \biggl(Ab+d+ \frac{Ad}{x^{*}}+bx^{*} \biggr) \\ &= \frac{2(1+hx^{*})(A+x^{*}) (b(x^{*})^{2}-Ad )}{x^{*}(4Ahx^{*}+2h{x^{*}}^{2}+3A+x^{*})} \\ &> 0. \end{aligned}$$
(39)

By Theorem 3.5, if \(a_{2}< a\leq a_{4}\), \(p_{0}-q_{0}\geq 0\); if \(a_{4}< a< a_{3}\), \(p_{0}-q_{0}<0\).

When \(p_{0}-q_{0} < 0\), \(\omega _{0}\) is a unique positive root of Eq. (33), that is, Eq. (27) has the roots of the form \(\pm i\omega _{0}\), where

$$ \omega _{0} = \biggl(\frac{q_{1}^{2}+2p_{0}-p_{1}^{2}+\sqrt{(p_{1}^{2}-2p_{0}-q_{1}^{2})^{2}-4p_{0}^{2}+4q_{0}^{2}}}{2} \biggr)^{\frac{1}{2}}. $$

From (32), we obtain

$$ \cos (\tau \omega _{0}) = \frac{q_{0}(\omega _{0}^{2}-p_{0})-p_{1}q_{1}\omega _{0}^{2}}{q_{1}^{2}\omega _{0}^{2}+q_{0}^{2}}. $$

Denote

$$ \tau _{0n} =\frac{1}{\omega _{0}}\arccos {\frac{q_{0}(\omega _{0}^{2}-p_{0})-p_{1}q_{1}\omega _{0}^{2}}{q_{1}^{2}\omega _{0}^{2}+q_{0}^{2}}}+ \frac{2n\pi }{\omega _{0}} ,\quad n=0,1,2,\ldots . $$
(40)

By Theorem 3.4.1 in Kuang [39], if \(p_{0}< q_{0}\), then \(E^{*}\) is still stable for \(\tau <\tau _{0} :=\tau _{00}\).

Taking the derivative of Eq. (27) with respect to τ, one has

$$ \biggl( \frac{\mathrm{d}\lambda }{\mathrm{d}\tau } \biggr) ^{-1}=\frac{2\lambda +p_{1}}{-\lambda (\lambda ^{2}+p_{1}\lambda +p_{0})}+ \frac{q_{1}}{\lambda (q_{1}\lambda +q_{0})}-\frac{\tau }{\lambda }. $$

Hence, a direct calculation shows that

$$\begin{aligned} \operatorname{sign} \biggl\{ \frac{d(\operatorname{Re}\lambda )}{d\tau } \biggr\} _{\lambda =i\omega _{0}} &= \operatorname{sign} \biggl\{ \operatorname{Re} \biggl( \frac{\mathrm{d}\lambda }{\mathrm{d}\tau } \biggr) ^{-1} \biggr\} _{\lambda =i\omega _{0}} \\ &= \operatorname{sign} \biggl\{ \frac{p_{1}^{2}+2(\omega _{0}^{2}-p_{0})}{p_{1}^{2}\omega _{0}^{2}+(\omega _{0}^{2}-p_{0})^{2}}+\frac{-q_{1}^{2}}{q_{1}^{2}\omega _{0}^{2}+q_{0}^{2}} \biggr\} . \end{aligned}$$

We derive from (32) that

$$ p_{1}^{2}\omega _{0}^{2}+\bigl(\omega _{0}^{2}-p_{0}\bigr)^{2} = q_{1}^{2}\omega _{0}^{2}+q_{0}^{2}, $$

which yields

$$ \operatorname{sign} \biggl\{ \frac{d(\operatorname{Re}\lambda )}{d\tau } \biggr\} _{\lambda =i\omega _{0}} = \operatorname{sign} \biggl\{ \frac{2\omega _{0}^{2}+p_{1}^{2}-2p_{0}-q_{1}^{2}}{q_{1}^{2}\omega _{0}^{2}+q_{0}^{2}} \biggr\} > 0. $$

When \(\omega =\omega _{0}\), \(\tau =\tau _{0}\), a Hopf bifurcation occurs.

The above analysis can be summarized in the following theorem.

Theorem 3.6

Let \(Q_{0}>0\) and \(A<\frac{b(x^{*})^{2}}{d}\) hold.

  1. (i)

    If \(a_{2} < a \leq a_{4} \), then \(E^{*}\) of system (4) is locally asymptotically stable for all \(\tau \geq 0\).

  2. (ii)

    If \(a_{4} < a < a_{3}\) holds, then \(E^{*}\) is a locally asymptotically stable equilibrium if \(0< \tau <\tau _{0}\) and unstable if \(\tau > \tau _{0}\).

Further, system (4) undergoes a Hopf bifurcation at \(E^{*}\) when \(\tau =\tau _{0}\).

4 Numeric simulations

We present some examples to verify our results.

Example 4.1

In system (4), assume that \(A=0.5\), \(d=1\), \(b=5\), \(m=5\), \(n=1\), \(h=0.1\), \(\tau =1\), and \(\delta =6\), then \(a_{1}=6.6623\).

  1. (i)

    If \(a=5\), that is, \(a< a_{1}\), it follows from Theorem 3.2 that \(E_{0}\) is globally asymptotically stable. Hence, both prey and predators are extinct (see Fig. 3).

    Figure 3
    figure 3

    Dynamic behaviors of system (4) with \(A=0.5\), \(d=1\), \(b=5\), \(m=5\), \(n=1\), \(h=0.1\), \(\tau =1\), \(\delta =6\), and \(a=5\)

  2. (ii)

    If \(a=8\), then system (4) has a boundary equilibrium \(E_{1}=(0.7702,0)\). Note that \(A=0.5<9.2975=\frac{b(x^{*})^{2}}{d}\), \(a_{1}< a<10.6848=a_{2}\). By Theorem 3.3, \(E_{1}\) is locally asymptotically stable (see Fig. 4).

    Figure 4
    figure 4

    Dynamic behaviors of system (4) with \(A=0.5\), \(d=1\), \(b=5\), \(m=5\), \(n=1\), \(h=0.1\), \(\tau =1\), \(\delta =6\), and \(a=8\)

Example 4.2

In system (4), assume that \(A=0.5\), \(d=1\), \(b=5\), \(m=5\), \(n=1\), \(h=0.1\), \(\delta =6\), and \(\tau =0\). By calculation, we have \(Q_{0}=4.4>0\), \(A<\frac{b(x^{*})^{2}}{d}=9.2975\), \(a_{2}=10.6848\), and \(a_{3}=27.2999\).

  1. (i)

    If \(a=15\), it is easy to show that \(a_{2}< a< a_{3}\). We can obtain a positive equilibrium \(E^{*}=(1.3636,0.7176)\). By Theorem 3.5, \(E^{*}\) is locally asymptotically stable (see Fig. 5).

    Figure 5
    figure 5

    Dynamic behaviors of system (4) with \(A=0.5\), \(d=1\), \(b=5\), \(m=5\), \(n=1\), \(h=0.1\), \(\delta =6\), \(\tau =0\), and \(a=15\)

  2. (ii)

    If \(a=30\), then \(a>a_{3}\). Also, we can obtain a positive equilibrium \(E^{*}=(1.3636,3.2121)\). By Theorem 3.5, \(E^{*}\) is unstable (see Fig. 6).

    Figure 6
    figure 6

    Dynamic behaviors of system (4) with \(A=0.5\), \(d=1\), \(b=5\), \(m=5\), \(n=1\), \(h=0.1\), \(\delta =6\), \(\tau =0\), and \(a=30\)

Example 4.3

In system (4), let \(a=4\), \(A=0.02\), \(d=2\), \(b=1\), \(m=n=4\), \(h=0.1\), \(\delta =3\). By calculation, we have \(Q_{0}=15.7>0\), \(A=0.02>0.0183=\frac{b(x^{*})^{2}}{d}\), \(a>a_{2}=2.4204\), and there is a positive equilibrium \(E^{*}=(0.1911,0.4102)\). By Theorem 3.5, \(E^{*}\) is unstable, that is, both the predator and prey species will be extinct (see Fig. 7).

Figure 7
figure 7

Dynamic behaviors of system (4) with \(a=4\), \(A=0.02\), \(d=2\), \(b=1\), \(m=n=4\), \(h=0.1\), \(\delta =3\), and \(\tau =0\)

Example 4.4

In system (4), assume that \(A=0.5\), \(d=1\), \(b=m=5\), \(n=1\), \(h=0.1\), \(\delta =6\). By calculation, we have \(a_{2}=10.6848\), \(a_{3}=27.2999\), \(a_{4}=18.4738\), \(Q_{0} = 4.400>0\), \(x^{*}=1.3636\), and \(A<\frac{b(x^{*})^{2}}{d}=9.2975\).

  1. (i)

    If \(a=15\) and \(\tau =3\). It is easy to show that \(a_{2}< a< a_{4}\) and the positive equilibrium is \(E^{*}=(1.3636,0.7176)\). By Theorem 3.6, \(E^{*}\) is locally asymptotically stable (see Fig. 8).

    Figure 8
    figure 8

    Dynamic behaviors of system (4) with \(a=15\), \(A=0.5\), \(d=1\), \(b=m=5\), \(n=1\), \(h=0.1\), \(\delta =6\), and \(\tau =3\)

  2. (ii)

    If \(a=15\) and \(\tau =15\). It is easy to show that \(a_{2}< a< a_{4}\) and the positive equilibrium is \(E^{*}=(1.3636,0.7176)\). By Theorem 3.6, \(E^{*}\) is locally asymptotically stable (see Fig. 9).

    Figure 9
    figure 9

    Dynamic behaviors of system (4) with \(a=15\), \(A=0.5\), \(d=1\), \(b=m=5\), \(n=1\), \(h=0.1\), \(\delta =6\), and \(\tau =15\)

  3. (iii)

    If \(a=20\), clearly, \(a_{4}< a< a_{3}\) and the positive equilibrium is \(E^{*}=(1.3636,1.5491)\). By calculation, we obtain \(\tau _{0}=0.1387\). According to Theorem 3.6, \(E^{*}\) is locally asymptotically stable if \(0<\tau <\tau _{0}\) and unstable if \(\tau >\tau _{0}\), and system (4) undergoes a Hopf bifurcation at \(E^{*}\) when \(\tau =\tau _{0}\), where \(\tau =0.07\) is shown in Fig. 10 and \(\tau =0.15\) is shown in Fig. 11.

    Figure 10
    figure 10

    Dynamic behaviors of system (4) with \(a=20\), \(A=0.5\), \(d=1\), \(b=m=5\), \(n=1\), \(h=0.1\), \(\delta =6\), and \(\tau =0.07\)

    Figure 11
    figure 11

    Dynamic behaviors of system (4) with \(a=20\), \(A=0.5\), \(d=1\), \(b=m=5\), \(n=1\), \(h=0.1\), \(\delta =6\), and \(\tau =0.15\)

5 Conclusion

In this paper, we investigate the stability and Hopf bifurcation of a delayed Holling type II predator–prey model when the prey is subject to Allee effect. Considering the birth rate a or the Allee effect A as a parameter, we get the threshold condition for the stability of the equilibria of system (4) and find that the Allee effect or the birth rate plays an important role in the stability of system (4).

Firstly, we analyze the logistic equation (1) with Allee effect, and show that the Allee effect may lead to the instability of the system. By analyzing the characteristic equations, we derive the local stability of the equilibria. Table 3 shows that \(E_{2}\) is always unstable if it exists. If the Allee effect is relatively small, with the increase of the birth rate a, the equilibrium \(E_{1}\) changes from the nonexistence to the local asymptotic stability, and eventually becomes instability. Table 4 shows that the equilibrium \(E_{1}\) changes from instability to local asymptotic stability, and eventually disappears with the increase of Allee effect.

We also investigate the local asymptotic stability of the coexistence equilibrium \(E^{*}\) and show that if the time delay passes through some critical values, the equilibrium \(E^{*}\) changes from a stable point to unstable point, and a Hopf bifurcation occurs; further we show periodic solutions.

Let \(\tau =0\), then system (4) is reduced to system (3). If system (3) has only three equilibrium points \(E_{i}, i=0,1,2\). \(E_{0}(0,0)\) is locally asymptotically stable. Further if \(E_{1}\) and \(E_{2}\) are unstable, then the trivial equilibrium \(E_{0}(0,0)\) is globally asymptotically stable. When system (3) has equilibrium points \(E_{i}, i=0,1,2\), and \(E^{*}\). If \(E_{1}\), \(E_{2}\), and \(E^{*}\) are unstable, then system (3) admits a limit cycle or \(E_{0}(0,0)\) is globally asymptotically stable. In Table 5, compared with Zu and Mimura [29], we give the threshold condition for the stability of \(E_{0}, i=0,1,2\), and \(E^{*}\) of system (4) with \(\tau =0\). Therefore, our main results complement and improve those in [29].

Table 5 Equilibria \(E_{i}, i=0,1,2\), and \(E^{*}\) of system (4) with \(\tau =0\)

In Theorem 3.6, we study the stability and Hopf bifurcation of system (4). The results of Theorems 3.13.6 are shown in Table 6. If the Allee effect is small, the equilibrium \(E_{1}\) changes from the nonexistence to the local asymptotic stability, and eventually becomes instability with the increase of the birth rate a. On the other hand, the equilibrium \(E^{*}\) changes from the nonexistence to the local asymptotic stability, and eventually a Hopf bifurcation occurs with the increase of the birth rate a. When the initial densities of prey and predators are not low, we show the results of Table 6 in Fig. 12 and Fig. 13 with the same initial values. Change the values of the birth rate a from zero to \(a_{1}\), the prey species will stabilize at zero, from \(a_{1}\) to \(a_{2}\), the prey species will stabilize at \(x_{1}\). From \(a_{2}\) to \(a_{3}\), the prey species will stabilize at \(x^{*}\). Finally, the prey species becomes oscillation if \(a_{4}< a< a_{3}\). The predator species has the similar case. Hence, we find that both the predator species and the prey species go to extinction when the prey population has a sufficiently small birth rate a. As a increases, the prey species could survive, but the predator is still extinct. Keeping the birth rate a increasing, the prey and the predator will coexist and stabilize at a fixed value. But if it is large enough, the stability of prey and predators will be destroyed, and the system becomes oscillation. Hence, in this paper, we show that we can choose a suitable value of the Allee effect or the birth rate to influence the stability of system (4).

Figure 12
figure 12

Dynamic behaviors of the prey in system (4) with \(A=0.5\), \(d=1\), \(b=m=5\), \(n=1\), \(h=0.1\), \(\delta =6\), \(\tau =0.5\), and \(a=4\) or 8 or 15 or 20

Figure 13
figure 13

Dynamic behaviors of the predator in system (4) with \(A=0.5\), \(d=1\), \(b=m=5\), \(n=1\), \(h=0.1\), \(\delta =6\), \(\tau =0.5\), and \(a=4\) or 8 or 15 or 20

Table 6 System (4) with \(Q_{0}>0\) and \(A<\frac{b(x^{*})^{2}}{d}\)