1 Introduction

The predator–prey models have a theoretical and practical significance. So many authors have studied the dynamics of these models. The Lotka–Volterra model is the first and the simplest model of predator–prey interactions. The model was developed independently by Lotka [1] and Volterra [2]. The Lotka–Volterra model assumes that the prey consumption rate by a predator is directly proportional to the prey abundance. This means that predator feeding is limited only by the amount of prey in the environment. While this may be realistic at low prey densities, it is certainly an unrealistic assumption at high prey densities where predators are limited, for example, by time and digestive constraints. So a number of changes in the model have been presented by researchers to include Holling type functional responses and density-dependent prey growth. Another way of modification of the Lotka–Volterra model can be considered by introducing the Allee effect. It is well known that the introduction of the Allee effect into the system is more realistic in modeling the prey–predator interaction.

The Allee effect is a crucial phenomenon that has drawn considerable attention from both theoretical and applied ecologists [3,4,5,6,7,8,9]. The Allee effect was first described by Allee in 1931 [10]. It describes a positive correlation between any measure of species fitness and population numbers. The main causes of the Allee effect are the difficulty in finding mates, inbreeding depression, social dysfunction at small population sizes, predator avoidance and food exploitation [11,12,13,14]. Empirical evidence of the Allee effect has been observed in many natural species, for example, plants [15], insects [16], marine invertebrates [17], birds and mammals [18]. Recent studies have shown that the Allee effect has important dynamical effects on the stability analysis of population models. The system with Allee effect shows either destabilization [11] or stabilization [19]. The local stability of a positive fixed point may be changed from stable to unstable or vice versa. In prey–predator systems, Allee effect may change the dynamics of the system in unexpected ways and induce complex dynamics. So, many authors have investigated the stabilizing or destabilizing effects on the predator–prey models with the Allee effect. But the bifurcation analysis of these systems with the Allee effect has been rarely performed. So, in this paper, a predator–prey system on which is imposed an Allee effect is considered.

In [20], the author has considered the following discrete-time predator–prey model which was proposed by Smith et al. [21]:

$$\begin{aligned}& \begin{aligned}&X_{t+1} =\alpha X_{t}(1-X_{t})-X_{t}Y_{t} , \\ &Y_{t+1} =\frac{1}{\beta }X_{t}Y_{t} , \end{aligned} \end{aligned}$$
(1)

where \(X_{t}\) and \(Y_{t}\) denotes the numbers of prey and predator respectively. The parameters α, β are positive real numbers. They have studied stability and Neimark–Sacker bifurcation of a discrete-time predator–prey model. The analysis of bifurcations has already received much attention during the last few years [20, 22,23,24,25,26,27,28,29]. Bifurcation and stability analysis are examined in detail in [20, 22,23,24, 30,31,32,33,34,35,36,37,38,39,40,41].

In this paper, modification of the model (1) is considered by introducing the Allee effect for the prey population as follows:

$$\begin{aligned}& \begin{aligned}&X_{t+1} =aX_{t}(1-X_{t})-X_{t}Y_{t} \biggl( \frac{X_{t}}{m+X_{t}} \biggr) , \\ &Y_{t+1} =\frac{1}{\beta }X_{t}Y_{t} , \end{aligned} \end{aligned}$$
(2)

where \(\frac{X}{m+X}\) is term for the Allee effect. \(m>0\) can be defined as Allee effect constant [11,12,13]. α and β are the growth rates of the prey and predator, respectively.

The aim of this paper is to investigate the dynamics of a modified a discrete-time predator–prey model with Allee effect on prey population by using bifurcation theory. The rest of the paper is organized as follows: In Sect. 2, the existence conditions and stability of the fixed points are discussed. In Sect. 3, choosing a parameter as bifurcation parameter, Neimark–Sacker bifurcation analysis is studied. Furthermore, by using normal form theory direction of Neimark–Sacker bifurcation is obtained. Finally, these theoretical results are supported by some numerical simulations.

2 The fixed points: existence and stability

In this section, we will study the existence of the fixed points of the discrete system with Allee effect and analyze the stability of these fixed points. To find the fixed points of the system (2), we can write

$$ X_{t} =X_{t+1}=X^{\ast },\qquad Y_{t} =Y_{t+1}=Y^{\ast } $$
(3)

in the model (2)

$$\begin{aligned} \begin{aligned}&X^{\ast } =aX^{\ast }\bigl(1-X^{\ast } \bigr)-X^{\ast }Y^{\ast } \biggl( \frac{X ^{\ast }}{m+X^{\ast }} \biggr) , \\ &Y^{\ast } =\frac{1}{\beta }X^{\ast }Y^{\ast }. \end{aligned} \end{aligned}$$
(4)

It is clear that the fixed points of the model (2) satisfy Eq. (4).

Lemma 1

  1. (i)

    The system (2) always has an axial fixed point \(E_{1}= ( 0,0 ) \).

  2. (ii)

    The system (2) has an axial fixed point \(E_{2}= ( \frac{a-1}{a},0 ) \) if \(a>1\).

  3. (iii)

    The system (2) has a unique positive fixed point \(E _{\ast }= ( \beta ,\frac{(\beta +m)(a(1-\beta )-1)}{\beta } ) \) if \(a>\frac{1}{1-\beta }\) and \(\beta <1\).

In order to analyze the stability of the fixed points of the model (2), we give Definition 1 and Lemma 2 as follows:

Definition 1

A fixed point \((N,P)\) is called

  1. (i)

    sink if \(\vert \lambda _{1} \vert <1\) and \(\vert \lambda _{2} \vert <1\), and it is locally asymptotically stable,

  2. (ii)

    source if \(\vert \lambda _{1} \vert >1\) and \(\vert \lambda _{2} \vert >1\), and it is locally unstable,

  3. (iii)

    saddle if \(\vert \lambda _{1} \vert <1\) and \(\vert \lambda _{2} \vert >1\) or (\(\vert \lambda _{1} \vert >1\) and \(\vert \lambda _{2} \vert <1\)),

  4. (iv)

    non-hyperbolic if either \(\vert \lambda _{1} \vert =1\) or \(\vert \lambda _{2} \vert =1\).

To investigate the stability of the one nontrivial fixed point of the system (2), we give Lemma 2.

Lemma 2

Assume \(F(\lambda )=\lambda ^{2}+B\lambda +C\), where B and C are two real constants and let \(F(1)>0\). Suppose \(\lambda _{1}\) and \(\lambda _{2}\) are two roots of \(F ( \lambda ) =0\). Then the following statements hold.

  1. (i)

    \(\vert \lambda _{1} \vert <1\) and \(\vert \lambda _{2} \vert <1\) if and only if \(F(-1)>0\) and \(C<1\);

  2. (ii)

    \(\vert \lambda _{1} \vert >1\) and \(\vert \lambda _{2} \vert >1\) if and only if \(F(-1)>0\) and \(C>1\);

  3. (iii)

    \(\vert \lambda _{1} \vert <1\) and \(\vert \lambda _{2} \vert >1\), or \(\vert \lambda _{1} \vert >1\) and \(\vert \lambda _{2} \vert <1\), if and only if \(F(-1)<0\);

  4. (iv)

    \(\lambda _{1}\) and \(\lambda _{2}\) are a pair of conjugate complex roots and \(\vert \lambda _{1} \vert = \vert \lambda _{2} \vert =1\) if and only if \(B ^{2}-4C<0\) and \(C=1\);

  5. (v)

    \(\lambda _{1}=-1\) and \(\vert \lambda _{2} \vert \neq 1\) if and only if \(F(-1)=0\) and \(B\neq 0,2\).

Now we will discuss the topological classification of the fixed points of the model (2) and we will apply Lemma 2 to prove the following lemmas.

The Jacobian matrix of the planar map in (2) evaluated at any point \((x,y)\) is given by

J(x,y)= ( a ( 1 − 2 x ) − 2 x y x + m + x 2 y ( x + m ) 2 − x 2 x + m y β x β ) ,
(5)

and we have

J ( E 1 ) = ( a 0 0 0 ) , J ( E 2 ) = ( 2 − a − ( a − 1 ) 2 a ( m a + a − 1 ) 0 a − 1 β a ) , J ( E ∗ ) = ( − a + 2 − β ( a ( β − 1 ) + 1 ) β + m − β 2 m + β − ( β + m ) ( a ( β − 1 ) + 1 ) β 2 1 ) .

Lemma 3

For the fixed point \(E_{1}(0,0)\), following topological classification holds:

  1. (i.1)

    \(E_{1}(0,0)\) is a sink if \(a<1\).

  2. (i.2)

    \(E_{1}(0,0)\) is a saddle if \(a>1\).

  3. (i.3)

    \(E_{1}(0,0)\) is a non-hyperbolic if \(a=1\).

Lemma 4

Assume that \(a>1\). For the fixed point \(E_{2}= ( \frac{a-1}{a},0 ) \), the following topological classification holds:

  1. (ii.1)

    \(E_{2}= ( \frac{a-1}{a},0 ) \) is a sink if \(1< a<\min \{\frac{1}{1-\beta },3\}\) and \(0<\beta <1\),

  2. (ii.2)

    \(E_{2}= ( \frac{a-1}{a},0 ) \) is a saddle if (\(\frac{1}{1-\beta }< a<3\) and \(0<\beta < \frac{2}{3}\)) or (\(3< a<\frac{1}{1-\beta }\) and \(\frac{2}{3}<\beta <1\)),

  3. (ii.3)

    \(E_{2}= ( \frac{a-1}{a},0 ) \) is a source if \(a>\max \{3,\frac{1}{1-\beta }\}\), \(0<\beta <1\),

  4. (ii.4)

    \(E_{2}= ( \frac{a-1}{a},0 ) \) is a non-hyperbolic if \(a=3\) or (\(a=\frac{1}{1-\beta }\), \(0< \beta <1\)).

The characteristic equation of matrix \(J(E_{\ast })\) can be written as follows:

$$\begin{aligned}& F(\lambda )=\lambda ^{2}-(a_{11}+a_{22})\lambda +(a_{11}a_{22}-a_{12}a _{21}) , \end{aligned}$$
(6)
$$\begin{aligned}& F(\lambda )=\lambda ^{2}-\biggl(\frac{-2\beta -3m+am+a\beta ^{2}}{m+\beta }\biggr) \lambda + \biggl(\frac{a\beta -2a\beta ^{2}-a\beta m+m}{m+\beta }\biggr) . \end{aligned}$$
(7)

From Lemma 2, we have

$$ F(1)=-1+a(1-\beta ). $$
(8)

Since \(a>\frac{1}{1-\beta }\), \(F(1)>0\).

Lemma 5

Assume that \(a>\frac{1}{1-\beta }\) and \(\beta <1\) then for the unique positive fixed point \(E_{\ast }\) of the system (2) the following holds true.

  1. (iii1)

    \(E_{\ast }\) is sink fixed point if the following condition holds:

    $$\begin{aligned}& \frac{1}{1-\beta }< a< \min \biggl\{ \frac{1}{1-2\beta -m} ,\frac{3\beta +5m}{3\beta ^{2}+\beta m+m-\beta }\biggr\} , \end{aligned}$$
  2. (iii2)

    \(E_{\ast }\) is source fixed point if the following condition holds:

    $$\begin{aligned}& \frac{1}{1-2\beta -m}< a< \frac{3\beta +5m}{3\beta ^{2}+ \beta m+m-\beta }, \end{aligned}$$
  3. (iii3)

    \(E_{\ast }\) is saddle fixed point if the following condition holds:

    $$\begin{aligned}& a>\frac{1}{1-2\beta -m}, \end{aligned}$$
  4. (iii4)

    non-hyperbolic fixed point if the following conditions hold:

    $$\begin{aligned}& a=\frac{3\beta +5m}{3\beta ^{2}+\beta m+m-\beta }\quad \textit{and} \quad m \neq \frac{4}{5}- \frac{9}{5}\beta , \end{aligned}$$
  5. (iii5)

    the roots of Eq. (7) are complex with modules one if and only if

    $$\begin{aligned}& a=\frac{1}{1-2\beta -m}\quad \textit{and} \quad m< \frac{4}{5}- \frac{9}{5} \beta . \end{aligned}$$

Example 1

For the parameter values \(a=2.5\), \(m=0.25\), \(\beta =0.23\) and initial condition \(( x_{0},y_{0} ) =(0.15,2.5)\), the positive fixed point of the model (2) is obtained: \(( x ^{\ast },y^{\ast } ) =(0.23,1.930434783)\). Figure 1 is shown that the fixed point \(( x^{\ast },y^{\ast } ) \) of the model (2) is local asymptotically stable where blue and red graphs represent \(x(t)\) and \(y(t)\) population, respectively.

Figure 1
figure 1

A stable fixed point for the system (2) for \(a=2.5\), \(m=0.25\), \(\beta =0.23\) and initial condition \(( x_{0},y_{0} ) =(0.15,2.5)\)

Full size image

3 Bifurcation analysis

3.1 Neimark–Sacker bifurcation at the point \(E_{\ast }\)

In this section we investigate the conditions for existence of the Neimark–Sacker bifurcation for unique positive fixed point \(E_{\ast }= ( \beta ,\frac{(\beta +m)(a(1-\beta )-1)}{\beta } ) \) of the model (2). Also, direction of Neimark–Sacker bifurcation is evaluated. We will take a as bifurcation parameter.

Let us consider the term of \(\varOmega _{NSB_{E^{\ast }}}\) as follows:

$$ \varOmega _{NSB_{E^{\ast }}}= \biggl\{ ( a,\beta ,m ) \in R ^{3}:a= \frac{1}{1-2\beta -m},m< \frac{4}{5}-\frac{9}{5}\beta \text{ and }\beta < \frac{1}{2} \biggr\} . $$
(9)

When the parameters change in small neighborhood of \(\varOmega _{NSB_{E^{\ast }}}\), two eigenvalues of \(J ( E^{\ast } ) \) are complex having magnitude one and Neimark–Sacker bifurcation can go out from the fixed point \(E^{\ast }\). The eigenvalues of Eq. (2) under these conditions are given by

$$ \lambda ,\overline{\lambda }=\frac{ ( -3m+5\beta -2 ) \pm i\sqrt{\beta +m}\sqrt{4-5m-9\beta }}{2(2\beta +m-1)}. $$
(10)

It is easy to see that

$$ \vert \lambda \vert = \vert \overline{\lambda } \vert =1. $$
(11)

From the transversality condition, we get

$$ \frac{d \vert \lambda _{i} ( a ) \vert }{da}\vert _{a=a_{NS}}=\frac{\beta (1-2\beta -m)}{2(\beta +m)}\neq 0,\quad i=1,2. $$
(12)

Moreover, the nonresonance condition \(B=-trJ_{E^{\ast }} ( a _{NS} ) \neq 0,1\) leads to

$$ m\neq \frac{2}{3}-\frac{5\beta }{3},\frac{3}{4}- \frac{7\beta }{4} ; $$
(13)

then we have

$$ \lambda ^{k} ( a_{NS} ) \neq 1\quad \text{for }k=1,2,3,4. $$
(14)

Assume that \(q,p\in \mathbb{C} ^{2}\) are two eigenvectors of \(J ( \varOmega _{NSB_{E^{\ast }}} ) \) and transposed matrix \(J^{T} ( \varOmega _{NSB_{E^{\ast }}} ) \) corresponding to λ and λ̅, respectively. We have

$$ q\sim \biggl( -\frac{\beta ^{2}}{2(m+\beta )}+\frac{1}{2}\frac{\beta ^{2}\sqrt{4-5m-9\beta }i}{(m+\beta )^{3/2}},1 \biggr) ^{T} $$
(15)

and

$$ p\sim \biggl( -\frac{(m+\beta )^{2}}{2\beta ^{2}(2\beta +m-1)}-\frac{ \frac{1}{2}(m+\beta )^{3/2}\sqrt{4-5m-9\beta }i}{\beta ^{2}(2\beta +m-1)},1 \biggr) ^{T}. $$
(16)

To achieve the normalization \(\langle p,q\rangle =1\), where \(\langle\, ,\,\rangle \) means the standard scalar product in \(C^{2}\), we can take the normalized vectors as

$$ \begin{aligned} &q= \biggl( -\frac{\beta ^{2}}{2(m+\beta )}+\frac{1}{2} \frac{\beta ^{2}\sqrt{4-5m-9 \beta }i}{(m+\beta )^{3/2}},1 \biggr) ^{T}, \\ &p=S_{1} \biggl( \frac{(S_{1}+\beta )^{2}}{2\beta ^{2}(S_{1}+2\beta -1)}+\frac{i(S _{1}+\beta )^{\frac{3}{2}}\sqrt{4-5S_{1}-9\beta }}{2\beta ^{2}(S_{1}+2 \beta -1)},1 \biggr) ^{T}, \end{aligned} $$
(17)

where \(S_{1}= ( -\frac{-5m-9\beta +4+\sqrt{4-5m-9\beta }\sqrt{m+ \beta }i}{2(-4+5m+9\beta )} ) \).

Using the transformation

$$ x_{t}=X_{t}-\beta ,\qquad y_{t}=Y_{t}- \biggl(\frac{m+\beta }{\beta }\biggr) \bigl(a(1-\beta )-1\bigr), $$
(18)

the fixed point \(E^{\ast }\) is shifted to the point \(( 0,0 ) \). From a Taylor expansion, the system (2) converts to

( x t y t ) →J ( E ∗ ) ( x t y t ) + ( F 1 ( x t , y t ) F 2 ( x t , y t ) ) ,
(19)

where

$$\begin{aligned}& F_{1} ( x_{t},y_{t} ) =\frac{-x_{t}(\beta ^{4}ax_{t}+3\beta ^{3}amx _{t}+2\beta ^{2}m^{2}ax_{t}+m^{2}a\beta x_{t}+m^{3}ax_{t}-y_{t}x_{t}-m ^{3}x_{t}}{\beta (\beta +m)^{3}} \\& \hphantom{F_{1} ( x_{t},y_{t} ) =}{} + \frac{\beta ^{4}y_{t}+3\beta ^{3}my _{t}+2\beta ^{2}m^{2}y_{t}+m^{2}ax_{t}^{2}+m^{2}x_{t}^{2}+m^{2}a\beta x_{t}^{2}+\beta m^{2}x_{t}y_{t})}{\beta (\beta +m)^{3}} \\& \hphantom{F_{1} ( x_{t},y_{t} ) =}{} +O \bigl( x _{t}^{4} \bigr), \\& F_{2} ( x_{t},y_{t} ) =\frac{1}{\beta }x_{t}y_{t}+O \bigl( x_{t}^{4} \bigr) . \end{aligned}$$

The system (19) can be expressed as

( x t + 1 y t + 1 ) =J ( E ∗ ) ( x t y t ) + 1 2 B( x t , x t )+ 1 6 C( x t , x t , x t )+O ( x t 4 ) ,
(20)

where B(x,y)= ( B 1 ( x , y ) B 2 ( x , y ) ) and C(x,y,u)= ( C 1 ( x , y , u ) C 2 ( x , y , u ) ) are symmetric multilinear vector functions of \(x,y,u\in R ^{2}\). These functions are defined as follows:

$$ \begin{aligned}&B_{1} ( x,y ) =\sum^{2}_{j,k=1} \frac{\partial ^{2}F_{1}}{ \partial \xi _{j}\partial \xi _{k}}\bigg\vert _{\xi =0} \\ &\hphantom{B_{1} ( x,y )}= \bigl(2x_{1}y_{1}a\beta ^{4}+2x_{1}y_{1}m\beta -3x_{1}y_{2}m\beta ^{3}-2x _{1}y_{2}\beta ^{2}m^{2}-3x_{2}y_{1}\beta ^{3}m \\ &\hphantom{B_{1} ( x,y )=}{}-2x_{2}y_{1}\beta ^{2}m ^{2}-2x_{1}y_{1}am^{3}\bigr)/\bigl(\beta (\beta +m)^{3}\bigr) \\ &\hphantom{B_{1} ( x,y )=}{}+ \bigl(6x_{1}y_{1}xm^{2}a-x_{1}y_{2}\beta ^{4}-x_{2}y_{1}\beta ^{4}-6x _{1}y_{1}a\beta ^{3}m-4x_{1}y_{1}a\beta ^{2}m^{2} \\ &\hphantom{B_{1} ( x,y )=}{}-2x_{1}y_{1}m^{2}a \beta +2x_{1}y_{1}m^{3}\bigr)/\bigl(\beta (\beta +m)^{3}\bigr), \\ &B_{2} ( x,y ) =\sum^{2}_{j,k=1} \frac{\partial ^{2}F_{2}}{\partial \xi _{j}\partial \xi _{k}}\bigg\vert _{\xi =0}=\frac{x_{1}y_{2}+x_{2y_{1}}}{\beta }, \\ &C_{1} ( x,y )= \sum^{2}_{j,k=1} \frac{\partial ^{3}F_{1}}{\partial \xi _{j}\partial \xi _{k}\partial \xi _{l}}\bigg\vert _{\xi =0} \\ &\hphantom{C_{1} ( x,y )} =- \bigl(2m^{2}(-3ax_{1}y_{1}v_{1}+3x_{1}y_{1}u_{1}+3a\beta x_{1}y _{1}u_{1}+\beta x_{1}y_{1}u_{2} \\ &\hphantom{C_{1} ( x,y )=}{}+\beta x_{1}y_{1}v_{1}+\beta x_{2}y _{1}u_{1})\bigr)/\bigl(\beta (\beta +m)^{3}\bigr), \\ &C_{2} ( x,y ) =\sum^{2}_{j,k=1} \frac{\partial ^{3}F_{2}}{\partial \xi _{j}\partial \xi _{k}\partial \xi _{l}}\bigg\vert _{\xi =0}=0. \end{aligned} $$
(21)

\(\forall X\in \mathbb{R} ^{2}\) can be uniquely represented near \(a_{1}\) by

$$ X=zq+\overline{z}\overline{q} $$

for some \(z\in \mathbb{C} \). The explicit formula of z is determined as \(z=\langle p,X\rangle \).

The system (19) can be transformed for all sufficiently small \(\vert a \vert \) into the form

$$ z\rightarrow \lambda ( a ) z+g ( z,\overline{z},a ) , $$
(22)

where \(\lambda ( a ) = ( 1+\varphi ( a ) ) e^{i\arctan ( a ) }\) with \(\varphi (a_{NS})=0\) and \(g ( z,\overline{z},a ) \) is smooth complex-valued function. After taking the Taylor expression of g with respect to \(( z, \overline{z} )\), we obtain

$$ g ( z,\overline{z},a ) =\underset{k+l\geq 2}{\sum } \frac{1}{k!l!}g_{kl} ( a ) z^{k}\overline{z}^{l}, \quad \text{with }g_{kl}\in k,l=0,1,\ldots $$
(23)

By symmetric multilinear vector functions, the Taylor coefficients \(g_{kl}\) can be expressed by the formulas

$$\begin{aligned}& \begin{aligned}&g_{20} ( a_{NS} ) =\bigl\langle p,B ( q,q ) \bigr\rangle , \\ &g_{11} ( a_{NS} ) =\bigl\langle p,B ( q,\overline{q} ) \bigr\rangle , \\ &g_{02} ( a_{NS} ) =\bigl\langle p,B (\overline{q}, \overline{q} ) \bigr\rangle , \\ &g_{21} ( a_{NS} ) =\bigl\langle p,C ( q,q,\overline{q} ) \bigr\rangle . \end{aligned} \end{aligned}$$
(24)

The coefficient \(\beta _{2}(a_{NS})\), which determines the direction of the appearance of the invariant curve in a generic system exhibiting the Neimark–Sacker bifurcation, can be calculated via

$$\begin{aligned} \beta _{2} ( a_{NS} ) =&\operatorname{Re} \biggl( \frac{e^{-i \arctan ( a_{NS} ) }}{2}g_{21} \biggr) \\ &{}-\operatorname{Re} \biggl( \frac{ ( 1-2e^{i\arctan ( a_{NS} ) } ) e^{-2i\arctan ( a_{NS} ) }}{2 ( 1-e^{i \arctan ( a_{NS} ) } ) }g_{20}g_{11} \biggr) \\ &{}-\frac{1}{2} \vert g_{11} \vert ^{2}- \frac{1}{4} \vert g _{02} \vert ^{2} \end{aligned}$$
(25)

where \(e^{i\arctan ( a_{NS} ) }=\lambda ( a_{NS} ) \).

We state the following theorem on the Neimark–Sacker bifurcation.

Theorem 1

Suppose that \(E_{\ast }\) is positive fixed point of the model (2). If (13) holds, \(1-2\beta -m\neq 0\), \(\beta _{2} ( a _{NS} ) \neq 0\) and the parameter a changes its value in small vicinity of \(\varOmega _{NSB_{E^{\ast }}}\), then the model (2) passes through a Neimark–Sacker bifurcation at only fixed point \(E_{\ast }\). Moreover, if \(\beta _{2} ( a_{NS} ) <0\) (resp., \(\beta _{2} ( a_{NS} ) >0\)), then the Neimark–Sacker bifurcation of model (2) at \(a=a_{NS}\) is supercritical (resp., subcritical) and there exists a unique closed invariant curve bifurcation from \(E_{\ast }\) for \(a=a_{NS}\), which is attracting (resp., repelling).

4 Numerical simulations

In this section, our aim is to present numerical simulations to validate the above theoretical results, especially the bifurcation diagrams and phase portraits for system (2) around fixed point \(E_{\ast }\). We will choose a as the bifurcation parameter. The Neimark–Sacker bifurcation point as \(a_{NS}=3.448275862\) is obtained. By taking the parameters values \(a_{NS}=3.448275862\), \(m=0.25\) and \(\beta =0.23\), the positive fixed point of the model (2) is evaluated as \(E_{\ast }= ( 0.23,3.454150957 ) \).

Using the above the parameters, we get the jacobian matrix as follows:

J E ∗ ( a N S )= [ − 0.6550835770 − 0.1102083333 15.01804764 1 ] .

The eigenvalues are evaluated as follows:

$$ \lambda _{1,2}=0.1724582115\pm 0.9850322777i. $$

Let \(q,p\in C^{2}\) be complex eigenvectors corresponding to \(\lambda _{1,2}\), respectively.

$$ q\sim ( -0.06558990232-0.05510315377i,i ) ^{T} $$

and

$$ p\sim ( 8.937911026-7.508885793i,-i ) ^{T}. $$

To obtain the normalization \(\langle p,q \rangle =1\), we can take normalized vectors as

$$ q= ( -0.06558990232-0.05510315377i,i ) ^{T} $$

and

$$ p=\bigl(-7.623124629+0.2149349453.10^{-8}i,-0.4200582087+0.5000000001i \bigr)^{T}. $$

Because of computing the coefficients of normal form, we transform the point \(( 0,0 ) \) to the fixed point \(E_{\ast }\) by a change of variables,

$$\begin{aligned} &X =x-0.23, \\ &Y =y-3.454150957. \end{aligned}$$

So, we can compute the coefficients of the normal of the system by using Eq. (24) as follows:

$$\begin{aligned}& g_{20}(\delta ) =-2.007675632-0.1335717640i, \\& g_{11}(\delta ) = 2.430928813+0.2395789296i, \\& g_{02}(\delta ) =1.186267438-2.549712245i, \\& g_{21}(\delta ) =0.1518172527+0.1907737437i. \end{aligned}$$

From (25), the critical part is obtained: \(\beta _{2} ( a _{NS} ) =-8.214126411<0\). Therefore, the Neimark–Sacker bifurcation is supercritical and it shows the correctness of Theorem 1. The bifurcation diagram and the phase portrait of the system (2) is shown in Figs. 2 and 3.

Figure 2
figure 2

Bifurcation diagram for the system (2) for values of \(m=0.25\), \(\beta =0.23\), \(a=3.3:0.0001:3.4\) and initial value \(( x_{0},y_{0} ) =(0.2,3.4)\)

Full size image
Figure 3
figure 3

Phase portraits of the system (2) for different values of a

Full size image

The bifurcation diagrams shown in Fig. 2 show that the stability of \(E_{\ast }\) holds for \(a<3.4482\) and loses its stability at \(a=3.4482\) and an attracting invariant curve appears if \(a>3.4482\).

The phase portraits of bifurcation diagrams in Fig. 2 for different values of a are displayed in Fig. 3, which clearly depicts the process of how a smooth invariant circle bifurcates from the stable fixed point \(E_{\ast }= ( 0.23,3.454150957 ) \). When a exceeds 3.448275862 there appears a circular curve enclosing the fixed point \(E_{\ast }\), and its radius becomes larger with respect to the growth of a.

5 Discussions

This paper is about to stability and bifurcation analysis of a discrete-time predator–prey model with Allee effect on prey population. We showed that the system (2) has three fixed points namely \(E_{1}\), \(E_{2}\) and \(E_{\ast} \). The topological classifications of these fixed points were given. By bifurcation theory [39,40,41] we showed that the system (2) undergoes a Neimark–Sacker bifurcation at unique coexistence fixed point if a varies around the set \(\varOmega _{NSB_{E_{\ast } }}\). The parametric conditions for existence and direction of Neimark–Sacker bifurcation in positive fixed point \(E_{\ast }\) were given. Also, we verified the theoretical results by presenting some numerical simulations and phase portrait diagrams using MATLAB. We displayed that when the bifurcation parameter a passes a critical bifurcation value, stability of the coexistence equilibrium point of the model (2) changes from stable to unstable and Neimark–Sacker bifurcation occurs at this critical value. Therefore, we can say that the parameter a has a strong effect on the stability of the system so as control two populations.