1 Introduction

During the past decade, a competitive system has been extensively investigated by many scholars [1–24], many excellent results concerned with the permanence, extinction, and global attractivity of the competition system have been obtained.

Traditional two-species Lotka–Volterra competitive system is as follows:

$$ \begin{gathered} \frac{dx_{1}}{dt} = x_{1}(b_{1}-a_{11}x_{1}-a_{12}x_{2}), \\ \frac{dx_{2}}{dt} = x_{2}(b_{2}-a_{21}x_{1}-a_{22}x_{2}), \end{gathered} $$
(1.1)

where \(x_{1}\) and \(x_{2}\) denote the population density of the first and second species at time t, respectively. \(b_{i}\), \(a_{ij}\), \(i, j=1, 2\), are positive constants. System (1.1) has been investigated in mathematical biology books [25]. Depending on the relationship of the coefficients of system (1.1), it has the following dynamic behaviors.

  1. (1)

    If

    $$ \frac{a_{11}}{a_{21}}> \frac{b_{1}}{b_{2}}> \frac{a_{12}}{a_{22}}, $$

    holds, system (1.1) has a unique positive equilibrium, which is globally attractive.

  2. (2)

    If

    $$ \frac{b_{1}}{b_{2}}> \frac{a_{11}}{a_{21}},\qquad \frac{b_{1}}{b_{2}}> \frac{a_{12}}{a_{22}}, $$

    holds, then the second species will be driven to extinction, while the first species will approach \(\frac{b_{1}}{a_{11}}\).

On the other hand, the study of resource-management, including fisheries, forestry, and wildlife management, has great importance as human activity is the main cause of the extinction of endangered species. It is necessary to harvest the population, but harvesting should be regulated so that both the ecological sustainability and conservation of the species can be implemented in a long run. Many scholars are interested in establishing appropriate biological models to further understand the scientific management of renewable resources. For example, based on model (1.1), Sharma and Samanta [26] further considered the harvesting and obtained the following model:

$$ \begin{gathered} \frac{dx(t; p)}{dt}=x(t)\bigl[(r_{1l})^{(1-p)}(r_{1u})^{p}-(b_{11l})^{(1-p)}(b_{11u})^{p}x(t) \\ \hphantom{\frac{dx(t; p)}{dt}=} {}-(b_{12l})^{(1-p)}(b_{12u})^{p}y(t) \bigr]-q_{1}Ex(t), \\ \frac{dy(t; p)}{dt}=y(t)\bigl[(r_{2l})^{(1-p)}(r_{2u})^{p}-(b_{22l})^{(1-p)}(b_{22u})^{p}y(t) \\ \hphantom{\frac{dy(t; p)}{dt}=} {}-(b_{21l})^{(1-p)}(b_{21u})^{p}x(t) \bigr]-q_{2}Ey(t), \end{gathered} $$
(1.2)

where \(p\in [0,1]\). Due to the difficulty in the estimation of the model parameters, the authors argued that taking account of the imprecise of biological parameter values makes some situations more realistic. They developed a method to handle imprecise parameters. In addition, they discussed the existence and stability of the system equilibria, as well as the bionomic equilibrium and optimal harvesting policy. The equilibrium solution of the control problem was given and the harvest strategy was optimized dynamically.

Ecosystem with harvesting has been extensively investigated by many scholars [27–37]. May et al. [31] proposed two types of harvesting: (1) constant harvest and (2) linear harvest. For the first case, it is impossible to harvest a certain number of species every year. Although this type of harvest may be relatively easy to study, it is not a reality. For the second case, the harvesting term takes the form \(h(x)=qEx\), obviously, when x or E tends to infinity, \(h(x)\) tends to infinity. Clearly this contradicts the facts, because in reality there is limited harvesting capacity or number of species, so the amount of species that can be harvested is limited. To overcome the drawback of the two kinds of harvesting, Clark [32] proposed the Michaelis–Menten type harvesting \(h(x)=\frac{qEx}{mE+nx}\) when x or E tends to infinity, \(h(x)\) tends to \(\frac{qE}{n}\) or \(\frac{qx}{m}\). In this type of harvesting, if the number of species tends to infinity, the final harvest depends on the harvesting capacity, and if the harvesting capacity tends to infinity, the final harvest depends on the number of species, which is in accordance with the actual situation. Since such kind of harvesting is more suitable, it brings many scholars to do works in this direction (see [33–37] and the references cited therein). For example, Yu, Chen, and Lai [35] introduced Michaelis–Menten type harvesting into a May type cooperative system and discussed the extinction of the first species and the global attraction of the unique positive equilibrium. Chen [36] studied the Lotka–Volterra commensal symbiosis model with Michaelis–Menten type harvesting; the modified model takes the following:

$$ \begin{gathered} \frac{dx}{dt}=\gamma _{1}x \biggl(1- \frac{x}{k_{1}}+\alpha \frac{y}{k_{1}} \biggr)- \frac{qEx}{m_{1}E+m_{2}x}, \\ \frac{dy}{dt}=\gamma _{2}y \biggl(1- \frac{y}{k_{2}} \biggr), \end{gathered} $$
(1.3)

where x and y denote the population density of the two species at time t, respectively. q denotes the fishing coefficient of the first species and E denotes the fishing effort. \(r_{1}\), \(r_{2}\), \(K_{1}\), \(K_{2}\), α, \(m_{1}\), \(m_{1}\) are all positive constants. The results show that the system has a globally asymptotically stable positive equilibrium. In addition, the two species can coexist stably when α and \(K_{2}\) are large enough. On the contrary, the first species will be driven to extinction.

After that, Liu et al. [37] considered the two-species amensalism model with Michaelis–Menten type harvesting and a cover for the first species; the modified model takes the following:

$$ \begin{gathered} \frac{dx}{dt}=a_{1}x-b_{1}x^{2}-c_{1}(1-k)xy- \frac{qE(1-k)x}{m_{1}E+m_{2}(1-k)x}, \\ \frac{dy}{dt}=a_{2}y-b_{2}y^{2}. \end{gathered} $$
(1.4)

Here, x and y denote the population density of the two species at time t, and k is the refuge (\(0< k<1\)). When the parameters satisfy certain conditions, saddle-node bifurcation and transcritical bifurcation will occur in the system, and the maximum threshold of species without extinction risk under continuous fishing is obtained. Scholars’ research shows that compared with the linear harvesting model, the Michaelis–Menten harvesting type model can not only reflect the harvesting process more realistically, but also show richer dynamic behavior.

Nevertheless, to the best of our knowledge, no scholars have proposed and studied the dynamic behaviors of the two-species competitive model with Michaelis–Menten type harvesting. So in this paper, based on model (1.1), we propose the following system:

$$ \begin{gathered} \frac{dx}{dt}=r_{1}x\biggl(1- \frac{x}{k_{1}}\biggr)- \alpha _{1}xy- \frac{q_{1}Ex}{m_{1}E+h_{1}x}, \\ \frac{dy}{dt}=r_{1}y\biggl(1- \frac{y}{k_{2}}\biggr)- \alpha _{2}xy, \end{gathered} $$
(1.5)

where \(r_{1}\), \(r_{2}\), \(k_{1}\), \(k_{2}\), \(\alpha _{1}\), \(\alpha _{2}\), \(q_{1}\), \(m_{1}\), \(h_{1}\), and E are all positive. For simplicity, we make the following nondimensional scheme:

$$ \bar{t}=r_{1}t,\qquad \bar{x}= \frac{1}{k_{1}}x,\qquad \bar{y}= \frac{1}{k_{2}}y. $$

Dropping the bars, we have the following system:

$$ \begin{gathered} \frac{dx}{dt}=x\biggl(1-x-a_{1}y- \frac{b_{1}}{c_{1}+x}\biggr), \\ \frac{dy}{dt}=\rho y(1-y-a_{2}x), \end{gathered} $$
(1.6)

where \(a_{1}=\frac{\alpha _{1}k_{2}}{r_{1}}\), \(b_{1}= \frac{q_{1}E}{k_{1}r_{1}h_{1}}\), \(c_{1}=\frac{m_{1}E}{h_{1}k_{1}}\), \(\rho =\frac{r_{2}}{r_{1}}\), \(a_{2}= \frac{k_{1}\alpha _{2}}{r_{2}}\), and the initial conditions

$$ x(0)=x_{0}>0,\qquad y(0)=y_{0}>0. $$
(1.7)

Biologically, we consider system (1.6) is defined on the set

$$ R_{0}^{+}\times R_{0}^{+}=\bigl\{ (x,y) \in R^{2}\mid x\geq 0,y\geq 0\bigr\} . $$

The organization of this paper is as follows. The basic properties of the model are discussed in the next section. We analyze the existence of the equilibria of the system in Sect. 3. The local stability of the equilibria of the system are investigated in Sect. 4. We consider the global stability of the positive equilibrium in Sect. 5. The possible bifurcation of the system is studied in Sect. 6. Numerical simulation is presented to show the feasibility of theoretical results in Sect. 7, and a brief discussion of our results is given in the last section.

2 Basic properties of the model

Lemma 2.1

([38])

When\(a,b>0\)and\(\frac{dx}{dt}\leq (\geq )x(t)(a-bx(t))\)with\(x(0)>0\), then

$$ \limsup_{t\rightarrow +\infty }x(t)\leq \frac{a}{b}\biggl(\liminf _{t\rightarrow +\infty }x(t)\geq \frac{a}{b}\biggr). $$

Definition 2.1

System (1.6) is said to be permanent if there exist two constants m and M (\(0< m< M\)) such that each positive solution \((x(t,x_{0},y_{0}),y(t,x_{0},y_{0}))\) of system (1.6) under the initial condition \((x_{0},y_{0})\in \operatorname{Int}(R_{+}^{2})\) satisfies

$$\begin{aligned}& \min \Bigl\{ \liminf_{t\rightarrow +\infty }x(t,x_{0},y_{0}), \liminf_{t\rightarrow +\infty }y(t,x_{0},y_{0})\Bigr\} \geq m, \\& \max \Bigl\{ \limsup_{t\rightarrow +\infty }x(t,x_{0},y_{0}), \limsup_{t\rightarrow +\infty }y(t,x_{0},y_{0})\Bigr\} \leq M. \end{aligned}$$

Theorem 2.1

All solutions\((x(t),y(t))\)of system (1.6) are positive under initial condition (1.7), i.e., \(x(t)>0\), \(y(t)>0\)for all\(t\geq 0\).

Proof

Since

$$ x(t)=x(0)\exp \biggl\{ \int _{0}^{t} \biggl(1-x(s)-a_{1}y(s)- \frac{b_{1}}{c_{1}+x(s)}\biggr) \,ds\biggr\} >0 $$

and

$$ y(t)=y(0)\exp \biggl\{ \int _{0}^{t} \rho \bigl(1-y(s)-a_{2}x(s) \bigr) \,ds\biggr\} >0. $$

So \(x(t)>0\), \(y(t)>0\) for all \(t\geq 0\) with initial condition (1.7).

This completes the proof. □

Theorem 2.2

All solutions\((x(t),y(t))\)of system (1.6) that satisfy initial condition (1.7) are bounded, for all\(t\geq 0\).

Proof

Based on the positivity of variable x, y, from system (1.6), we have

$$ \frac{dx}{dt}=x\biggl(1-x-a_{1}y- \frac{b_{1}}{c_{1}+x}\biggr)\leq x(1-x). $$
(2.1)

From Lemma 2.1, we can obtain

$$ {\limsup_{t\rightarrow +\infty }x(t)}\leq 1. $$
(2.2)

Meanwhile, from system (1.6), we have

$$ \frac{dy}{dt}=\rho y(1-y-a_{2}x)\leq \rho y(1-y). $$
(2.3)

Again from the same Lemma 2.1, we can get

$$ {\limsup_{t\rightarrow +\infty }y(t)}\leq 1. $$
(2.4)

This completes the proof. □

Theorem 2.3

Assume that\(a_{1}+\frac{b_{1}}{c_{1}}<1\), \(0< a_{2}<1\), system (1.6) is permanent with initial condition (1.7).

Proof

From (2.2) and (2.4), for \(\varepsilon >0\) small enough, there is \(T>0\) such that, for \(t>T\), we have

$$ x(t)\leq 1+\varepsilon ,\qquad y(t)\leq 1+\varepsilon . $$

Then from the first equation of system (1.6), one could get

$$ \frac{dx}{dt}=x\biggl(1-x-a_{1}y- \frac{b_{1}}{c_{1}+x}\biggr)\geq x\biggl(1-a_{1}(1+\varepsilon )- \frac{b_{1}}{c_{1}}-x\biggr). $$

From Lemma 2.1, when \(1-a_{1}(1+\varepsilon )-\frac{b_{1}}{c_{1}}>0\), we can get

$$ {\liminf_{t\rightarrow +\infty }x(t)}\geq 1-a_{1}(1+ \varepsilon )- \frac{b_{1}}{c_{1}}. $$

Let \(\varepsilon \rightarrow 0\) in this inequality, we can obtain

$$ {\liminf_{t\rightarrow +\infty }x(t)\geq 1-a_{1}- \frac{b_{1}}{c_{1}}}=\omega _{1}. $$

In the same way, we can obtain the following results for \(y(t)\):

$$ {\liminf_{t\rightarrow +\infty }y(t)\geq \omega _{2}}, $$

where \(\omega _{2}=1-a_{2}>0\), i.e., \(0< a_{2}<1\). So we choose \(m=\min (\omega _{1},\omega _{2})\), \(M=1\).

This completes the proof. □

3 The existence of equilibria

The equilibria of system (1.6) are determined by the following equations:

$$ \begin{gathered} x\biggl(1-x-a_{1}y- \frac{b_{1}}{c_{1}+x} \biggr)=0, \\ \rho y(1-y-a_{2}x)=0. \end{gathered} $$
(3.1)

Obviously, system (1.6) always has two boundary equilibria \(E_{0}(0,0)\) and \(E_{1}(0,1)\) for all parameters. For other possible boundary equilibria and positive equilibria, we consider the following cases:

(i) When \(x\neq 0\), \(y=0\), the other boundary equilibria of system (1.6) satisfy the following equation:

$$ x^{2}-(1-c_{1})x+b_{1}-c_{1}=0. $$
(3.2)

Let \(\Delta _{1}\) denote the discriminant with express \(\Delta _{1}\) in terms of \(b_{1}\), i.e.,

$$ \Delta _{1}(b_{1})=(1+c_{1})^{2}-4b_{1}. $$

If the equilibria for system (1.6) exist, then \(\Delta _{1}(b_{1})\geq 0\), i.e.,

\((H_{1})\):

\(b_{1}\leq \frac{1+c_{1}}{4}:=b_{1}^{*}\),

and

$$ x_{21}= \frac{1-c_{1}-\sqrt{\Delta _{1}(b_{1})}}{2},\qquad x_{22}= \frac{1-c_{1}+\sqrt{\Delta _{1}(b_{1})}}{2}. $$

If \((1-c_{1})^{2}=\Delta _{1}(b_{1})\), one could calculate that

\((H_{2})\):

\(b_{1}=c_{1}\).

Therefore, we can calculate that

\((H_{3})\):

if \(b_{1}>c_{1}\), then \((1-c_{1})^{2}>\Delta _{1}(b_{1})\);

\((H_{4})\):

if \(b_{1}< c_{1}\), then \((1-c_{1})^{2}<\Delta _{1}(b_{1})\).

Besides, we have \(c_{1}\leq b_{1}^{*}\) and \(c_{1}= b_{1}^{*}\) if and only if \(c_{1}=1\).

From conditions \((H_{1})\), \((H_{2})\), \((H_{3})\), and \((H_{4})\), we can get that:

(1) If \(0< b_{1}< c_{1}\), \(x_{21}<0\), \(x_{22}>0\).

(2) If \(b_{1}=c_{1}\), when \(0< c_{1}<1\), \(x_{21}=0\), \(x_{22}>0\); when \(c_{1}=1\), \(x_{21}=x_{22}=0\); when \(c_{1}>1\), \(x_{21}<0\), \(x_{22}=0\).

(3) If \(c_{1}< b_{1}< b_{1}^{*}\), when \(0< c_{1}<1\), \(x_{21}>0\), \(x_{22}>0\); when \(c_{1}>1\), \(x_{21}<0\), \(x_{22}<0\).

(4) If \(b_{1}=b_{1}^{*}\), when \(0< c_{1}<1\), \(x_{23}>0\).

(5) If \(b_{1}>b_{1}^{*}\), system (1.6) has no other boundary equilibria.

(ii) When \(x\neq 0\), \(y\neq 0\), the possible positive equilibria of system (1.6) are determined by the following equation:

$$ \begin{gathered} 1-x-a_{1}y- \frac{b_{1}}{c_{1}+x}=0, \\ 1-y-a_{2}x=0, \end{gathered} $$
(3.3)

i.e.,

$$ Ax^{2}-Bx+C=0, $$
(3.4)

where \(A=a_{1}a_{2}-1\), \(B=a_{1}+c_{1}-a_{1}a_{2}c_{1}-1\), \(C=c_{1}-a_{1}c_{1}-b_{1}\).

Let the discriminant of (3.4) be denoted by \(\Delta _{2}\), i.e.,

$$ \Delta _{2}=B^{2}-4AC=(-c_{1}A-a_{1}+1)^{2}+4b_{1}A. $$

It is obvious that \(\Delta _{2}>0\) if \(A>0\).

When \(\Delta _{2}\geq 0\), the existence of positive equilibria of system (1.6), and

$$ x_{31}= \frac{B-\sqrt{\Delta _{2}}}{2A}, x_{32}= \frac{B+\sqrt{\Delta _{2}}}{2A}. $$
(3.5)

(6) If \(\Delta _{2}>0\), let us discuss the following cases:

Case 1: When (a) \(A>0\), \(B>0\), \(C<0\) or (b) \(A>0\), \(B<0\), \(C<0\), then \(x^{*}=x_{32}>0\), \(x_{31}<0\) and system (1.6) has only one positive equilibrium \(E^{*}(x^{*},y^{*})=(x^{*},1-a_{2}x^{*})\) if \(\frac{1}{a_{2}}>x^{*}\).

Case 2: When (c) \(A<0\), \(B>0\), \(C>0\) or (d) \(A<0\), \(B<0\), \(C>0\), then \(x^{*}=x_{31}>0\), \(x_{32}<0\) and system (1.6) has only one positive equilibrium \(E^{*}(x^{*},y^{*})=(x^{*},1-a_{2}x^{*})\) if \(\frac{1}{a_{2}}>x^{*}\).

Case 3: When \(A<0\), \(B<0\), \(C<0\), then \(x_{31}>x_{32}>0\) and system (1.6) has two positive equilibria:

$$ E_{31}(x_{31},y_{31})=(x_{31},1-a_{2}x_{31}) $$

and

$$ E_{32}(x_{32},y_{32})=(x_{32},1-a_{2}x_{32}). $$

Both \(E_{31}\) and \(E_{32}\) will exist if \(\frac{1}{a_{2}}>\max \{x_{31}, x_{32}\}\).

Case 4: When \(A>0\), \(B<0\), \(C=0\), then \(x_{32}=0>x_{31}\) and system (1.6) has no positive equilibria.

Case 5: When \(A<0\), \(B>0\), \(C=0\), then \(x_{31}=0>x_{32}\) and system (1.6) has no positive equilibria.

Case 6: When \(A<0\), \(B<0\), \(C=0\), then \(x_{31}>0=x_{32}\) and system (1.6) has only one positive equilibrium \(E_{31}(x_{31},y_{31})=(x_{31},1-a_{2}x_{31})\) if \(\frac{1}{a_{2}}>x_{31}\).

(7) If \(\Delta _{2}=0\), \(B<0\), then \(x_{31}=x_{32}=x_{33}>0\) and system (1.6) has only one positive equilibrium \(E_{33}(x_{33},y_{33})=(\frac{B}{2A},1-a_{2} \frac{B}{2A})\) if \(\frac{1}{a_{2}}>\frac{B}{2A}\).

(8) If \(\Delta _{2}<0\), then system (1.6) has no positive equilibria.

From what has been discussed above, we can get the following theorem.

Theorem 3.1

System (1.6) always has two boundary equilibria\(E_{0}(0,0)\)and\(E_{1}(0,1)\)for all parameters. The other possible boundary equilibria and positive equilibria are as follows:

  1. (i)

    For other possible boundary equilibria:

    1. (1)

      If\(0< b_{1}< c_{1}\), system (1.6) has only one other boundary equilibrium\(E_{22}(x_{22},0)\).

    2. (2)

      If\(b_{1}=c_{1}\)and\(0< c_{1}<1\), system (1.6) has only one other boundary equilibrium\(E_{22}(x_{22},0)\).

    3. (3)

      If\(c_{1}< b_{1}< b_{1}^{*}\)and\(0< c_{1}<1\), system (1.6) has two other boundary equilibria\(E_{21}(x_{21},0)\)and\(E_{22}(x_{22},0)\).

    4. (4)

      If\(b_{1}=b_{1}^{*}\)and\(0< c_{1}<1\), system (1.6) has only one other boundary equilibrium\(E_{23}(x_{23},0)\).

    5. (5)

      If\(b_{1}>b_{1}^{*}\), system (1.6) has no other boundary equilibria.

  2. (ii)

    For positive boundary equilibria:

    1. (6)

      If\(\Delta _{2}\geq 0\), we have the following.

      1. (a)

        When\(A>0\), \(B>0\), \(C<0\)or\(A>0\), \(B<0\), \(C<0\)or\(A<0\), \(B<0\), \(C\geq 0\)or\(A<0\), \(B>0\), \(C>0\), then system (1.6) has only one positive equilibrium\(E^{*}(x^{*},y^{*})\)if\(\frac{1}{a_{2}}>x^{*}\).

      2. (b)

        When\(A<0\), \(B<0\), \(C<0\), then system (1.6) has two positive equilibria\(E_{31}(x_{31},y_{31})\)and\(E_{32}(x_{32},y_{32})\)if\(\frac{1}{a_{2}}>\max \{x_{31}, x_{32}\}\).

    2. (7)

      If\(\Delta _{2}=0\), then system (1.6) has only one positive equilibrium\(E_{33}(x_{33},y_{33})\).

    3. (8)

      If\(\Delta _{2}<0\), then system (1.6) has no positive equilibria.

4 Stability of equilibria

Theorem 4.1

For the boundary equilibria\(E_{0}\)and\(E_{1}\)of system (1.6), which always exist, we have:

  1. (1)

    \(E_{0}\)is always unstable.

  2. (2)

    For\(E_{1}\), we have the following results:

    1. (a)

      If\(b_{1}< c_{1}(1-a_{1})\), \(E_{1}\)is a saddle;

    2. (b)

      If\(b_{1}>c_{1}(1-a_{1})\), \(E_{1}\)is a stable node;

    3. (c)

      If\(b_{1}< c_{1}(1-a_{1})\), \(E_{1}\)is a saddle node with\(a_{1}a_{2}-1+\frac{1-a_{1}}{c_{1}}\neq 0\); \(E_{1}\)is a saddle with\(a_{1}a_{2}-1+\frac{1-a_{1}}{c_{1}}=0\).

Proof

Firstly, we calculate the Jacobi matrix of system (1.6):

$$ J(x,y)= \begin{pmatrix} 1-2x-a_{1}y- \frac{b_{1}c_{1}}{(c_{1}+x)^{2}}&-a_{1}x \\ -\rho a_{2}y& \rho (-a_{2}x-2y+1)\end{pmatrix}. $$
(4.1)

(1) The Jacobian matrix of system (1.6) at \(E_{0}\) is

$$ J(E_{0})= \begin{pmatrix} 1- \frac{b_{1}}{c_{1}}&0 \\ 0& \rho \end{pmatrix}. $$
(4.2)

Obviously, the eigenvalues of \(J(E_{0})\) are \(\lambda _{1}=1-\frac{b_{1}}{c_{1}}\) and \(\lambda _{2}=\rho >0\), so \(E_{0}\) is always unstable.

(2) The Jacobian matrix of system (1.6) at \(E_{1}\) is given by

$$ J(E_{1})= \begin{pmatrix} 1-a_{1}- \frac{b_{1}}{c_{1}}&0 \\ -\rho a_{2}& -\rho \end{pmatrix}. $$
(4.3)

It is obvious that the eigenvalues of \(J(E_{1})\) are \(\lambda _{1}=1-a_{1}-\frac{b_{1}}{c_{1}}\) and \(\lambda _{2}=-\rho <0\), so the stability of \(J(E_{1})\) depends on \(\lambda _{1}\). \(E_{1}\) is a saddle if \(b_{1}< c_{1}(1-a_{1})\) and a stable node if \(b_{1}>c_{1}(1-a_{1})\). When \(b_{1}=c_{1}(1-a_{1})\), we cannot directly come to the conclusion.

Let us first move \(E_{1}\) to the origin by transforming \((X,Y)=(x,y-1)\) and make Taylor’s expansion around the origin, under which system (1.6) is as follows:

$$ \begin{gathered} \frac{dX}{dt}=\biggl( \frac{b_{1}}{c_{1}^{2}}-1 \biggr)x^{2}-a_{1}xy- \frac{b_{1}}{c_{1}^{3}}x^{3}+ \frac{b_{1}}{c_{1}^{4}}x^{4}, \\ \frac{dY}{dt}=-\rho a_{2}x-\rho y-\rho y^{2}-\rho a_{2}xy. \end{gathered} $$
(4.4)

Next, making the following transformation

$$ \begin{pmatrix} X \\ Y \end{pmatrix} ={ \begin{pmatrix} a_{2}&0 \\ -a_{2}^{2}&-a_{2} \end{pmatrix} \begin{pmatrix} X_{1} \\ Y_{1} \end{pmatrix}}, $$
(4.5)

and letting \(\tau =\rho t\), system (4.4) becomes

$$ \begin{gathered} \frac{dX_{1}}{d\tau }=a_{01}X_{1}Y_{1}+a_{02}X_{1}^{2}+a_{03}X_{1}^{3}+P_{1}(X_{1},Y_{1}), \\ \frac{dY_{1}}{d\tau }=Y_{1}+b_{01}X_{1}Y_{1}+b_{02}X_{1}^{2}+b_{03}Y_{1}^{2}+b_{04}X_{1}^{3} +Q_{1}(X_{1},Y_{1}):=P(X_{1},Y_{1}), \end{gathered} $$
(4.6)

where

$$\begin{aligned}& a_{01}=-\frac{a_{1}a_{2}}{\rho },\qquad a_{02}=\frac{a_{2} (c_{1}(a_{1}a_{2}-1)+1-a_{1} )}{\rho c_{1}}, \qquad a_{03}=-\frac{b_{1}a_{2}^{2}}{\rho c_{1}^{3}},\\& b_{01}=\frac{a_{2}^{2}(a_{1}-\rho )}{\rho }, \qquad b_{02}=-\frac{a_{2}^{2} (c_{1}(a_{1}a_{2}-1)+1-a_{1} )}{\rho c_{1}},\qquad b_{03}=a_{1},\qquad b_{04}=\frac{b_{1}a_{2}^{3}}{\rho c_{1}^{3}}. \end{aligned}$$

And \(P_{1}(X_{1},Y_{1})\) and \(Q_{1}(X_{1},Y_{1})\) are power series in \((X_{1},Y_{1})\) with terms \(X_{1}^{i}Y_{1}^{j}\) satisfying \(i+j\geq 4\).

From the implicit function theorem, there exists a function \(Y_{1}=\varphi (X_{1})\) that satisfies \(\varphi (0)=0\) and \(P(X_{1},\varphi (X_{1}))=0\). By using the second equation of system (4.6), we can obtain

$$ Y_{1}=\varphi (X_{1})=-b_{02}X_{1}^{2}+o \bigl(X_{1}^{3}\bigr). $$
(4.7)

Substituting (4.7) into the first equation of system (4.6),we have

$$ \frac{dX_{1}}{d\tau }=a_{02}X_{1}^{2}- \biggl(a_{01}b_{02}+ \frac{a_{2}^{2}(1-a_{1})}{\rho c_{1}^{2}} \biggr)X_{1}^{3}+ \cdots. $$

By Theorem 7.1 in Chap. 2 in [39], when \(a_{02}\neq 0\), i.e., \(m=2\), \(E_{1}\) is a saddle node; when \(a_{02}=0\), then we have \(m=3\), \(a_{m}=\frac{a_{2}^{2}(a_{1}-1)}{\rho c_{1}^{2}}<0\) and \(E_{1}\) is a saddle.

This completes the proof. □

Theorem 4.2

Assume that other boundary equilibria\(E_{21}\), \(E_{22}\), and\(E_{23}\)of system (1.6) exist, we have:

  1. (1)

    \(E_{21}\)is always unstable.

  2. (2)

    For\(E_{22}\), we have the following results:

    1. (a)

      If\(\frac{1}{a_{2}}>x_{22}\), \(E_{22}\)is a saddle;

    2. (b)

      If\(\frac{1}{a_{2}}< x_{22}\), \(E_{22}\)is a stable node;

    3. (c)

      If\(\frac{1}{a_{2}}=x_{22}\), \(E_{22}\)is a saddle node.

  3. (3)

    If\(a_{2}(c_{1}-1)\neq -2\), \(E_{23}\)is a saddle node; on the contrary, \(E_{23}\)consists of a hyperbolic sector and an elliptic sector.

Proof

When other boundary equilibrium points exist, we have

$$ 1-x_{2i}-a_{1}y_{2i}- \frac{b_{1}}{c_{1}+x_{2i}}=0, $$

here, \(i=1,2,3\).

So (4.1) can be written as

$$ J(E_{2i})= \begin{pmatrix} \frac{x_{2i}}{(c_{1}+x_{2i})^{2}} (b_{1}-(c_{1}+x_{2i})^{2} )&-a_{1}x_{2i} \\ 0& \rho (1-a_{2}x_{2i})\end{pmatrix}. $$
(4.8)

(1) It is easy to get from (4.8) that the eigenvalues of \(J(E_{21})\) are \(\lambda _{1}=\frac{x_{21}}{(c_{1}+x_{21})^{2}} (b_{1}-(c_{1}+x_{21})^{2} )\) and \(\lambda _{2}=\rho (1-a_{2}x_{21})\), while

$$ \begin{aligned} \lambda _{1}&= \frac{x_{21}}{(c_{1}+x_{21})^{2}} \bigl(b_{1}-(c_{1}+x_{21})^{2} \bigr) \\ &= \frac{x_{21}\sqrt{\triangle _{1}(b_{1})}(\sqrt{\triangle _{1}(b_{1})+4b_{1}}-\sqrt{\triangle _{1}(b_{1})})}{2(c_{1}+x_{21})^{2}}>0, \end{aligned} $$

so \(E_{21}\) is unstable.

(2) From (4.8), one could obtain \(\lambda _{1}=\frac{x_{22}}{(c_{1}+x_{22})^{2}} (b_{1}-(c_{1}+x_{22})^{2} )\) and \(\lambda _{2}=\rho (1-a_{2}x_{22})\) are the eigenvalues of \(J(E_{22})\); nevertheless,

$$ \begin{aligned} \lambda _{1}&= \frac{x_{22}}{(c_{1}+x_{22})^{2}} \bigl(b_{1}-(c_{1}+x_{22})^{2} \bigr) \\ &= \frac{-x_{22}\sqrt{\triangle _{1}(b_{1})}(\sqrt{\triangle _{1}(b_{1})+4b_{1}}+\sqrt{\triangle _{1}(b_{1})})}{2(c_{1}+x_{22})^{2}}< 0, \end{aligned} $$

so the stability of \(J(E_{22})\) depends on \(\lambda _{2}\). If \(\lambda _{2}>0\), i.e., \(\frac{1}{a_{2}}>x_{22}\), then \(E_{22}\) is a saddle; if \(\lambda _{2}<0\), i.e., \(\frac{1}{a_{2}}< x_{22}\), then \(E_{22}\) is a stable node; if \(\lambda _{2}=0\), i.e., \(\frac{1}{a_{2}}=x_{22}\), it is difficult for us to draw the conclusion.

Now, let us use Theorem 7.1 in Chap. 2 in [39] to determine its stability. First of all, we make transformation \((X,Y)=(x-x_{22},y)\) to move \(E_{22}\) to the original, and then expand in power series up to the third order around the origin, which makes the system as follows:

$$ \begin{gathered} \frac{dX}{dt}=c_{01}X-Y-a{1}XY+c_{02}X^{2}+c_{03}X^{3}, \\ \frac{dY}{dt}=-\rho a_{2}XY-\rho Y^{2}, \end{gathered} $$
(4.9)

where \(c_{01}=x_{22}(\frac{b_{1}}{(c_{1}+x_{22})^{2}}-1)\), \(c_{02}=\frac{b_{1}c_{1}}{(c_{1}+x_{22})^{3}}-1\), \(c_{03}=-\frac{b_{1}c_{1}}{(c_{1}+x_{22})^{4}}\).

Next, we make the following transformation:

$$ \begin{pmatrix} X \\ Y \end{pmatrix} ={ \begin{pmatrix} 1&-1 \\ 0&-c_{01} \end{pmatrix} \begin{pmatrix} X_{2} \\ Y_{2} \end{pmatrix}}, $$
(4.10)

and let \(\tau =c_{01}t\), system (4.9) becomes

$$ \begin{gathered} \frac{dX_{2}}{d\tau }=X_{2}+d_{01}X_{2}Y_{2}+d_{02}X_{2}^{2}+d_{03}Y_{2}^{2} +P_{2}(X_{2},Y_{2}) :=P(X_{2},Y_{2}), \\ \frac{dY_{2}}{d\tau }=e_{01}X_{2}Y_{2}+e_{02}Y_{2}^{2}+Q_{2}(X_{2},Y_{2}), \end{gathered} $$
(4.11)

where \(d_{01}=\frac{a_{1}c_{01}-\rho a_{2}-2c_{02}}{c_{01}}\), \(d_{02}=\frac{c_{02}}{c_{01}}\), \(d_{03}=\rho -a_{1}+\frac{c_{02}+\rho a_{2}}{c_{01}}\), \(e_{01}=-\frac{\rho a_{2}}{c_{01}}\), \(e_{02}=\rho (\frac{a_{2}}{c_{01}}+1)\), and \(P_{2}(X_{2},Y_{2})\) and \(Q_{2}(X_{2},Y_{2})\) are power series in \((X_{2},Y_{2})\) with terms \(X_{2}^{i}Y_{2}^{j}\) satisfying \(i+j\geq 3\).

According to the implicit function theorem, there is a function \(X_{2}=\varphi (Y_{2})\) such that \(\varphi (0)=0\) and \(P(\varphi (Y_{2}),Y_{2})=0\). From the first equation of system (4.11), we have

$$ X_{2}=\varphi (Y_{2})=-d_{03}Y_{2}^{2}+o \bigl(Y_{2}^{3}\bigr). $$
(4.12)

Substituting (4.12) into the second equation of system (4.11), we can get

$$ \frac{dY_{2}}{d\tau }=e_{02}Y_{2}^{2}+\cdots. $$

By Theorem 7.1 in Chap. 2 in [39], because \(e_{02}>0\) and \(E_{22}\) is the saddle node.

(3) From (4.8) and \(x_{23}=\frac{1-c_{1}}{2}\), one could get the eigenvalues of \(J(E_{23})\) are \(\lambda _{1}=0\) and \(\lambda _{2}=\rho (1-a_{2}x_{23})\). In order to get the stability of \(E_{23}\), firstly, we make transformation \((X,Y)=(x-x_{23},y_{23})\) to move \(E_{23}\) to the original, and then expand in power series up to the third order around the origin, which makes the system as follows:

$$ \begin{gathered} \frac{dX}{dt}=a_{01}Y+a{02}XY+a_{03}X^{2}+a_{04}X^{3}, \\ \frac{dY}{dt}=b_{01}Y+b_{02}XY+b_{03}Y^{2}, \end{gathered} $$
(4.13)

where \(a_{01}=\frac{a_{1}(c_{1}-1)}{2}\), \(a_{02}=-a_{1}\), \(a_{03}=\frac{c_{1}-1}{c_{1}+1}\), \(a_{04}=\frac{-4c_{1}}{(c_{1}+1)^{2}}\), \(b_{01}=\frac{a_{2}(c_{1}-1)+2}{2}\), \(b_{02}=-a_{2}\rho \), \(b_{03}=-\rho \). Secondly, we discuss it in two cases.

Case 1: If \(\lambda _{2}\neq 0\), the stability of \(E_{23}\) can be proved by the same method in Theorem 4.2(2), so \(E_{23}\) is a saddle node if \(x_{23}\neq \frac{1}{a_{2}}\).

Case 2: If \(\lambda _{2}=0\), let \(\tau =-\frac{a_{1}}{a_{2}}t\), then system (4.13) becomes

$$ \begin{gathered} \frac{dX}{d\tau }=Y+a_{2}XY+c_{01}X^{2}+c_{02}X^{3}+P_{3}(X,Y):=P(X,Y), \\ \frac{dY}{d\tau }=d_{01}XY+d_{02}Y^{2}+Q_{3}(X,Y). \end{gathered} $$
(4.14)

Here, \(c_{01}=\frac{2}{a_{1}(c_{1}+1)}\), \(c_{02}=\frac{4a_{2}c_{1}}{a_{1}(c_{1}+1)^{2}}\), \(d_{01}=\frac{\rho a_{2}^{2}}{a_{1}}\), \(d_{02}=\frac{\rho a_{2}}{a_{1}}\), and \(P_{3}(X,Y)\) and \(Q_{3}(X,Y)\) are power series in \((X,Y)\) with terms \(X^{i}Y^{j}\) satisfying \(i+j\geq 3\).

According to the implicit function theorem, there is a function \(Y=\varphi (X)\) such that \(\varphi (0)=0\) and \(P(X, \varphi (X))=0\). From the first equation of system (4.14), we have

$$ Y=\varphi (X)= \frac{a_{2}(c_{1}-1)}{a_{1}(c_{1}+1)}X^{2}+o\bigl(X^{3}\bigr). $$
(4.15)

Substituting (4.15) into the second equation of system (4.14),we can get

$$ \frac{dY}{d\tau }= \frac{a_{2}^{3}\rho (c_{1}-1)}{a_{1}^{2}(c_{1}+1)}X^{3}+\cdots $$

and

$$ v(X)= \frac{a_{2}}{a_{1}}\biggl( \frac{2(1-c_{1})}{1+c_{1}}+\rho a_{2} \biggr)X+\cdots. $$

By Theorem 7.3 in Chap. 2 in [39], because \(k=2m+1=3\), \(m=1\), \(a_{k}=\frac{a_{2}^{3}\rho (c_{1}-1)}{a_{1}^{2}(c_{1}+1)}<0\), \(N=1\), \(B_{N}= \frac{a_{2}}{a_{1}}(\frac{2(1-c_{1})}{1+c_{1}}+\rho a_{2})>0\), \(\lambda =B_{N}^{2}+4(m+1)a_{2m+1}= (\frac{a_{2}}{a_{1}}(\frac{2(1-c_{1})}{1+c_{1}}-\rho a_{2}) )^{2}>0\), then \(E_{23}\) consists of a hyperbolic sector and an elliptic sector.

This completes the proof. □

Theorem 4.3

When the positive equilibria exist, we have:

  1. (1)

    \(E_{31}\)is a stable node.

  2. (2)

    \(E_{32}\)is a saddle.

  3. (3)

    \(E_{33}\)is a saddle node.

Proof

Notice that when the positive equilibria exist, (4.1) can be simplified as follows:

$$ J(E_{3i})= \begin{pmatrix} x_{3i} ( \frac{b_{1}}{(c_{1}+x_{3i})^{2}}-1)&-a_{1}x_{3i} \\ -\rho a_{2}y_{3i}& -\rho y_{3i}\end{pmatrix}, $$
(4.16)

where \(i=1,2,3\).

Thus,

$$\begin{aligned}& \operatorname{tr}J(E_{3i})=x_{3i} \biggl( \frac{b_{1}}{(c_{1}+x_{3i})^{2}}-1 \biggr)-\rho y_{3i}, \\& \begin{aligned} \operatorname{det}J(E_{3i})&=-\rho x_{3i}y_{3i} \biggl( \frac{b_{1}}{(c_{1}+x_{3i})^{2}}-1 \biggr)-\rho a_{1}a_{2}x_{3i}y_{3i} \\ &=-\rho x_{3i}y_{3i} \biggl( \frac{b_{1}}{(c_{1}+x_{3i})^{2}}-1+a_{1}a_{2} \biggr). \end{aligned} \end{aligned}$$

Using

$$ 1-x_{3i}-a_{1}y_{3i}- \frac{b_{1}}{c_{1}+x_{3i}}=0 $$

and

$$ 1-y_{3i}-a_{2}x_{3i}=0, $$

we have

$$ \operatorname{det}J(E_{3i})= \frac{-\rho x_{3i}y_{3i}}{c_{1}+x_{3i}}(2Ax_{3i}-B). $$

(1) According to (3.5), we get that

$$ \operatorname{det}J(E_{32})= \frac{-\rho x_{32}y_{32}\sqrt{\triangle _{2}}}{c_{1}+x_{32}}< 0, $$

so \(E_{32}\) is a saddle.

(2) Similarly, we have

$$ \operatorname{det}J(E_{31})= \frac{\rho x_{31}y_{31}\sqrt{\triangle _{2}}}{c_{1}+x_{31}}>0. $$

Therefore, we consider the sign of \(\operatorname{tr}J(E_{31})\). Let \(N(x)=\frac{b_{1}}{(c_{1}+x)^{2}}-1\), \(x\in (0,+\infty )\), then we have

$$ \frac{dN(x)}{dx}= \frac{-2b_{1}}{(c_{1}+x)^{3}}< 0. $$

Note that \(N(x)\) is monotonically decreasing in the interval \((0,+\infty )\). It is clear that

$$ 0< x_{32}< x_{33}< x_{31} $$

and

$$ N(x_{33})=\frac{b_{1}}{(c_{1}+x_{33})^{2}}-1=-a_{1}a_{2}< 0. $$

So we can obtain \(N(x_{31})<0\), and it is easy to calculate that

$$ \operatorname{tr}J(E_{31})=x_{31}N(x_{31})-\rho y_{31}< 0. $$

The above analysis shows that \(E_{31}\) is a stable node.

(3) From \(x_{33}=\frac{B}{2A}\), we can get

$$\begin{aligned}& \operatorname{det}J(E_{33})= \frac{-\rho x_{33}y_{33}}{c_{1}+x_{33}}\biggl(2A* \frac{B}{2A}-B\biggr)=0, \\& \begin{aligned} \operatorname{tr}J(E_{33})&=x_{33} \biggl( \frac{b_{1}}{(c_{1}+x_{33})^{2}}-1\biggr)-\rho y_{33} \\ &=-a_{1}a_{2}x_{33}-\rho y_{33}< 0, \end{aligned} \end{aligned}$$

so the eigenvalues of \(J(E_{33})\) are \(\lambda _{1}<0\) and \(\lambda _{2}=0\), we use the same method as Theorem 4.2(2) and easily get \(E_{33}\) is a saddle node.

In order to verify the above results, let \(a_{1}=0.4\), \(a_{2}=2\), \(b_{1}=1.51\), \(c_{1}=2.5\), \(\rho =1\). By simple computation, we have \(A<0\), \(B<0\), \(C<0\), and \(\Delta _{2}>0\), \(E_{31}(0.3618,0.27639)\) is a stable node, \(E_{32}(0.138197,0.72361)\) is a saddle. From Fig. 1, it is easy to get the red line that divides the first quadrant into two parts, recorded as I (left one) and II (right one). Assume that the initial conditions are located in region I, all the solutions tend to \(E_{1}(0,1)\) which is a stable manifold and \(E_{32}(0.138197,0.72361)\) is an unstable manifold. From the biological point of view, when the initial values are located in region I, the first species will be driven to extinction. On the contrary, assume that the initial conditions are located in region II, all the solutions tend to \(E_{31}(0.3618,0.27639)\) which is a stable manifold, \(E_{32}(0.138197,0.72361)\) and \(E_{22}(0.49599,0)\) are unstable manifolds. From the biological point of view, when the initial values are located in region II, two species can always coexist. This is the bistable phenomenon which is shown in Figs. 1 and 2.

Figure 1
figure 1

\(E_{31}(x_{31},y_{31})\) is a stable node, \(E_{32}(x_{32},y_{32})\) is a saddle, and the red line is the separatrix

Full size image
Figure 2
figure 2

\(E_{31}(x_{31},y_{31})\) is a stable node, \(E_{32}(x_{32},y_{32})\) is a saddle, and the red line is the separatrix. Figure 2 is an enlarged version of Fig. 1

Full size image

This completes the proof. □

Theorem 4.4

For the unique positive equilibrium that exists for system (1.6), our results are as follows:

  1. (1)

    When\(A>0\), \(B>0\), \(C<0\)or\(A>0\), \(B<0\), \(C<0\)or\(A<0\), \(B<0\), \(C\geq 0\), \(E^{*}\)is a saddle;

  2. (2)

    When\(A<0\), \(B>0\), \(C>0\), \(E^{*}\)is a stable node.

Proof

Through the discussion in Theorem 4.3, we can get

$$\begin{aligned}& \operatorname{det}J\bigl(E^{*}\bigr)= \frac{-\rho x^{*}y^{*}}{c_{1}+x^{*}} \bigl(2Ax^{*}-B\bigr), \\& \begin{aligned} \operatorname{tr}J\bigl(E^{*}\bigr)&=x^{*} \biggl( \frac{b_{1}}{(c_{1}+x^{*})^{2}}-1\biggr)- \rho y^{*} \\ &= \frac{x^{*}}{c_{1}+x^{*}} \bigl((A-1)x^{*}-(B+a_{1}a_{2}c_{1}) \bigr)-\rho y^{*}. \end{aligned} \end{aligned}$$

In order to analyze the stability of \(E^{*}\), let us consider the functions

$$ f(x)=2Ax-B $$

and

$$ g(x)=(A-1)x-(B+a_{1}a_{2}c_{1}). $$

Next let us discuss them in different cases:

Case 1: When \(A>0\), \(B>0\), \(C<0\) or \(A>0\), \(B<0\), \(C<0\), note that \(x^{*}=x_{32}>\frac{B}{2A}\), thus it is easy for us to get \(f(x^{*})>0\), i.e., \(\operatorname{det}J(E^{*})<0\), so \(E^{*}\) is a saddle.

Case 2: When \(A<0\), \(B<0\), \(C\geq 0\), we have \(x^{*}=x_{31}<\frac{B}{2A}\), which gives \(f(x^{*})>0\), i.e., \(\operatorname{det}J(E^{*})<0\), so \(E^{*}\) is a saddle.

Case 3: When \(A<0\), \(B>0\), \(C>0\), we have \(x^{*}=x_{31}>0\), which implies that \(f(x^{*})<0\), \(g(x^{*})<0\), i.e., \(\operatorname{det}J(E^{*})>0\), \(\operatorname{tr}J(E^{*})<0\), so \(E^{*}\) is a stable node.

In order to verify the above results, let \(a_{1}=0.6\), \(a_{2}=2\), \(b_{1}=0.42\), \(c_{1}=0.4\), \(\rho =1\). By simple computation, we have \(A>0\), \(B<0\), \(C<0\). \(E^{*}(x^{*},y^{*})\) and \(E_{21}(0.03542,0)\) are saddle, \(E_{1}(0,1)\) and \(E_{22}(0.56458,0)\) are stable nodes. From Fig. 3, it is easy to get the red line that divides the first quadrant into two parts, recorded as I (left one) and II (right one). Assume that the initial conditions are located in region I, all the solutions tend to \(E_{1}(0,1)\) which is a stable manifold, \(E^{*}(x^{*},y^{*})\) and \(E_{21}(0.03542,0)\) are unstable manifolds. From the biological point of view, when the initial values are located in region I, the first species will be driven to extinction. On the contrary, assume that the initial conditions are located in region II, all the solutions tend to \(E_{22}(0.56458,0)\) which is a stable manifold, \(E^{*}(x^{*},y^{*})\) and \(E_{21}(0.03542,0)\) are unstable manifolds. From the biological point of view, when the initial values are located in region II, the second species will be driven to extinction. This is the bistable phenomenon which is shown in Fig. 3.

Figure 3
figure 3

\(E^{*}(x^{*},y^{*})\) is a saddle, the red line is the separatrix

Full size image

This completes the proof. □

5 Global stability of equilibrium

Theorem 5.1

When\(E^{*}\)is locally stable, which is globally asymptotically stable.

Proof

We will adapt the idea of Yu [40] to prove Theorem 5.1. More precisely, we will consider a Lyapunov function

$$ V(x,y)=\bigl(c_{1}+x^{*}\bigr) \biggl(x-x^{*}-x^{*} \ln \frac{x}{x^{*}}\biggr)+ \rho a_{2}\biggl(y-y^{*}-y^{*} \ln \frac{y}{y^{*}}\biggr). $$

It is easy to see that the function \(V(x,y)\) is zero at the equilibrium \(E^{*}(x^{*},y^{*})\), which is positive everywhere in the first quadrant except at \(E^{*}\). Then the time derivative of \(V(x,y)\) along the trajectories of (1.6) is

$$\begin{aligned} D^{+}V(t)&=\bigl(c_{1}+x^{*}\bigr) \frac{x-x^{*}}{x} \biggl(x\biggl(1-x-a_{1}y- \frac{b_{1}}{c_{1}+x} \biggr) \biggr) \\ &\quad {}+\rho a_{2}\frac{y-y^{*}}{y} \bigl(\rho y(1-y-a_{2}x) \bigr) \\ &=\bigl(c_{1}+x^{*}\bigr) \bigl(x-x^{*}\bigr) \biggl(x^{*}+a_{1}y^{*}+\frac{b_{1}}{c_{1}+x^{*}}-x-a_{1}y- \frac{b_{1}}{c_{1}+x}\biggr) \\ &\quad {}+\rho ^{2}a_{2}\bigl(y-y^{*}\bigr) \bigl(y^{*}+a_{2}x^{*}-y-a_{2}x\bigr) \\ &=\bigl(x-x^{*}\bigr)^{2} \biggl(\frac{b_{1}}{c_{1}+x}- \bigl(c_{1}+x^{*}\bigr) \biggr)-\rho ^{2}a_{2} \bigl(y-y^{*}\bigr)^{2} \\ &\quad {}- \bigl(a_{1}\bigl(c_{1}+x^{*}\bigr)+ \rho a_{2}^{2} \bigr) \bigl(x-x^{*}\bigr) \bigl(y-y^{*}\bigr) \\ &< \bigl(x-x^{*}\bigr)^{2}\biggl(\frac{b_{1}}{c_{1}}-c_{1} \biggr)-\rho ^{2}a_{2}\bigl(y-y^{*} \bigr)^{2}. \end{aligned}$$

Note that \(B>0\), \(C>0\) when \(E^{*}\) exists, and we have \(\frac{b_{1}}{c_{1}}<1-a_{1}<c_{1}(1-a_{1}a_{2})\), i.e., \(\frac{b_{1}}{c_{1}}-c_{1}<0\), so one could obtain \(D^{+}V(t)<0\) strictly for all \(x,y>0\) except the equilibrium \(E^{*}(x^{*},y^{*})\), where \(D^{+}V(t)=0\). Hence, \(V(x,y)\) satisfies Lyapunov’s asymptotic stability theorem, then the equilibrium \(E^{*}(x^{*},y^{*})\) of system (1.6) is globally asymptotically stable. From the biological point of view, two species can always coexist (see Fig. 4).

Figure 4
figure 4

\(E^{*}(x^{*},y^{*})\) is globally asymptotically stable

Full size image

This completes the proof. □

6 Bifurcation analysis

In this section, we are interested in studying the various possible bifurcations of system (1.6). From Theorem 3.1, we know that system (1.6) may undergo saddle-node bifurcation at \(E_{23}\) and \(E_{23}\), respectively, and transcritical bifurcation around the equilibria \(E_{0}\) and \(E_{1}\), which is a very interesting phenomenon.

6.1 Saddle-node bifurcation

From Theorem 3.1, it is easy to find that when \(c_{1}< b_{1}< b_{1}^{*}\) and \(0< c_{1}<1\) the system has two different boundary equilibria, which may coincide if \(b_{1}=b_{1}^{*}\) and \(0< c_{1}<1\) or which may disappear if \(b_{1}>b_{1}^{*}\). The emergence or appearance of the equilibria is due to the saddle-node bifurcation at \(E_{23}\) (see Fig. 5 and Fig. 6).

Figure 5
figure 5

Dynamics behaviors of system (1.6)

Full size image
Figure 6
figure 6

Dynamics behaviors of system (1.6)

Full size image

Theorem 6.1

System (1.6) undergoes a saddle-node bifurcation around\(E_{23}\)when\(b_{1}=b_{1SN}=\frac{(1+c_{1})^{2}}{4}\)and\(0< c_{1}<1\), where\(b_{1}\)is the bifurcation parameter.

Proof

By the proof of Theorem 4.2, we have an eigenvalue of \(J(E_{23})\) that is zero, named \(\lambda _{1}\). Let \(V_{1}\) and \(W_{1}\) represent the eigenvectors of \(\lambda _{1}\) for the matrices \(J(E_{23})\) and \(J(E_{23})^{T}\), respectively. After simple calculation, we have

$$ V_{1}={ \begin{pmatrix} V_{11} \\ V_{12} \end{pmatrix}}={ \begin{pmatrix} 1 \\ 0 \end{pmatrix}};\qquad W_{1}={ \begin{pmatrix} W_{11} \\ W_{12} \end{pmatrix}}={ \begin{pmatrix} 1 \\ \frac{a_{1}(1-c_{1})}{\rho (a_{2}(c_{1}-1)+2)} \end{pmatrix}}. $$

Moreover,

$$\begin{aligned}& \begin{aligned} F_{b_{1}}(E_{23};b_{1SN})={ \begin{pmatrix} \frac{-x}{c_{1}+x} \\ 0 \end{pmatrix}_{x=x_{23}}} &={ \begin{pmatrix} \frac{c_{1}-1}{c_{1}+1} \\ 0 \end{pmatrix}}, \end{aligned} \\& \begin{aligned} D^{2}F(E_{23};b_{1SN}) (V_{1},V_{1})&={ \begin{pmatrix} \frac{\partial ^{2}F_{1}}{\partial x^{2}}V_{11}^{2}+2 \frac{\partial ^{2}F_{1}}{\partial x \,\partial y}V_{11}V_{12}+ \frac{\partial ^{2}F_{1}}{\partial ^{2} y}V_{12}^{2} \\ \frac{\partial ^{2}F_{2}}{\partial x^{2}}V_{11}^{2}+2 \frac{\partial ^{2}F_{2}}{\partial x \,\partial y}V_{11}V_{12}+ \frac{\partial ^{2}F_{2}}{\partial ^{2} y}V_{12}^{2} \end{pmatrix}_{(E_{23};b_{1SN})}} \\ &={ \begin{pmatrix} \frac{2b_{1}c_{1}}{(c_{1}+x_{23})^{3}}-2 \\ 0 \end{pmatrix}}. \end{aligned} \end{aligned}$$

Clearly, we can get that \(V_{1}\) and \(W_{1}\) satisfy

$$ \begin{gathered} W_{1}^{T}F_{b_{1}}(E_{23};b_{1SN})= \frac{c_{1}-1}{c_{1}+1}\neq0, \\ W_{1}^{T}\bigl[D^{2}F(E_{23};b_{1SN}) (V_{1},V_{1})\bigr]= \frac{2(c_{1}-1)}{c_{1}+1}\neq0. \end{gathered} $$

This implies that when \(b_{1}=b_{1SN}\), the saddle-node bifurcation occurs at \(E_{23}\).

This completes the proof. □

Similarly, the conditions for the existence of positive equilibria of system (1.6) are given in Theorem 3.1, and we could find that when \(\Delta _{2}>0\), \(A<0\), \(B<0\), \(C<0\), the system has two different positive equilibria, which may coincide if \(\Delta _{2}=0\) and \(A<0\), \(B<0\) or which may disappear if \(\Delta _{2}<0\). The emergence or appearance of the equilibria is due to the saddle-node bifurcation at \(E_{33}\) (see Fig. 5 (a) and Fig. 7).

Figure 7
figure 7

Dynamics behaviors of system (1.6)

Full size image

Theorem 6.2

System (1.6) undergoes a saddle-node bifurcation around\(E_{33}\)when\(b_{1}=\tilde{b_{1SN}}\)and\(A<0\), \(B<0\), where\(b_{1}\)is the bifurcation parameter.

Proof

By the proof of Theorem 4.3, we know that an eigenvalue of \(J(E_{23})\) is zero, named \(\lambda _{1}\). Let \(V_{2}\) and \(W_{2}\) represent the eigenvectors of \(\lambda _{1}\) for the matrices \(J(E_{33})\) and \(J(E_{33})^{T}\), respectively. After simple calculation, we have

$$ V_{2}={ \begin{pmatrix} V_{21} \\ V_{22} \end{pmatrix}}={ \begin{pmatrix} 1 \\ -a_{2} \end{pmatrix}};\qquad W_{2}={ \begin{pmatrix} W_{21} \\ W_{22} \end{pmatrix}}={ \begin{pmatrix} 1 \\ \frac{-a_{1}x_{33}}{\rho y_{33}} \end{pmatrix}}. $$

Moreover,

$$\begin{aligned}& \begin{aligned} F_{b_{1}}(E_{33};b_{1SN})={ \begin{pmatrix} \frac{-x}{c_{1}+x} \\ 0 \end{pmatrix}_{x=x_{33}}} &={ \begin{pmatrix} \frac{-x_{33}}{c_{1}+x_{33}} \\ 0 \end{pmatrix}}, \end{aligned} \\& \begin{aligned} D^{2}F(E_{33};b_{1SN}) (V_{2},V_{2})&={ \begin{pmatrix} \frac{\partial ^{2}F_{1}}{\partial x^{2}}V_{21}^{2}+2 \frac{\partial ^{2}F_{1}}{\partial x \,\partial y}V_{21}V_{22}+ \frac{\partial ^{2}F_{1}}{\partial ^{2} y}V_{22}^{2} \\ \frac{\partial ^{2}F_{2}}{\partial x^{2}}V_{21}^{2}+2 \frac{\partial ^{2}F_{2}}{\partial x \,\partial y}V_{21}V_{22}+ \frac{\partial ^{2}F_{2}}{\partial ^{2} y}V_{22}^{2} \end{pmatrix}_{(E_{33};b_{1SN})}} \\ &={ \begin{pmatrix} 2(\frac{b_{1}c_{1}}{(c_{1}+x_{33})^{3}}+a_{1}a_{2}-1) \\ 0 \end{pmatrix}}. \end{aligned} \end{aligned}$$

So, we can obtain \(V_{2}\) and \(W_{2}\) satisfy

$$ \begin{gathered} W_{2}^{T}F_{b_{1}}(E_{33};b_{1SN})= \frac{-x_{33}}{c_{1}+x_{33}}\neq0, \\ W_{2}^{T}\bigl[D^{2}F(E_{33};b_{1SN}) (V_{2},V_{2})\bigr]= \frac{AB}{2(a_{1}-1)-B}\neq0, \end{gathered} $$

which means that when \(b_{1}=b_{1SN}\), the saddle-node bifurcation occurs at \(E_{33}\).

This completes the proof. □

6.2 Transcritical bifurcation

Through the discussion of Theorem 3.1, we find an interesting phenomenon: when \(b_{1}=c_{1}\), \(E_{21}\) will coincide with \(E_{0}\) if \(0< c_{1}<1\); \(E_{22}\) will coincide with \(E_{0}\) if \(c_{1}>1\). Therefore, the emergence of this phenomenon is owing to the transcritical bifurcation at \(E_{0}\) (see Fig. 8). Then we obtain the following.

Figure 8
figure 8

The phase portraits of transcritical bifurcation of system (1.6) around \(E_{0}\)

Full size image

Theorem 6.3

System (1.6) undergoes a transcritical bifurcation around\(E_{0}\)with\(b_{1}\)as a bifurcation parameter, when\(b_{1}=b_{1SN}=c_{1}\).

Proof

Here, we use Sotomayor’s theorem to verify the transversality conditions for transcritical bifurcation. The Jacobian matrix of system (1.6) evaluated at the point \(E_{0}\) is given by (3.2). Obviously, the eigenvalue \(\lambda _{1}=0\) of \(J(E_{0})\) if \(b_{1}=c_{1}\). Let \(V_{3}\) and \(W_{3}\) be the eigenvectors of \(J(E_{0})\) and \(J(E_{0})^{T}\) corresponding to \(\lambda _{1}=0\), respectively. Then we can obtain

$$ V_{3}={ \begin{pmatrix} V_{31} \\ V_{32} \end{pmatrix}}={ \begin{pmatrix} 1 \\ 0 \end{pmatrix}},\qquad W_{3}={ \begin{pmatrix} W_{31} \\ W_{32} \end{pmatrix}}={ \begin{pmatrix} -1 \\ 0 \end{pmatrix}}. $$

Furthermore,

$$\begin{aligned}& \begin{aligned} F_{b_{1}}(E_{0};b_{1TC})={ \begin{pmatrix} \frac{-x}{c_{1}+x} \\ 0 \end{pmatrix}_{x=x_{0}}} &={ \begin{pmatrix} 0 \\ 0 \end{pmatrix}}, \end{aligned} \\& DF_{b_{1}}(E_{0};b_{1TC})V_{3}={ \begin{pmatrix} \frac{-c_{1}}{(c_{1}+x)^{2}}&0 \\ 0&0 \end{pmatrix}} { \begin{pmatrix} 1 \\ 0 \end{pmatrix}_{(E_{0};b_{1TC})}}={ \begin{pmatrix} -\frac{1}{c_{1}} \\ 0 \end{pmatrix}}, \\& \begin{aligned} D^{2}F(E_{0};b_{1TC}) (V_{3},V_{3})&={ \begin{pmatrix} \frac{\partial ^{2}F_{1}}{\partial x^{2}}V_{31}^{2}+2 \frac{\partial ^{2}F_{1}}{\partial x \,\partial y}V_{31}V_{32}+ \frac{\partial ^{2}F_{1}}{\partial ^{2} y}V_{32}^{2} \\ \frac{\partial ^{2}F_{2}}{\partial x^{2}}V_{31}^{2}+2 \frac{\partial ^{2}F_{2}}{\partial x \,\partial y}V_{31}V_{32}+ \frac{\partial ^{2}F_{2}}{\partial ^{2} y}V_{32}^{2} \end{pmatrix}_{(E_{0};b_{1TC})}} \\ &={ \begin{pmatrix} 2 ( \frac{1}{c_{1}}-1 ) \\ 0 \end{pmatrix}}. \end{aligned} \end{aligned}$$

Thus, we have

$$ \begin{gathered} W_{3}^{T}F_{b_{1}}(E_{0};b_{1TC})=0, \\ W_{3}^{T}\bigl[DF_{b_{1}}(E_{0};b_{1TC})V_{3} \bigr]= \frac{1}{c_{1}}\neq0, \\ W_{3}^{T}\bigl[D^{2}F(E_{0};b_{1TC}) (V_{3},V_{3})\bigr]=2 \biggl(1- \frac{1}{c_{1}} \biggr) \neq0. \end{gathered} $$

So from Sotomayor’s theorem system (1.6) undergoes a transcritical bifurcation at \(E_{0}\).

This completes the proof. □

In the same way, for the positive equilibria of system (1.6), when \(b_{1}=c_{1}(1-a_{1})\) and \(A<0\), \(E_{32}\) will coincide with \(E_{1}\) if \(B>0\); \(E_{31}\) will coincide with \(E_{1}\) if \(B<0\). Hence, the appearance of this phenomenon is owing to the transcritical bifurcation at \(E_{1}\) (see Fig. 9). Then we can get the following.

Figure 9
figure 9

The phase portraits of transcritical bifurcation of system (1.6) around \(E_{1}\)

Full size image

Theorem 6.4

System (1.6) undergoes a transcritical bifurcation around\(E_{1}\)with\(b_{1}\)as a bifurcation parameter, when\(b_{1}=b_{1SN}=c_{1}(1-a_{1})\)and\(A<0\).

Proof

Here, we use Sotomayor’s theorem to verify the transversality conditions for transcritical bifurcation. The Jacobian matrix of system (1.6) evaluated at the point \(E_{1}\) is given by (3.2). Clearly, the eigenvalue \(\lambda _{1}=0\) of \(J(E_{1})\) if \(b_{1}=c_{1}(1-a_{1})\). Let \(V_{4}\) and \(W_{4}\) be the eigenvectors of \(J(E_{1})\) and \(J(E_{1})^{T}\) corresponding to \(\lambda _{1}=0\), respectively. Then we can get

$$ V_{4}={ \begin{pmatrix} V_{41} \\ V_{42} \end{pmatrix}}={ \begin{pmatrix} 1 \\ -a_{2} \end{pmatrix}},\qquad W_{4}={ \begin{pmatrix} W_{41} \\ W_{42} \end{pmatrix}}={ \begin{pmatrix} 1 \\ 0 \end{pmatrix}}. $$

Furthermore,

$$\begin{aligned}& \begin{aligned} F_{b_{1}}(E_{1};b_{1TC})={ \begin{pmatrix} \frac{-x}{c_{1}+x} \\ 0 \end{pmatrix}_{x=x_{1}}} &={ \begin{pmatrix} 0 \\ 0 \end{pmatrix}}, \end{aligned} \\& DF_{b_{1}}(E_{1};b_{1TC})V_{4}={ \begin{pmatrix} \frac{-c_{1}}{(c_{1}+x)^{2}}&0 \\ 0&0 \end{pmatrix}} { \begin{pmatrix} 1 \\ -a_{2} \end{pmatrix}_{(E_{1};b_{1TC})}}={ \begin{pmatrix} -\frac{1}{c_{1}} \\ 0 \end{pmatrix}}, \\& \begin{aligned} D^{2}F(E_{1};b_{1TC}) (V_{4},V_{4})&={ \begin{pmatrix} \frac{\partial ^{2}F_{1}}{\partial x^{2}}V_{41}^{2}+2 \frac{\partial ^{2}F_{1}}{\partial x \,\partial y}V_{41}V_{42}+ \frac{\partial ^{2}F_{1}}{\partial ^{2} y}V_{42}^{2} \\ \frac{\partial ^{2}F_{2}}{\partial x^{2}}V_{41}^{2}+2 \frac{\partial ^{2}F_{2}}{\partial x \,\partial y}V_{41}V_{42}+ \frac{\partial ^{2}F_{2}}{\partial ^{2} y}V_{42}^{2} \end{pmatrix}_{(E_{1};b_{1TC})}} \\ &={ \begin{pmatrix} - \frac{2B}{c_{1}} \\ 0 \end{pmatrix}}. \end{aligned} \end{aligned}$$

Thus, we have

$$ \begin{gathered} W_{4}^{T}F_{b_{1}}(E_{1};b_{1TC})=0, \\ W_{4}^{T}\bigl[DF_{b_{1}}(E_{1};b_{1TC})V_{4} \bigr]=- \frac{1}{c_{1}}\neq0, \\ W_{4}^{T}\bigl[D^{2}F(E_{1};b_{1TC}) (V_{4},V_{4})\bigr]=- \frac{2B}{c_{1}}\neq0. \end{gathered} $$

So from Sotomayor’s theorem system (1.6) undergoes a transcritical bifurcation at \(E_{1}\).

This completes the proof. □

7 Numeric simulations

Now, let us give the following examples to illustrate the main results.

Example 7.1

Let \(a_{1}=0.6\), \(a_{2}=1\), \(b_{1}=0.4\), \(c_{1}=1.2\), \(\rho =1\), then we have \(E^{*}\) is globally asymptotically stable (Fig. 4).

Example 7.2

Let \(a_{1}=0.5\), \(c_{1}=0.5\), \(\rho =1\), by simple computation, we have:

  1. (1)

    For \(a_{2}=1\), \(b_{1}=0.6\), we get \(b_{1}^{*}=0.375< b_{1}\), \(\Delta _{2}=-0.6375<0\), and the system has no other boundary equilibria and positive equilibria.

  2. (2)

    For \(a_{2}=3\), we get \(E_{23}\) is a saddle.

  3. (3)

    For \(a_{2}=4\), we get \(E_{23}\) consists of a hyperbolic sector and an elliptic sector.

Figure 5 shows the above results.

Example 7.3

Let \(a_{1}=2\), \(a_{2}=2.5\), \(c_{1}=0.5\), \(\rho =1\), by simple computation, we have \(\frac{1}{a_{2}}=0.4\),

  1. (1)

    For \(b_{1}=0.55\), we get \(x_{22}=0.3618<\frac{1}{a_{2}}\) and \(E_{22}\) is a saddle, \(E_{21}\) is unstable.

  2. (2)

    For \(b_{1}=0.54\), we get \(x_{22}=0.4=\frac{1}{a_{2}}\) and \(E_{22}\) is a saddle node, \(E_{21}\) is unstable.

  3. (3)

    For \(b_{1}=0.52\), we get \(x_{22}=0.4562>\frac{1}{a_{2}}\) and \(E_{22}\) is a stable node, \(E_{21}\) is unstable.

Figure 6 shows the above results.

Example 7.4

Let \(a_{1}=0.4\), \(a_{2}=2\), \(c_{1}=2.5\), \(\rho =1\), we have:

  1. (1)

    For \(b_{1}=1.5125\), we get \(\Delta _{2}=0\) and the unique positive equilibrium \(E_{33}\) is a saddle node.

  2. (2)

    For \(b_{1}=1.51\), we get \(\Delta _{2}>0\) and \(E_{31}\) is a stable node, \(E_{32}\) is a saddle.

Figure 7 shows the above results.

Example 7.5

Let \(a_{1}=2\), \(a_{2}=2\), \(\rho =1\), by simple computation, we have:

  1. (1)

    For \(b_{1}=0.4\), \(c_{1}=0.4\), we get \(E_{21}\) coincides with \(E_{0}\).

  2. (2)

    For \(b_{1}=1\), \(c_{1}=1\), we get both \(E_{21}\) and \(E_{22}\) coincide with \(E_{0}\).

  3. (3)

    For \(b_{1}=1.5\), \(c_{1}=1.5\), we get \(E_{22}\) coincides with \(E_{0}\).

Figure 8 shows that system (1.6) undergoes transcritical bifurcation at \(E_{0}\).

Example 7.6

Let \(a_{1}=0.4\), \(a_{2}=2\), \(\rho =1\), by simple computation, we have:

  1. (1)

    For \(c_{1}=2.8\), we get \(b_{1}=1.68\) and \(E_{32}\) coincide with \(E_{1}\).

  2. (2)

    For \(c_{1}=3\), we get \(b_{1}=1.8\) and both \(E_{31}\) and \(E_{32}\) coincide with \(E_{1}\).

  3. (3)

    For \(c_{1}=4\), we get \(b_{1}=2.4\) and \(E_{31}\) coincides with \(E_{1}\).

Figure 9 shows that system (1.6) undergoes transcritical bifurcation at \(E_{1}\).

8 Conclusion

In this paper, we have considered a two-species competitive system with Michaelis–Menten type harvesting. The model shows rich dynamic behaviors. We have studied the permanence condition of the system, and by analyzing the stability of the system equilibria, we obtained that the system cannot collapse for any parameter value as the origin is always unstable. In addition, from the global asymptotic stability of the positive equilibrium, it can be seen that two species can coexist stably under appropriate conditions. We also get that there are two different cases of bistability in the system: on the one hand, a boundary equilibrium and a positive equilibrium are globally asymptotically stable; on the other hand, the two boundary equilibria are globally asymptotically stable.

Qualitative analysis indicates that Michaelis–Menten type harvesting plays an important role in the dynamic behaviors and bifurcations of the system. Firstly, the parameters \(b_{1}\) and \(c_{1}\) of the Michaelis–Menten type harvesting term will affect the number and stability of the system equilibria, compared with system (1.1), the boundary equilibria and positive equilibria of system (1.6) are both increased. Secondly, from the Michaelis–Menten type harvesting term \(h(x)=\frac{b_{1}x}{c_{1}+x}\), we know that system (1.6) is supplementary to system (1.1). Because system (1.6) becomes unharvested situation if \(b_{1}=0\), system (1.6) becomes constant harvested situation if \(c_{1}=0\). Thirdly, we give a strict proof of the bifurcation of system (1.6) by Sotomayor’s theorem, which has important ecological significance. Through saddle-node bifurcation and transcritical bifurcation, one could obtain the maximum threshold without extinction risk of species in continuous harvest. This provides important reference for decision makers to make reasonable strategies to ensure the sustainable development of ecosystem and maximize economic benefits.