Qualitative analysis of a harvested predator-prey system with Holling-type III functional response incorporating a prey refuge

Abstract

A predator-prey model with simplified Holling type III response function incorporating a prey refuge under sparse effect is considered. Through qualitative analysis of the model, at least two limit cycles exist around the positive equilibrium point with the result of focus value, the Hopf bifurcation under a prey refuge is obtained. We also show the influence of prey refuge. Numerical simulations are carried out to illustrate the feasibility of the obtained results and the dependence of the dynamic behavior on the prey refuge. Through the results of computer simulation, it is further shown that under certain conditions the model has three limit cycles surrounding the positive equilibrium point.

1 Introduction

The dynamical relationship between predators and their preys is one of the dominant subjects in ecology and mathematical ecology due to its universal importance, see [1–3]. Some of the empirical and theoretical work have investigated the effect of prey refuges, the refuges used by prey have a stabilizing effect on the considered interactions, and prey extinction can be prevented by the addition of refuges [4–8].

Huang et al. [9] studied the stability analysis of a prey-predator model with Holling type III response function incorporating a prey refuge:

$$\left\{\begin{array}{c}\frac{dx}{dt}=ax-b{x}^2-\frac{\alpha x^{2}y}{{\beta }^2+x^2},\hfill \\ \frac{dy}{dt}=-cy+\frac{k\alpha x^{2}y}{{\beta }^2+x^2}.\hfill \end{array}\right.$$

(1.1)

Motivated by the study of Huang et al. [9] and Ji and Wu [10], we consider the following predator-prey model with Holling type III response function incorporating a prey refuge under sparse effect:

$$\left\{\begin{array}{c}\frac{dx}{dt}=r{x}^2\left(1-\frac{x}{k}\right)-\frac{{\left(1-m\right)}^2{x}^{2}y}{a+{\left(1-m\right)}^2{x}^2},\hfill \\ \frac{dy}{dt}=y\left[-d+\frac{u{\left(1-m\right)}^2{x}^2}{a+{\left(1-m\right)}^2{x}^2}\right].\hfill \end{array}\right.$$

(1.2)

where x(t) is the population density of the prey and y(t) is the population density of the predator at time t; r > 0 represents the intrinsic growth rate of the prey; k is the carrying capacity of the prey in the absence of predator and harvesting; m is a constant number of prey using refuges, which protects m of prey from predation; the term x²/(a+x²) denotes the functional response of the predator, which is known as Holling type III response function; u > 0 is the conversion factor denoting the number of newly born predators for each captured prey; d > 0 is the death rate of the predator.

From the point of view of human needs, the exploitation of biological resources and the harvest of population are commonly practiced in fishery, forestry, and wildlife management. Concerning the conservation for the long-term benefits of humanity, there is a wide range of interest in the use of bioeconomic modeling to gain insight into the scientific management of renewable resources.

The problem of predator-prey interactions under a prey refuge have been studied by some authors. For example, Hassel [2] showed that adding a large refuge to a model, which exhibited divergent oscillations in the absence of a refuge, replaced the oscillatory behavior with a stable equilibrium. McNair [11] obtained that a prey refuge with legitimate entry-exit dynamics was quite capable of amplifying rather than damping predator-prey oscillations. McNair [12] showed that several kinds of refuges could exert a locally destabilizing effect and create stable, large-amplitude oscillations which would damp out if no refuge was present. Even now, prey refuges are widely believed to prevent prey extinction and damp predator-prey oscillations. For example, Kar [6] considered a Lotka-Volterra type predator-prey system incorporating a constant proportion of prey using refuges m, which protects m of prey from predation, with Holling type II response function and Holling type III response function, respectively. Our results indicate that refuge had a stabilizing effect on prey-predator interactions and the dynamic behavior very much depends on the prey refuge parameter m, point that increasing the amount of refuge could increase prey densities and lead to population outbreaks.

This article is organized as follows. Basic properties such as the existence, stability, and instability of the equilibria of the model and the boundedness of the solutions of the system (1.2) with positive initial values are given in Section 2. In Section 3, sufficient conditions for the global stability of the unique positive equilibrium are obtained. Section 4 is devoted to deriving the existence of limit cycle. In Section 5, we study the Hopf bifurcation of system (1.2). In Section 6, we analyze the influence of prey refuge and give numerical stimulations.

2 Basic properties of the model

Let $\overline{R_2^{+}}=\left\{\left(x,y\right)|x\ge 0,y\ge 0\right\}$. For practical biological meaning, we simply study system (1.2) in $\overline{R_2^{+}}$. The main aim of this article is to study the existence and non-existence of positive equilibrium of (1.2) by the effects of a prey refuge, that is to say, the existence and non-existence of positive equilibrium of system (1.2) depend on the constant m ∈ [0, 1).

We make the following substitution for model (1.2)

$$dt=\left[a+{\left(1-m\right)}^2{x}^2\right]d\tau .$$

Denoting new argument τ with t again, it gives

$$\left\{\begin{array}{c}\frac{dx}{dt}=x^2\left\{r\left(1-\frac{x}{k}\right)\left[a+{\left(1-m\right)}^2{x}^2\right]-{\left(1-m\right)}^{2}y\right\},\hfill \\ \frac{dy}{dt}=y\left[-da+\left(u-d\right){\left(1-m\right)}^2{x}^2\right].\hfill \end{array}\right.$$

(2.1)

Solutions of system (2.1) are discussed as follows:

• (1) If u ≤ d, system (2.1) possesses only two equilibrium points on the region $\overline{R_2^{+}}$, they are the trivial solution O(0, 0) and the semi-trivial solution in the absence of predator P₁(k, 0);

• (2) if u > d, system (2.1) admits three equilibrium points on the region $\overline{R_2^{+}}$, they are the trivial solution O(0, 0), the semi-trivial solution in the absence of predator P₁(k, 0) and the unique positive constant solution P₂(x₁, y₁), where

  $$x_1=\frac{1}{1-m}\sqrt{\frac{da}{u-d},}{y}_1=r\left[1-\frac{1}{k\left(1-m\right)}\sqrt{\frac{da}{u-d}}\right]\frac{ua}{\left(u-d\right){\left(1-m\right)}^2}.$$

For the existence of positive constant solution P₂(x₁, y₁), it is necessary to assume that $1-\frac{1}{k\left(1-m\right)}\sqrt{\frac{da}{u-d}}>0$, then we derive that $0\le m<1-\frac{1}{k}\sqrt{\frac{da}{u-d}}$.

It turns out that the non-constant positive solutions of (2.1) may exist for some ranges of the parameter m. From the first equation of system (1.2), it is easy to derive that

$$\underset{t\to +\infty }{lim\; sup}x\left(t\right)\le k.$$

Lemma 2.1. The solution (x(t), y(t)) of system (1.2) with the initial value x(0) > 0, y(0) > 0 is positive and bounded for all t ≥ 0.

Proof. We see that $\frac{dx}{dt}{|}_{x=k}=-\frac{{\left(1-m\right)}^2{k}^{2}y}{a+{\left(1-m\right)}^2{k}^2}<0$ with y > 0, so x = k is a untangent line of system (1.2). And the positive trajectory of system (1.2) goes through from its right side to its left side when it meets the line x = k.

Construct Dulac function w(x, y) = y + ux - l, computing w = 0 along the trajectories of system (1.2)

$$\frac{dw}{dt}=\frac{dy}{dt}+u\frac{dx}{dt}x=-dy+ur{x}^2\left(1-\frac{x}{k}\right)=ur{x}^2\left(1-\frac{x}{k}\right)-d\left(l-ux\right).$$

If l > 0 is large enough, we have $\frac{dw}{dt}<0$, where 0 < x < k. So the line y + ux = l goes through from its upside to its downside in the region {(x, y) | 0 < x < k, 0 < y < +∞}. For system (1.2), constructing a Bendixson ring $\widehat{OABC}$ including P₂(x₁, y₁). Define $\overline{OA}$, $\overline{AB}$, $\overline{BC}$ as the lengths of lines L₁ = y = 0, L₂ = x - k = 0, L₃ = y + ux - l, respectively. The boundary line of the Bendixson ring $\widehat{OABC}\prime \text{s}$ is J. So, the orbits of system (1.2) enter into the interior of the Bendixson ring when it meets the boundary line J.

If the initial value p which is in the first quadrant is not in the $\widehat{OABC}$, we can construct a curve J _p in the same way, denoting p ∈ J _p . Thus the positive trajectory L _p ⁺ which is pass through p goes through into the interior of J _p at the end. Note that the point P₂(x₁, y₁) is the unique equilibrium point in the Bendixson ring $\widehat{OABC}$. Through the Limit collection theory, we know that the trajectory L _p ⁺ goes through into the interior of the Bendixson ring $\widehat{OABC}$ at the end. Therefore, all of the solutions (x(t), y(t)) of system (1.2) with the initial value x(0) > 0, y(0) > 0 are positive and bounded. This completes the proof.

Lemma 2.2. (1) O(0, 0) is a stable node point;

• (2) if u ≤ d holds, P₁(k, 0) is a stable focus or a stable node;

• (3) if u > d holds, then P₁(k, 0) is a stable focus or a stable node for $1-\frac{1}{k}\sqrt{\frac{da}{u-d}}<m<1$; P₁(k, 0) is a stable node point for $0\le m<1-\frac{1}{k}\sqrt{\frac{da}{u-d}}$; P₁(k, 0) is a stable node point for $m=1-\frac{1}{k}\sqrt{\frac{da}{u-d}}$. P₂(x₁, y₁) is an unstable focus or an unstable node for $0\le m<1-\frac{u+2d}{2dk}\sqrt{\frac{da}{u-d}}$; P₂(x₁, y₁) is a stable focus or a stable node for $1-\frac{u+2d}{2dk}\sqrt{\frac{da}{u-d}}<m<1-\frac{1}{k}\sqrt{\frac{da}{u-d}}$; P₂(x₁, y₁) is a center or a focus for $m=1-\frac{u+2d}{2dk}\sqrt{\frac{da}{u-d}}$.

Proof. The Jacobian matrix of system (2.1) is given by

$$J\left(x,y\right)=\left(\begin{array}{cc}r\left[2ax-\frac{3a}{k}{x}^2+4{\left(1-m\right)}^2{x}^3-\frac{5{\left(1-m\right)}^2}{k}{x}^4\right]-2{\left(1-m\right)}^{2}xy\hfill & -{\left(1-m\right)}^2{x}^2\hfill \\ 2\left(u-d\right){\left(1-m\right)}^{2}xy\hfill & -da+\left(u-d\right){\left(1-m\right)}^2{x}^2\hfill \end{array}\right).$$

• (1) Since detJ(0, 0) = 0, O(0, 0) is a higher-order singular point. We make a transformation $d\tau =-\frac{1}{da}dt$. Substituting this into (2.1), then replacing τ with t gives

  $$\left\{\begin{array}{c}\frac{dx}{dt}=-\frac{r}{d}{x}^2+\frac{r}{dk}{x}^3-\frac{r{\left(1-m\right)}^2}{da}{x}^4+\frac{r{\left(1-m\right)}^2}{dak}{x}^5+\frac{{\left(1-m\right)}^2}{da}{x}^{2}y\equiv P_2\left(x,y\right),\hfill \\ \frac{dy}{dt}=y-\frac{\left(u-d\right){\left(1-m\right)}^2}{da}{x}^{2}y=y+Q_2\left(x,y\right).\hfill \end{array}\right.$$

  (2.2)

From y + Q₂(x, y) = 0, we can solve that y = φ(x) ≡ 0. Furthermore, we can derive that $P_2\left(x,\phi \left(x\right)\right)=-\frac{r}{d}{x}^2+\frac{r}{dk}{x}^3-\frac{r{\left(1-m\right)}^2}{da}{x}^4+\frac{r{\left(1-m\right)}^2}{dak}{x}^5$, so we have ${\mathrm{\Delta }}_n=-\frac{r}{d}\ne 0$, n = 2. According to Zhang et al. [13], we can derive that O(0, 0) is a stable node point.

• (2) The Jacobian matrix of system (2.1) for the equilibrium point P₁(k, 0) is given by

  $$J\left(k,0\right)=\left(\begin{array}{cc}r\left[-ak-{\left(1-m\right)}^2{k}^3\right]\hfill & -{\left(1-m\right)}^2{k}^2\hfill \\ 0\hfill & -da+\left(u-d\right){\left(1-m\right)}^2{k}^2\hfill \end{array}\right).$$

If u ≤ d, the eigenvalues of matrix r[-ak - (1 - m)²k³] and -da + (u - d) (1 - m)²k² are negative, hence P₁(k, 0) is a stable focus or a stable node.

• (3) If u > d, according to (2), we know that if -da + (u - d)(1 - m)²k² < 0, namely $1-\frac{1}{k}\sqrt{\frac{da}{u-d}}<m<1$, then p = detJ(k, 0) > 0, hence P₁(k, 0) is a stable focus or a stable node; If -da + (u - d) (1 - m)²k² > 0, namely $0\le m<1-\frac{1}{k}\sqrt{\frac{da}{u-d}}$, then detJ(k, 0) < 0, hence P₁(k, 0) is a saddle point. If $m=1-\frac{1}{k}\sqrt{\frac{da}{u-d}}$, then detJ(k, 0) = 0, thus P₁(k, 0) is a higher-order singular point. In this situation P₁(k, 0) and point P₂(x₁, y₁) are the same point. So P₁(k, 0) is a stable node point.

As $detJ\left(x_1,y_1\right)=2\left(u-d\right){\left(1-m\right)}^4{x}_1^3{y}_1>0$,

$$\begin{array}{cc}\hfill P& =-r\left[2a{x}_1+4{\left(1-m\right)}^2{x}_1^3-\frac{3a}{k}{x}_1^2-\frac{5{\left(1-m\right)}^2}{k}{x}_1^4\right]+2{\left(1-m\right)}^2{x}_1{y}_1\hfill \\ =-r\left[-\frac{a}{k}{x}_1^2+2{\left(1-m\right)}^2{x}_1^3-\frac{3{\left(1-m\right)}^2}{k}{x}_1^4\right]\hfill \\ =-r{x}_1^2\left[2\left(1-m\right)\sqrt{\frac{da}{u-d}}-\frac{au+2da}{k\left(u-d\right)}\right],\hfill \end{array}$$

if $2\left(1-m\right)\sqrt{\frac{da}{u-d}}-\frac{au+2da}{k\left(u-d\right)}>0$, namely $0\le m<1-\frac{u+2d}{2dk}\sqrt{\frac{da}{u-d}}$, we derive that P < 0, then P₂ (x₁, y₁) is an unstable focus or an unstable node; if $2\left(1-m\right)\sqrt{\frac{da}{u-d}}-\frac{au+2da}{k\left(u-d\right)}<0$, namely $1-\frac{u+2d}{2dk}\sqrt{\frac{da}{u-d}}<m<1-\frac{1}{k}\sqrt{\frac{da}{u-d}}$, we derive that P > 0, then P₂(x₁, y₁) is a stable focus or a stable node; If $2\left(1-m\right)\sqrt{\frac{da}{u-d}}-\frac{au+2da}{k\left(u-d\right)}=0$, namely P = 0, then we can derive that P₂(x₁, y₁) is a center or a focus. The proof is completed.

From Lemma 2.2, if $P=-r{x}_1^2\left[2\left(1-m\right)\sqrt{\frac{da}{u-d}}-\frac{au+2da}{k\left(u-d\right)}\right]=0$ holds, namely $m=1-\frac{u+2d}{2dk}\sqrt{\frac{da}{u-d}}$, P₂(x₁, y₁) is a center focus. We can make further conclusions:

Lemma 2.3. (1) if C₀ > 0 holds, P₂(x₁, y₁) is a stable fine focus with order one;

• (2) if C₀ < 0 holds, P₂(x₁, y₁) is an unstable fine focus with order one;

• (3) if C₀ = 0 and C₁ > 0 hold, P₂(x₁, y₁) is a stable fine focus with second-order;

• (4) if C₀ = 0 and C₁ < 0 hold, P₂(x₁, y₁) is an unstable fine focus with second-order.

where $C_0=\frac{\mathrm{\Pi }}{2}\left(3G+3F+I+2DE-5EG-3D-HG\right)$ and

$$\begin{array}{cc}\hfill C_1& =\frac{\mathrm{\Pi }}{8}\left(10M-\frac{28}{3}EDN-12E^{2}F-\frac{10}{3}E{D}^3-48E^{3}D+\frac{112}{3}{E}^{2}HG-\frac{106}{3}{E}^{2}HD\right.\hfill \\ -\frac{32}{3}ED{H}^2+76E^{3}G+52D{E}^2-52G{E}^2+36EDH-36EGH-2EFH+\frac{26}{3}EG{H}^2\hfill \\ +\frac{70}{3}{G}^{2}F-\frac{100}{3}ED{G}^2-\frac{2}{3}{D}^{2}F+4FN+2F{H}^2-\frac{10}{3}DL-\frac{40}{3}DGF+\frac{92}{3}EG{D}^2\hfill \\ \left.+4D^3-14G{D}^2-\frac{2}{3}ENG+\frac{40}{3}GL+10D{G}^2-6DN+10D{H}^2-10G{H}^2\right).\hfill \end{array}$$

Proof. First use the coordinate translation, that is translation the origin of coordinates into the point P₂(x₁, y₁). Then we assume

$$\bar{x}=x-x_1,\mathit{ȳ}=y-y_1,d\bar{t}={\left(1-m\right)}^2{x}_1^{2}dt.$$

Replacing $\bar{x}$, $\mathit{ȳ}$, $\bar{t}$ with x, y, t, respectively, it gives

$$\left\{\begin{array}{c}\frac{dx}{dt}=-y-\frac{2}{x_1}xy+\frac{r}{{\left(1-m\right)}^2{x}_1^2}\left[a+6x_1^2{\left(1-m\right)}^2-\frac{3a{x}_1+10{\left(1-m\right)}^2{x}_1^3}{k}-{\left(1-m\right)}^2{y}_1\right]x^2\hfill \\ -\frac{1}{x_1^2}{x}^{2}y+\frac{r}{{\left(1-m\right)}^2{x}_1^2}\left[4x_1{\left(1-m\right)}^2-\frac{a+10{\left(1-m\right)}^2{x}_1^2}{k}\right]x^3+\frac{r}{x_1^2}\left[1-\frac{5x_1}{k}\right]x^4-\frac{r}{k{x}_1^2}{x}^5,\hfill \\ \frac{dy}{dt}=\frac{2\left(u-d\right)y_1}{x_1}x+\frac{\left(u-d\right)y_1}{x_1^2}{x}^2+\frac{2\left(u-d\right)}{x_1}xy+\frac{u-d}{x_1^2}{x}^{2}y.\hfill \end{array}\right.$$

(2.3)

We denote$A=\sqrt{\frac{2\left(u-d\right)y_1}{x_1}}>0$ and make the following transformations u = x, $v=\frac{1}{A}y$, dτ = -Adt, and replacing u, v, τ with x, y, t, respectively, we have

$$\left\{\begin{array}{c}\frac{dx}{dt}=y+Dxy-E{x}^2+N{x}^{2}y-F{x}^3-L{x}^4+M{x}^5=y+{\displaystyle \sum _{j=2}^5}{P}_j\left(x,y\right)\equiv \widehat{P}\left(x,y\right),\hfill \\ \frac{dy}{dt}=-x-G{x}^2-Hxy-I{x}^{2}y=-x+{\displaystyle \sum _{j=2}^3}{Q}_j\left(x,y\right)\equiv \widehat{Q}\left(x,y\right).\hfill \end{array}\right.$$

(2.4)

where

$$\begin{array}{c}D=\frac{2}{x_1},E=\frac{r}{A{\left(1-m\right)}^2{x}_1^2}\left[a+6x_1^2{\left(1-m\right)}^2-\frac{3a{x}_1+10{\left(1-m\right)}^2{x}_1^3}{k}-{\left(1-m\right)}^2{y}_1\right],\hfill \\ N=\frac{1}{x_1^2},F=\frac{r}{A{\left(1-m\right)}^2{x}_1^2}\left[4x_1{\left(1-m\right)}^2-\frac{a+10{\left(1-m\right)}^2{x}_1}{k}\right],L=\frac{r}{A{x}_1^2}\left[1-\frac{5x_1}{k}\right],\hfill \\ M=\frac{r}{Ak{x}_1^2},G=\frac{\left(u-d\right)y_1}{A^2{x}_1^2},H=\frac{2\left(u-d\right)}{A{x}_1},I=\frac{u-d}{A{x}_1^2}.\hfill \end{array}$$

It is obvious that D² = 4N and HD = 4I. Then we make use of the method Poincare to calculate the focus value.

Construct a form progression $F\left(x,y\right)=x^2+y^2+{\sum }_{k=3}^{\infty }{F}_k\left(x,y\right)$, where F _k (x, y) is the k th homogeneous multinomials with x and y.

Considering $\frac{dF}{dt}|\left(2.4\right)=\frac{\partial F}{\partial t}\cdot \widehat{P}\left(x,y\right)+\frac{\partial F}{\partial t}\widehat{Q}\left(x,y\right)=0$, we can obtain that three multinomials and four multinomials of F (x, y) are equal to zero separately.

Noting that 2xP₂(x, y) + 2yQ₂(x, y) = -H₃(x, y) we can obtain

$$H_3=2E{x}^3+\left(2G-2D\right)x^{2}y+2Hx{y}^2$$

Let F₃(x, y) = a₀x³ + a₁x²y + a₂xy² + a₃y³, then we can obtain the following form:

$$\begin{array}{cc}\hfill y\frac{\partial F_3}{\partial x}-x\frac{\partial F_3}{\partial y}& =y\left(3a_0{x}^2+2a_{1}xy+a_2{y}^2\right)-x\left(a_1{x}^2+2a_{2}xy+3a_3{y}^2\right)\hfill \\ =-a_1{x}^3+\left(3a_0-2a_2\right)x^{2}y+\left(2a_1-3a_3\right)x{y}^2+a_2{y}^3.\hfill \end{array}$$

From $y\frac{\partial F_3}{\partial x}-x\frac{\partial F_3}{\partial y}=H_3$, we can derive that

$$-a_1{x}^3+3\left(a_0-2a_2\right)x^{2}y+\left(2a_1-3a_3\right)x{y}^2+a_2{y}^3=2E{x}^3+\left(2G-2D\right)x^{2}y+2Hx{y}^2.$$

Through the comparison method of correlates, we can obtain that

$$\left\{\begin{array}{c}{a}_0=\frac{2G-2D}{3},\hfill \\ a_1=-2E,\hfill \\ a_2=0,\hfill \\ a_3=-\frac{2H+4E}{3},\hfill \end{array}\right.$$

then $F_3\left(x,y\right)=\frac{2G-2D}{3}{x}^3-2E{x}^{2}y-\frac{2H+4E}{3}{y}^3$ and

$$\begin{array}{cc}\hfill -H_4& =2x{P}_3+2y{Q}_3+\frac{\partial F_3}{\partial x}\cdot P_2+\frac{\partial F_3}{\partial y}\cdot Q_2\hfill \\ =\left(2EG+2D-2G-2F\right)x^4+\left(2HG+4EG-2I-4DE\right)x^2{y}^2\hfill \\ +\left(2N+2GD+4E^2+2EH-2D^2\right)x^{3}y+\left(2H^2+4EH\right)x{y}^3.\hfill \end{array}$$

Let x = r cos θ, y = r sin θ, then we can derive that

$$C_0={\int }_0^{2\mathrm{\Pi }}{H}_4\left(cos\theta ,sin\theta \right)d\theta =\frac{\mathrm{\Pi }}{2}\left(3G+3F+I+2DE-5EG-3D-HG\right).$$

Hence, then point O(0, 0) of system (2.3) is an unstable fine focus with order one when C₀ > 0. But considering the time change dτ = -Adt, P₂(x₁, y₁) is a stable fine focus with order one. And P₂(x₁, y₁) is an unstable fine focus with order one when C₀ < 0. If C₀ = 0, namely 3G + 3F + I + 2DE - 5EG - 3D - HG = 0, we denote

$$F_4\left(x,y\right)=b_0{x}^4+b_1{x}^{3}y+b_2{x}^2{y}^2+b_{3}x{y}^3+b_4{y}^4.$$

By substituting F₄(x, y) = b₀x⁴ + b₁x³y + b₂x²y² + b₃xy³ + b₄y⁴ into $y\frac{\partial F_4}{\partial x}-x\frac{\partial F_4}{\partial y}=H_4$,

$$\begin{array}{c}{b}_3{y}^4-b_1{x}^4+\left(4b_0-2b_2\right)x^{3}y+\left(3b_1-3b_3\right)x^2{y}^2+\left(2b_2-4b_4\right)x{y}^3\\ =\left(2G+2F-2EG-2D\right)x^4+\left(2I+4DE-2HG-4EG\right)x^2{y}^2\\ +\left(2D^2-2N-2GD-4E^2-2EH\right)x^{3}y-\left(2H^2+4EH\right)x{y}^3.\end{array}$$

Through the comparison method of correlates, we can obtain that

$$\left\{\begin{array}{c}{b}_0=\frac{1}{2}{D}^2-\frac{1}{2}N-\frac{1}{2}GD-E^2-\frac{3}{2}EH-\frac{1}{2}{H}^2,\hfill \\ b_1=2EG+2D-2G-2F=\frac{1}{3}\left(2I+4DE-2HG-4EG\right),\hfill \\ b_2=-H^2-2EH,\hfill \\ b_3=0,\hfill \\ b_4=0,\hfill \end{array}\right.$$

then we have

$$F_4\left(x,y\right)=\left(\frac{1}{2}{D}^2-\frac{1}{2}N-\frac{1}{2}GD-E^2-\frac{3}{2}EH-\frac{1}{2}{H}^2\right)x^4+\left(2EG+2D-2G-2F\right)x^{3}y-\left(H^2+2EH\right)x^2{y}^2.$$

Also

$$\begin{array}{cc}\hfill -H_5& =2x{P}_4+2y{Q}_4+P_3\frac{\partial F_3}{\partial x}+Q_3\frac{\partial F_3}{\partial y}+P_2\frac{\partial F_4}{\partial x}+Q_2\frac{\partial F_4}{\partial y}\hfill \\ =\left(-2L-2GF+2DF-2D^{2}E+2NE+2GDE+4E^3+6E^{2}H+2E{H}^2\right)x^5\hfill \\ +(2GN-4DN+10EF+2EI+2D^3-2G{D}^2-4E^{2}D-6EHD-2H^{2}D\hfill \\ -6E^{2}G-6DE+6GE+2G{H}^2+2EGH-2DH+2GH+2FH)x^{4}y\hfill \\ +(-4EN+6EGD+6D^2-6GD-6FD+6E{H}^2+4E^{2}H\hfill \\ +2H^3)x^3{y}^2+\left(2HI+4EI-2D{H}^2-4DEH\right)x^2{y}^3.\hfill \end{array}$$

Noting F₅(x, y) = d₀x⁵ + d₁x⁴y + d₂x³y ² + d₃x²y³ + d₄xy⁴ + d₅y⁵ into $y\frac{\partial F_5}{\partial x}-x\frac{\partial F_5}{\partial y}=H_5$, through the comparison method of correlates, we can obtain that

$$\left\{\begin{array}{c}{d}_0=-\frac{2}{5}GN-\frac{4}{5}DN-2EF+6EI+\frac{8}{5}GN+\frac{4}{5}{E}^{2}D+\frac{8}{3}HI+\frac{6}{5}{E}^{2}G+\frac{6}{5}DE,\hfill \\ -\frac{6}{5}EG-\frac{2}{5}G{H}^2-\frac{2}{5}EGH+\frac{8}{5}I-\frac{2}{5}GH-\frac{2}{5}FH,\hfill \\ d_1=-2L-2GF+2DF-6EN+2GDE+4E^3+6E^{2}H+2E{H}^2,\hfill \\ d_2=2HI+2EI,\hfill \\ d_3=-\frac{28}{3}EN+\frac{14}{3}EGD+8N-2GD+\frac{2}{3}FD+\frac{14}{3}E{H}^2+\frac{28}{3}{E}^{2}H+\frac{2}{3}{H}^3\hfill \\ -\frac{8}{3}L-\frac{8}{3}GF+\frac{16}{3}{E}^3,\hfill \\ d_4=0,\hfill \\ d_5=-\frac{52}{15}EN+\frac{28}{15}EGD+\frac{16}{5}N-\frac{4}{5}GD+\frac{4}{15}FD+\frac{28}{15}E{H}^2\hfill \\ +\frac{56}{15}{E}^{2}H+\frac{4}{15}{H}^3-\frac{16}{15}L-\frac{16}{15}GF+\frac{32}{15}{E}^3.\hfill \end{array}\right.$$

(2.5)

Substituting (2.5) into

$$-H_6=2x{P}_5+2y{Q}_5+P_2\frac{\partial F_5}{\partial x}+Q_2\frac{\partial F_5}{\partial y}+P_3\frac{\partial F_4}{\partial x}+Q_3\frac{\partial F_4}{\partial y}+P_4\frac{\partial F_3}{\partial x}+Q_4\frac{\partial F_3}{\partial y},$$

we can derive that

$$\begin{array}{cc}\hfill C_1& =\frac{\mathrm{\Pi }}{8}(10M-\frac{28}{3}EDN-12E^{2}F-\frac{10}{3}E{D}^3-48E^{3}D+\frac{112}{3}{E}^{2}HG-\frac{106}{3}{E}^{2}HD\hfill \\ -\frac{32}{3}ED{H}^2+76E^{3}G+52D{E}^2-52G{E}^2+36EDH-36EGH-2EFH\hfill \\ +\frac{26}{3}EG{H}^2+\frac{70}{3}{G}^{2}F-\frac{100}{3}ED{G}^2-\frac{2}{3}{D}^{2}F+4FN+2F{H}^2-\frac{10}{3}DL\hfill \\ -\frac{40}{3}DGF+\frac{92}{3}EG{D}^2+4D^3-14G{D}^2-\frac{2}{3}ENG+\frac{40}{3}GL+10D{G}^2\hfill \\ -6DN+10D{H}^2-10G{H}^2).\hfill \end{array}$$

Hence point O(0, 0) of system (2.3) is an unstable fine focus with second-order when C₁ > 0. But considering the time change dτ = -Adt, we know that P₂(x₁, y₁) is a stable fine focus with second-order. And P₂(x₁, y₁) is an unstable fine focus with second-order when C₁ < 0. The proof is completed.

Remark: If C₁ = 0 holds, P₂(x₁, y₁) is possible a fine focus with third-order.

3 Global stability of the unique positive equilibrium

Theorem 3.1. Suppose that $1-\frac{\sqrt{3a}}{k}\le m<1$ holds and there is no close orbit around system (2.1) in the first quadrant. Assume that:

(H₁) $d<u\le \frac{4}{3}d$;

(H₂) $u>\frac{4}{3}d$ and u ≠ 4d, $1-\frac{\sqrt{3a}}{k}\le m<1-\frac{1}{k}\sqrt{\frac{da}{u-d}}$;

(H₃) u = 4d, $1-\frac{\sqrt{3a}}{k}<m<1-\frac{\sqrt{3a}}{3k}.$

Then the equilibrium P₁(k, 0) of system (2.1) is globally asymptotically stable on the first quadrant, if (H₁) holds. And the positive equilibrium P₂(x₁, y₁) of system (2.1) is globally asymptotically stable, if one of (H 2) and (H 3) holds.

Proof. By Lemmas 2.1 and 2.2, the solution (x(t), y(t)) of system (1.2) with the initial values x(0) > 0, y(0) > 0 is unanimous bounded for all t ≥ 0 and the point P₂(x₁, y₁) is globally asymptotically stable, we should proof that the system (2.1) is not exist limit cycle if $1-\frac{\sqrt{3a}}{k}\le m<1$ holds.

Define a Dulac function B(x, y) = x^-2y^-1, then from system (2.1), we have

$$\begin{array}{cc}\hfill D& =\frac{\partial \left(BP\right)}{\partial x}+\frac{\partial \left(BQ\right)}{\partial y}\hfill \\ =\frac{r}{y}\left[-\frac{a}{k}-\frac{3{\left(1-m\right)}^2}{k}{x}^2+2{\left(1-m\right)}^{2}x\right]\hfill \\ \le \frac{r}{y}\left[-\frac{a}{k}-\frac{3{\left(1-m\right)}^2}{k}{\left(\frac{k}{3}\right)}^2+2{\left(1-m\right)}^2\left(\frac{k}{3}\right)\right]\hfill \\ \le \frac{r}{y}\left[-\frac{a}{k}+\frac{{\left(1-m\right)}^{2}k}{3}\right].\hfill \end{array}$$

Hence if $m\ge 1-\frac{\sqrt{3a}}{k}$ and D < 0 for all x ≥ 0, system (2.1) does not exist any close orbit. Then we can obtain that if $1-\frac{1}{k}\sqrt{\frac{ad}{u-d}}\le 1-\frac{\sqrt{3a}}{k}$ holds, namely $d<u\le \frac{4}{3}d$, the positive equilibrium P₂(x₁, y₁) does not exist and P₁(k, 0) of system (2.1) is globally asymptotically stable; If $u>\frac{4}{3}d$ and u ≠ 4d, then $1-\frac{\sqrt{3a}}{k}>1-\frac{u+2d}{2k}\sqrt{\frac{a}{d\left(u-d\right)}}$ or if u = 4d, then $1-\frac{\sqrt{3a}}{k}=1-\frac{u+2d}{2k}\sqrt{\frac{a}{d\left(u-d\right)}}$. So the positive equilibrium P₂(x₁, y₁) of system (2.1) is globally asymptotically stable, if one of (H 2) and (H 3) holds. The proof is completed.

4 Existence of limit cycle

Theorem 4.1. Suppose that $0\le m<1-\frac{u+2d}{2k}\sqrt{\frac{a}{d\left(u-d\right)}}$. Then system (2.1) exists at least one limit cycle in the first quadrant.

Proof. In the proof of Lemma 2.1, we can obtain the boundary line J of the Bendixson ring B = {(x, y) | 0 < x < k, 0 < y < l - ux}. Let J be the outer boundary of system (2.1). Due to the Lemma 2.1, we know that there exists an unique unstable singular point P₂(x₁, y₁) in the Bendixson ring B. By Poincare-Bendixson theorem, System (2.1) exists at least one limit cycle in the first quadrant. This completes the proof.

5 Hopf bifurcation

By the study of Lou et al. [14] and the Lemma 2.3 we have the following theorem:

Theorem 5.1. (1) If C₀ > 0 and $0<1-\frac{u+2d}{2d}\cdot \frac{1}{k}\sqrt{\frac{da}{u-d}}-m\ll 1$ hold, then system (2.1) exists a limit cycle around the small neighborhood of P₂(x₁, y₁). Further if the limit cycle is unique, then it is stable.

• (2) If C₀ < 0 and $0<m-\left(1-\frac{u+2d}{2d}\cdot \frac{1}{k}\sqrt{\frac{da}{u-d}}\right)\ll 1$ hold, then system (2.1) exists a limit cycle around the small neighborhood of P₂(x₁, y₁). Further if the limit cycle is unique, then it is unstable.

• (3) If C₁ < 0, 0 < δ₃ ≪ 1, 0 < δ₄ < δ₃ ≪ 1 and 0 < C₀ < δ₃, -δ₄ < P < 0 hold, then system (2.1) exists at least two limit cycles around the small neighborhood of P₂(x₁, y₁). By selecting the suitable values of the parameters, we can obtain these two limit cycle.

• (4) If C₁ > 0, 0 < δ₅ ≪ 1, 0 < δ₆ < δ₅ ≪ 1 and -δ₅ < C₀ < 0, 0 < P < δ₆ hold, then system (2.1) exists at least two limit cycles around the small neighborhood of P₂(x₁, y₁). By selecting the suitable values of the parameters, we can obtain these two limit cycle.

6 The effect of prey refuge and harvesting efforts and examples

6.1 The influence of prey refuge on model (1.2)

By the variable transformation dt = a + (1 - m)²x²dτ, the positive equilibrium P₂(x₁, y₁) of system (1.2) takes the form $x_1=\frac{1}{1-m}\sqrt{\frac{da}{u-d}}$, and $y_1=r\left[1-\frac{1}{k\left(1-m\right)}\sqrt{\frac{da}{u-d}}\right]\frac{ua}{\left(u-d\right){\left(1-m\right)}^2}$, where 0 < x₁ < k.

For x₁ < k, we have $\frac{1}{1-m}\sqrt{\frac{da}{u-d}}<k$, namely $0<m<1-\frac{1}{k}\sqrt{\frac{da}{u-d}}$. Then we obtain

$$\frac{d{x}_1}{dm}=\frac{1}{{\left(1-m\right)}^2}\sqrt{\frac{da}{u-d}}>0.$$

(6.1)

The above inequality shows that x₁ is a strictly increasing function with respect to the parameter m and that the increasing of the prey refuge increases the density of the prey.

One could see that y₁ is also a continuous differential function of the parameter m.

Simple computation shows that

$$\frac{d{y}_1}{dm}=r\frac{ua}{\left(u-d\right){\left(1-m\right)}^3}\left[2-\frac{3}{k\left(1-m\right)}\sqrt{\frac{da}{u-d}}\right].$$

(6.2)

We discuss (6.2) in the following two cases.

Case 1: Assume that the inequality $1-\frac{3}{2k}\sqrt{\frac{da}{u-d}}<m<1-\frac{1}{k}\sqrt{\frac{da}{u-d}}$ holds, then $\frac{d{y}_1}{dm}<0$ for all m > 0, thus y₁ is a strictly decreasing function with respect to the parameter m. That is, increasing the amount of prey refuge can decrease the density of the predator. In this case, y₁ reaches the maximum value $r\left[1-\frac{1}{k}\sqrt{\frac{da}{u-d}}\right]\frac{ua}{u-d}$ at m = 0.

Case 2: Assume that the inequality $0<m<1-\frac{3}{2k}\sqrt{\frac{da}{u-d}}$ holds, then $\frac{d{y}_1}{dm}>0$ for all m > 0, thus y₁ is a strictly increasing function of parameter m. That is, increase the amount of prey refuge can increase the density the predator. This analysis shows that increasing the amount of prey refuge can increase the density of the predator due to the predator still has enough food for predation with m being small.

6.2 Example and simulations

Example 1. Let r = 0.05, k = 4, a = 3, d = 2, u = 4 in system (2.1). By simple computation, we have

$$\begin{array}{c}1-\frac{\sqrt{3a}}{k}=1-\frac{3}{4}=0.25,\hfill \\ 1-\frac{1}{k}\sqrt{\frac{da}{u-d}}=1-\frac{\sqrt{3}}{4}=0.567,\hfill \\ 1-\frac{u+2d}{2d}\cdot \frac{1}{k}\sqrt{\frac{da}{u-d}}=1-\frac{\sqrt{3}}{2}=0.134,\hfill \\ \frac{4}{3}d=\frac{8}{3}<4,u>\frac{4}{3}d.\hfill \end{array}$$

We known that $1-\frac{\sqrt{3a}}{k}<m<1-\frac{1}{k}\sqrt{\frac{da}{u-d}}$, if we take m = 0.49 and m = 0.43 separately. By Theorem 3.1, system (2.1) admits a globally asymptotically stable equilibrium P₂(x₁, y₁) in the region $R_2^{+}$, which is shown in Figures 1 and 2.

Figure 1

P ₂ ( x ₁ , y ₁ ) is globally asymptotically stable with m = 0.49.

Figure 2

P ₂ ( x ₁ , y ₁ ) is globally asymptotically stable with m = 0.43.

If we take m = 0.134 and m = 0.113 separately, by Theorem 5.1, system (2.1) admits one and two limit cycles surrounding equilibrium P₂(x₁, y₁), which is shown in Figures 3 and 4.

Figure 3

There is a stable limit cycle surrounding P ₂ ( x ₁ , y ₁ ) with m = 0.134.

Figure 4

There are two limit cycles surrounding P ₂ ( x ₁ , y ₁ ) with m = 0.113.

Example 2. Let r = 0.05, k = 4, $a=\frac{10}{3}$, d = 2, u = 4 in the system (2.1). By simple computation, we have

$$\begin{array}{c}1-\frac{\sqrt{3a}}{k}=1-\frac{\sqrt{10}}{4}=0.209,\hfill \\ 1-\frac{1}{k}\sqrt{\frac{da}{u-d}}=1-\frac{\sqrt{30}}{12}=0.544,\hfill \\ 1-\frac{u+2d}{2d}\cdot \frac{1}{k}\sqrt{\frac{da}{u-d}}=1-\frac{\sqrt{30}}{2}=0.087.\hfill \end{array}$$

If we take m = 0.065, by Theorem 5.1, system (2.1) admits three limit cycles surrounding equilibrium P₂(x₁, y₁), which is shown in Figure 5.

Figure 5

There are three limit cycles surrounding P ₂ ( x ₁ , y ₁ ) with m = 0.065.

Figures 1, 2, 3, 4, and 5 show the dependence of the dynamic behavior of system (1.2) on the prey refuge m. Figures 2, 3, 4, and 5 show that when m is small enough, there are three or two limit cycles surrounding the unique positive equilibrium, when m is large there is a stable limit cycle surrounding the unique positive equilibrium and when m is large enough, the limit cycle is broken and both the prey and predator population converge to their equilibrium values, respectively, which means that if we change the value of m, it is possible to prevent the cyclic behavior of the predator-prey system and to drive it to a required stable state.

7 Conclusion

This article considers a predator-prey model with Holling type III response function incor-porating a prey refuge under sparse effect. We give the complete qualitative analysis of the instability and global stability properties of the equilibria and the existence of limit cycles for the model. Our results, examples, and simulations indicate that dynamic behavior of the model very much depends on the prey refuge parameter m and increasing the amount of refuge could increase prey densities and lead to population outbreaks.