Bifurcation analysis of a modified Leslie–Gower model with Holling type-IV functional response and nonlinear prey harvesting

Abstract

In this work, an attempt is made to understand the dynamics of a modified Leslie–Gower model with nonlinear harvesting and Holling type-IV functional response. We study the model system using qualitative analysis, bifurcation theory and singular optimal control. We show that the interior equilibrium point is locally asymptotically stable and the system under goes a Hopf bifurcation with respect to the ratio of intrinsic growth of the predator and prey population as bifurcation parameter. The existence of bionomic equilibria is analyzed and the singular optimal control strategy is characterized using Pontryagin’s maximum principle. The existence of limit cycles appearing through local Hopf bifurcation and its stability is also examined and validated numerically by computing the first Lyapunov number. Optimal singular equilibrium points are obtained numerically for various discount rates.

1 Introduction

The Leslie–Gower model [1, 2] (LG model) shows how asymptotic solutions converge to a stable equilibrium (independent of the initial conditions) state. The equilibrium point depends on the intrinsic factors which govern the system dynamics (in the sense of biology). It marks a significant improvement over the famous Lotka–Volterra model and it is limited in its explanatory capability [3]. Korobeinikov [4] established the global stability of a positive equilibrium point and showed that the limit cycle could be admitted by the model system. This limit cycle also exists if we take the Holling type-II or -III functional response. Aziz and Okiye [5] have designed and studied the modified LG model with cyrtoid type functional response. Huang and Xiao [6] investigated a predator–prey system with Holling type-IV functional response. The qualitative analysis and bifurcation theory along with numerical simulations indicated that it has a unique stable limit cycle. Yafia et al. [7] studied the limit cycle in a modified LG model with Holling type-II scheme bifurcated from small and large time delays. Ji et al. [8] extended the study of the modified LG model designed by Aziz and Camara for stochastic perturbation. Huang et al. [9] studied controlling bifurcation in a delayed fractional predator–prey system with incommensurate orders. Rihan et al. [10] investigated fractional-order delayed predator–prey systems with Holling type-II functional response. Song et al. [11] have studied dynamic analysis of a fractional-order delayed predator–prey system with harvesting. Xu et al. [12] studied the stability and Hopf bifurcation of a population model with Holling type-IV functional response and time delay. Jana et al. [13] have studied a hybrid type tri-trophic food chain model (Holling type-IV and Beddington–DeAngelis type functional responses) and try to understand the role of top predator interference and gestation delay and observed the subcritical Hopf bifurcation phenomena in the designed model system and the bifurcating periodic solution is unstable for the considered set of parameter values. Yu [14] studied the global stability of a modified LG model with Beddington–DeAngelis type functional response. Recently, Agrawal et al. [15] investigated the occurence of double Hopf bifurcation at positive equilibrium point when they choose appropriate measure of the tolerance of the prey. Furthermore, some dynamic behaviors, such as stability switches, chaos, bifurcation and double Hopf bifurcation scenarios are observed using numerical simulations.

Presently, the population dynamics is a very importance topic in the field of bio-economics and related to the optimal management of renewable resources. It is more realistic to introduce the harvesting factor in the model system. We also introduce the idea of maximal sustainable yield in the management of renewable resources. It suggests exploiting the surplus production on the basis of biological growth model. Clark [16] reviewed how harvesting affects the fisheries management using ecological and economic models. Hoekstra et al. [17] studied the conservation and harvesting of a population dynamic model system and illustrated that several types of optimal harvesting solutions are possible and depend on both ecological parameters such as the predators search and handling time of prey and economic parameters (e.g., the maximum harvest rate, the discount rate and the cost of fishing). We know that mainly three types of harvesting have been reported in the literature: (i) constant rate of harvesting, (ii) proportional harvesting $h(x)=qEx$, where q is the catch ability coefficient, E is the effort made for harvesting and the product qE is the fishing mortality [18], and (iii) nonlinear harvesting $h(x)=qEx/(m_{1}E+m_{2}x)$, where $m_{1}$ and $m_{2}$ are suitable positive constants, proportional to the ratio of the stock level to the harvesting rate at high levels of effort and to the ratio of the effort level to the harvesting rate at high stock levels, respectively. Azar et al. [19] studied the Lotka–Volterra type two prey one predator model where the predator is harvested with two different schemes: (i) constant harvesting quota and (ii) constant harvesting effort, and one reported that a constant harvesting quota on the predator may destabilize the system. Zhu and Lan [20] made a Hopf bifurcation analysis of the LG model with constant prey harvesting. The dynamics of a model with proportionate harvesting has been studied by Mena-Lorca et al. [21]. Zhang et al. [22] studied the harvested LG model and found that harvesting has no influence on the persistence of the model system. So, the predator density strictly decreases with harvesting effort but it has no effect on the prey density. Kar and Ghorai [23] analyzed the dynamics of a delayed predator–prey model with harvesting in a modified LG model. Gupta et al. [24] studied the bifurcation analysis and control of the LG model with Michaelis–Menten type prey harvesting and observed that for a wide range of initial values the system goes to extinction. Recently, Huang et al. [25] studied the effect of constant yield predator harvesting on the dynamics of a LG model. The model exhibits different types of bifurcations including the saddle-node bifurcation, attracting and repelling Bogdanov–Takens bifurcations, and all types of Hopf bifurcation with the variation of the control parameters. Saleh [26] studied the dynamics of a modified LG model with quadratic predator harvesting.

We have proposed a new 2D modified Leslie–Gower type predator–prey model with Holling type-IV functional response in this manuscript. We have also incorporated the nonlinear harvesting in the prey population. Andrews [27] explored a function of the form $f(x)=mx/((x^{2}/i)+x+a)$ and named it the Holling type-IV functional response or Monod–Haldane [28] function, which is similar to the Monod (i.e., the Michaelis–Menten) function for low concentration but includes the inhibitory effect at high concentrations. The parameters m and a can be interpreted as the maximum per capita predation rate and the half saturation constant in the absence of any inhibitory effect. The parameter i is a measure of the predator’s immunity from or tolerance of the prey. We have analyzed this model for its rich dynamics and studied the bio-economic equilibria using singular optimal control strategies.

This work is organized as follows. In the next section, model formulation and biological meaning of the parameters are given. In Sect. 3, we presented the detail analysis of the model system. In Sect. 4, we discuss the linear stability analysis and Hopf bifurcation. Stability of the limit cycle is also presented in this section. Section 5 discusses the bionomic equilibria and optimal harvesting policy. In Sect. 6, some numerical simulation results are presented to illustrate or complement our mathematical findings. Conclusions and discussions are given in the final section which summarizes our findings.

2 Model formulation

The Leslie–Gower type model described [29] in extended form is given by

$$ \textstyle\begin{cases} \frac{du}{dt}= (r(1-\frac{u}{K})-\frac{mv}{\frac{u^{2}}{i}+u+a} )u, \\ \frac{dv}{dt}=s(1-\frac{nv}{u})v, \end{cases} $$

(1)

subject to positive initial condition $u(0)>0$, $v(0)>0$. Here $u\equiv u(t)$ and $v\equiv v(t)$ are the prey and predator population density, respectively.

Model system (1) is defined on the set

$$ \Omega =\bigl\{ (u, v)\in R^{2}|u\geq 0, v\geq 0\bigr\} $$

(2)

with all the parameters r, K, m, i, s, a and n being positive. Further, the biological meanings of the parameters are listed in Table 1.

Table 1 Parameters and their biological meanings in this paper

The interaction between prey and predator is expressed by a Holling type-IV functional response, that is, $f(u)= \frac{mu}{\frac{u^{2}}{i}+u+a}$ [3]. Taylor [30] has suggested that the subsistence of the predator depends on prey population therefore, the conventional environmental carrying capacity $K_{v}$ of the predator is taken to be proportional to the prey abundance u, thus $K_{v}= \frac{u}{n}$.

2.1 The model with prey harvesting

We introduce the nonlinear harvesting $H(u)= \frac{qEu}{m_{1} E+m_{2} u}$ in the model system (1). Therefore the modified system of differential equations is

$$ \textstyle\begin{cases} \frac{du}{dt}= (r(1-\frac{u}{K})-\frac{mv}{\frac{u^{2}}{i}+u+a}-\frac{qE}{m _{1} E+m_{2} u} )u, \\ \frac{dv}{dt}=s(1-\frac{nv}{u})v, \end{cases} $$

(3)

subject to the positive initial condition $u(0)=u_{0}>0$ and $v(0)=v_{0}>0$. The parameter r, K, m, i, s, a and n have the same biological meaning as in model (1), q is the catchability coefficient, E is the effort made to harvest individuals and $m_{1}$, $m_{2}$ are suitable constants. All parameter are assumed to be positive. System (3) is defined on the set Ω.

We introduce the following substitutions to bring the system of equations into non-dimensional form:

$$ \begin{gathered} u=Kx ,\qquad t=rT ,\qquad h=\frac{qE}{rm_{2}K} ,\qquad c=\frac{m_{1}E}{m_{2}K} ,\qquad v=\frac{rKy}{m} ,\qquad \alpha =\frac{i}{K} ,\\ \beta =\frac{nr}{m} ,\qquad \gamma = \frac{a}{K} ,\qquad \delta =\frac{s}{r} . \end{gathered} $$

The constants $m_{1}$, $m_{2}$ are chosen in such a way that $(\frac{m_{1}}{m_{2}})E< u$ where u is the prey biomass at a given instant of time. The model system takes the following form:

$$ \textstyle\begin{cases} \frac{dx}{dt}=x(1-x)-\frac{xy}{\frac{x^{2}}{\alpha }+x+\gamma }- \frac{hx}{x+c}\equiv x\psi (x, y), \\ \frac{dy}{dt}=\delta y(1-\beta \frac{y}{x})\equiv y\phi (x, y), \end{cases} $$

(4)

subject to the initial condition

$$ x(0)=x_{0}>0, \quad\quad y(0)=y_{0}>0. $$

(5)

Here,

$$\begin{aligned}& \psi (x, y)=(1-x)-\frac{y}{\frac{x^{2}}{\alpha }+x+\gamma }- \frac{h}{x+c}, \end{aligned}$$

(6)

$$\begin{aligned}& \phi (x, y)=y\biggl(1-\beta \frac{y}{x}\biggr), \end{aligned}$$

(7)

and α, β, γ, δ, h and c are positive constants.

3 Analysis of the model system

3.1 Positivity and boundedness of solution

Lemma 1

(a) All solutions $(x(t), y(t))$ of system (4) with the initial condition (5) are positive for all ${t\geq 0}$. (b) All solutions $(x(t),y(t))$ of system (4) with the initial condition (5) are bounded for all ${t\geq 0}$.

Proof

(a) Integrating Eq. (4) with initial condition (5) we have

$$\begin{aligned}& x(t)=x_{0}\exp\biggl( \int_{0}^{t}\psi \bigl(x(s), y(s)\bigr)\,ds\biggr)>0, \\& y(t)=y_{0}\exp\biggl( \int_{0}^{t}\psi \bigl(x(s), y(s)\bigr)\,ds\biggr)>0. \end{aligned}$$

Hence the whole solution starting in $\operatorname {Int}(\Omega )=\{(x, y)\in R^{2}|x>0, y>0\}$ remains in $\operatorname {Int}(\Omega )$ for all $t\geq 0$. Since the trajectories which start in positive direction of the x-axis and remain on it at all future time, the positive x-axis is an invariant set and similarly the positive y-axis is an invariant set for the system (4). Combining the two we observe that the set Ω defined in (2) is an invariant set for system (4).

(b) To prove the boundedness of solutions of Eq. (4), we use the positivity of variables x, y and results of Lin and Ho [31]. From (4) we can write

$$ \frac{dx}{dt}=x(1-x)-\frac{xy}{\frac{x^{2}}{\alpha }+x+y}- \frac{hx}{c+x}\leq x(1-x), $$

which gives

$$ x\leq \frac{1}{1+\frac{1-x_{0}}{x_{0}}e^{-t}}. $$

Therefore,

$$ x(t)\leq \max\{1, x_{0}\}\equiv M_{1}. $$

Further, from Eq. (4) we have

$$ \frac{dy}{dt}=\delta y\biggl(1-\beta \frac{y}{x}\biggr)\leq \delta y \biggl(1-\beta \frac{y}{M _{1}}\biggr), $$

which gives

$$ \frac{y}{1-\frac{\beta y}{M_{1}}}\leq \frac{y_{0}}{1-\frac{\beta y _{0}}{M_{1}}}e^{\delta t}. $$

Therefore, $y(t)\leq \max\{\frac{M_{1}}{\beta }, y_{0}\}\equiv M_{2}$. This completes the proof of the boundedness of solutions and hence the system under consideration is a dissipative system. □

3.2 Equilibrium analysis

To find the equilibrium points of the model system (4), we need to study the zero growth isoclines and the point of interaction. The equilibrium points of system (4) are obtained by

$$ \psi (x, y)=0, \quad\quad \phi (x, y)=0. $$

The axial equilibrium points are given by the roots of the quadratic equation,

$$ x^{2}+(c-1)x+h-c=0, $$

(8)

and the interior equilibria are given by the roots of the following bi-quadratic equation:

$$ Px^{4}+Qx^{3}+Rx^{2}+Sx+T=0, $$

(9)

where $P=\beta$, $Q=\beta (\alpha +c-1)$, $R=\alpha \beta c-c\beta + \alpha +h\beta +\alpha \beta \gamma -\alpha \beta$, $S=\alpha (c+ \beta h-\beta \gamma +c\beta \gamma -c\beta )$ and $T=\alpha \beta \gamma (h-c)$.

The roots of Eqs. (8) and (9) depend on the parameters h and c, so we shall consider the following cases.

Case I. When $h>c$

Axial Equilibria

The possible equilibrium points on the boundary of the first quadrant for system (4) are $E_{1}=(x_{1}, 0)$, $E_{2}=(x_{2}, 0)$ where $x_{1}$, $x_{2}$ are the positive real roots of the quadratic equation (8) with $x_{1}< x_{2}$ and are given by

$$ x_{1}=\frac{(1-c)-\sqrt{(1+c)^{2}-4h}}{2}, \quad\quad x_{2}=\frac{(1-c)+ \sqrt{(1+c)^{2}-4h}}{2}. $$

The two real positive roots $x_{1}$ and $x_{2}$ will exist if $c<1$ and $(1+c)^{2}-4h>0$. Note that the other cases are not biologically feasible.

Interior Equilibria

From (9), it is obvious that $P>0$ and $T>0$. If either (i) $Q, R, S<0$, or (ii) $Q, S>0$, $R<0$, or (iii) $Q<0$, $S, R>0$, or (iv) $Q, R>0$, $S<0$, or (v) $Q>0$, $S, R<0$, or (vi) $Q, R<0$, $S>0$, then by Descartes’ rule Eq. (9) has either two positive real roots or no real root and if either $Q, S<0$, $R>0$, then by Descartes’ rule Eq. (9) has either four positive real root or two positive real roots or no real root. Note that the other cases are not biologically feasible.

Case II. When $h< c$

Axial Equilibria

In this case $x=\tilde{x}$ is the only positive real root of Eq. (8). For $h< c$, the product of roots is negative. Therefore, the two roots are either of opposite sign or complex conjugates. The root $\tilde{x}=\frac{(1-c)+\sqrt{(1+c)^{2}-4h}}{2}$ and the real positive root x̃ will exist if $c<1$ and $(1+c)^{2}-4h>0$. Hence the axial equilibria are $\tilde{E}=( \tilde{x}, 0)$.

Interior Equilibria

From Eq. (9), obviously $P>0$ and $T<0$. If either (i) $Q, R, S<0$, or (ii) $Q, R, S>0$, then by Descartes’ rule Eq. (9) has only one positive real root and if either (i) $Q>0$, $R>0$, $S<0$, or (ii) $Q>0$, $R<0$, $S>0$, or (iii) $Q<0$, $R<0$, $S>0$, or (iv) $Q<0$, $R>0$, $S<0$, or (v) $Q>0$, $R<0$, $S<0$, or (vi) $Q<0$, $R>0$, $S>0$, then by Descartes’ rule Eq. (9) has either three positive real roots or one positive real root. Note that the other cases are not biologically feasible.

Case III. When $h=c$

Axial Equilibria

In this case $x=1-c$ is the only positive real root of Eq. (8). Therefore the axial equilibrium point is $E=(1-c, 0)$ provided $c<1$.

Interior Equilibria

From Eq. (9), it is obvious that $P>0$. If either (i) $Q, R, S<0$, or (ii) $Q>0$, $R<0$, $S<0$, or (iii) $Q>0$, $R>0$, $S<0$, then by Descartes’ rule Eq. (9) has only one positive real root and if either $Q<0$, $R>0$, $S<0$, then by Descartes’ rule Eq. (9) has either three positive real roots or one positive real root and if either (i) $Q>0$, $R<0$, $S>0$, or (ii) $Q<0$, $R>0$, $S>0$, or (iii) $Q<0$, $R<0$, $S>0$, then by Descartes’ rule Eq. (9) has either two positive real roots or no positive real root. Note that the other cases are not biologically feasible.

4 Linear stability analysis and Hopf bifurcation

Theorem 1

• (a) For $h>c$, the axial equilibrium point $E_{1}=(x_{1}, 0)$ is a repeller and $E_{2}=(x_{2}, 0)$ is a saddle point.

• (b) For $h< c$, the axial equilibrium point $\tilde{E}=(\tilde{x}, 0)$ is always a saddle point.

• (c) For $h=c$, the axial equilibrium point $E=(1-c, 0)$ is always a saddle point.

Proof

The Jacobian of system (4) evaluated at a point $(x, 0)$ is

$$ \left ( \begin{matrix} 1-2x-\frac{hc}{(c+x)^{2}} &-\frac{x}{\frac{x^{2}}{\alpha }+x+\gamma } \\ {0} &\delta \end{matrix} \right ) . $$

The eigenvalues of the Jacobian matrix are $1-2x-\frac{hc}{(c+x)^{2}}$ and δ.

• (a) For the equilibrium point $E_{1}=(x_{1}, 0)$, the eigenvalues $1-2x-\frac{hc}{(c+x)^{2}}$ and δ are both positive and so the equilibrium point $E_{1}$ is a repeller. Similarly at the equilibrium point $E_{2}=(x_{2}, 0)$, the eigenvalue $1-2x-\frac{hc}{(c+x)^{2}}$ and $\delta >0$ and the equilibrium point $E_{2}$ is a saddle point.

• (b) Since $\tilde{E}=E_{2}$, Ẽ is saddle point.

• (c) For the axial equilibrium point $E=(1-c, 0)$, the eigenvalues are $1-2x-\frac{hc}{(c+x)^{2}}=-(1-c)^{2}<0$ and $\delta >0$ and the equilibrium point E is a saddle point.

 □

Theorem 2

• (a) The equilibrium point $E_{*}(x_{*}, y_{*} )$ (for all three cases) is locally asymptotically stable if $\frac{1}{2}< x_{*}<\sqrt{\alpha \gamma }$.

• (b) System (4) undergoes a Hopf bifurcation with respect to the bifurcation parameter $\delta =\tilde{\delta }$ around the equilibrium point $E_{*} (x_{*}, y_{*})$ if $x_{*}>\frac{1}{2}$ and $\alpha \beta \tilde{\delta }(\frac{x^{2}}{\alpha }+x+\gamma )^{2}= \alpha (\frac{x^{2}}{\alpha }+x+\gamma )[\beta (1-2x_{*})(x_{*}+c)(\frac{x ^{2}}{\alpha }+x+\gamma )-c\beta (1-x_{*})(\frac{x^{2}}{\alpha }+x+ \gamma )+cx_{*}]-x_{*}(\alpha \gamma -x_{*}^{2}\beta )(x_{*}+c)$.

The proof is given in Appendix A1.

Theorem 3

System (4) undergoes a saddle-node bifurcation around $E_{*}(x_{*}, y_{*})$ with respect to the bifurcation parameter h if:

• (i) $2\beta x_{*}^{5}+\beta (4\alpha +c-1)x_{*}^{4}+\alpha \beta (2c+2 \alpha -2+4\gamma )x_{*}^{3} +\alpha (2c\beta \gamma -\beta +c\alpha \beta +\alpha -c+4\alpha \beta \gamma -\beta \gamma )x_{*}^{2} +(c \beta \gamma -\beta +2\beta \gamma^{2}-\beta \gamma +2\gamma )x_{*}+ \alpha^{2}(\beta c\gamma +\beta \gamma +c\gamma )=0$,

• (ii) $2\beta x_{*}^{6}+\beta (4\alpha +c-1+\delta )x_{*}^{5}+(2c \alpha \beta +2\alpha^{2}\beta +2\alpha \beta \delta -\alpha +2\alpha \beta \gamma +c\beta \delta +2\alpha \beta \gamma -2\alpha \beta )x _{*}^{4} +\alpha (2c\beta \gamma +c\beta -\beta \gamma -\beta -2c+c \alpha \beta +4\alpha \beta \gamma -\alpha \beta +\alpha \beta \delta -2c\beta \delta )x_{*}^{3} +\alpha (2c\alpha \beta \gamma -2\alpha \beta \gamma -c\alpha +\alpha \beta \gamma^{2}+2\alpha \beta \gamma \delta +\alpha \gamma +2c\beta \gamma \delta +c\alpha \beta \delta )x _{*}^{2} +\alpha^{2}\beta (c\gamma^{2}+\gamma^{2}\delta +2c\gamma \delta -\gamma^{2})x_{*}+c\alpha^{2}\beta \gamma^{2}\delta >0$, and

• (iii) $\sqrt{\frac{\alpha \gamma }{3}}< x_{*}<(2hc)^{1/3}-c$.

The proof is given in Appendix A2.

4.1 Stability of limit cycles

Following Perko [32], we now compute the Lyapunov coefficient σ at the point $E_{*}(x_{*}, y_{*})$ of system (4) to discuss the stability of the limit cycle. Let us translate the equilibrium $E_{*}(x_{*}, y_{*})$ of system (4) to the origin by using the transformation $\tilde{x}=x-x_{*}$ and $\tilde{y}=y-y _{*}$. Then system (4) in a neighborhood of the origin can be written as

$$\begin{aligned} \begin{gathered} \frac{d\tilde{x}}{dt} =(\tilde{x}+x_{*}) (1-\tilde{x}-x_{*})- \frac{( \tilde{x}+x_{*})(\tilde{y}+y_{*})}{\frac{(\tilde{x}+x_{*})^{2}}{ \alpha }+(\tilde{x}+x_{*})+\gamma }-\frac{h(\tilde{x}+x_{*})}{c+( \tilde{x}+x_{*})}, \\ \frac{d\tilde{y}}{dt} =\delta (\tilde{y}+y_{*}) \biggl(1-\beta \frac{ \tilde{y}+y_{*}}{\tilde{x}+x_{*}} \biggr). \end{gathered} \end{aligned}$$

(10)

Now

$$\begin{aligned}& \begin{aligned} \frac{d\tilde{x}}{dt} &=x_{*}-x_{*}^{2}- \frac{\alpha x_{*}(\tilde{y}+y _{*})}{x_{*}^{2}+\alpha x_{*}+\alpha \gamma }-\frac{hx_{*}}{c+x_{*}} \\ &\quad {} +\tilde{x} \biggl[1-2x_{*}-\frac{\alpha (\tilde{y}+y_{*})}{x_{*}^{2}+ \alpha x_{*}+\alpha \gamma }+\frac{\alpha x_{*}(2x_{*}+\alpha )( \tilde{y}+y_{*})}{(x_{*}^{2}+\alpha x_{*}+\alpha \gamma )^{2}}- \frac{hc}{(c+x _{*})^{2}} \biggr] \\ &\quad {} +\tilde{x}^{2} \biggl[-1+\frac{\alpha x_{*}(3x_{*}+\alpha )(\tilde{y}+y _{*})}{(x_{*}^{2}+\alpha x_{*}+\alpha \gamma )^{2}}-\frac{\alpha x _{*}(2x_{*}+\alpha \gamma )^{2}(\tilde{y}+y_{*})}{(x_{*}^{2}+\alpha x _{*}+\alpha \gamma )^{3}}+ \frac{hc}{(c+x_{*})^{3}} \biggr] \\ &\quad {} +\tilde{x}^{3} \biggl[-\frac{2\alpha x_{*}(2x_{*}+\alpha )(y_{*}+ \tilde{y})}{(x_{*}^{2}+\alpha x_{*}+\alpha \gamma )^{2}}-\frac{\alpha (2x_{*}+\alpha )^{2}(y_{*}+\tilde{y})}{(x_{*}^{2}+\alpha x_{*}+\alpha \gamma )^{3}} \\ &\quad {} +\frac{\alpha x_{*}(2x_{*}+\alpha )^{3}(y_{*}+\tilde{y})}{(x_{*} ^{2}+\alpha x_{*}+\alpha \gamma )^{4}}-\frac{hc}{(c+x_{*})^{4}} \biggr]+F ^{1}(\tilde{x}, \tilde{y}), \end{aligned} \\& \begin{aligned} \frac{d\tilde{y}}{dt} &=y_{*}\delta -\frac{\beta \delta y_{*}^{2}}{x _{*}} \biggl(1- \frac{\tilde{x}}{x_{*}}+\frac{\tilde{x}^{2}}{x_{*}^{2}}-\frac{ \tilde{x}^{3}}{x_{*}^{3}} \biggr)+\delta \tilde{y} \biggl[1-\frac{2\beta y_{*}}{x_{*}} \biggl(1-\frac{\tilde{x}}{x_{*}}+\frac{\tilde{x}^{2}}{x _{*}^{2}}- \frac{\tilde{x}^{3}}{x_{*}^{3}} \biggr) \biggr] \\ &\quad {} +\delta \tilde{y}^{2} \biggl[-\frac{\beta }{x_{*}} \biggl(1- \frac{ \tilde{x}}{x_{*}}+\frac{\tilde{x}^{2}}{x_{*}^{2}}-\frac{\tilde{x}^{3}}{x _{*}^{3}} \biggr) \biggr]+F^{2}(\tilde{x}, \tilde{y}), \end{aligned} \end{aligned}$$

where $F^{1}(\tilde{x}, \tilde{y})$ and $F^{2}(\tilde{x}, \tilde{y})$ are power series in powers of $\tilde{x}^{i}\tilde{y}^{j}$ satisfying $i+j>4$. Then we obtain

$$\begin{aligned}& \frac{d\tilde{x}}{dt} = a_{10}\tilde{x}+a_{01} \tilde{y}+a_{20} \tilde{x}^{2}+a_{11}\tilde{x} \tilde{y}+a_{02}\tilde{y}^{2}+a_{30} \tilde{x}^{3}+a_{21}\hat{x}^{2}\hat{y}+a_{03} \tilde{y}^{3}+F^{1}( \tilde{x}, \tilde{y}), \\& \frac{d\tilde{y}}{dt} = b_{10}\tilde{x}+b_{01} \tilde{y}+b_{20} \tilde{x}^{2}+b_{11}\tilde{x} \tilde{y}+b_{02}\tilde{y}^{2}+b_{30} \tilde{x}^{3}+b_{21}\hat{x}^{2}\hat{y}+b_{03} \tilde{y}^{3}+F^{2}( \tilde{x}, \tilde{y}), \end{aligned}$$

where

$$\begin{aligned}& a_{10} = 1-2x_{*}-\frac{\alpha y_{*}}{x_{*}^{2}+\alpha x_{*}+\alpha \gamma }+\frac{\alpha x_{*}(2x_{*}+\alpha )y_{*}}{(x_{*}^{2}+\alpha x _{*}+\alpha \gamma )^{2}}- \frac{hc}{(c+x_{*})^{2}}, \\& a_{01} = -\frac{\alpha x_{*}}{x_{*}^{2}+\alpha x_{*}+\alpha \gamma }, \\& a_{20} = -1+\frac{\alpha x_{*}y_{*}}{(x_{*}^{2}+\alpha x_{*}+\alpha \gamma )^{2}}+\frac{\alpha (2x_{*}+\alpha )y_{*}}{(x_{*}^{2}+\alpha x _{*}+\alpha \gamma )^{2}}-\frac{\alpha x_{*}(2x_{*}+\alpha \gamma )^{2}y _{*}}{(x_{*}^{2}+\alpha x_{*}+\alpha \gamma )^{3}}+ \frac{hc}{(c+x_{*})^{3}}, \\& a_{11} = \frac{\alpha (x_{*}^{2}-\gamma )}{(x_{*}^{2}+\alpha x_{*}+ \alpha \gamma )^{2}},\quad\quad a_{02}=0, \\& a_{30} = -\frac{2\alpha x_{*}(2x_{*}+\alpha )y_{*}}{(x_{*}^{2}+\alpha x_{*}+\alpha \gamma )^{2}}-\frac{\alpha (2x_{*}+\alpha )^{2}y_{*}}{(x _{*}^{2}+\alpha x_{*}+\alpha \gamma )^{3}}+\frac{\alpha x_{*}(2x_{*}+ \alpha )^{3}y_{*}}{(x_{*}^{2}+\alpha x_{*}+\alpha \gamma )^{4}}- \frac{hc}{(c+x _{*})^{4}}, \\& a_{21} = \frac{\alpha x_{*}}{(x_{*}^{2}+\alpha x_{*}+\alpha \gamma )^{2}}+\frac{ \alpha (2x_{*}+\alpha )}{(x_{*}^{2}+\alpha x_{*}+\alpha \gamma )^{2}}-\frac{ \alpha x_{*}(2x_{*}+\alpha \gamma )^{2}}{(x_{*}^{2}+\alpha x_{*}+ \alpha \gamma )^{3}},\quad\quad a_{12}=0, \\& a_{03}=0, \\& b_{10} = \frac{\beta \delta y_{*}^{2}}{x_{*}^{2}}, \quad\quad b_{01}=\delta \biggl(1- \frac{2 \beta y_{*}}{x_{*}}\biggr), \quad\quad b_{20}=- \frac{\beta \delta y_{*}^{2}}{x_{*}^{3}},\quad\quad b_{11}=\frac{2\beta \delta y_{*}}{x_{*}^{2}}, \\& b_{02}=-\frac{\beta \delta }{x_{*}}, \quad\quad b_{30} = \frac{\beta \delta y_{*}^{2}}{x_{*}^{4}},\quad\quad b_{21}=-\frac{2 \beta \delta y_{*}}{x_{*}^{3}},\quad\quad b_{12}= \frac{\beta \delta }{x_{*}^{2}}, \quad\quad b_{03}=0, \end{aligned}$$

with $F^{1}(\tilde{x}, \tilde{y})=\sum_{i+j=4}^{\infty }a_{ij} \tilde{x}^{i}\tilde{y}^{j}$ and $F^{2}(\tilde{x}, \tilde{y})=\sum_{i+j=4} ^{\infty }b_{ij}\tilde{x}^{i}\tilde{y}^{j}$. Hence the first Lyapunov coefficient σ for the planar system is given by

$$\begin{aligned} \sigma =&-\frac{3\pi }{2a_{01}\Delta^{3/2}}\bigl[\bigl\{ a_{10}b_{10} \bigl(a_{11} ^{2}+a_{11} b_{02}+a_{02}b_{11} \bigr)+a_{10}a_{01}\bigl(b_{11}^{2}+a_{20}b_{11}+a _{11}b_{02}\bigr) \\ & {} +b_{10}^{2}(a_{11}a_{02}+2a_{02}b_{02})-2a_{10}b_{10} \bigl(b_{20}^{2}-a _{20}a_{02} \bigr)-2a_{10}a_{01}\bigl(a_{20}^{2}-b_{20}b_{02} \bigr) \\ & {} -a_{01}^{2}(2a_{20}b_{20}+b_{11}b_{20})+ \bigl(a_{01}b_{10}-2a_{10}^{2}\bigr) (b _{11}b_{02}-a_{11}a_{20})\bigr\} \\ & {} -\bigl(a_{10}^{2}+a_{01}b_{10}\bigr) \bigl\{ 3(b_{01}b_{03}-a_{01}a_{30})+2a_{10}(a _{21}+b_{12})+(b_{10}a_{12}-a_{01}b_{21}) \bigr\} \bigr], \end{aligned}$$

where

$$\begin{aligned} \Delta =&\frac{\delta x_{*}}{\alpha^{2}\beta (x_{*}+c)(\frac{x_{*} ^{2}}{\alpha }+x_{*}+\gamma )^{2}}\bigl[2\beta x_{*}^{5}+\beta (4 \alpha +c-1)x _{*}^{4}+\alpha \beta (2c+2\alpha -2+4\gamma )x_{*}^{3} \\ & {} +\alpha (2c\beta \gamma -\beta +c\alpha \beta +\alpha -c+4\alpha \beta \gamma -\beta \gamma )x_{*}^{2} \\ & {} +\alpha^{2}\bigl(c\beta \gamma -\beta +2\beta \gamma^{2}- \beta \gamma +2 \gamma \bigr)x_{*}+\alpha^{2}(c\beta \gamma - \beta \gamma +c \gamma )\bigr]. \end{aligned}$$

Since the expression for the Lyapunov number σ is complicated we cannot say anything about the sign of σ and therefore we have analyzed it numerically.

5 Bionomic equilibria

The bionomic equilibrium is obtained when the total revenue obtained by selling the harvested biomass equals the total cost utilized in harvesting. The net profit at any time [33] is given by

$$ P(x, E)= \biggl(\frac{pqx}{m_{1}E+m_{2}x}-C \biggr)E. $$

(11)

Note that if the harvesting cost is greater than the revenue for prey species (i.e. $C>\frac{pqx}{m_{1}E+m_{2}x}$), then the harvesting in prey species is not profitable and it is of no interest. Hence, we consider that the cost must be less than the revenue for prey species (i.e. $C<\frac{pqx}{m_{1}E+m_{2}x}$).

The bionomic equilibrium $(x, y,E)$ is given by the positive solutions of $\frac{dx}{dt}=\frac{dy}{dt}=P=0$. That is,

$$\begin{aligned}& r \biggl(1-\frac{x}{K} \biggr)-\frac{mx}{\frac{x^{2}}{i}+x+a}- \frac{qE}{m_{1}E+m_{2}x}=0, \end{aligned}$$

(12)

$$\begin{aligned}& x=ny, \end{aligned}$$

(13)

$$\begin{aligned}& \frac{pqx}{m_{1}E+m_{2}x}-C=0. \end{aligned}$$

(14)

Thus, the bionomic equilibria are the points of intersection of biological equilibrium line and zero profit line. Solving Eqs. (12) and (13), we obtain the value of $x_{\infty }$ and $y_{\infty }$ and from (14) we obtain $E_{\infty }=\frac{pq-Cm_{2}}{cm_{1}}x_{\infty }$ if $Cm_{2}< pq$.

5.1 Optimal harvesting policy

In this section, we explain the optimal harvesting policy or singular optimal control to be adopted by a regulatory agency. With the help of Pontryagin’s maximum principle [34], we maximize the current value of the continuous time stream of revenues, which is given by

$$ J(x_{*}, E)= \int_{0}^{\infty }e^{-\mu t}P(x, E)\,dt, $$

(15)

where μ denotes the continuous annual discount rate which is fixed by harvesting agencies. The control variable E is subject to the constraint $0\leq E\leq E_{m}ax$, where $E_{m}ax$ is a feasible upper limit for the harvesting effort.

Therefore, the optimal control problem over an infinite time horizon is given by

$$ \max_{0\leq E\leq E_{\max}} \int_{0}^{\infty }e^{-\mu t}P(x, E)\,dt, $$

(16)

subject to Eqs. (3) with $(x, y)\neq (0, 0)$ and $x(0)=x_{0}$, $y(0)=y_{0}$.

The Hamiltonian function is given by

$$ \begin{aligned} H(x,y,E,t)&= \biggl(\frac{pqx}{m_{1}E+m_{2}x}-C \biggr)Ee^{-\mu t} \\ &\quad{} +\lambda _{1} \biggl(r \biggl(1-\frac{x}{K} \biggr)-\frac{mx}{\frac{x^{2}}{i}+x+a}- \frac{qE}{m _{1}E+m_{2}x} \biggr)+\lambda_{2}sy \biggl(1-\frac{ny}{x} \biggr), \end{aligned} $$

where $\lambda_{i}=\lambda_{i}(t)$, $i=1, 2$, are adjoint variables.

Differentiating the Hamiltonian H with respect to the control variable E, we get

$$ \frac{\partial H}{\partial E}= \biggl( \frac{pqm_{2}x}{(m_{1}E+m_{2}x)^{2}}-C \biggr)e^{-\mu t}-\lambda_{1}\frac{qm _{2}x}{(m_{1}E+m_{2}x)^{2}}. $$

(17)

The considered control problem admits a singular solution on the control set $[0,E_{\max}]$, if $\frac{\partial H}{\partial E}=0$, which gives

$$ \lambda_{1}e^{\mu t}=p-\frac{C(m_{1}E+m_{2}x)^{2}}{qm_{2}x^{2}}, $$

(18)

where $\lambda_{1}e^{\mu t}$ is the usual shadow price [30].

In order to find the path of a singular control, Pontryagin’s maximum principle [34] is utilized and the adjoint variables must satisfy the adjoint equations given by

$$ \frac{d\lambda_{1}}{dt}=-\frac{\partial H}{\partial x}, \quad\quad \frac{d\lambda _{2}}{dt}=- \frac{\partial H}{\partial y}. $$

(19)

For a singular optimal equilibrium solution, we use the steady state equations (12) and (13) in terms of $x^{*}$ and $y^{*}=\frac{x^{*}}{n}$, hence $x^{*}$, $y^{*}$ and E can be taken as constant [35]. Thus Eq. (19) along with the steady state equations (12) and (13) give

$$\begin{aligned}& \begin{aligned}[b] \frac{d\lambda_{1}}{dt}&=\lambda_{1} \biggl( \frac{rx^{*}}{K}-\frac{ \frac{m}{n}x^{*^{2}}(\frac{2x^{*}}{i}+1)}{(\frac{x^{*^{2}}}{i}+x^{*}+a)^{2}}-\frac{qm _{2}Ex^{*}}{(m_{1}E+m_{2}x^{*})^{2}} \biggr) \\ &\quad{} - \frac{\lambda_{2}s}{n}-\frac{pqm _{1}E^{2}}{(m_{1}E+m_{2}x^{*})^{2}}e^{-\mu t}, \end{aligned} \end{aligned}$$

(20)

$$\begin{aligned}& \frac{d\lambda_{2}}{dt}=s\lambda_{2}+\lambda_{1} \biggl(\frac{mx^{*}}{\frac{x ^{*^{2}}}{i}+x^{*}+a} \biggr). \end{aligned}$$

(21)

Due to the presence of the term $e^{-\mu t}$, no steady state is possible for the above system. Following Srinivasu et al. [36], we consider the transformation

$$ \lambda_{i}(t)=\tau_{i}(t)e^{-\mu t}, \quad i=1, 2, $$

where $\tau_{i}$ represents the present value of the adjoint variable $\lambda_{i}$. Using Eq. (18), Eq. (21) can be written in terms of $\tau_{2}$ as follows:

$$ \frac{d\tau_{2}}{dt}-(s+\mu )\tau_{2}=-R \bigl(x^{*}\bigr), $$

(22)

where

$$ R\bigl(x^{*}\bigr)= \biggl(\frac{mx^{*}}{\frac{x^{*^{2}}}{i}+x^{*}+a} \biggr) \biggl( \frac{C(m _{1}E+m_{2}x^{*})^{2}}{qm_{2}x^{*^{2}}}-p \biggr). $$

This shadow price $\tau_{i}(t)=\lambda_{i}(t)e^{-\mu t}$, $i=1,2$, should remain constant over time in singular equilibrium to satisfy the transversality conditions at ∞ (i.e. $\lim_{t\rightarrow \infty }\lambda_{i}(t)=0$ for $i=1,2$). Thus, the solution of Eq. (23) satisfying the transversality condition for the discounted autonomous infinite horizon problem (16) is $\tau_{2}=\frac{R(x ^{*})}{(s+\mu )}$.

Using the above value of $\tau_{2}$ Eq. (20) can be written in terms of $\tau_{1}$ as follows:

$$ \frac{d\tau_{1}}{dt}-\bigl(\xi \bigl(x^{*}\bigr)+\mu \bigr)\tau_{1}=-\eta \bigl(x^{*}\bigr), $$

(23)

where

$$\begin{aligned}& \xi \bigl(x^{*}\bigr)=\frac{rx^{*}}{K}-\frac{\frac{m}{n}x^{*^{2}}( \frac{2x^{*}}{i}+1)}{\frac{x^{*^{2}}}{i}+x^{*}+a}- \frac{qm_{2}Ex^{*}}{(m _{1}E+m_{2}x^{*})^{2}}, \\& \eta \bigl(x^{*}\bigr)=\frac{pqm_{1}E^{2}}{(m_{1}E+m_{2}x^{*})^{2}}+\frac{sR(x ^{*})}{n(s+\mu )}. \end{aligned}$$

The solution of Eq. (23) satisfying the transversality condition at ∞ is given by

$$ \tau_{2}=\frac{\eta (x^{*})}{\xi (x^{*})+\mu }. $$

(24)

From Eqs. (18) and (24) we obtain

$$ \frac{pqm_{2}x^{*^{2}}}{(m_{1}E+m_{2}x^{*})^{2}}-C= \biggl(\frac{\eta (x ^{*})}{\xi (x^{*})+\mu } \biggr) \frac{qm_{2}x^{*^{2}}}{(m_{1}E+m_{2}x ^{*})^{2}}. $$

(25)

This gives the desired singular path.

Equation (25) along with the steady state equations (12) and (13) reduces to the following polynomial equation in $x^{*}$:

$$ f\bigl(x^{*}\bigr)=A_{1}x^{*}{^{5}}+A_{2}x^{*}{^{4}}+A_{3}x^{*}{^{3}}+A_{4}x ^{*}{^{2}}+A_{5}x^{*}+A_{6}=0, $$

(26)

where $A_{1}$, $A_{2}$, $A_{3}$, $A_{4}$, $A_{5}$, $A_{6}$ are found using MATHEMATICA and are given by

$$\begin{aligned}& A_{1} = \frac{Hr}{K}, \quad\quad A_{2}=-G+EH+ \frac{2irH}{K}, \\& A_{3} = 2iG+2iEH-2imH+\frac{2iarH}{K}+\frac{i^{2}rH}{K}- \frac{imsF}{n(s+ \mu )}, \\& A_{4} = -2iaG+2aiEH-i^{2}G+i^{2}EH-i^{2}mH+ \frac{2i^{2}arH}{K}-\frac{i ^{2}msF}{n(s+\mu )}, \\& A_{5} = -2iaG^{2}+2i^{2}aEH+\frac{i^{2}a^{2}rH}{K}- \frac{i^{2}msF}{n(s+ \mu )}, \\& A_{6} = -i^{2}aG+i^{2}a^{2}EH,\\& \quad E=\mu - \frac{m_{2}C}{m_{1}p^{2}q}(pq-Cm _{2}), F = \frac{p^{2}q}{Cm_{2}}-p, G=\frac{(pq-Cm_{2})^{2}}{pqm_{1}}, H=\frac{(m _{2}C-pq)p}{m_{2}C}. \end{aligned}$$

Equation (26) will have at least one positive real root for $x^{*}$ if $A_{1}>0$ and $A_{6}<0$. Equivalently, we can say that if $m_{2}C>pq$ and $G>aEH$, then Eq. (26) will have at least one positive real root for $x^{*}$, which together with the steady state equations (12) and (13) gives the singular equilibrium point

$$ \bigl(x^{*}, y^{*}, E^{*}\bigr)= \biggl(x^{*}, \frac{x^{*}}{n}, \frac{n(m_{1}rx ^{*}+qK-m_{1}rK)(\frac{x^{*}{^{2}}}{i}+x^{*}+a)+mx^{*}}{m_{2}x^{*}(nr(K-x ^{*})(\frac{x^{*}{^{2}}}{i}+x^{*}+a)-mx^{*}K)} \biggr). $$

Notice that

$$ \begin{aligned} \frac{\partial^{2}H(x^{*}, y^{*}, E^{*}, t)}{\partial x^{2}}&=-\frac{2 \lambda_{1}r}{K}+\frac{2ix^{*}(x^{*}{^{3}}+ax^{*}+2ai)}{n(x^{*}{^{2}}+ix ^{*}+ai)^{3}}\\&\quad{} -\frac{2m_{1}E^{*}{^{2}}}{x^{*}{^{2}}(m_{1}E^{*}+m_{2}x ^{*})} \biggl(\frac{\partial H}{\partial E}+Ce^{-\mu t} \biggr)-\frac{2 \lambda_{2}s}{nx^{*}} \end{aligned} $$

and

$$ \frac{\partial^{2}H(x^{*}, y^{*}, E^{*}, t)}{\partial y^{2}}=-\frac{2 \lambda_{2}ns}{x^{*}}. $$

Therefore, in the case of singular control (i.e. $\frac{\partial H}{ \partial E}=0$)

$$ \frac{\partial^{2}H(x^{*}, y^{*}, E^{*}, t)}{\partial x^{2}}< 0, \quad\quad \frac{ \partial^{2}H(x^{*}, y^{*}, E^{*}, t)}{\partial y^{2}}< 0, $$

for all $t\in [0, \infty )$ provided

$$ -\frac{2\lambda_{1}r}{K}+\frac{2ix^{*}(x^{*}{^{3}}+ax^{*}+2ai)}{n(x ^{*}{^{2}}+ix^{*}+ai)^{3}}-\frac{2m_{1}E^{*}{^{2}}}{x^{*}{^{2}}(m_{1}E ^{*}+m_{2}x^{*})}Ce^{-\mu t}- \frac{2\lambda_{2}s}{nx^{*}}< 0, $$

(27)

and $\lambda_{i}(t)$, $i=1, 2$ are positive i.e. $R(x^{*} )>0$ and $\xi (x^{*})+\mu >0$, which gives

$$ \frac{C(m_{1}E+m_{2}x^{*})^{2}}{qm_{2}x^{*}{^{2}}}>p,\quad\quad \frac{rx^{*}}{K}+\mu > \frac{\frac{m}{n}x^{*}{^{2}}(\frac{2x^{*}}{i}+1)}{(\frac{x ^{*}{^{2}}}{i}+x^{*}+a)^{2}}+ \frac{qm_{2}Ex^{*}}{(m_{1}E+m_{2}x^{*})^{2}}. $$

(28)

Thus, the maximized Hamiltonian $H^{*}$ is concave in both x and y for all $t\in [0, \infty )$ provided (27) and (28) are satisfied. Hence, the Arrow sufficiency condition for an infinite time horizon is satisfied [34] under certain constraints.

The generalized Legendre–Clebsch condition for the optimal control problem (16) is trivially satisfied as $\frac{\partial H}{\partial E}=0$ for all $t\in [0, \infty )$ along the optimal singular solution. Hence, from the Arrow sufficient conditions for infinite time horizon and generalized Legendre–Clebsch condition, the singular solution $(x^{*},y^{*},E^{*})$ is a part of the optimal solution (piecewise continuous curve) locally.

Further, by differentiating (17) with respect to t along a singular solution together with $\frac{dx}{dt}=0$ and $\frac{d \lambda_{1}}{dt}=\frac{-\mu \eta (x)}{\xi (x)+\mu }e^{-\mu t}$, we obtain

$$ \frac{d}{dt}\frac{dH}{dE}= \biggl( \frac{qm_{2}x^{2}}{(m_{1}x+m_{2}E)^{2}} \biggl(-P+\frac{\eta (x)}{ \xi (x)+\mu } \biggr)+C \biggr)\mu e^{-\mu t} $$

(29)

for all $t\in [0, \infty )$.

From (29), we infer that the singular optimal solution is $(x^{*}, y^{*}, E^{*})$ where $x^{*}$, the positive root of the equation, is

$$ \frac{qm_{2}x^{2}}{(m_{1}x+m_{2}E)^{2}} \biggl(-P+\frac{\eta (x)}{ \xi (x)+\mu } \biggr)+C=0, $$

which coincides with Eq. (25) for the singular solution. The maximum present value can be found by the evaluation of the performance measure in the obtained optimal values from the previous analysis [37]. In this case

$$ J\bigl(x^{*}, E^{*}\bigr)= \int_{0}^{\infty }E^{*} \biggl( \frac{pqx^{*}}{m_{1}E+m _{2}x^{*}}-C \biggr)e^{-\mu t}\,dt. $$

(30)

This gives

$$ J\bigl(x^{*}, E^{*}\bigr)= \frac{E^{*} (\frac{pqx^{*}}{m_{1}E+m_{2}x^{*}}-C )}{\mu }. $$

(31)

Therefore

$$ J\bigl(x^{*}, E^{*}\bigr)= \frac{H^{*}(x^{*}, y^{*}, E^{*}, 0)}{\mu }. $$

(32)

The transversality condition is

$$ \lim_{t\rightarrow \infty }e^{-\mu t}H^{*}\bigl(x^{*}, y^{*}, E^{*}, t\bigr)=0, $$

and it is trivially satisfied. This result agrees with the Michel theorem for the discounted autonomous infinite horizon model [38].

6 Numerical simulation results

The model system (4) is integrated numerically using the Runge–Kutta method for different sets of parameters values.

• (i) For $\alpha =0.1$, $\beta =2$, $c=0.004$, $\delta =0.5$, $\gamma =1$, $h=0.09$, the equilibrium points are $E_{1}(0.101018, 0.050509)$ and $E_{2}(0.847354, 0.423677)$.

  For the equilibrium point $E_{1}(0.101018, 0.050509)$, $\operatorname {Tr}N=0.233986$ and $\det N=-0.346001$. For the equilibrium point $E_{2}(0.847354, 0.423677)$, $\operatorname {Tr}N=-1.16308$ and $\det N=0.355004$. The equilibrium point $E_{1}(0.101018, 0.050509)$ is a saddle point and the equilibrium point $E_{2}(0.847354, 0.423677)$ is a stable focus, which is shown in Fig. 1.

  Figure 1

  

  The prey and predator nullclines are represented by red and blue lines respectively. The equilibrium point $E_{1} (0.101018,0.050509)$ is saddle point and the equilibrium point $E_{2}(0.847354,0.423677)$ is stable focus

• (ii) For $\alpha =0.2$, $\beta =0.125$, $c=0.049$, $\delta =0.12$, $\gamma =1$, $h=0.05$, which gives the two equilibrium point $E_{11}(0.0018, 0.0144)$ and $E_{21}(0.0695, 0.556)$.

  The equilibrium point $E_{11}(0.0018, 0.0144)$ is a saddle point as the eigenvalues of Jacobian matrix are −0.107753 and 0.0208392. A stable limit cycle appears in a small neighborhood of $E_{21}(0.0695, 0.556)$ as the Lyapunov number $\sigma =-30387.3\pi <0$ (see Fig. 2).

  Figure 2

  

  The figure shows that the limit cycle around the equilibrium point $E_{21} (0.0695,0.556)$ is stable

• (iii) For $\alpha =0.5$, $\beta =7.5$, $c=0.1$, $\delta =0.002$, $\gamma =1$, $h=0.05$, we have only one equilibrium point, $E_{*}(0.9169, 0.1223)$.

  For the equilibrium point $E_{*}(0.9169, 0.1223)$, $\operatorname {Tr}N=-0.8342$ and $\det N=0.0017$ and so the equilibrium point $E_{*}(0.9169, 0.1223)$ is a stable focus, which is shown in Fig. 3.

  Figure 3

  

  The prey and predator nullclines are represented by red and blue lines respectively. The equilibrium point $E^{*}(0.9169,0.1223)$ is stable focus

• (iv) For $\alpha =0.2$, $\beta =0.125$, $c=0.1$, $\delta =0.12$, $\gamma =1$, $h=0.05$, the equilibrium point is $E_{*}(0.09177, 0.73416)$.

  An unstable limit cycles appears in a small neighborhood of $E_{*}(0.09177, 0.73416)$ as the Lyapunov number $\sigma =728.498 \pi >0$ (see Fig. 4).

  Figure 4

  

  The figure shows that the limit cycle around the equilibrium point $E_{*}(0.09177, 0.73416)$ is unstable

• (v) For $\alpha =0.5$, $\beta =7.5$, $c=0.1$, $\delta =0.002$, $\gamma =1$, $h=0.1$, we have only one equilibrium point, which is $E^{**}(0.8617, 0.1149)$.

  For the equilibrium point $E^{**}(0.8617, 0.1149)$, $\operatorname {Tr}N=-0.7312$ and $\det N=0.0015$, so the equilibrium point $E^{**}(0.8617, 0.1149)$ is a stable focus, which is shown in Fig. 5.

  Figure 5

  

  The prey and predator nullclines are represented by red and blue lines respectively. The equilibrium point $E^{**}(0.8617,0.1149)$ is stable focus

• (vi) For $\alpha =0.2$, $\beta =0.125$, $c=0.01$, $\delta =0.12$, $\gamma =1$, $h=0.01$, the equilibrium point is $E_{3}(0.1196, 0.9568)$.

  An unstable limit cycles appears in a small neighborhood of $E_{3}(0.1196, 0.9568)$ as the Lyapunov number $\sigma =330.29\pi >0$ (see Fig. 6).

  Figure 6

  

  The figure shows that the limit cycle around the equilibrium point $E_{3}(0.1196, 0.9568)$ is unstable

Also it is too difficult to solve Eq. (26) analytically and therefore we consider the following numerical examples.

Case (i) :

For $h>c$: For the parameters $a=1$, $n=1$, $m=0.4$, $s=0.01$, $r=0.02$, $i=0.1$, $p=0.5$, $q=0.9$, $K=100$, $C=0.03$, $m_{1}=0.1$, $m_{2}=0.03$, the optimal singular solutions for different rates of μ are given in Table 2.

Table 2 Optimal singular solution for different discount rate μ

Case (ii) :

For $h< c$: For the parameters $a=1$, $n=1$, $m=0.4$, $s=0.01$, $r=0.02$, $i=0.1$, $p=0.5$, $q=0.0009$, $K=100$, $C=0.03$, $m_{1}=0.1$, $m_{2}=0.03$, the optimal singular solutions for different rates of μ are given in Table 3.

Table 3 Optimal singular solution for different discount rate μ

Case (iii) :

For $h=c$: For the parameters $a=1$, $n=1$, $m=1.04$, $s=0.1$, $r=0.02$, $i=0.9$, $p=0.5$, $q=0.002$, $K=100$, $C=0.003$, $m_{1}=0.1$, $m_{2}=0.03$, the optimal singular solutions for different rates of μ are given in Table 4.

Table 4 Optimal singular solution for different discount rate μ

Note that $y_{(j)}^{*}=x_{(j)}^{*}$, $j=1,2,3,4,5$, since in each case $n=1$.

It is clearly seen that we can find an optimum equilibrium for various discount rates μ in various cases. From the above tables we conclude that there is at least one optimum singular equilibrium point in each from which $(x_{(1)}^{*},y_{(1)}^{*},E_{(1)}^{*})$ is feasible from an ecological point of view. Once the optimal singular solution is known then one can easily find the optimal paths (see Figs. 7–9).

Figure 7

Plot $E^{*}$ vs $x^{*}$ for the case (i) with $\mu= 0.2$

Figure 8

Plot $E^{*}$ vs $x^{*}$ for the case (ii) with $\mu=0.2$

Figure 9

Plot $E^{*}$ vs $x^{*}$ for the case (iii) with $\mu= 0.2$

7 Conclusions and discussions

In this paper, a mathematical model to study the dynamical behavior of a modified Leslie–Gower model with Holling type-IV functional response and nonlinear prey harvesting has been proposed and analyzed. The existence of equilibrium points and their stability analysis have been discussed with the help of stability theory. The proposed model system undergoes a Hopf bifurcation and a saddle-node bifurcation around the equilibrium point for the control parameter δ, the ratio of intrinsic growth rates of the predator and prey population and $h=\frac{qE}{rm_{2}K}$, respectively.

We have discussed the bionomical equilibrium of the model and explain the optimal harvesting policy to be adopted by a regulatory agency. By constructing an appropriate Hamiltonian function and using Pontryagin’s maximum principle, the optimal harvesting policy has been discussed. We also found an optimal equilibrium solution. The effect of harvesting the prey, predator or both on the stability of the system depends on a pretty fine-tuned balancing of the parameter values and also on which functions/functional responses are chosen to represent the ecological harvesting policy [39]. We have established the stability of a limit cycle using the first Lyapunov number around the equilibrium point. A stable limit cycle seems to be possible only when the per capita consumption of prey by the predator is bounded by some maximum value, as with the nonlinear prey harvesting. Optimal singular equilibrium points have been obtained numerically for various discount rate in various cases. Therefore, at least one optimum singular equilibrium point in each from which $(x_{(1)}^{*}, y_{(1)}^{*}, E_{(1)}^{*})$ is feasible from an ecological point of view.

Appendices

Appendix A1

Proof of Theorem 2

The Jacobian matrix of system (4) evaluated at the point $E_{*}(x_{*}, y_{*} )$ is

$$\begin{aligned} N =&\left ( \begin{matrix} 1-2x-\frac{\frac{\gamma x_{*}}{\beta }-\frac{x_{*}^{2}}{\alpha \beta }}{(\frac{x_{*}^{2}}{\alpha }+x_{*}+\gamma )^{2}- \frac{hc}{(c+x_{*})^{2}}} &-\frac{x}{\frac{x^{2}}{\alpha }+x+\gamma } \\ \frac{\delta \beta y_{*}^{2}}{x_{*}^{2}} &\delta (1-\frac{2\beta y_{*}}{x_{*}}) \end{matrix} \right ) \\ =&\left ( \begin{matrix} 1-2x-\frac{\frac{\gamma x_{*}}{\beta }-\frac{x_{*}^{2}}{\alpha \beta }}{(\frac{x_{*}^{2}}{\alpha }+x_{*}+\gamma )^{2}- \frac{hc}{(c+x_{*})^{2}}} &-\frac{x}{\frac{x^{2}}{\alpha }+x+\gamma } \\ \frac{\delta }{\gamma } &-\delta \end{matrix} \right ) . \end{aligned}$$

Now,

$$\begin{aligned} \det N =&-\delta \biggl[\frac{(1-2x_{*})(x_{*}+c)(\frac{x_{*}^{2}}{ \alpha }+x_{*}+\gamma )-c(1-x_{*})(\frac{x_{*}^{2}}{\alpha }+x_{*}+ \gamma )+\frac{cx_{*}}{\beta }}{(x_{*}+c)(\frac{x_{*}^{2}}{\alpha }+x _{*}+\gamma )} \\ & {} +\frac{\delta }{\beta } \biggl[\frac{x_{*}^{2}+2\gamma x_{*}}{(\frac{x _{*}}{\alpha }+x_{*}+\gamma )^{2}} \biggr] \biggr]>0 \end{aligned}$$

and

$$\begin{aligned} \begin{aligned} \operatorname {tr}N &= \biggl[\frac{(1-2x_{*})(x_{*}+c)(\frac{x_{*}^{2}}{\alpha }+x _{*}+\gamma )-c(1-x_{*})(\frac{x_{*}^{2}}{\alpha }+x_{*}+\gamma )+\frac{cx _{*}}{\beta }}{(x_{*}+c)(\frac{x_{*}^{2}}{\alpha }+x_{*}+\gamma )} \biggr] \\ &\quad {} -\frac{x_{*}(\alpha \gamma -x_{*}^{2})}{\alpha \beta (\frac{x_{*} ^{2}}{\alpha }+x_{*}+\gamma )^{2}}-\delta < 0. \end{aligned} \end{aligned}$$

Hence by the Routh–Hurwitz criterion the equilibrium point $E_{*}(x_{*} ,y_{*} )$ is asymptotically stable.

(b) We know that if $\operatorname {tr}N=0$, then the two eigenvalues will be purely imaginary, provided $\det N>0$. Therefore, by the implicit function theorem, a Hopf bifurcation occurs where a periodic orbit is created as the stability of the equilibrium point $E_{*}(x_{*}, y_{*})$ changes. Note that $\operatorname {tr}N=0$ gives the Hopf bifurcation point $\delta = \tilde{\delta }$. Further, from the given condition (i) $\operatorname {tr}N=0$, (ii) $\det N>0$ at $\delta =\tilde{\delta }$, and (iii) $\frac{d}{d\delta }\operatorname {tr}N=-1<0$ at $\delta =\tilde{\delta }$.

This guarantees the existence of a Hopf bifurcation around the equilibrium point $E_{*}(x_{*}, y_{*} )$. □

Appendix A2

Proof of Theorem 3

Let $\phi =(\phi^{(1)}, \phi^{(2)})^{T}$ where $\phi^{(1)}(x, y)=x(1-x)-\frac{xy}{\frac{x^{2}}{\alpha }+x+\gamma }- \frac{hx}{c+x}$ and $\phi^{(2)}(x, y)=\delta y(1-\beta \frac{y}{x})$. Now, we calculate the Jacobian matrix J at the point $E_{*}(x_{*}, y _{*} )$ to be

$$ J\equiv D\phi (x_{*}, y_{*})=\left ( \begin{matrix} A &B \\ \frac{\delta }{\beta } &-\delta \end{matrix} \right ) , $$

where

$$ A=1-2x_{*}-\frac{\frac{\gamma x_{*}}{\beta }-\frac{x_{*}^{3}}{\alpha \beta }}{(\frac{x_{*}^{2}}{\alpha }+x_{*}+\gamma )^{2}}-\frac{hc}{(c+x _{*})^{2}}, \quad\quad B=- \frac{x_{*}}{\frac{x_{*}^{2}}{\alpha }+x_{*}+\gamma }. $$

Let $h=\tilde{h}$ be such that the matrix J has a simple zero eigenvalue at $h=\tilde{h}$. This requires that $\det J=\frac{x_{*} \delta }{\alpha^{2}\beta (x_{*}+c)(\frac{x_{*}^{2}}{\alpha }+x_{*}+ \gamma )^{2}}[2\beta x_{*}^{5}+\beta (4\alpha +c-1)x_{*}^{4}+\alpha \beta (2c+2\alpha -2+4\gamma )x_{*}^{3} +\alpha (2c\beta \gamma - \beta +c\alpha \beta +\alpha -c+4\alpha \beta \gamma -\beta \gamma )x _{*}^{2} +(c\beta \gamma -\beta +2\beta \gamma^{2}-\beta \gamma +2 \gamma )x_{*}+\alpha^{2}(\beta c\gamma +\beta \gamma +c\gamma )]=0$.

In addition, $\operatorname {tr}J<0$, if $2\beta x_{*}^{6}+\beta (4\alpha +c-1+ \delta )x_{*}^{5}+(2c\alpha \beta +2\alpha^{2}\beta +2\alpha \beta \delta -\alpha +2\alpha \beta \gamma +c\beta \delta +2\alpha \beta \gamma -2\alpha \beta )x_{*}^{4} +\alpha (2c\beta \gamma +c\beta - \beta \gamma -\beta -2c+c\alpha \beta +4\alpha \beta \gamma -\alpha \beta +\alpha \beta \delta -2c\beta \delta )x_{*}^{3} +\alpha (2c \alpha \beta \gamma -2\alpha \beta \gamma -c\alpha +\alpha \beta \gamma^{2}+2\alpha \beta \gamma \delta +\alpha \gamma +2c\beta \gamma \delta +c\alpha \beta \delta )x_{*}^{2} +\alpha^{2}\beta (c\gamma^{2}+ \gamma^{2}\delta +2c\gamma \delta -\gamma^{2})x_{*}+c\alpha^{2}\beta \gamma^{2}\delta >0$.

Therefore, one of the eigenvalues of J is negative at $h=\tilde{h}$.

Let $v=(\beta , 1)^{T}$ and $w= (1, -\frac{x_{*}}{\delta (\frac{x _{*}^{2}}{\alpha }+x_{*}+\gamma )} )$ be the eigenvector of J and $J^{T}$ corresponding to the zero eigenvalue, respectively. Thus, $\Omega_{1}=w^{T}\phi_{h}(E, \tilde{h})= (1, -\frac{x_{*}}{ \delta (\frac{x_{*}^{2}}{\alpha }+x_{*}+\gamma )} ) \bigl( {\scriptsize\begin{matrix}{} -\frac{x_{*}}{x_{*}+c} \cr 0 \end{matrix}} \bigr) <0$ at $h=\tilde{h}$.

Now

$$ \Omega_{2}=w^{T}\bigl[D^{2}\phi (E, \tilde{h}) (v, v)\bigr]= \biggl(1, -\frac{x _{*}}{\delta (\frac{x_{*}^{2}}{\alpha }+x_{*}+\gamma )} \biggr) \left ( \begin{matrix} \beta^{2}\phi_{xx}^{(1)}+2\beta \phi_{xy}^{(1)}+\phi_{yy}^{(1)} \\ \beta^{2}\phi_{xx}^{(2)}+2\beta \phi_{xy}^{(2)}+\phi_{yy}^{(2)} \end{matrix} \right ) , $$

where

$$\begin{aligned}& \phi_{xx}^{(1)} = -2+\frac{2hc}{(c+x_{*})^{3}}+ \frac{\alpha x_{*}(6 \alpha \gamma x_{*}-2x_{*}^{3}+2\alpha^{2}\gamma )}{\beta (x_{*}^{2}+ \alpha x_{*}+\alpha \gamma )^{3}}, \\& \phi_{xy}^{(1)} = \frac{\alpha (\alpha \gamma -x_{*}^{2})}{(x_{*}^{2}+ \alpha x_{*}+\alpha \gamma )^{2}}, \quad\quad \phi_{yy}^{(1)}=0, \\& \phi_{xx}^{(2)} = -\frac{2\delta }{\beta x_{*}},\quad\quad \phi_{xy}^{(2)}=\frac{2 \delta }{x_{*}}, \quad\quad \phi_{yy}^{(2)}=- \frac{2\beta \delta }{x_{*}}. \end{aligned}$$

Therefore,

$$ \Omega_{2}=\frac{2\beta^{2}(2hc-(c+x_{*})^{3})}{(c+x_{*})^{3}}+\frac{ \alpha^{2}(2\beta \gamma (3x_{*}^{2}-\alpha \gamma )+2\beta x_{*}^{2})}{(x _{*}^{2}+\alpha x_{*}+\alpha \gamma )^{3}}>0. $$

Thus, from Sotomayor’s theorem, the system undergoes a saddle-node bifurcation around $E_{*}(x_{*}, y_{*})$ at ${h=\tilde{h}}$. Hence, we conclude that when the bifurcation parameter h passes from one side of $h=\tilde{h}$ to the other side, the number of interior equilibria of system (4) changes. Biologically, for a range of initial data $0< h<\tilde{h}$, i.e. co-existence for model (4) is possible in the form of an equilibrium for a certain choice of initial values, but for ${h>\tilde{h}}$ the system collapses, i.e. the two species do not exist. □