1 Introduction

Since the early days of theoretical ecology after the pioneering works of Lotka [1, 2] and Volterra [3], the models describing the interaction between two or more populations have attracted the attention of many researchers [4,5,6,7,8,9, and references therein]. There are three types of possible interactions between different populations: (i) prey-predator interaction, when one population needs the other to survive; (ii) competition, when populations fight each other for resources available; (iii) cooperation, when populations benefit from each other due to their coexistence, while for reasons of overpopulation, they do not benefit from their growth [5, 10, 11].

In the first case, there is a reduction of preys over the time with respect to the number of predators and a growth with respect to the number of preys themselves, while the predator population tends to grow with the increase of the number of preys, but since an intraspecific competition for survival is established between the predators, they tend to decrease with the increase of the predators themselves. After a certain period of time, the extinction of the prey population occurs and then, as a direct consequence, the predators cannot survive due to lack of resources, so they in turn become extinct. In the second case, since the competition is stronger if there are more individuals, each population gets a disadvantage when the others increase, and instead benefits from its own growth.

Intraguild predation (IGP) is killing and sometimes eating a potential competitor of different species. This interaction represents a combination of predation and competition, because two species (IG predator and IG prey) rely on the same prey resources (basal prey) and also benefit from preying upon one another. IGP is common in nature and can be asymmetrical (one species feeds upon the other) or symmetrical (both species prey upon each other). Since the dominant intraguild predator gains the dual benefits of feeding and eliminating a potential competitor, IGP interactions can have considerable effects on the structure of ecological communities. The early works on IGP are those by Holt and Polis [12, 13] and stated that the coexistence in the IGP models can exist only if the IG prey is a better competitor for the basal prey resource than the IG predator.

In this paper, a Leslie–Gower predator–prey model with intraguild predation describing the evolution between three interacting species—namely prey, mesopredator and top predator—is investigated, generalising the results found in [14], where the authors assumed that the top predator population feeds only on the mesopredator population and influences its consumption rate. Therefore, the novelty of the presented system is the modelling of a complete food web: a Holling type II functional response [15] is considered to model the interaction between prey and top-predator populations and a qualitative analysis is performed. In particular, the stability of the coexistence equilibrium is investigated.

The paper is organized as follows. In Sect. 2 the mathematical model is presented. In Sect. 3 we proved the existence of an absorbing set in the phase space. In Sect. 4 the biologically meaningful equilibria are determined and discussed. In Sects. 5 and 6 the linear instability analysis of the coexistence equilibrium is performed, while in Sect. 7 we found conditions for the nonlinear (local) asymptotic stability of the coexistence equilibrium. In Sect. 8 the numerical results on the stability results we found are presented. The paper ends with a concluding Section that recaps all the obtained results.

2 Mathematical model

Let us derive a predator–prey model describing the evolution of three interacting species, namely prey, mesopredator and top predator. According to [14], we employ a Tanner model (see [16, and references therein]) to describe the interaction dynamics between top predator and mesopredator populations, a Gause model (see [17, and references therein]) to describe the interference of top predator on the interaction dynamics between prey and mesopredator populations, while the interactions between top predator and prey populations are modeled using a Holling-type II functional response, see [15]. Let us suppose that the prey population grows logistically in the absence of the populations of mesopredator and top predator with an intrinsic growth rate \(\tilde{r}\) and environmental carrying capacity K. Moreover, the top predator population grows logistically, with environmental carrying capacity not constant but proportional to y, via the positive constant \(\tilde{m}\). The mathematical model is given by

$$\begin{aligned} {\left\{ \begin{array}{ll} \dfrac{dx}{d\tilde{t}}=x\left[ \tilde{r}\left( 1-\dfrac{x}{K}\right) -\dfrac{\tilde{a}y}{1+\tilde{b}x +\tilde{c}z}-k_1\dfrac{z}{\alpha +x}\right] ,\\ \dfrac{dy}{d\tilde{t}}=y\left[ \dfrac{\tilde{a}\tilde{q}x}{1+\tilde{b}x+\tilde{c}z}-\dfrac{\tilde{e}z}{1+\tilde{n}y}-\tilde{d}\right] ,\\ \dfrac{dz}{d\tilde{t}}=z\left[ \tilde{s}\left( 1-\dfrac{z}{\tilde{m}y}\right) +k_2\dfrac{x}{\alpha +x}\right] , \end{array}\right. } \end{aligned}$$
(1)

where \(x=x(\tilde{t})\), \(y=y(\tilde{t})\), \(z=z(\tilde{t})\) represent the populations (or densities) of prey, mesopredator and top predator at time \(\tilde{t}\), respectively; \(\frac{\tilde{a}x}{1+\tilde{b}x+\tilde{c}z}\) is the functional response of the mesopredator to the resource, where the term \(\tilde{c}z\) models the interference of top predator on mesopredator; \(\frac{\tilde{e}y}{1+\tilde{n}y}\) is the Holling type II functional response describing mesopredator and top predator dynamics, with \(\tilde{e}\) the constant predation rate of top predator on mesopredator; let \(\alpha \) be a half predation constant, so \(k_1\frac{x}{\alpha +x}\) and \(k_2\frac{x}{\alpha +x}\) are the Holling type II functional responses, with \(k_1\) predation rate of top predator on prey and with \(k_2\) interaction rate between prey and top predator. Let us define \(\Lambda =(K,\tilde{a},\tilde{b},\tilde{c},\tilde{d},\tilde{e},\tilde{m},\tilde{n},\tilde{q},\tilde{r},\tilde{s},k_1,k_2,\alpha )\in \mathbb {R}^{14}_+\) the set of dimensional ecological parameters, whose definitions are collected in Table 1.

In order to reduce the number of fundamental parameters of the system, let us introduce the following variables

$$\begin{aligned} \begin{aligned} x&=Ku, \quad y=\frac{\tilde{b}K\tilde{r}}{\tilde{a}}v, \quad z=\frac{\tilde{m}\tilde{b}\tilde{r}K}{\tilde{a}}w, \quad \tilde{t}=\frac{1}{\tilde{r}}t, \quad h_1 \!=\! \frac{k_1\tilde{m}\tilde{b}}{\tilde{a}}, \quad h_2 \!=\! \frac{k_2}{\tilde{r}} \\ b\!&=\! \frac{1}{\tilde{b}K}, \ n\!=\! \frac{\tilde{a}}{\tilde{n}\tilde{b}\tilde{r}K}, \ h \!=\! \frac{\alpha }{K}, \ q \!=\! \frac{\tilde{a}\tilde{q}}{\tilde{r}\tilde{b}}, \ d \!=\! \frac{\tilde{d}\tilde{b}}{\tilde{a}\tilde{q}}, \ s \!=\! \frac{\tilde{s}}{\tilde{r}}, \ c \!=\! \frac{\tilde{c}\tilde{m}\tilde{r}}{\tilde{a}}, \ e \!=\! \frac{\tilde{e}\tilde{m}\tilde{b}}{\tilde{n}\tilde{q}\tilde{a}}. \end{aligned} \end{aligned}$$
(2)

Then, model (1) becomes

$$\begin{aligned} {\left\{ \begin{array}{ll} \dfrac{du}{dt}=u\left[ 1-u-\dfrac{v}{b+u+cw}-\dfrac{h_1w}{h+u}\right] ,\\ \dfrac{dv}{dt}=qv\left[ \dfrac{u}{b+u+cw}-\dfrac{ew}{n+v}-d\right] ,\\ \dfrac{dw}{dt}=w\left[ s\left( 1-\dfrac{w}{v}\right) +\dfrac{h_2u}{h+u}\right] , \end{array}\right. } \end{aligned}$$
(3)

which is defined on \(\Omega =\{(u,v,w)\in \mathbb {R}^3_+:u\ge 0,v>0, w\ge 0\}\), with \(\Lambda '=(b,c,d,e,h,h_1,h_2,n,q,s)\in \mathbb {R}^{10}_+\). To problem (3) we append the following smooth non-negative initial data

$$\begin{aligned} u(t_0)=u_0 \ge 0, \qquad v(t_0)=v_0>0, \qquad w(t_0)=w_0\ge 0. \end{aligned}$$
(4)
Table 1 Definitions of the fundamental dimensional parameters of system (1) with their respective units

Remark 2.1

In view of (3)–(4), one easily obtains that

$$\begin{aligned} \begin{aligned}&u(t)=u_0\exp {\int _0^t\left[ 1-u-\dfrac{v}{b+u+cw}-\dfrac{h_1w}{h+u}\right] d\tau },\\&v(t)=v_0\exp {\int _0^tq\left[ \dfrac{u}{b+u+cw}-\dfrac{ew}{n+v}-d\right] d\tau },\\&w(t)=w_0\exp {\int _0^t\left[ s\left( 1-\dfrac{w}{v}\right) +\dfrac{h_2u}{h+u}\right] d\tau }, \end{aligned} \end{aligned}$$
(5)

hence \(\{u_0>0,v_0>0,w_0>0\} \ \Rightarrow \ \{u(t)>0,v(t)>0,w(t)>0\}\), \(\forall t>t_0,\) i.e. the positive octant is invariant.

3 Absorbing sets

In this section, we will investigate the asymptotic behaviour of solutions to (3)–(4).

Theorem 3.1

All the solutions of system (3)–(4) in \(\Omega \) are bounded.

Proof

The proof can be obtained following, step by step, the procedure used to prove Theorem 3.1 in [14]. For the sake of completeness, we provide a sketch of the proof.

From (3)\(_1\) one has

$$\begin{aligned} \dfrac{du}{dt} \le u(1-u) \end{aligned}$$
(6)

and hence

$$\begin{aligned} {\dot{g}}+g>1, \qquad g(t)=\displaystyle \frac{1}{u(t)}. \end{aligned}$$
(7)

Integrating (7) with respect to time t, one obtains \(g(t)>1+e^{t_0-t}(g(t_0)-1)\), i.e. \( u(t)<\dfrac{u(t_0)}{u(t_0)+e^{t_0-t}(1-u(t_0))}\). Therefore, u(t) is bounded in its maximal domain and it follows that

$$\begin{aligned} \displaystyle \limsup _{t\rightarrow \infty }u(t)=1. \end{aligned}$$
(8)

Introducing the function \(S(t)=qu(t)+v(t)\) and choosing \(0<\varepsilon <d q\), one has

$$\begin{aligned} \displaystyle \limsup _{t\rightarrow \infty }S(t)=q\dfrac{(\varepsilon +1)^2}{4\varepsilon } \ \Longrightarrow \ \limsup _{t\rightarrow \infty }v(t)=q\dfrac{(\varepsilon -1)^2}{4\varepsilon }. \end{aligned}$$
(9)

Let us set \(M=q\dfrac{(\varepsilon -1)^2}{4\varepsilon }\). From (3)\(_3\) and accounting for the boundedness of functions u(t) and v(t), it follows that

$$\begin{aligned} \dfrac{dw}{dt}\le w\left[ s+\dfrac{h_2}{h}-\dfrac{s}{M}w\right] . \end{aligned}$$
(10)

Let us consider the change of variable \(p(t)=1/w(t)\) and set \(\gamma =s+h_2/h\). Therefore, (10) becomes \({\dot{p}}+\gamma p\ge s/M,\) which, by integration with respect to time t, leads to \(p(t)\ge e^{\gamma (t_0-t)}(p(t_0)-\delta )+\delta ,\) where we set \(\delta =s/M\gamma \). Then it follows

$$\begin{aligned} w(t) \le \dfrac{w(t_0)}{e^{\gamma (t_0-t)}(1-\delta w(t_0))+\delta w(t_0)}. \end{aligned}$$
(11)

Therefore, w(t) is bounded in its maximal domain and, from (11) it follows that

$$\begin{aligned} \displaystyle \limsup _{t\rightarrow \infty }w(t)=q\dfrac{(\varepsilon -1)^2}{4\varepsilon }\left( 1+\dfrac{h_2}{sh}\right) . \end{aligned}$$
(12)

\(\square \)

Theorem 3.2

\(\forall \bar{\varepsilon }\) the set

$$\begin{aligned} S_{\bar{\varepsilon }}=\left\{ (u,v,w)\in \mathbb {R}^3_+: \frac{1}{2} \left( u^2+v^2+w^2 \right) \le \frac{a_2}{a_1}+\bar{\varepsilon }\right\} \end{aligned}$$
(13)

with

$$\begin{aligned} a_1=2dq, \quad a_2=1+dq+q\left( q\frac{(\varepsilon -1)^2}{4\varepsilon }\right) ^2+(s+dq+h_2)\left( q\frac{(\varepsilon -1)^2}{4\varepsilon }\left( 1+\frac{h_2}{sh}\right) \right) ^2\nonumber \\ \end{aligned}$$
(14)

is an absorbing set of the phase space.

Proof

Multiplying (3)\(_1\) by u, (3)\(_2\) by v and (3)\(_3\) by w and adding the obtained results, one obtains

$$\begin{aligned} \dfrac{d{\mathcal {E}}}{dt}\le -2dq{\mathcal {E}}+(1+dq)u^2+qv^2+(s+dq+h_2)w^2 \end{aligned}$$
(15)

with \({\mathcal {E}}(t)=\frac{1}{2}(u^2+v^2+w^2)\). By virtue of (8), (9) and (12), from (15) it follows that \(\dfrac{d{\mathcal {E}}}{dt}\le -a_1 {\mathcal {E}}+a_2,\) where the positive constants \(a_1,a_2\) are defined by (14). Following the procedure in [18, 19] one can prove that \(S_{\bar{\varepsilon }}\) is an absorbing set. \(\square \)

4 Biologically meaningful equilibria

In this Section, we will determine the steady states of (3)–(4), i.e. the non-negative solutions of the system

$$\begin{aligned} {\left\{ \begin{array}{ll} u\left[ 1-u-\dfrac{v}{b+u+cw}-\dfrac{h_1w}{h+u}\right] =0,\\ qv\left[ \dfrac{u}{b+u+cw}-\dfrac{ew}{n+v}-d\right] =0,\\ w\left[ s\left( 1-\dfrac{w}{v}\right) +\dfrac{h_2u}{h+u}\right] =0. \end{array}\right. } \end{aligned}$$
(16)

System (16) admits as non-null solution having at least one null component (boundary equilibria) \(E_1=\left( \dfrac{bd}{1-d},\dfrac{(1-d-bd)b}{(1-d)^2},0\right) \), that is the no-top predators solution, existing only if \(d<\displaystyle \frac{1}{1+b}\). From a biological point of view, boundary equilibria represent the situation in which at least one species becomes extinct. Avoiding this situation, we are interested in finding conditions guaranteeing the existence of coexistence equilibria, i.e. the solutions of (16) with all positive component. Let us denote \(E_c=(u_c,v_c,w_c)\) the coexistence equilibrium. Let us remark that, since \(\{u_c>0, v_c>0, w_c>0\}\), (16)\(_1\) admits a positive root only if \(u_c<1\). Analogously, (16)\(_3\) admits a positive solution only if \(w_c>v_c\). From (16)\(_2\), one has that:

  • since \(\displaystyle \frac{u_c}{b+u_c+cw_c}<\dfrac{1}{1+b}\), there exists a positive solution only if \(d<\dfrac{1}{1+b}\);

  • since \(\dfrac{u_c}{b+u_c+cw_c}<\dfrac{u_c}{b+u_c}\), there exists a positive solution only if \(u_c>\dfrac{bd}{1-d}\).

Then, the condition

$$\begin{aligned} d<\dfrac{1}{1+b}, \end{aligned}$$
(17)

is necessary for the existence of the coexistence equilibria. Furthermore, if \(E_c\) exists, then, necessarily

$$\begin{aligned} \dfrac{bd}{1-d}<u_c<1, \quad v_c<w_c. \end{aligned}$$
(18)

Theorem 4.1

The coexistence equilibrium \(E_c=(u_c,v_c,w_c)\) exists if (17)–(18) hold together with

$$\begin{aligned} v_c<\tilde{v}_M:=\dfrac{ch-h-bh_1+\sqrt{(h-ch+bh_1)^2+4bchh_1}}{2ch_1}. \end{aligned}$$
(19)

Proof

From (16)\(_3\), it immediately follows

$$\begin{aligned} w=v+\dfrac{h_2uv}{s(h+u)}. \end{aligned}$$
(20)

Substituting (20) in (16)\(_2\), one gets

$$\begin{aligned} A_1(u)v^2+A_2(u)v+A_3(u)=0 \end{aligned}$$
(21)

where

$$\begin{aligned} \begin{aligned} A_1(u)&=-cd-ce-\dfrac{ceh_2^2u^2}{s^2(h+u)^2}-\dfrac{cdh_2u}{s(h+u)}-\dfrac{2ceh_2u}{s(h+u)}, \\ A_2(u)&=-bd-be-cdn+u-du-eu-\dfrac{beh_2u}{s(h+u)}-\dfrac{cdnh_2u}{s(h+u)}-\dfrac{eh_2u^2}{s(h+u)},\\ A_3(u)&=n\left[ -bd+u(1-d)\right] . \end{aligned} \end{aligned}$$
(22)

Since \(A_1(u)<0\) and, in view of (18)\(_1\), \(A_3(u)>0\), by Descartes’ rule there exists a unique positive root of (21). Besides, substituting (20) in (16)\(_1\), one obtains

$$\begin{aligned} su^5+B_0(v)u^4+B_1(v)u^3+B_2(v)u^2+B_3(v)u+B_4(v)=0 \end{aligned}$$
(23)

where \( B_4(v)=-bh^3s+h^3sv-ch^3sv+bh^2h_1sv+ch^2h_1sv^2\). Since \(s>0\), if \(B_4(v)<0\) then exists at least one positive root \(\bar{u}=f_2(v)\) of (23). Simple calculations show that \(B_4(v)<0\Leftrightarrow v<{\tilde{v}}_M.\) \(\square \)

5 Preliminaries to stability

Let \(E_*=(u^*,v^*,w^*)\) be a generic equilibrium and let us introduce the perturbation fields \( \{U=u-u^*, \, V=v-v^*, \, W=w-w^*\} \). Hence, the perturbation equations associated to system (3) are

$$\begin{aligned} {\left\{ \begin{array}{ll} {\dot{U}}=a_{11}U+a_{12}V+a_{13}W+F_1, \\ {\dot{V}}=a_{21}U+a_{22}V+a_{23}W+F_2, \\ {\dot{W}}=a_{31}U+a_{32}V+a_{33}W+F_3, \end{array}\right. } \end{aligned}$$
(24)

with

$$\begin{aligned} \hspace{-11.111pt}\begin{aligned}&a_{11} \!=\! 1-2u^*-\dfrac{v^*(b \!+\! cw^*)}{(b\!+\!u^*\!+\!cw^*)^2}-\dfrac{hh_1w^*}{(h\!+\!u^*)^2}, \quad a_{12} \!=\! -\dfrac{u^*}{b \!+\! u^* \!+\! cw^*}(<0), \\&a_{13} \!=\! \dfrac{cu^*v^*}{(b \!+\! u^* \!+\! cw^*)^2} \!-\! \dfrac{h_1u^*}{h+u^*}, \quad a_{21} \!=\! \dfrac{qv^*(b\!+\!cw^*)}{(b\!+\!u^*\!+\!cw^*)^2}(>0),\\&a_{22} \!=\! \dfrac{qu^*}{b\!+\!u^*\!+\!cw^*} \!-\! \dfrac{enqw^*}{(n\!+\!v^*)^2} \!-\! dq, \quad a_{23} \!= \!- \dfrac{cqu^*v^*}{(b\!+\!u^*\!+\!cw^*)^2} \!-\! \dfrac{eqv^*}{n\!+\!v^*}(<0),\\&a_{31} \!=\! \dfrac{hh_2w^*}{(h+u^*)^2}(\ge 0), \quad a_{32} \!=\! \dfrac{sw^{*2}}{v^{*2}}(\ge 0), \quad a_{33} \!=\! s-\dfrac{2sw^*}{v^*}+\dfrac{h_2u^*}{h+u^*}, \end{aligned} \end{aligned}$$
(25)

while the nonlinear terms \(F_i(U,V,W)\), \(i=1,2,3\), are

$$\begin{aligned} \hspace{-5.0pt}{} & {} \begin{aligned} F_1&\!=\! -2\left[ 1\!-\!\dfrac{(v^* \!+\! \theta _2 V)(b \!+\! c(w^* \!+\! \theta _3 W))}{(b \!+\! u^* \!+\! \theta _1 U \!+\! c(w^* \!+\! \theta _3 W))^3}\!-\!\dfrac{hh_1(w^* \!+\! \theta _3 W)}{(h \!+\! u^* \!+\! \theta _1 U)^3}\right] U^2\\&\quad -\!\dfrac{2c^2(u^* \!+\! \theta _1 U)(v^* \!+\! \theta _2 V)}{(b \!+\! u^* \!+\! \theta _1 U \!+\! c(w^* \!+\! \theta _3 W))^3}W^2\!-\!\dfrac{b \!+\! c(w^* \!+\! \theta _3 W)}{(b \!+\! u^* \!+\! \theta _1 U \!+\! c(w^* \!+\! \theta _3 W))^2}UV\\&\quad + \dfrac{c(u^* \!+\! \theta _1 U)}{(b+u^* \!+\! \theta _1 U \!+\! c(w^* \!+\! \theta _3 W))^2}VW\!-\!\bigg [\dfrac{c(v^* \!+\! \theta _2 V)}{(b \!+\! u^* \!+\! \theta _1 U \!+\! c(w^* \!+\! \theta _3 W))^2}\\&\quad -\dfrac{2c(v^* \!+\! \theta _2 V)(b \!+\! c(w^* \!+\! \theta _3 W))}{(b \!+\! u^* \!+\! \theta _1 U \!+\! c(w^* \!+\! \theta _3 W))^3} \!+\! \dfrac{hh_1}{(h \!+\! u^* \!+\! \theta _1 U)^2}\bigg ]UW \end{aligned}\nonumber \\ \end{aligned}$$
(26)
$$\begin{aligned} \hspace{-4.44443pt}{} & {} \begin{aligned} F_2&\!=\! -\dfrac{2q(v^* \!+\! \theta _2 V)(b \!+\! c(w^* \!+\! \theta _3 W))}{(b \!+\! u^* \!+\! \theta _1 U \!+\! c(w^* \!+\! \theta _3 W))^3}U^2 \!+\! \dfrac{2enq(w^* \!+\! \theta _3 W)}{(n \!+\! v^* \!+\! \theta _2 V))^3}V^2\\&\qquad + \dfrac{2c^2q(u^* \!+\! \theta _1 U)(v^* \!+\! \theta _2 V)}{(b \!+\! u^* \!+\! \theta _1 U \!+\! c(w^* \!+\! \theta _3 W))^3}W^2\!+\! \dfrac{q(b \!+\! c(w^* \!+\! \theta _3 W))}{(b \!+\! u^* \!+\! \theta _1 U \!+\! c(w^* \!+\! \theta _3 W))^2}UV\\&\quad -\dfrac{cq(u^* \!+\! \theta _1 U)}{(b+u^* \!+\! \theta _1 U \!+\! c(w^* \!+\! \theta _3 W))^2}VW\!+\!\bigg [\dfrac{cq(v^* \!+\! \theta _2 V)}{(b \!+\! u^* \!+\! \theta _1 U \!+\! c(w^* \!+\! \theta _3 W))^2}\\&\quad -\dfrac{2cq(v^* \!+\! \theta _2 V)(b \!+\! c(w^* \!+\! \theta _3 W))}{(b \!+\! u^* \!+\! \theta _1 U \!+\! c(w^* \!+\! \theta _3 W))^3}\bigg ]UW \end{aligned}\nonumber \\ \end{aligned}$$
(27)
$$\begin{aligned} \hspace{-63.88885pt}{} & {} \begin{aligned} F_3&\!=\! -\dfrac{2hh_2(w^* \!+\! \theta _3 W)}{(h \!+\! u^* \!+\! \theta _1 U)^3}U^2-\dfrac{2s(w^* \!+\! \theta _3 W)^2}{(v^* \!+\! \theta _2 V)^3}V^2-\dfrac{2s}{v^* \!+\! \theta _2 V}W^2\\&\!+\! \dfrac{2s(w^* \!+\! \theta _3 W)}{(v^* \!+\! \theta _2 V)^2}VW \!+\! \dfrac{hh_2}{(h \!+\! u^* \!+\! \theta _1 U)^2}UW \end{aligned} \end{aligned}$$
(28)

with \(\theta _i\in (0,1)\), \((i=1,2,3)\). The Jacobian matrix linked to system (24) is

$$\begin{aligned} J = \begin{pmatrix} a_{11} &{} a_{12} &{} a_{13} \\ a_{21} &{} a_{22} &{} a_{23} \\ a_{31} &{} a_{32} &{} a_{33} \end{pmatrix}. \end{aligned}$$
(29)

6 Linear stability

In order to analyse the linear stability of the meaningful equilibria \(E_1\) and \(E_c\), let us evaluate the roots of the characteristic equation associated to J, namely:

$$\begin{aligned} \lambda ^3-I_1 \lambda ^2 + I_2 \lambda - I_3=0, \end{aligned}$$
(30)

where \(I_i \ (i = 1, 2, 3)\) are the principal invariants of J. If all roots of (30) have negative real part, then the corresponding equilibria are linearly stable. The necessary and sufficient conditions guaranteeing all roots of (30) to have negative real part are the Routh–Hurwitz conditions (see [4, 5, 20]):

$$\begin{aligned} I_1<0, \quad I_1 I_2 - I_3<0, \quad I_3<0. \end{aligned}$$
(31)

Let us remark that if at least one of (31) is reversed, the instability of the meaningful equilibria is guaranteed. Let us first investigate the linear stability of \(E_1\). The Jacobian matrix (29) evaluated in \(E_1\) is given by

$$\begin{aligned} J(E_1)\!=\!\begin{pmatrix} \dfrac{d(1\!-\!b\!-\!d\!-\!bd)}{1\!-\!d} &{} \!-\!d &{} \dfrac{cd(1\!-\!d\!-\!bd)}{1\!-\!d}\!-\!\dfrac{h_1bd}{h(1\!-\!d)\!+\!bd} \\ q(1\!-\!d\!-\!bd) &{} 0 &{} \dfrac{(1\!-\!d\!-\!bd)[b^2cd^2\!-\!b(1\!-\!d)(cd\!+\!e) \!-\!cd(1\!-\!d)^2n]q}{(1\!-\!d)[b\!-\!bd\!-\!b^2d\!+\!(1\!-\!d)^2n]} \\ 0 &{} 0 &{} s\!+\!\dfrac{h_2bd}{h(1\!-\!d)\!+\!bd} \end{pmatrix} \end{aligned}$$
(32)

while the associated eigenvalues are

$$\begin{aligned} \begin{aligned}&\lambda _1 = s+\dfrac{bdh_2}{bd+h(1-d)}, \\&\lambda _{2,3} = \dfrac{1}{2(1\!-\!d)} \left[ d(1\!-\!b\!-\!d\!-\!bd) \pm \sqrt{d^2(1\!-\!b\!-\!d\!-\!bd)^2\!-\!4dq(1\!-\!d)^2(1\!-\!d\!-\!bd)} \right] \end{aligned} \end{aligned}$$
(33)

Since \(d<1\) is a necessary condition for the existence of \(E_1\), it follows that \(\lambda _1>0\), hence we recover the stability results found in [14], i.e. the following theorem holds.

Theorem 6.1

The boundary equilibrium \(E_1\) is linearly unstable and satisfies the following statements:

  1. 1.

    If \(0<q \le \dfrac{d(1-b-d-bd)^2}{4(1-d)^2(1-d-bd)}\), then \(\lambda _{2,3}\) are real and \(E_1\) is a repeller point if \(0<b<\dfrac{1-d}{1+d}\) and a saddle point if \(\dfrac{1-d}{1+d}<b<\dfrac{1-d}{d}\).

  2. 2.

    If \(q>\dfrac{d(1-b-d-bd)^2}{4(1-d)^2(1-d-bd)}\), then \(\lambda _{2,3}\) are complex and \(E_1\) is a saddle point.

In the sequel, we analyse the stability of the coexistence equilibria. Conditions (31) are necessary and sufficient to guarantee the linear stability of \(E_c\). Since they are not easy to handle and difficult to be interpreted from a biological point of view, we look for simpler sufficient conditions guaranteeing their validity. Let us firstly remark that, if

$$\begin{aligned} {\left\{ \begin{array}{ll} a_{11}<0, \ a_{22}<0, \ a_{33}<0, \\[2mm] a_{13}+\dfrac{a_{12}a_{23}a_{31}}{a_{21}a_{32}}<0,\,\, -a_{31}(a_{11}+a_{33})-a_{21}a_{32}>0. \end{array}\right. } \end{aligned}$$
(34)

then (31) are verified and the coexistence equilibrium is, therefore, linearly stable.

Setting

$$\begin{aligned} \hspace{-10.0pt}v_c^{(1)} \!=\! \dfrac{n\left[ e\!-\!2(1\!-\!d)\right] \!-\! n\sqrt{e^2 \!-\! 4e(1\!-\!d)}}{2(1-d)},\ v_c^{(2)} \!=\! \dfrac{n\left[ e\!-\! 2(1\!-\!d)\right] \!+\! n\sqrt{e^2\!-\!4e(1\!-\!d)}}{2(1-d)}, \end{aligned}$$
(35)

the following theorem holds.

Theorem 6.2

If (18) holds together with

$$\begin{aligned} \begin{array}{l} \dfrac{1}{2b+1}<d<\dfrac{1}{b+1},\, \,h_2<\min \left\{ s,\dfrac{d-1+2bd}{1-d}\right\} ,\,\, e>4(1-d),\\[3mm] q<\dfrac{h h_2 v_c^{(1)}}{(h+1)^2(s+h_2)},\,\, n<\dfrac{2(1-d){\tilde{v}}_M}{e-2(1-d)+\sqrt{e^2-4e(1-d)}},\\[4mm] v_c^{(1)}<v_c<v_c^{(2)}, \,\, h_1>(1+h)\left\{ \dfrac{c v_c^{(2)}}{b^2}+\left( \dfrac{c}{b^2}+\dfrac{e}{n}\right) \dfrac{(b+1+2c v_c^{(2)})v_c^{(2)}}{b h}\right\} , \end{array}\end{aligned}$$
(36)

then the coexistence equilibrium \(E_c\) is linearly stable.

Proof

The proof follows by observing that (18) and (36) guarantee the validity of (34), which implies the linear stability of \(E_c\). In fact, in view of (25), since \(v_c>0\) and \(w_c>0\), then \(a_{11}<1-2u_c\). From (18)\(_1\), if \(\dfrac{bd}{1-d}>\dfrac{1}{2}\) (i.e. (36)\(_1\) holds), then \(a_{11}<0\). Analogously, in view of (18), one has that \( a_{33}=s-2\,s\dfrac{w_c}{v_c}+\dfrac{h_2u_c}{h+u_c}<-s+h_2, \) that is negative if (36)\(_2\) holds. Concerning \(a_{22}\), from (18) one has

$$\begin{aligned} a_{22}=q\left\{ \dfrac{u_c}{b+u_c+cw_c}-\dfrac{en w_c}{(n+v_c)^2}-d\right\} <q\left\{ (1-d)-\dfrac{e n v_c}{(n+v_c)^2}\right\} . \end{aligned}$$
(37)

Simple calculation show that \(a_{22}<0\) if \((1-d) v_c^2+n\left[ 2(1-d)-e\right] v_c+(1-d)n^2<0,\) that is verified if (36)\(_3\) and (36)\(_6\) hold. We observe that (36)\(_5\) guarantees that \(v_c^{(2)}\!<\!\tilde{v}_M\).

It remains to verify (34)\(_4\) and (34)\(_5\). From (25), (34)\(_4\) is verified if

$$\begin{aligned} \dfrac{cv_c}{(b\!+\!u_c\!+\!cw_c)^2}\!-\!\dfrac{h_1}{h\!+\!u_c}\!+\!\dfrac{hh_2}{(h\!+\!u_c)^2}\dfrac{v_c^2(b\!+\!u_c\!+\!cw_c)}{sw_c(b\!+\!cw_c)}\left[ \dfrac{cu_c}{(b\!+\!u_c\!+\!cw_c)^2}\!+\!\dfrac{e}{n\!+\!v_c}\right] \!<\!0. \end{aligned}$$
(38)

In view of (18) and (36)\(_6\), a sufficient condition guaranteeing (38), is given by (36)\(_7\) Finally, since

$$\begin{aligned} \begin{array}{l} -a_{31}(a_{11}+a_{33})-a_{21}a_{32}= w_c\left\{ \dfrac{hh_2(-1+2u_c)}{(h+u_c)^2}+ \dfrac{h h_2 v_c(b+cw_c)}{(h+u_c)^2(b+u_c+cw_c)^2}\right. \\ +\dfrac{h^2 h_1h_2 w_c}{(h+u_c)^4}-\dfrac{sh h_2}{(h+u_c)^2} +\left. \dfrac{2s w_c h h_2}{v_c(h+u_c)^2}-\dfrac{h_2^2 h u_c}{(h+u_c)^3}-\dfrac{qsw_c(b+cw_c)}{v_c(b+u_c+cw_c)^2}\right\} , \end{array} \end{aligned}$$
(39)

(36)\(_2\) and (36)\(_4\) imply (34)\(_5\). \(\square \)

7 Nonlinear stability

In this Section, we perform the nonlinear stability analysis of \(E_c\). Setting \(\{ U=\mu _1Y_1, \, V=\mu _2Y_2, \, W=\mu _3Y_3\}, \) being \(\mu _1,\mu _2,\mu _3\) positive constants to be suitably chosen later, system (24) becomes

$$\begin{aligned} {\left\{ \begin{array}{ll} {\dot{Y}}_1=b_{11}Y_1+b_{12}Y_2+b_{13}Y_3+\dfrac{1}{\mu _1}\bar{F}_1, \\ {\dot{Y}}_2=b_{21}Y_1+b_{22}Y_2+b_{23}Y_3+\dfrac{1}{\mu _2}\bar{F}_2, \\ {\dot{Y}}_3=b_{31}Y_1+b_{32}Y_2+b_{33}Y_3+\dfrac{1}{\mu _3}\bar{F}_3, \end{array}\right. } \end{aligned}$$
(40)

where \(b_{ij}=a_{ij}\dfrac{\mu _j}{\mu _i}\) and \(\bar{F}_i=F_i(\mu _1Y_1,\mu _2Y_2,\mu _3Y_3)\) for \(i=1,2,3\). Let us introduce the functional [21, 22]:

$$\begin{aligned} E=\frac{1}{2}Y_1^2+\frac{1}{2}\left[ \hat{A}\left( Y_2^2+Y_3^2\right) +\left( b_{22}Y_3-b_{32}Y_2\right) ^2+\left( b_{23}Y_3-b_{33}Y_2\right) ^2\right] \end{aligned}$$
(41)

being \(\hat{A}=b_{22}b_{33}-b_{23}b_{32}=a_{22}a_{33}-a_{23}a_{32}.\) Choosing \(\displaystyle \left( \frac{\mu _2}{\mu _3}\right) ^2=-\frac{a_{23}a_{33}}{a_{22}a_{32}} (> 0)\), the time derivative of E along the solutions of (40) is

$$\begin{aligned} {\dot{E}}=b_{11}Y_1^2+\hat{A}\hat{I}\left( Y_2^2+Y_3^2\right) +\Phi +\Phi ^* \end{aligned}$$
(42)

where

$$\begin{aligned} {\left\{ \begin{array}{ll} \hat{I}=b_{22}+b_{33}, \quad A_1=\hat{A}+b_{32}^2+b_{33}^2, \quad A_2=\hat{A}+b_{22}^2+b_{23}^2, \\[2mm] \Phi =\dfrac{1}{\mu _1}Y_1\bar{F}_1+\dfrac{A_1}{\mu _2}Y_2\bar{F}_2+\dfrac{A_2}{\mu _3}Y_3\bar{F}_3, \\[3mm] \Phi ^*=(A_1b_{21}+b_{12})Y_1Y_2+(A_2b_{31}+b_{13})Y_1Y_3. \end{array}\right. } \end{aligned}$$
(43)

Theorem 7.1

There exists a positive constant \(\delta \) such that

$$\begin{aligned} \Phi \le \delta \left( Y_1^2+Y_2^2+Y_3^2\right) ^\frac{3}{2}. \end{aligned}$$
(44)

Proof

Setting

$$\begin{aligned} \Phi _1=\alpha _1UF_1+\alpha _2VF_2+\alpha _3WF_3, \end{aligned}$$
(45)

with \(F_i\) given by (26)–(28) and \(\alpha _i\) positive constants \((i=1,2,3)\), from (8), (9), (12), it turns out that

$$\begin{aligned}&v^*+\theta _2V=v^*(1-\theta _2)+\theta _2v\le v^*(1-\theta _2)+\theta _2\frac{q(\varepsilon -1)^2}{4\varepsilon }:=H_1, \end{aligned}$$
(46)
$$\begin{aligned}&w^*+\theta _3W=w^*(1-\theta _3)+\theta _3w\le w^*(1-\theta _3)+\theta _3\frac{q(\varepsilon -1)^2}{4\varepsilon }\left( 1+\frac{h_2}{sh}\right) :=H_2, \end{aligned}$$
(47)
$$\begin{aligned}&b+c(w^*+\theta _3W)=b+c[w^*(1-\theta _3)+\theta _3w]\le b+cH_2, \end{aligned}$$
(48)
$$\begin{aligned}&u^*+\theta _1U=u^*(1-\theta _1)+\theta _1u\le 1. \end{aligned}$$
(49)

Furthermore,

$$\begin{aligned} h+u^*+\theta _1U&=h+u^*(1-\theta _1)+\theta _1u>h, \end{aligned}$$
(50)
$$\begin{aligned} b+u^*+\theta _1U+c(w^*+\theta _3W)&=b+u^*(1-\theta _1)+\theta _1u+c[w^*(1-\theta _3)+\theta _3w]>b, \end{aligned}$$
(51)
$$\begin{aligned} n+v^*+\theta _2V&=n+v^*(1-\theta _2)+\theta _2v>n. \end{aligned}$$
(52)

Hence, according to (46)–(52), from (45) it follows

$$\begin{aligned} \begin{aligned} \Phi _1&\le c_1|U|^3+c_2|V|^3+c_3|W|^3+c_4U^2|V|+c_5U^2|W|\\&\quad +c_6|U|V^2+c_7V^2|W|+c_8|U|W^2+c_9|V|W^2, \end{aligned}\nonumber \\ \end{aligned}$$
(53)

where

$$\begin{aligned} \begin{aligned} c_1&=2\alpha _1\left[ 1+\frac{H_1(b+cH_2)}{b^3}+\frac{h_1H_2}{h^2}\right] , \ c_2=\alpha _2\frac{2eqH_2}{n^2}, \ c_3=\alpha _3\frac{2s}{H_1}, \\ c_4&=\frac{1}{2}\left[ \alpha _1\frac{c}{b^2}\!+\!\alpha _2\frac{cqH_1}{b^2} \left( \frac{2(b+cH_2)}{b}\!+\!1\right) \right] \!+\!\alpha _1\frac{b+cH_2}{b^2}\! +\!\alpha _2\frac{2qH_1(b+cH_2)}{b^3}, \\ c_5&=\alpha _1\left[ \frac{cH_1}{b^2}\left( 1+\frac{2(b+cH_2)}{b}\right) +\frac{h_1}{h}\right] +\alpha _3\frac{2h_2H_2}{h^2} \\ c_6&=\alpha _2\frac{q(b+cH_2)}{b^2}, \ c_7=\alpha _2\frac{cq}{b^2}+\alpha _3\frac{2sH_2^2}{H_1^3}, \ c_8=\alpha _1\frac{2c^2H_1}{b^3}+\alpha _3\frac{h_2}{h}, \\ c_9&=\frac{1}{2}\left[ \alpha _1\frac{c}{b^2}+\alpha _2\frac{cqH_1}{b^2} \left( \frac{2(b+cH_2)}{b}+1\right) \right] +\alpha _2\frac{2c^2qH_1}{b^3}+\alpha _3\frac{2sH_2}{H_1^2}. \end{aligned} \end{aligned}$$
(54)

Setting \(d_1=\max \{c_1,c_4,c_5\}, \ d_2=\max \{c_2,c_6,c_7\}, \ d_3=\max \{c_3,c_8,c_9\},\) from (53) one obtains \( \Phi _1\le (|U|+|V|+|W|)\left( d_1U^2+d_2V^2+d_3W^2\right) \). Since \(|U|+|V|+|W|\le 2\sqrt{2}\left( U^2+V^2+W^2\right) ^\frac{1}{2},\) setting \(\delta =2\sqrt{2}\max \{d_1,d_2,d_3\}\), it follows that \( \Phi _1\le \delta \left( U^2+V^2+W^2\right) ^\frac{3}{2},\) from which one immediately recovers (44). \(\square \)

Fig. 1
figure 1

a: Dynamic of the solutions of system (3) plotted with respect to time t choosing as initial conditions a random perturbation of \(E^*_1\) and for parameters \(S \cup \{h_1=h_2=0.6\} \). b: Dynamic of the solutions of system (3) in the phase space uvw

Fig. 2
figure 2

a: Dynamic of the solutions of system (3) plotted with respect to time t choosing as initial conditions \(u(0)=0.2,v(0)=w(0)=0.229\) and for parameters \(S \cup \{h_1=0.6,h_2=0.9\} \). b: Dynamic of the solutions of system (3) in the phase space uvw

Fig. 3
figure 3

a: Dynamic of the solutions of system (3) plotted with respect to time t choosing as initial conditions \(u(0)=0.2,v(0)=w(0)=0.229\) and for parameters \(S \cup \{h_1=1,h_2=0.5\} \). b: Dynamic of the solutions of system (3) in the phase space uvw

Fig. 4
figure 4

a: Dynamic of the solutions of system (3) plotted with respect to time t choosing as initial conditions a random perturbation of \(E_{s3}\) and for parameters \(S \cup \{h_1=1,h_2=0.5\} \). b: Dynamic of the solutions of system (3) in the phase space uvw

Regarding the nonlinear term \(\Phi ^*\), one has that

$$\begin{aligned} \Phi ^*\le m|Y_1|(|Y_2|+|Y_3|) \end{aligned}$$
(55)

with \(m=\sup \{|A_1b_{21}\!+\!b_{12}|,|A_2b_{31}\!+\!b_{13}|\}:=\mu _1m_1\) and \(m_1=\sup \left\{ \left| A_1a_{21}\frac{1}{\mu _2}\!+\!a_{12}\frac{\mu _2}{\mu _1^2}\right| ,\left| A_2a_{31}\frac{1}{\mu _3}\!+\!a_{13}\frac{\mu _3}{\mu _1^2}\right| \right\} .\) By virtue of the generalized Cauchy inequality, from (55) one gets

$$\begin{aligned} \Phi ^*\le \frac{\mu _1^2m_1^2}{2\hat{A}\hat{I}}Y_1^2+\frac{\hat{A}\hat{I}}{2}\left( Y_2^2+Y_3^2\right) . \end{aligned}$$
(56)

Theorem 7.2

If the conditions (17)–(18) and (34) hold together with \(E(0)^{1/2}<k(\Lambda ')\), k being a suitable positive constant depending on the model parameters, then the coexistence equilibrium \(E_c\) is nonlinearly locally asymptotically stable.

Proof

Let us choose \(\displaystyle \mu _1^2=\frac{|a_{11}||\hat{A}\hat{I}|}{m_1^2}\). Therefore, by virtue of (44) and (56), from (42) one gets

$$\begin{aligned} {\dot{E}}\le -\frac{1}{2}|a_{11}|Y_1^2-\frac{|\hat{A}\hat{I}|}{2} \left( Y_2^2+Y_3^2\right) +\delta \left( Y_1^2+Y_2^2+Y_3^2\right) ^\frac{3}{2}. \end{aligned}$$
(57)

Since \( q_1\left( Y_1^2+Y_2^2+Y_3^2\right) \le E\le p_1\left( Y_1^2+Y_2^2+Y_3^2\right) \), where \(q_1=\min \left\{ \dfrac{1}{2},\dfrac{\hat{A}}{2}\right\} \) and \(p_1=\max \left\{ \dfrac{1}{2},\dfrac{A_1}{2},\dfrac{A_2}{2}\right\} ,\) from (57) it follows \( {\dot{E}}\le -\left( \dfrac{\sigma }{q_1}-\dfrac{\delta }{p_1^\frac{3}{2}}E^\frac{1}{2}\right) E\), with \(\sigma =\min \left\{ \dfrac{|a_{11}|}{2},\dfrac{|\hat{A}\hat{I}|}{2}\right\} .\) Then, by recursive arguments, the condition \( E(0)^\frac{1}{2}<\dfrac{\sigma p_1^\frac{3}{2}}{q_1\delta }\), implies that \({\dot{E}}<0\) and hence the thesis follows. \(\square \)

Fig. 5
figure 5

a: Dynamic of the solutions of system (3) in the phase space uvw, choosing as initial conditions \(u(0)=0.2,v(0)=w(0)=0.229\) and for parameters \(S \cup \{h_1=0.4,h_2=1\} \). b: Dynamic of the solutions of system (3) in the phase space uvw, choosing as initial conditions \(u(0)=0.2,v(0)=w(0)=0.229\) and for parameters \(S \cup \{h_1=0.9,h_2=0.3\} \)

Fig. 6
figure 6

a: Dynamic of the solutions of system (3) in the phase space uvw, choosing as initial conditions \(u(0)=0.2,v(0)=w(0)=0.229\) and for parameters \(S \cup \{h_1=0.001,h_2=0.9\} \). b: Dynamic of the solutions of system (3) in the phase space uvw, choosing as initial conditions \(u(0)=0.2,v(0)=w(0)=0.229\) and for parameters \(S \cup \{h_1=0.9,h_2=0.001\} \)

Fig. 7
figure 7

a: Dynamic of the solutions of system (3) in the phase space uvw, choosing as initial conditions \(u(0)=0.2,v(0)=w(0)=0.229\) and for parameters \(S_1 \cup \{h_1=0.5,h_2=0.8\} \). b: Dynamic of the solutions of system (3) in the phase space uvw, choosing as initial conditions \(u(0)=0.2,v(0)=w(0)=0.229\) and for parameters \(S_1 \cup \{h_1=0.9,h_2=0.3\} \)

8 Numerical results

To evaluate the effect of the interaction between prey and top predator populations on the system dynamics, let us analyse the behaviour of the solutions to (3)–(4) for quoted values of \(h_1\), that represent the predation rate of top predator on prey, and \(h_2\), that represent the interaction rate between prey and top predator. In order to compare our results with those ones obtained in [14], let us first employ the following set of parameters: \(S=\{ b=0.0499,c=0.135,d=0.0788,e=1.3423,n=0.25,q=0.632,s=1,h=0.5 \}\). For \(h_1=h_2=0\), we recover the results found in [14], as expected. Let us first evaluate the behaviour of the system in the case \(h_1=h_2\not =0\). Choosing \(h_1=h_2=0.6\) there are three interior equilibria, i.e. \(E^*_1=[0.072456,0.11111,0.11954]\), \(E^*_2=[0.19226,0.17085,0.19933]\), \(E^*_3=[0.50505,0.20026,0.26064]\). In Fig. 1a the dynamic of the solutions are plotted with respect to time t choosing as initial conditions a random perturbation of \(E^*_1\), while in Fig. 1b the dynamic of the solutions are plotted in the phase space uvw. For \(E^*_1\), one gets \(I_1=-0.6264\), \(I_2=0.0178\), \(I_3=-0.0333\) and \(I_1I_2-I_3=0.0221\), while for \(E^*_3\), one gets \(I_1=-1.2195\), \(I_2=0.5711\), \(I_3=-0.0713\) and \(I_1I_2-I_3=-0.6252\). A limit cycle around the unstable equilibrium \(E^*_1\) is observable, while \(E^*_3\) is locally stable.

Let us now consider the case \(h_1<h_2\). For parameters \(S \cup \{h_1=0.6,h_2=0.9\} \), the coexistence equilibrium is \(E_{s1}=[0.63925,0.16578,0.2495]\), the principal invariants are \(I_1=-1.6645\), \(I_2=0.9867\), \(I_3=-0.1826\), while \(I_1I_2-I_3=-1.4597\), therefore \(E_{s1}\) is linearly stable. In Figs. 2a and b, the dynamic of the solutions to (3) are plotted under initial conditions \(u(0)=0.2,v(0)=w(0)=0.229\). The stability of \(E_{s1}\) is depicted in Fig. 2. This behaviour is also obtained for other values of \(h_1\) and \(h_2\) satisfying \(h_1<h_2\) (see Figs. 5a and 6a), therefore, in that case, the coexistence equilibrium is stable.

Finally, let us consider the case \(h_1>h_2\). For parameters \(S \cup \{h_1=1,h_2=0.5\} \), the coexistence equilibrium is \(E_{s2}=[0.049455,0.087578,0.09152]\), while \(I_1=-0.6727,\) \(I_2=0.0245,\) \(I_3=-0.0664\) and \(I_1I_2-I_3=0.0499\). In Fig. 3 the solutions are plotted choosing as initial conditions \(u(0)=0.2,v(0)=w(0)=0.229\), while in Fig. 4 the chosen initial conditions are a random perturbation of the interior equilibrium \(E_{s2}\). From Figs. 3b and 4b, we can see a limit cycle near to \(E_{s2}\). This behaviour is also depicted in Figs. 5b and 6b, therefore if \(h_1\) and \(h_2\) verified \(h_1>h_2\), the coexistence equilibrium is unstable and a limit cycle is present.

Let us underline that we also numerically found a set of parameters \(S_1\) such that the validity of the Routh-Hurwitz conditions—hence the stability of the coexistence equilibrium – does not depend on parameters \(h_1\) and \(h_2\). In particular, let us fix \(S_1=\{ b=0.0499,c=15,d=0.9,e=1.3423,n=0.25,q=0.632,s=1,h=0.5\}\). In Fig. 7, the solutions are plotted in the phase space choosing as initial conditions \(u(0)=0.2,v(0)=w(0)=0.229\). In particular, in Fig. 7a we chose \(h_1=0.5\) and \(h_2=0.8\), the equilibrium is \(E_{s3}=[0.99735,0.00118524,0.0028394]\), while the principal invariants are \(I_1=-2.5280\), \(I_2=1.5747\), \(I_3=-0.0491\), and \(I_1I_2-I_3=-3.9315\), therefore \(E_{s3}\) is linearly stable. In Fig. 7b we chose \(h_1=0.9\) and \(h_2=0.3\), the equilibrium is \(E_{s4}=[0.99612,0.0023637,0.0028358]\), while the principal invariants are \(I_1=-2.1927\), \(I_2=1.2298\), \(I_3=-0.0384\), and \(I_1I_2-I_3=-2.6581\), therefore \(E_{s4}\) is linearly stable. From a biological point of view, the independence of the stability of the coexistence equilibrium on parameters \(h_1\) and \(h_2\) is motivated by the choice of the interference coefficient c: the interference of the top-predator on the capture by the mesopredator population is so high that the interaction between top-predator and prey does not affect the system dynamics.

9 Conclusion

In this paper, a Leslie–Gower predator–prey model describing a complete food web has been analysed. The boundedness of the solutions to the governing system has been proved, founding an absorbing set in the phase space. Sufficient conditions for the existence and the linear stability of biological meaningful equilibria have been determined. Moreover, via nonlinear stability analysis of the coexistence equilibrium, conditions guaranteeing the nonlinear, local, asymptotic stability of the interior equilibrium have been found. Finally, numerical investigations of the stability results have been presented, in order to analyse how the interaction between prey and top-predator populations affects the system dynamics.