1 Introduction

Plankton includes plants and animals that float along at the mercy of the sea’s tides and currents. There are two main types of plankton, phytoplankton and zooplankton. Phytoplankton include microscopic organisms such as diatoms and dinoflagellates as well as blue-green algae. Zooplankton depends on the phytoplankton and other particulate matter that is found in the water for food. Phytoplankton is the chief source of food for the zooplankton. The significance of plankton for the wealth of the ocean ecosystems and ultimately for the planet itself is nowadays widely recognized. Therefore, many scholars have studied models of plankton [1, 2]. Caraballo et al. [1] considered a model in which one predator (zooplankton) has two preys of different sizes (phytoplankton). The authors first proved the existence of a global pullback attractor and then estimated the fractal dimension of the attractor by using Leonov’s theorem and constructing a proper Lyapunov function. In addition, many science researcher have devoted their efforts to investigating plankton dynamics through laboratory experiments and field studies. The studies show that many types of phytoplankton can release toxic chemicals, which have an important effect on plankton; see [3,4,5,6,7,8] and the references therein. Mathematical modeling is an important tool to study plankton dynamics and to understand the various mechanisms involved in toxin-producing phytoplankton [9, 10]. For example, Chattopadhayay et al. [6] using field studies and mathematical models confirmed that toxin-producing plankton may act as a biological control for planktonic blooms. Turner and Tester [7] revealed that the interplay between toxic phytoplankton and the zooplankton grazing on them gives rise to various situations, since, in some cases, the toxic phytoplankton does not cause any harm, but in other cases it causes serious damage if grazed. Therefore, the study of the toxin-producing phytoplankton population is an interesting and important topic in marine ecology. Mukhopadhyay et al. [11] investigated a nutrient-plankton model in an aquatic environment in the context of phytoplankton bloom and they observed that the zooplankton populations try to avoid the areas where toxin-producing phytoplankton density is high. Jin and Ma [12] considered the two plankton species having competitive and allelopathic effects on each other. In particular, Banerjee and Venturino [13] proposed the following model:

$$\begin{aligned} \textstyle\begin{cases} \frac{d\varPi _{1}}{d\tau }=\rho _{1}\varPi _{1}(1-\frac{\varPi _{1}}{K_{1}})- \tilde{a}\varPi _{1}\varPi _{2}-\alpha _{1}\varPi _{1}\zeta , \\ \frac{d\varPi _{2}}{d\tau }=\rho _{2}\varPi _{2}(1-\frac{\varPi _{2}}{K_{2}})- \tilde{b}\varPi _{1}\varPi _{2}-\frac{\alpha _{2}\varPi _{2}\zeta }{\beta _{2}+\varPi _{2}^{2}}, \\ \frac{d\zeta }{d\tau }=e_{1}\alpha _{1}\varPi _{1}\zeta -\frac{e_{2}\alpha _{2}\varPi _{2}\zeta }{\beta _{2}+\varPi _{2}^{2}}-\mu \zeta . \end{cases}\displaystyle \end{aligned}$$
(1.1)

Here \(\varPi _{1}\), \(\varPi _{2}\) and ζ, respectively, denote the phytoplankton, toxin-producing phytoplankton and zooplankton populations. \(\rho _{i}\) denotes the growth rate of species i (\(i=1,2\)), \(K_{i}\) is the environmental carry capacity of species i (\(i=1,2\)), ã represents the action of the second population upon the growth rate of the first population, stands for the action of the first population upon the growth rate of the second population, \(\alpha _{1}\) is the capture rate, \(\frac{\alpha _{2}\varPi _{2}}{\beta _{2}+ \varPi _{2}^{2}}\) is the Monod–Haldane function, which represents zooplankton populations consuming the toxic-phytoplankton populations, \(e_{1}\) denotes the nutrition conversion rate for the \(\varPi _{1}\) to the ζ, \(e_{2}\) represents the effect of consuming \(\varPi _{2}\) on ζ, μ is the death rate of the zooplankton population. All parameters are assumed to be nonnegative and constant in time. And we assume that the zooplankton is able to recognize the two phytoplankton populations and when the toxic phytoplankton is ingested too much, and it will kill too much zooplankton; then the latter will decrease its consumption. This is modeled via a Monod–Haldane type functional response \(\frac{\alpha _{2}\varPi _{2}}{\beta _{2}+\varPi _{2}^{2}}\) [14], which increases to a maximum and then decreases again for larger values of the toxin-producing phytoplankton population. For more information, see Ref. [13] and the references therein.

Model (1.1) can be simplified to a dimensionless form with a reduced number of parameters by means of the transformation \(P(t)=K_{1}\varPi _{1}(t)\), \(T(t)=\sqrt{\beta _{2}}\varPi _{2}(t)\), \(Z(t)=e_{2}\sqrt{ \beta _{2}}\zeta (t)\), \(t=\frac{\tau }{e_{2}\alpha _{2}\beta _{2}\sqrt{ \beta _{2}}}\), then we obtain the following system:

$$\begin{aligned} \textstyle\begin{cases} \frac{dP(t)}{dt}=r_{1}P(t)(1-P(t))-aP(t)T(t)-cP(t)Z(t), \\ \frac{dT(t)}{dt}=r_{2}T(t)(1-\frac{T(t)}{H})-bP(t)T(t)-\frac{T(t)Z(t)}{1+T ^{2}(t)}, \\ \frac{dZ(t)}{dt}=eP(t)Z(t)-\frac{T(t)Z(t)}{1+T^{2}(t)}-mZ(t). \end{cases}\displaystyle \end{aligned}$$
(1.2)

Here

$$\begin{aligned}& r_{1}=\frac{\rho _{1}}{e_{2}\alpha _{2}\beta _{2}\sqrt{\beta _{2}}},\qquad r _{2}= \frac{\rho _{2} }{e_{2}\alpha _{2}\beta _{2}\sqrt{\beta _{2}}},\qquad H=\frac{K _{2}}{\sqrt{\beta _{2}}} ,\qquad b= \frac{\tilde{b}K_{1}}{\alpha _{2}\beta _{2}\sqrt{\beta _{2}}e_{2}}, \\& a=\frac{\tilde{a}}{\alpha _{2}\beta _{2}e_{2}},\qquad c=\frac{\alpha _{1}}{ \alpha _{2}\beta _{2}},\qquad e= \frac{e_{1}K_{1}\alpha _{1}}{e_{2}\alpha _{2} \beta _{2}\sqrt{\beta _{2}}},\qquad m=\frac{\mu }{e_{2}\alpha _{2}\beta _{2}\sqrt{ \beta _{2}}} , \end{aligned}$$

are positive constants.

It is well known that environmental fluctuation plays an important role in the study of ecosystems. Therefore, many authors have introduced stochastic white noise into deterministic models to describe the effect of random noise [15,16,17,18,19,20,21,22,23,24,25,26,27]. For example, Han et al. [16] obtained two systems admitting a unique random attractor under some conditions. They also obtained conditions under which coexistence of species exists for the random system. Yuan et al. [19] considered a nutrient-phytoplankton model with toxin-producing phytoplankton under environmental fluctuations. The obtained results suggested that toxin-producing phytoplankton and environmental fluctuations play a key role in the termination of algal blooms. Colucci et al. [20] studied semi-Kolmogorov models for plankton. For the random semi-Kolmogorov system they also obtained sufficient conditions for the existence of a global random attractor. Therefore, it is meaningful to further incorporate the environmental fluctuation into the underlying model (1.2), which could provide us a deeper understanding for the realistic aquatic ecosystems. Inspired by the above-mentioned facts, we get the following stochastic model:

$$\begin{aligned} \textstyle\begin{cases} dP(t)=[r_{1}P(t)(1-P(t))-aP(t)T(t)-cP(t)Z(t)]\,dt+\sigma _{1}P(t)\,dB_{1}(t), \\ dT(t)=[r_{2}T(t)(1-\frac{T(t)}{H})-bP(t)T(t)- \frac{T(t)Z(t)}{1+T^{2}(t)}]\,dt+\sigma _{2}T(t)\,dB_{2}(t), \\ dZ(t)=[eP(t)Z(t)-\frac{T(t)Z(t)}{1+T^{2}(t)}-mZ(t)]\,dt+\sigma _{3}Z(t)\,dB _{3}(t), \end{cases}\displaystyle \end{aligned}$$
(1.3)

where \(B_{i}(t)\) are standard Brownian motions. \(\sigma _{i}^{2}\) denotes the intensity of the white noise. Recently, a few studies about the effect of environmental fluctuations on aquatic ecosystems have been carried out, it is worth noting that in this paper, we split the phytoplankton, identifying the toxic-producing subpopulation, in other words, the considered system includes three species: phytoplankton, toxic phytoplankton, and zooplankton; we investigate the effect of the environmental fluctuations on this system. To the best of our knowledge, the results about the stochastic phytoplankton-toxin phytoplankton–zooplankton model are few. We will devote our main attention to the analysis of model (1.3). First, we study the persistence or extinction of the populations. Then we prove the existence of a stationary distribution.

The rest of this paper is organized as follows. Some related preliminaries are given in the next section. In Sect. 3, we prove the existence of the global positive solution and establish the threshold between weakly persistence in the mean and extinction in Sect. 4. Sufficient conditions for the stationary distribution and its ergodic of model (1.3) are established by using a Lyapunov function in Sect. 5. Finally, in order to illustrate our results, some numerical simulations and discussions are presented in Sect. 6.

2 Preliminaries

From now on, unless otherwise specified, let \((\varOmega ,\mathscr{F},\{ \mathscr{F}_{t}\}_{t\geq 0},P)\) denote a complete probability space with a filtration \(\{\mathscr{F}_{t}\}_{t\geq 0}\) satisfying the usual conditions (i.e. it is right continuous and \(\mathscr{F}_{0}\) contains all P-null sets). Let us consider an n-dimensional stochastic differential equation:

$$ dx(t)=f \bigl(x(t),t \bigr)\,dt+g \bigl(x(t),t \bigr)\,dW(t),\quad t\geq 0, $$

with the initial \(x(0)=x_{0}\in R^{n}\), where \(W(t)\) denotes an m-dimensional standard Wiener process. A definite a \(C^{2}\)-function V, applying Itõ’s formula, yields

$$\begin{aligned} dV \bigl(x(t),t \bigr) =&(V_{t} \bigl(x(t),t \bigr)+V_{x} \bigl(x(t),t \bigr)f \bigl(x(t),t \bigr) \\ &{}+\frac{1}{2}\operatorname{trace} \bigl( \bigl(g^{T} \bigl(x(t),t \bigr)V_{xx} \bigl(x(t),t \bigr)g \bigl(x(t),t \bigr) \bigr) \bigr)\,dt \\ &{}+V_{x} \bigl(x(t),t \bigr)g \bigl(x(t),t \bigr)\,dW(t) \\ :=&LV \bigl(x(t),t \bigr)\,dt+V_{x} \bigl(x(t),t \bigr)g \bigl(x(t),t \bigr)\,dW(t), \end{aligned}$$

where

$$\begin{aligned} V_{t}=\frac{\partial V}{\partial t},\qquad V_{x}= \biggl( \frac{\partial V}{\partial x_{1}},\ldots,\frac{\partial V}{\partial x_{n}} \biggr)^{T},\qquad V_{xx}= \biggl(\frac{ \partial ^{2}V}{\partial x_{i}\,\partial x_{j}} \biggr)_{n\times n}. \end{aligned}$$

Definition 2.1

([28, 29])

  1. (i)

    Population \(x(t)\) is said to be extinct if \(\lim_{t\rightarrow \infty }x(t)=0\) almost surely.

  2. (ii)

    Population \(x(t)\) is said to be weakly persistent in the mean if there exists a constant \(N>0\) such that \(\limsup_{t\rightarrow +\infty }\frac{1}{t}\int _{0}^{t}{x(s)\,ds}\leq N\) almost surely.

  3. (iii)

    Population \(x(t)\) is said to be strongly persistent in the mean if there exists a constant \(N>0\) such that \(\limsup_{t\rightarrow +\infty }\frac{1}{t}\int _{0}^{t}{x(s)\,ds}\geq N\) almost surely.

  4. (iv)

    Population \(x(t)\) is said to be persistent in the mean if there exists a constant \(N>0\) such that \(\lim_{t\rightarrow +\infty }\frac{1}{t}\int _{0}^{t}{x(s)\,ds}= N\) almost surely.

Next, we will present some lemmas which will be used in the following sections.

Lemma 2.1

([30])

Let \(M=\{M_{t}\}_{t}\geq 0\) be a real-valued continuous local martingale, then

$$\begin{aligned} \lim_{t\rightarrow \infty }\langle M,M\rangle _{t}=0 \quad \textit{a.s.} \quad \Rightarrow\quad \lim_{t\rightarrow \infty } \frac{M_{t}}{\langle M,M \rangle _{t}}=0 \quad \textit{a.s.}, \end{aligned}$$

and

$$\begin{aligned} \limsup_{t\rightarrow \infty }\frac{\langle M,M\rangle _{t}}{t}< \infty \quad \textit{a.s.} \quad \Rightarrow\quad \lim_{t\rightarrow \infty } \frac{M_{t}}{t}=0 \quad \textit{a.s.} \end{aligned}$$

Lemma 2.2

([31])

Let \(X(t)\in R_{+}\) and \(g(t)\) be two stochastic process satisfying \(\lim_{t\rightarrow \infty } \frac{g(t)}{t}=0\) a.s.

  1. (i)

    If there exist three positive constants \(t_{0}\), β and \(\beta _{0}\) such that, for all \(t\geq t_{0}\),

    $$ \ln X(t)\leq \beta t-\beta _{0} \int _{0}^{t}{X(s)\,ds}+g(t), $$

    then

    $$ \limsup_{t\rightarrow +\infty }\frac{1}{t} \int _{0}^{t}{X(s)\,ds} \leq \frac{\beta }{\beta _{0}}, \quad \textit{a.s.} $$
  2. (ii)

    If there exist three positive constants \(t_{0}\), β and \(\beta _{0}\) such that, for all \(t\geq t_{0}\),

    $$ \ln X(t)\geq \beta t-\beta _{0} \int _{0}^{t}{X(s)\,ds}+g(t), $$

    then

    $$ \liminf_{t\rightarrow +\infty }\frac{1}{t} \int _{0}^{t}{X(s)\,ds} \geq \frac{\beta }{\beta _{0}}, \quad \textit{a.s.} $$

3 Existence and uniqueness of globally positive solution

In this section, we show that there is a unique local positive solution for the system (1.3) and then we prove that this solution is global by constructing a suitable Lyapunov function. To the best of our knowledge, in order to guarantee that a stochastic differential equation has a unique local positive solution, the coefficients of the equation should satisfy the linear growth condition and the local Lipschitz condition. Therefore, we establish the following theorem.

Theorem 3.1

For any positive initial value \((P(0),T(0),Z(0)) \in R_{+}^{3}\), there exists a unique global positive solution of model (1.3), and the solution remains in the region \(R_{+}^{3}\) with probability one.

Proof

Let \(u(t)=\ln P(t)\), \(v(t)=\ln T(t)\), \(w(t)=\ln Z(t)\), by use of the Itô formula, we get

$$\begin{aligned} \textstyle\begin{cases} du(t)=[r_{1}(1-e^{u}(t))-ae^{v}(t)-ce^{w}(t)-\frac{\sigma _{1}^{2}}{2}]\,dt+ \sigma _{1}\,dB_{1}(t), \\ dv(t)=[r_{2}(1-\frac{e^{v}(t)}{H})-be^{u}(t)- \frac{e^{w}(t)}{1+e^{v}(t)}-\frac{\sigma _{2}^{2}}{2}]\,dt+\sigma _{2}\,dB _{2}(t), \\ dw(t)=[ee^{u}(t)-\frac{e^{v}(t)}{1+e^{v}(t)}-m]\,dt+\sigma _{3}\,dB_{3}(t), \end{cases}\displaystyle \end{aligned}$$
(3.1)

with initial values \(u(0)=\ln P(0)\), \(v(0)=\ln T(0)\), \(w(0)=\ln Z(0)\). Obviously, model (3.1) meets the local Lipschitz condition. Hence there exists a unique local solution \((u(t),v(t),w(t))\) for \(t\in [0,\tau _{e})\), where \(\tau _{e}\) is the explosion time. The system (1.3) is equivalent to the system (3.1). So, for the system (1.3) there exists a local solution \((P(t),T(t),Z(t))\) on \(t\in [0,\tau _{e})\), for an arbitrary initial value \((P(0),T(0),Z(0))\in R_{+}^{3}\). Now, we need to prove \(\tau _{e}=\infty \) a.s. Let \(k_{0}>0\) be sufficiently large such that each component of the initial value \((P(0),T(0),Z(0))\) lies within the interval \([\frac{1}{k_{0}},k_{0}]\) when \(k\geq k_{0}\). Define the stopping time

$$\begin{aligned} \tau _{k}=\inf \biggl\{ t\in [0,\tau _{e}):\min \bigl\{ P(t),T(t),Z(t) \bigr\} \leq \frac{1}{k} \text{ or } \max \bigl\{ P(t),T(t),Z(t) \bigr\} \geq k \biggr\} . \end{aligned}$$

Throughout this paper we set \(\inf \emptyset =\infty \) (as usual ∅ denotes the empty set). Clearly, \(\tau _{k}\) is increasing as \(k\rightarrow \infty \). Set \(\tau _{\infty }= \lim_{k \rightarrow \infty } \tau _{k}\), hence \(\tau _{\infty } \leq \tau _{e}\) a.s. If we can show that \(\tau _{\infty }= \infty \) a.s., then \(\tau _{e} =\infty \) a.s. The proof will go by contradiction. If this statement is false, then there exist a pair of constants \(T >0\) and \(\varepsilon \in (0,1)\) such that \(P\{\tau _{k} \leq T\} \geq \varepsilon \) for any \(k\geq k_{0}\).

Define a \(\mathcal{C}^{2}\)-function \(V:R^{3}_{+}\rightarrow R_{+}\) as follows:

$$\begin{aligned} V(P,T,Z)=V_{1}(P,T,Z)+V_{2}(P,T,Z)+V_{3}(P,T,Z), \end{aligned}$$

where \(V_{1}(P,T,Z)=AP-\ln P-1-\ln A\), \(V_{2}(P,T,Z)=T-1-\ln T\), \(V_{3}(P,T,Z)=B(Z-1-\ln Z)\), A, B are positive constants to be determined later. Obviously, this function is nonnegative.

An application of the Itô formula to \(V_{1}\) yields

$$\begin{aligned} dV_{1} =& \biggl(Ar_{1}P-Ar_{1}P^{2}-AaPT-AcPZ-r_{1}+r_{1}P+aT+cZ+ \frac{ \sigma _{1}^{2}}{2} \biggr)\,dt \\ &{}+\sigma _{1}A(P-1)\,dB_{1}(t). \end{aligned}$$

Analogously

$$\begin{aligned} dV_{2} =& \biggl(r_{2}T-\frac{r_{2}}{H}T^{2}-bPT- \frac{TZ}{1+T^{2}}-r_{2}+\frac{r _{2}}{H}T+bP+ \frac{Z}{1+T^{2}}+\frac{\sigma _{2}^{2}}{2} \biggr)\,dt \\ &{}+\sigma _{2}(T-1)\,dB_{2}(t) \end{aligned}$$

and

$$\begin{aligned} dV_{3} =& \biggl(BePZ-B\frac{TZ}{1+T^{2}}-BmZ-BeP+B \frac{T}{1+T^{2}}+Bm+\frac{B \sigma _{3}^{2}}{2} \biggr)\,dt \\ &{}+\sigma _{3}B(Z-1)\,dB_{3}(t). \end{aligned}$$

Hence

$$\begin{aligned} LV =& \biggl(Ar_{1}P-Ar_{1}P^{2}-AaPT-AcPZ-r_{1}+r_{1}P+aT+cZ+ \frac{\sigma _{1}^{2}}{2}+r_{2}T \\ &{}-\frac{r_{2}}{H}T^{2}-bPT-\frac{TZ}{1+T^{2}}-r_{2}+ \frac{r_{2}}{H}T+bP+\frac{Z}{1+T ^{2}}+\frac{\sigma _{2}^{2}}{2}+BePZ \\ &{}-B\frac{TZ}{1+T^{2}}-BmZ-BeP+B\frac{T}{1+T^{2}}+Bm+ \frac{B\sigma _{3}^{2}}{2} \biggr) \\ \leq &-Ar_{1}P^{2}+(Ar_{1}+r_{1}+b-Be)P- \frac{r_{2}}{H}T^{2}+ \biggl(a+r _{2}+ \frac{r_{2}}{H} \biggr)T+(c+1-Bm)Z \\ &{}+ \biggl(-r_{1}-r_{2}+\frac{B}{2}+Bm \biggr)+(Be-Ac)PZ+\frac{A\sigma _{1}^{2}+ \sigma _{2}^{2}+B\sigma _{3}^{2}}{2}. \end{aligned}$$

Choose \(A=\frac{ce+2e}{cm}\), \(B=\frac{(c+2)e+m}{m^{2}} \), then

$$\begin{aligned} LV \leq &-Ar_{1}P^{2}+(Ar_{1}+r_{1}+b-Be)P- \frac{r_{2}}{H}T^{2}+ \biggl(a+r _{2}+ \frac{r_{2}}{H} \biggr)T \\ &{}+ \biggl(-r_{1}-r_{2}+\frac{B}{2}+Bm \biggr)+\frac{\sigma _{1}^{2}+\sigma _{2}^{2}+B \sigma _{3}^{2}}{2} \\ \leq &M. \end{aligned}$$

Then \(dV\leq M\,dt+\sigma _{1}A(P-1)\,dB_{1}(t)+\sigma _{2}(T-1)\,dB_{2}(t)+ \sigma _{3}B(Z-1)\,dB_{3}(t)\).

Integrating both sides from 0 to \(\tau _{k} \wedge T\), and taking the expectation, we get

$$\begin{aligned} EV \bigl(P(\tau _{k} \wedge T),T(\tau _{k} \wedge T),Z( \tau _{k} \wedge T) \bigr) \leq V \bigl(P(0),T(0),Z(0) \bigr)+MT. \end{aligned}$$

By the definition of stopping time, substituting k and \(\frac{1}{k}\) into \(V(P(0),T(0),Z(0))\), which implies that

$$\begin{aligned}& V \bigl(P(\tau _{k} \wedge T), T(\tau _{k} \wedge T), Z(\tau _{k} \wedge T) \bigr) \\& \quad \geq \min \biggl\{ Ak-1-\ln k-\ln A, \frac{A}{k}-1+\ln k-\ln A, \\& \qquad k-1-\ln k,\frac{1}{k}-1+\ln k, B(k-1-\ln k), B \biggl( \frac{1}{k}-1+\ln k \biggr) \biggr\} \\& \quad =F(k). \end{aligned}$$

Next,

$$\begin{aligned} P(\tau _{k}\leq T)F(k)\leq V \bigl(P(0),T(0),Z(0) \bigr)+MT. \end{aligned}$$

Letting \(k\rightarrow \infty \) gives

$$\begin{aligned} \lim_{t\rightarrow \infty }P(\tau _{k}\leq T)=0, \end{aligned}$$

which contradicts the assumption; then we must have \(P(\tau _{\infty }= \infty )=1\). Therefore, the solution will not explode in a finite time with probability one. This completes the proof. □

4 Weakly persistent in the mean and extinction

In this section, we will study the long time behavior of stochastic model (1.3) and show that the population will be weakly persistent in the mean when the intensity of noises is small. Otherwise, if the intensity of noises is sufficiently large, the population will become extinct with probability one.

Theorem 4.1

Let \((P(t),T(t),Z(t))\) be a solution of model (1.3).

  1. (i)

    If \(r_{1}>\frac{\sigma _{1}^{2}}{2}\), \(r_{2}> \frac{\sigma _{2}^{2}}{2}\) and \(r_{1}(e-m)>e\frac{\sigma _{1}^{2}}{2}+r _{1}\frac{\sigma _{3}^{2}}{2}\), then the populations \(P(t)\), \(T(t)\) and \(Z(t)\) are weakly persistent in the mean.

  2. (ii)

    If \(r_{1}>\frac{\sigma _{1}^{2}}{2}\), \(r_{2}< \frac{\sigma _{2}^{2}}{2}\) and \(r_{1}(e-m)>e\frac{\sigma _{1}^{2}}{2}+r _{1}\frac{\sigma _{3}^{2}}{2}\), then the populations \(P(t)\) and \(Z(t)\) are weakly persistent in the mean, the population \(T(t)\) is extinct.

  3. (iii)

    If \(r_{1}<\frac{\sigma _{1}^{2}}{2}\), \(r_{2}> \frac{\sigma _{2}^{2}}{2}\), then the populations \(P(t)\) and \(Z(t)\) are extinct, the population \(T(t)\) is weakly persistent in the mean.

  4. (iv)

    If \(r_{1}<\frac{\sigma _{1}^{2}}{2}\), \(r_{2}< \frac{\sigma _{2}^{2}}{2}\), then the population \(P(t)\), \(T(t)\) and \(Z(t)\) are extinct.

Proof

By using Itô’s formula in (1.3), we have

$$\begin{aligned}& d\ln P= \biggl[r_{1}(1-P)-aT-cZ-\frac{\sigma _{1}^{2}}{2} \biggr]\,dt+ \sigma _{1}\,dB_{1}(t), \end{aligned}$$
(4.1)
$$\begin{aligned}& d\ln T= \biggl[r_{2} \biggl(1-\frac{T}{H} \biggr)-bP- \frac{Z}{1+T^{2}}-\frac{\sigma _{2} ^{2}}{2} \biggr]\,dt+\sigma _{2} \,dB_{2}(t), \end{aligned}$$
(4.2)
$$\begin{aligned}& d\ln Z= \biggl(eP-\frac{T}{1+T^{2}}-m-\frac{\sigma _{3}^{2}}{2} \biggr)\,dt+ \sigma _{3}\,dB _{3}(t), \end{aligned}$$
(4.3)

and from (4.1), we furthermore obtain

$$\begin{aligned} d\ln P\leq \biggl(r_{1}-\frac{\sigma _{1}^{2}}{2}-r_{1}P \biggr)\,dt+\sigma _{1}\,dB_{1}(t), \end{aligned}$$
(4.4)

integrating both sides of (4.4) from 0 to t gives the following inequality:

$$\begin{aligned} \ln P(t)= \biggl(r_{1}-\frac{\sigma _{1}^{2}}{2} \biggr)t-r_{1} \int _{0}^{t}{P(s)\,ds}+ \varTheta (t), \end{aligned}$$
(4.5)

denoting \(\varTheta (t)=M_{1}(t)+\ln P(0)=\sigma _{1}B_{1}(t)+\ln P(0)\). We can get \(\lim_{t\rightarrow \infty }\frac{\varTheta (t)}{t}=0\). By Lemmas 2.1 and 2.2, we have

$$\begin{aligned} \limsup_{t\rightarrow +\infty }\frac{1}{t} \int _{0}^{t}{P(s)\,ds} \leq \frac{r_{1}-\frac{\sigma _{1}^{2}}{2}}{r_{1}} \quad \text{a.s.} \end{aligned}$$
(4.6)

So, following the above conditions \(r_{1}>\frac{\sigma _{1}^{2}}{2}\), we see that the population \(P(t)\) is weakly persistent in the mean.

From (4.2), we have

$$\begin{aligned} d\ln T\leq \biggl(r_{2}-\frac{\sigma _{2}^{2}}{2}-\frac{r_{2}}{H}T \biggr)\,dt+\sigma _{2}\,dB_{2}(t), \end{aligned}$$
(4.7)

integrating both sides of (4.7) from 0 to t gives the following inequality:

$$\begin{aligned} \ln T(t)\leq \biggl(r_{2}-\frac{\sigma _{2}^{2}}{2} \biggr)t- \frac{r_{2}}{H} \int _{0}^{t}{T(s)\,ds}+f(t), \end{aligned}$$
(4.8)

letting \(f(t)=M_{2}(t)+\ln T(0)=\sigma _{2}B_{2}(t)+\ln T(0)\). It is easy to get \(\lim_{t\rightarrow \infty }\frac{f(t)}{t}=0\). Similarly, we have

$$\begin{aligned} \limsup_{t\rightarrow +\infty }\frac{1}{t} \int _{0}^{t}{T(s)\,ds} \leq \frac{(r_{2}-\frac{\sigma _{2}^{2}}{2})H}{r_{2}} \quad \text{a.s.} \end{aligned}$$
(4.9)

For \(r_{2}>\frac{\sigma ^{2}_{2}}{2}\), the population \(T(t)\) is weakly persistent in the mean.

According to a similar method in [32], we can get

$$\begin{aligned} \limsup_{t\rightarrow +\infty }\frac{\ln P(t)}{t}=0 \quad \text{a.s.} \end{aligned}$$
(4.10)

and

$$\begin{aligned} \limsup_{t\rightarrow +\infty }\frac{\ln Z(t)}{t}=0 \quad \text{a.s.} \end{aligned}$$
(4.11)

From (4.1), we can get

$$\frac{\ln P(t)-\ln P(0)}{t}=\biggl(r_{1}-\frac{\sigma _{1}^{2}}{2}\biggr)- \frac{r _{1}}{t} \int _{0}^{t}{P(s)\,ds}-\frac{a}{t} \int _{0}^{t}{T(s)\,ds}- \frac{c}{t} \int _{0}^{t}{Z(s)\,ds}+\frac{\sigma _{1}B_{1}}{t}. $$

Applying Lemma 2.1, combining with (4.10), yields

$$\begin{aligned} \lim_{t\rightarrow \infty }\frac{1}{t} \int _{0}^{t}{P(s)\,ds}= \lim _{t\rightarrow \infty } \frac{1}{r_{1}} \biggl[ \biggl(r_{1}- \frac{\sigma _{1}^{2}}{2} \biggr)-\frac{a}{t} \int _{0}^{t}{T(s)\,ds}-\frac{c}{t} \int _{0}^{t} {Z(s)\,ds} \biggr] \quad \text{a.s.} \end{aligned}$$
(4.12)

Similarly, from (4.3), we have

$$\begin{aligned} \lim_{t\rightarrow \infty }\frac{e}{t} \int _{0}^{t}{P(s)\,ds}= \lim _{t\rightarrow \infty } \frac{1}{t} \int _{0}^{t}\frac{T(s)\,ds}{1+T ^{2}(s)}+m+ \frac{\sigma _{3}^{2}}{2} \quad \text{a.s.} \end{aligned}$$
(4.13)

Together with (4.12) and (4.13), this yields

$$ \limsup_{t\rightarrow +\infty }\frac{1}{t} \int _{0}^{t}{Z(s)\,ds} \leq \frac{1}{c} \biggl(r_{1}-\frac{\sigma _{1}^{2}}{2} \biggr)- \frac{r_{1}}{ce} \biggl(m+\frac{ \sigma _{3}^{2}}{2} \biggr) \quad \text{a.s.} $$

The condition \(r_{1}(e-m)>e\frac{\sigma _{1}^{2}}{2}+r_{1}\frac{\sigma _{3}^{2}}{2}\) implies that \(\frac{1}{c}(r_{1}- \frac{\sigma _{1}^{2}}{2})-\frac{r_{1}}{ce}(m+\frac{\sigma _{3}^{2}}{2})>0\), hence, the population \(Z(t)\) is weakly persistent in the mean, thus we complete the proof of (i).

In (ii), we have only to prove that \(r_{2}<\frac{\sigma _{2}^{2}}{2}\) implies \(\lim_{t\rightarrow +\infty }T(t)=0\) a.s.

If \(r_{2}<\frac{\sigma _{2}^{2}}{2}\), from (4.8), we can obtain

$$\begin{aligned} \ln T(t)< \biggl(r_{2}-\frac{\sigma _{2}^{2}}{2} \biggr)t+f(t), \end{aligned}$$
(4.14)

dividing both sides of (4.14) by t and taking the limit, by Lemma 2.1, we have \(\lim_{t\rightarrow +\infty }\frac{\ln T(t)}{t}<0\) a.s.; thus the population \(T(t)\) is extinct.

(iii) If \(r_{1}<\frac{\sigma _{1}^{2}}{2}\), \(r_{2}> \frac{\sigma _{2}^{2}}{2}\), from the above proof, we know that the population \(P(t)\) is extinct, the population \(T(t)\) is weakly persistent in the mean. From (4.3), it is easy to see that

$$\begin{aligned} \ln Z(t)< e \int _{0}^{t}P(s)\,ds- \biggl(m+ \frac{\sigma _{3}^{2}}{2} \biggr)t+G(t). \end{aligned}$$
(4.15)

We denote \(G(t)=\sigma _{3}B_{3}(t)+\ln Z(0)\). We get \(\lim_{t\rightarrow \infty }\frac{G(t)}{t}=0\). Dividing both sides of (4.15) by t and taking the limit, we have \(\lim_{t\rightarrow +\infty }\frac{\ln Z(t)}{t}<0\) a.s., that is, the population \(Z(t)\) is extinct.

The result of (iv) comes from the proofs (ii) and (iii), so we omit it. The proof is complete. □

Remark 4.1

From (i), (ii) of Theorem 4.1, we know all populations preserve some stability under small noise and large noise may lead to the extinction of the toxic-phytoplankton population. From (iii) of Theorem 4.1, we find that the noise-induced extinction of \(P(t)\) contributes the density increase of \(T(t)\), thus noise-induced extinction of nontoxic phytoplankton may lead to more serious outbreaks of the toxic phytoplankton and furthermore if there is excessive consumption of toxin phytoplankton by zooplankton this will lead to its extinction. The results also reflect that if the reproduction rate of the population is dominated by the environmental noise, the population will go in extinction. Otherwise, the population will be weakly persistent in the mean.

Remark 4.2

By \(r_{1}=\frac{\rho _{1}}{e_{2}\alpha _{2}\beta _{2}\sqrt{\beta _{2}}}\), \(r_{2}=\frac{\rho _{2} }{e_{2}\alpha _{2}\beta _{2}\sqrt{\beta _{2}}}\), \(e=\frac{e_{1}K_{1}\alpha _{1}}{e_{2}\alpha _{2}\beta _{2}\sqrt{\beta _{2}}}\), \(m=\frac{\mu }{e_{2}\alpha _{2}\beta _{2}\sqrt{\beta _{2}}}\), we can know that the \(r_{i}\) (\(i=1,2\)) are proportional to \(\rho _{i}\) (the growth rate of species i), respectively; m is proportional to μ (the death rate); e is proportional to the product of \(e_{1}\) (the nutrition conversion rate) and \(\alpha _{1}\) (the capture rate). From (i), (ii), (iii) and (iv) of Theorem 4.1, one can see that a small noise (\(\sigma _{i}\)) and a large growth rate (\(\rho _{i}\)) lead to the persistence of phytoplankton populations. Meanwhile, a high nutrient conversion rate (\(e_{1}\)) from P to Z, a high predation rate (\(\alpha _{1}\)) of Z to P, a small death rate (μ) and a small noise (\(\sigma _{1}\) and \(\sigma _{3}\)) contribute to the persistence of the zooplankton population.

5 Existence of an ergodic stationary distribution

In the study of a zooplankton and phytoplankton deterministic model, the stability of each equilibrium is our main concern. For a stochastic model, we concentrate on the existence of the unique stationary distribution. For the plankton system, one of the most important things is to investigate the persistence of the population. Meanwhile, the existence of stationary distribution implies that the populations are persistent. Therefore, it is very meaningful to obtain the unique ergodic stationary distribution. In the following, based on the theory of Has’minskii [33], we will prove that the system (1.3) has a stationary distribution which is ergodic when the white noises are not particularly large. Let \(X(t)\) be a homogeneous Markov process in \(R^{d}\) of the stochastic differential equation \(dX(t)=b(X)\,dt+\sum_{r=1}^{k}g_{r}(X)\,dB_{r}(t)\), then the diffusion matrix is defined as \(A(x)=(a_{ij}(x))\), \(a_{ij}(x)=\sum_{r=1}^{k}g^{i} _{r}(x)g^{j}_{r}(x)\).

Lemma 5.1

([33])

The Markov process \(X(t)\) has a unique ergodic stationary distribution \(\mu (\cdot )\) if there exists a bounded open domain \(U\subset R^{n}\) with regular boundary Γ, having the following properties:

\(A_{1}\)::

the diffusion matrix \(A(x)\) is strictly positive definite for all \(x\in U\);

\(A_{2}\)::

there exists a nonnegative \(C^{2}\)-function V such that LV is negative for any \(R^{d}\backslash U\).

Assumption 5.1

Assume that

$$\begin{aligned} \varPi =&r_{1}-\frac{\sigma _{1}^{2}}{2}+r_{2}- \frac{\sigma _{2}^{2}}{2}- \frac{1}{2}-m-\frac{\sigma _{3}^{2}}{2}- \biggl( \frac{H(r_{2}+a+ \frac{r_{2}}{H})^{2}}{4r_{2}}+\frac{(\frac{ec+2e+cm}{cm}r_{1}+b)^{2}cm}{4(c+2)er _{1}} \biggr) \\ =&M-M_{1}-M_{2}>0, \end{aligned}$$

where \(M=r_{1}-\frac{\sigma _{1}^{2}}{2}+r_{2}- \frac{\sigma _{2}^{2}}{2}-\frac{1}{2}-m-\frac{\sigma _{3}^{2}}{2}\), \(M_{1}=\frac{H(r_{2}+a+\frac{r_{2}}{H})^{2}}{4r_{2}}\), \(M_{2}=\frac{( \frac{ec+2e+cm}{cm}r_{1}+b)^{2}cm}{4(c+2)er_{1}}\).

Theorem 5.1

Under Assumption 5.1, the system (1.3) has a unique ergodic stationary distribution \(\mu (\cdot )\) for any initial value \((P(0),T(0),Z(0))\in R^{3}_{+}\).

Proof

In order to prove Theorem 5.1, one only needs to validate conditions \(A_{1}\) and \(A_{2}\) of Lemma 5.1. The diffusion matrix of the system (1.3) is given by

$$\begin{aligned} A= \begin{pmatrix} \sigma _{1}^{2}P^{2} & 0 & 0 \\ 0 & \sigma _{2}^{2}T^{2} & 0 \\ 0 & 0 & \sigma _{3}^{2}Z^{2} \end{pmatrix} . \end{aligned}$$
(5.1)

Clearly, the matrix A is positive definite for any compact \(R_{+}^{3}\), so the condition \(A_{1}\) of Lemma 5.1 holds.

Construct a \(C^{2}\)-function \(W:R_{+}^{3} \rightarrow R\)

$$\begin{aligned} W(P,T,Z) =&(c_{1}P+T+c_{2}Z)-(\ln P-1+\ln c_{1}+\ln T+\ln Z-1+\ln c _{2}) \\ :=&W_{1}(P,T,Z)+W_{2}(P,T,Z), \end{aligned}$$

where \(c_{1}\) and \(c_{2}\) are positive constants which will be determined later. Obviously, \(W(P,T,Z)\) is a nonnegative function. According to the Itô formula, then

$$\begin{aligned} LW_{1} =&c_{1}r_{1}P-c_{1}r_{1}P^{2}-c_{1}aPT-c_{1}cPZ+r_{2}T- \frac{r _{2}}{H}T^{2}-bPT-\frac{TZ}{1+T^{2}} \\ &{}+c_{2}ePZ-c_{2}\frac{TZ}{1+T^{2}}-c_{2}mZ \\ \leq &-c_{1}r_{1}P^{2}+c_{1}r_{1}P- \frac{r_{2}}{H}T^{2}+r_{2}T-c_{2}mZ-(c _{1}c-c_{2}e)PZ \end{aligned}$$

and

$$\begin{aligned} LW_{2} =&-r_{1}+r_{1}P+aT+cZ+ \frac{\sigma _{1}^{2}}{2}-r_{2}+\frac{r _{2}}{H}T+bP+ \frac{Z}{1+T^{2}}+\frac{\sigma _{2}^{2}}{2}-eP \\ &{}+\frac{T}{1+T^{2}}+m+\frac{\sigma _{3}^{2}}{2} \\ \leq &(r_{1}+b)P+ \biggl(a+\frac{r_{2}}{H} \biggr)T+(c+1)Z+ \biggl(-r_{1}-r_{2}+ \frac{1}{2}+m+ \frac{\sigma _{1}^{2}+\sigma _{2}^{2}+\sigma _{3}^{2}}{2} \biggr). \end{aligned}$$

Hence, we get

$$\begin{aligned} LW \leq &-c_{1}r_{1}P^{2}+ \bigl[(c_{1}+1)r_{1}+b \bigr]P-\frac{r_{2}}{H}T^{2}+ \biggl(r _{2}+a+\frac{r_{2}}{H} \biggr)T \\ &{}+(-c_{2}m+c+1)Z-(c_{1}c-c_{2}e)PZ- \biggl(r_{1}-\frac{\sigma _{1}^{2}}{2}+r _{2}- \frac{\sigma _{2}^{2}}{2}-\frac{1}{2}-m-\frac{\sigma _{3}^{2}}{2} \biggr). \end{aligned}$$

Choose \(c_{1}=\frac{e}{c}c_{2}\) and \(c_{2}=\frac{c+2}{m}\), then

$$\begin{aligned} \begin{aligned}[b] LW&\leq -\frac{ce+2e}{cm}r_{1}P^{2}+ \biggl(\frac{ce+cm+2e}{cm}r_{1}+b \biggr)P-\frac{r _{2}}{H}T^{2}+ \biggl(r_{2}+a+\frac{r_{2}}{H} \biggr)T-Z \\ &\quad {}- \biggl(r_{1}-\frac{\sigma _{1}^{2}}{2}+r_{2}- \frac{\sigma _{2}^{2}}{2}-\frac{1}{2}-m-\frac{\sigma _{3}^{2}}{2} \biggr). \end{aligned} \end{aligned}$$
(5.2)

Now, we define a closed set

$$\begin{aligned} D_{\epsilon }= \biggl\{ (P,T,Z)\in R_{+}^{3}: \epsilon \leq P \leq \frac{1}{ \epsilon }, \epsilon \leq T \leq \frac{1}{\epsilon }, \epsilon \leq Z \leq \frac{1}{\epsilon } \biggr\} , \end{aligned}$$
(5.3)

where \(0<\epsilon <1\) is a sufficiently small number such that

$$\begin{aligned}& \biggl(\frac{ce+cm+2e}{cm}r_{1}+b \biggr)\epsilon < \frac{1}{2} \biggl[M-\frac{(r_{2}+a+\frac{r _{2}}{H})^{2}H}{4r_{2}} \biggr], \end{aligned}$$
(5.4)
$$\begin{aligned}& \biggl(r_{2}+a+\frac{r_{2}}{H} \biggr)\epsilon < \frac{1}{2} \biggl[M-\frac{( \frac{ec+2e+cm}{cm}r_{1}+b)^{2}cm}{4(c+2)er_{1}} \biggr], \end{aligned}$$
(5.5)
$$\begin{aligned}& -\epsilon < \frac{1}{2}(M-M_{1}-M_{2}), \end{aligned}$$
(5.6)
$$\begin{aligned}& -\frac{ce+2e}{2cm}r_{1}\frac{1}{\epsilon ^{2}}+K_{4} \leq -1, \end{aligned}$$
(5.7)
$$\begin{aligned}& -\frac{r_{2}}{2H\epsilon ^{2}}+K_{5}\leq -1, \end{aligned}$$
(5.8)
$$\begin{aligned}& -\frac{1}{\epsilon }+K_{6}\leq -1. \end{aligned}$$
(5.9)

In addition, we denote

$$\begin{aligned}& D_{\epsilon }^{1}= \bigl\{ (P,T,Z)\in R_{+}^{3}:0< P< \epsilon \bigr\} ,\qquad D_{\epsilon }^{2}= \bigl\{ (P,T,Z)\in R_{+}^{3}:0< T< \epsilon \bigr\} , \\& D_{\epsilon }^{3}= \bigl\{ (P,T,Z)\in R_{+}^{3}:0< Z< \epsilon \bigr\} ,\qquad D_{\epsilon }^{4}= \biggl\{ (P,T,Z)\in R_{+}^{3}:P>\frac{1}{\epsilon } \biggr\} , \\& D_{\epsilon }^{5}= \biggl\{ (P,T,Z)\in R_{+}^{3}:T> \frac{1}{\epsilon } \biggr\} ,\qquad D _{\epsilon }^{6}= \biggl\{ (P,T,Z)\in R_{+}^{3}:Z>\frac{1}{\epsilon } \biggr\} . \end{aligned}$$

Then \(R_{+}^{3}\setminus U_{\epsilon }=D_{\epsilon }^{1}\cup D_{ \epsilon }^{2}\cup D_{\epsilon }^{3}\cup D_{\epsilon }^{4}\cup D_{ \epsilon }^{5}\cup D_{\epsilon }^{6}\) and on each \(D_{\epsilon }^{i}\) we have the following.

Case 1. On \(D_{\epsilon }^{1}\), we have \(0< P<\epsilon \),

$$\begin{aligned} LW \leq &-\frac{r_{2}}{H} \biggl(T-\frac{H(r_{2}+a+\frac{r_{2}}{H})}{2r_{2}} \biggr)^{2}+\frac{H(r _{2}+a+\frac{r_{2}}{H})^{2}}{4r_{2}}+ \biggl(\frac{ce+cm+2e}{cm}r_{1}+b \biggr) \epsilon -M \\ \leq & \biggl(\frac{H(r_{2}+a+\frac{r_{2}}{H})^{2}}{4r_{2}} \biggr)+ \biggl( \frac{ce+cm+2e}{cm}r_{1}+b \biggr)\epsilon -M \\ \leq &K_{1}, \end{aligned}$$

where \(K_{1}=\frac{1}{2}[ \frac{H(r_{2}+a+\frac{r_{2}}{H})^{2}}{4r_{2}}-M]<0\). In fact, it follows from (5.4) and Assumption 5.1 that \(K_{1}<0\).

Case 2. On \(D_{\epsilon }^{2}\), we have \(0< T<\epsilon \),

$$\begin{aligned} LW \leq &-\frac{ce+2e}{cm}r_{1} \biggl[P- \frac{(\frac{ec+cm+2e}{cm}r_{1}+b)cm}{2(c+2)er _{1}} \biggr]^{2}+\frac{(\frac{ec+cm+2e}{cm}r_{1}+b)^{2}cm}{4(c+2)er_{1}} \\ &{}-\frac{r_{2}}{H}T^{2}+ \biggl(r_{2}+a+ \frac{r_{2}}{H} \biggr)T-M \\ \leq &\frac{(\frac{ec+cm+2e}{cm}r_{1}+b)^{2}cm}{4(c+2)er_{1}}+ \biggl(r_{2}+a+\frac{r _{2}}{H} \biggr)\epsilon -M \\ \leq &K_{2}, \end{aligned}$$

where \(K_{2}=\frac{1}{2}[\frac{(\frac{ec+2e+cm}{cm}r_{1}+b)^{2}cm}{4(c+2)er _{1}}-M]<0\). In fact, it follows from (5.5) and Assumption 5.1 that \(K_{2}<0\).

Case 3. On \(D_{\epsilon }^{3}\), we have \(0< Z<\epsilon \),

$$\begin{aligned} LW \leq &M_{1}+M_{2}-\epsilon -M \\ \leq &K_{3}, \end{aligned}$$

where \(K_{3}=\frac{1}{2}(M_{1}+M_{2}-M)<0\). In fact, it follows from (5.6) and Assumption 5.1 that \(K_{3}<0\).

Case 4. On \(D_{\epsilon }^{4}\), we have \(P>\frac{1}{\epsilon }\),

$$\begin{aligned} LW \leq &-\frac{ce+2e}{cm}r_{1}P^{2}+ \biggl( \frac{ce+cm+2e}{cm}r_{1}+b \biggr)P-\frac{r _{2}}{H}T^{2}+ \biggl(r_{2}+a+\frac{r_{2}}{H} \biggr)T-Z-M \\ \leq &-\frac{ce+2e}{2cm}r_{1}\frac{1}{\epsilon ^{2}}+K_{4}, \end{aligned}$$

where \(K_{4}=\sup_{(P,T,Z)\in R_{+}^{3}}\{-\frac{ce+2e}{2cm}r_{1}P ^{2}+(\frac{ce+cm+2e}{cm}r_{1}+b)P-\frac{r_{2}}{H}T^{2}+(r_{2}+a+\frac{r _{2}}{H})T-Z-M\}\). In fact, it follows from (5.7) that \(LW\leq -1\).

Case 5. On \(D_{\epsilon }^{5}\), we have \(T>\frac{1}{\epsilon }\),

$$\begin{aligned} LW \leq &-\frac{r_{2}}{2H}T^{2}-\frac{ce+2e}{cm}r_{1}P^{2}+ \biggl( \frac{ce+cm+2e}{cm}r_{1}+b \biggr)P-\frac{r_{2}}{2H}T^{2} \\ &{}+ \biggl(r_{2}+a+\frac{r_{2}}{H} \biggr)T-Z-M \\ \leq &-\frac{r_{2}}{2H\epsilon ^{2}}+K_{5}, \end{aligned}$$

where \(K_{5}=\sup_{(P,T,Z)\in R_{+}^{3}}\{-\frac{ce+2e}{cm}r_{1}P^{2}+( \frac{ce+cm+2e}{cm}r_{1}+b)P-\frac{r_{2}}{2H}T^{2}+(r_{2}+a+\frac{r _{2}}{H})T-Z-M\}\). In fact, it follows from (5.8) that \(LW\leq -1\).

Case 6. On \(D_{\epsilon }^{5}\), we have \(Z>\frac{1}{\epsilon }\),

$$\begin{aligned} LW \leq &M_{1}+M_{2}-Z-M \\ \leq &-\frac{1}{\epsilon }+M_{1}+M_{2}-M \\ \leq &-\frac{1}{\epsilon }+K_{6}, \end{aligned}$$

where \(K_{6}=M_{1}+M_{2}-M\). In fact, it follows from (5.9) that \(LW\leq -1\).

In conclusion, we can obtain that \(LW\leq \min \{K_{1},K_{2},K_{3},-1 \}\), \((P,T,Z)\in R_{+}^{3}\setminus D_{\epsilon }\). Then, the conclusion follows from \(A_{2}\) of Lemma 5.1.

Thus, according to Lemma 5.1, the system (1.3) admits a unique ergodic stationary distribution \(\mu (\cdot )\). This completes the proof. □

Remark 5.1

From the expression of Π, we can see that, as \(\sigma _{i}\) gets smaller, Π gets bigger. When \(\sigma _{i}\) (\(i=1,2,3\)) is big enough, Π is less than 0. Hence, environmental noise plays an important role in population persistence.

6 Numerical analysis and discussions

In this paper, according to the fact that the environmental fluctuation has a great effect on the aquatic ecosystem, we propose and investigate a stochastic phytoplankton-toxin phytoplankton–zooplankton model. Without the effects of environmental fluctuations, Banerjee and Venturino [13] gave a comprehensive analysis and showed that the toxic phytoplankton does not drive the zooplankton population towards extinction. With the effects of environmental fluctuations, we have proved the existence of global positive solution of the investigated model and obtain the sufficient conditions for weakly persistence in the mean and extinction of the plankton. We also investigate the existence of ergodic stationary distribution by constructing a suitable Lyapunov function. The results show that a large environmental fluctuation may lead to the extinction of the population, and small environmental fluctuation can keep population weakly persistent in the mean. We also find that the noise-induced extinction of one phytoplankton population may lead to the density increase of the other phytoplankton population in two competitive phytoplankton populations. Thus noise-induced extinction of non-toxic-phytoplankton population may lead to an outburst of toxic-phytoplankton population. In the following, we will verify our main results with the help of a numerical method which is based on the work of [34, 35].

The following example concerns the extinction and weakly persistent in the mean of the populations.

Example 6.1

Let the parameters be \(r_{1}=0.6\), \(r_{2}=0.7\), \(a=0.1\), \(b=0.9\), \(c=1.35\), \(e=1.63\), \(m=0.3\), \(H=0.33\).

(i) Fix \(\sigma _{1}=0.1\) and \(\sigma _{3}=0.05\), let \(\sigma _{2}\) vary to see the effect of noise on the dynamics of model (1.3). We first take \(\sigma _{2}=0.1\), it is easy to compute that \(r_{1}-\frac{\sigma _{1} ^{2}}{2}=0.595>0\), \(r_{2}-\frac{\sigma _{2}^{2}}{2}=0.695>0\) and \(r_{1}(e-m)-(e\frac{\sigma _{1}^{2}}{2}+r_{1}\frac{\sigma _{3}^{2}}{2})=0.7891>0\), in view of (i) of Theorem 4.1, the populations \(P(t)\), \(T(t)\) and \(Z(t)\) are weakly persistent in the mean (see Fig. 1). Changing \(\sigma _{2}\) from \(\sigma _{2}=0.1\) to \(\sigma _{2}=1.2\), we obtain that \(r_{1}-\frac{\sigma _{1}^{2}}{2}=0.595>0\), \(r_{2}- \frac{\sigma _{2}^{2}}{2}=-0.02<0\) and \(r_{1}(e-m)-(e\frac{\sigma _{1} ^{2}}{2}+r_{1}\frac{\sigma _{3}^{2}}{2})=0.7891>0\) which satisfy the conditions (ii) of Theorem 4.1, the populations \(P(t)\) and \(Z(t)\) are weakly persistent in the mean and the population \(T(t)\) goes to extinction (see Fig. 2). From Fig. 1, we can find that phytoplankton, toxic-phytoplankton and zooplankton population reach the steady state value after some transient oscillations for the deterministic model, while all populations fluctuate around the steady state values of the corresponding deterministic models for a stochastic model. Figures 1 and 2 show that, under small noise, all populations preserve some stability and large noise may lead to the extinction of the toxic-phytoplankton population. The results also reflect that if the reproduction rate of the population is dominated by the environmental noise, the population will go to extinction. Otherwise, the population will be weakly persistent in the mean.

Figure 1
figure 1

The figure shows the sample path of \(P(t)\), \(T(t)\) and \(Z(t)\) for deterministic model (1.2) and corresponding stochastic model (1.3) with the initial value \((P(0),T(0),Z(0))=(0.6,0.3,0.5)\), and \(\sigma_{1}=0.1\), \(\sigma_{2}=0.1\) and \(\sigma_{3}=0.05\)

Figure 2
figure 2

The figure shows the sample path of \(P(t)\), \(T(t)\) and \(Z(t)\) for deterministic model (1.2) and corresponding stochastic model (1.3) with the initial value \((P(0),T(0),Z(0))=(0.6,0.3,0.5)\), and \(\sigma_{1}=0.1\), \(\sigma_{2}=1.2\) and \(\sigma_{3}=0.05\)

Next, we focus on Figs. 1 and 5, the populations \(P(t)\), \(T(t)\) and \(Z(t)\) are persistent for corresponding deterministic model of model (1.3). From Fig. 1, we can see that the populations \(P(t)\), \(T(t)\) and \(Z(t)\) are weakly persistent for stochastic model (1.3). However, if we only increase the values of \(\sigma _{i}\) (\(i=1,2,3\)) from 0.1, 0.1, 0.05 to 1.1, 1.2, 1, respectively, all the other parameters being the same as for Fig. 1 (see Fig. 5), we can find that all the populations \(P(t)\), \(T(t)\) and \(Z(t)\) go extinct, which furthermore indicates that a large fluctuation plus conditions that ensure the persistence of the deterministic model allow extinction for the solutions of the stochastic model.

(ii) Fix \(\sigma _{2}=0.1\) and \(\sigma _{3}=0.05\), letting \(\sigma _{1}\) vary from \(\sigma _{1}=0.1\) to \(\sigma _{1}=1.1\), when \(\sigma _{1}=0.1\), we check that \(r_{1}-\frac{\sigma _{1}^{2}}{2}=-0.005<0\), \(r_{2}-\frac{ \sigma _{2}^{2}}{2}=0.695>0\) and by (iii) of Theorem 4.1, both the populations \(P(t)\) and \(Z(t)\) are in extinction, the population \(T(t)\) is weakly persistent in the mean (see Fig. 3). The results imply that if zooplankton feeds enough on the toxin phytoplankton, it is driven to extinction. Figure 3 also shows that the noise-induced extinction of \(P(t)\) contributes the density increase of \(T(t)\), which fluctuates around a much higher density level than that of the deterministic counterpart. Thus a noise-induced extinction of nontoxic phytoplankton may lead to more serious outbreaks of the toxic phytoplankton. From Figs. 2 and 3, we also find the interesting phenomenon that the noise-induced extinction of one phytoplankton population may lead to a density increase of the other phytoplankton population in two competitive phytoplankton populations. Furthermore, if we fix \(\sigma _{1}=1.1\) and \(\sigma _{3}=0.05\) and change \(\sigma _{2}\) from \(\sigma _{2}=0.1\) to \(\sigma _{2}=1.2\), we can easily derive that \(r_{1}-\frac{\sigma _{1} ^{2}}{2}=-0.005<0\), \(r_{2}-\frac{\sigma _{2}^{2}}{2}=-0.02<0\); according to (iv) of Theorem 4.1, the populations \(P(t)\), \(T(t)\) and \(Z(t)\) are in extinction, which is shown in Fig. 4.

Figure 3
figure 3

The figure shows the sample path of \(P(t)\), \(T(t)\) and \(Z(t)\) for deterministic model (1.2) and corresponding stochastic model (1.3) with the initial value \((P(0),T(0),Z(0))=(0.6,0.3,0.5)\), and \(\sigma_{1}=1.1\), \(\sigma_{2}=0.1\) and \(\sigma_{3}=0.05\)

Figure 4
figure 4

The figure shows the sample path of \(P(t)\), \(T(t)\) and \(Z(t)\) for deterministic model (1.2) and corresponding stochastic model (1.3) with the initial value \((P(0),T(0),Z(0))=(0.6,0.3,0.5)\), and \(\sigma_{1}=1.1\), \(\sigma_{2}=1.2\) and \(\sigma_{3}=0.05\)

To further illustrate the existence of a stationary distribution, we consider the following example.

Example 6.2

We choose the parameters as \(r_{1}=0.6\), \(r_{2}=0.7\), \(a=0.1\), \(b=0.9\), \(c=1.35\), \(e=1.63\), \(m=0.3\), \(H=0.33\), \(\sigma _{1}=0.1\), \(\sigma _{2}=0.1\), \(\sigma _{3}=0.05\). By a simple computation, we obtain \(M=0.4887\), \(M_{1}=0.1626\), and \(M_{2}= 0.0874\). Furthermore, \(\varPi = 0.2387>0\). That is to say, the condition of Theorem 5.1 holds, for the system (1.3) exists a unique ergodic stationary distribution \(\mu (\cdot )\) (see Fig. 5).

Figure 5
figure 5

The figure shows the sample path of \(P(t)\), \(T(t)\) and \(Z(t)\) for deterministic model (1.2) and corresponding stochastic model (1.3) with the initial value \((P(0),T(0),Z(0))=(0.6,0.3,0.5)\), and \(\sigma_{1}=1.1\), \(\sigma_{2}=1.2\) and \(\sigma_{3}=1\)

Figure 6
figure 6

The figure shows the frequency histogram of \(P(t)\), \(T(t)\) and \(Z(t)\) for model (1.3) with the initial value \((P(0),T(0),Z(0))=(0.2,0.1,0.4)\), and \(\sigma_{1}=0.1\), \(\sigma_{2}=0.1\), \(\sigma_{3}=0.05\)