Algal competition in a water column with excessive dioxide in the atmosphere

Abstract

This paper deals with a resource competition model of two algal species in a water column with excessive dioxide in the atmosphere. First, the uniqueness of positive steady state solutions to the single-species model with two resources is established by the application of the degree theory and the strong maximum principle for the cooperative system. Second, some asymptotic behavior of the single-species model is given by comparison principle and uniform persistence theory. Third, the coexistence solutions to the competition system of two species with two substitutable resources are obtained by global bifurcation theory, various estimates and the strong maximum principle for the cooperative system. Numerical simulations are used to illustrate the outcomes of coexistence and competitive exclusion.

Appendix: Some mathematical proofs

1.1 The washout solution

The purpose of this subsection is to study the washout solution of (4)–(5). Noting that (8) is a cooperative system, we first introduce the maximum principle for cooperative weakly coupled elliptic systems, which is adapted from Amann (2004) and López-Gómez and Molina-Meyer (1994) and plays a crucial role throughout this paper. This maximum principle helps us to decouple the full elliptic system into some scalar systems or some lower dimensional systems.

Let $E_1=C^{2+\sigma }([0, 1], {\mathbb {R}}^2), E_2=C^{\sigma }([0, 1], {\mathbb {R}}^2)$, and ${\mathcal {L}}_0:E_1\rightarrow E_2$ be given by

$$\begin{aligned} {\mathcal {L}}_0=\left( \begin{array}{cc} D\frac{d^2}{dx^2}-\omega _r&{}\quad \omega _s\\ \omega _r&{}\quad D\frac{d^2}{dx^2}-\omega _s\end{array}\right) . \end{aligned}$$

Consider the eigenvalue problem

$$\begin{aligned}&-{\mathcal {L}}_0(\phi _1, \phi _2)^\intercal =\lambda (\phi _1, \phi _2)^\intercal ,\quad x\in (0, 1),\nonumber \\&-(\phi _1)_x(0)+\alpha \phi _1(0)=0, \phi _1(1)=0,\quad (\phi _2)_x(0)=0, \phi _2(1)=0. \end{aligned}$$

(12)

It follows from Theorem 2.6 or Remark 2.4 of López-Gómez and Molina-Meyer (1994) that the operator $(-{\mathcal {L}}_0)^{-1}:E_2\rightarrow E_1$ with the above boundary conditions is well defined and it is compact and strongly order preserving, and the principal eigenvalue of $-{\mathcal {L}}_0$, denoted by $\lambda _1(-{\mathcal {L}}_0)$, is strictly positive and has a positive eigenfunction. That is, $-{\mathcal {L}}_0$ subject to the boundary conditions: $-(\phi _1)_x(0)+\alpha \phi _1(0)=0, \phi _1(1)=0,\ (\phi _2)_x(0)=0, \phi _2(1)=0$ satisfies the strong maximum principle (cf. Theorem 13 of Amann 2004).

Proof of Theorem 2.1

The steady-state system corresponding to (8) is

$$\begin{aligned}&DR_{xx}+\omega _sS-\omega _rR=0,\quad x\in (0, 1),\nonumber \\&DS_{xx}-\omega _sS+\omega _rR=0, \quad x\in (0, 1),\nonumber \\&R_x(0)=\alpha (R(0)-\hat{R}),\quad R(1)=R^0,\quad S_x(0)=0,\quad S(1)=S^0. \end{aligned}$$

(13)

It follows from Theorem 13 of Amann (2004) or Theorem 2.6 and Remark 2.4 of López-Gómez and Molina-Meyer (1994) that the principle eigenvalue of $-{\mathcal {L}}_0$ is positive, and hence the strong maximum principle holds for (13). In view of Theorem 15 of Amann (2004), (13) has a unique solution, which is positive and denoted by $(R^*, S^*)$.

Note that the solutions of (8) generate a monotone semi-dynamical system on $C([0, 1],{\mathbb {R}}^2_+)$. Hence, (ii) is a direct result of (i) and Theorem 2.2.6 of Zhao (2003).

Proof of Theorem 2.2

The linearized eigenvalue problem of (6)–(7) with respect to the washout solution $(R^*, S^*, 0, 0)$ is

$$\begin{aligned}&D\phi _{1xx}+\omega _s\phi _2-\omega _r\phi _1-(f_1(R^*)-r_1)\psi _1-(f_2(R^*)-r_2)\psi _2=-\lambda \phi _1,\nonumber \\&D\phi _{2xx}-\omega _s\phi _2+\omega _r\phi _1-g_1(S^*)\psi _1-g_2(S^*)\psi _2=-\lambda \phi _2,\nonumber \\&D\psi _{1xx}-\nu _1 \psi _{1x}+[f_1(R^*)+g_1(S^*)-r_1-m_1]\psi _1=-\lambda \psi _1,\nonumber \\&D\psi _{2xx}-\nu _2 \psi _{2x}+[f_2(R^*)+g_2(S^*)-r_2-m_2]\psi _2=-\lambda \psi _2, \end{aligned}$$

(14)

with boundary conditions

$$\begin{aligned}&{-}\phi _{1x}(0)+\alpha \phi _1(0)=0,\quad \phi _1(1)=0,\nonumber \\&\phi _{2x}(0)=0,\quad \phi _2(1)=0,\nonumber \\&D\psi _{ix}(0)-\nu _i \psi _{i}(0)=0,\quad D\psi _{ix}(1)-\nu _i \psi _{i}(1)=0,\quad i=1, 2. \end{aligned}$$

(15)

Clearly, the eigenvalues of the linearized eigenvalue problem (14)–(15) consist of the eigenvalues of the following three operators: $-{\mathcal {L}}_0, L_1=-D\frac{d^2}{dx^2}+\nu _1\frac{d}{dx}-(f_1(R^*)+g_1(S^*)-r_1-m_1)$ and $L_2=-D\frac{d^2}{dx^2}+\nu _2\frac{d}{dx}-(f_2(R^*)+g_2(S^*)-r_2-m_2)$. Recall that the smallest eigenvalue $\lambda _1(-{\mathcal {L}}_0)>0$ of $-{\mathcal {L}}_0$. Meanwhile, the smallest eigenvalue of $L_i(i=1, 2)$ is larger than 0 provided $m_i>m_i^*$, and less than 0 provided $m_i<m_i^*$. Hence, all eigenvalues of the linearized eigenvalue problem (14)–(15) are larger than zero provided $m_1>m_1^*, m_2>m_2^*$, and the linearized eigenvalue problem (14)–(15) has an eigenvalue less than zero provided $m_1<m_1^*$ or $m_2<m_2^*$. That is, the washout equilibrium solution $(R^*, S^*, 0, 0)$ is linearly stable provided $m_1>m_1^*, m_2>m_2^*$; and unstable provided $m_1<m_1^*$ or $m_2<m_2^*$.

1.2 Single population system

The purpose of this subsection is to study the dynamical behavior of the single population system (9) and to establish Theorems 2.3–2.4.

First, we study the well-posedness of the initial boundary value problem (9). Let $X^+=C([0,1],{\mathbb {R}}^3_+)$ be the positive cone in the Banach space $X=C([0,1],{\mathbb {R}}^3)$ with the usual supremum norm. To simplify notations, we set

$$\begin{aligned} \Phi _1=R,\, \Phi _2=S,\, \Phi _3=e^{-\frac{\nu }{D}x}B\quad \text{ and }\quad \Phi =(\Phi _1, \Phi _2, \Phi _3). \end{aligned}$$

Note that the initial conditions in (9) satisfying $(\Phi _1^0, \Phi _2^0, \Phi _3^0)=(R_0(x), S_0(x), e^{-\frac{\nu }{D}x}B_0(x))\in X^+$. For the local existence and positivity of solutions, we appeal to the theory developed by Martin and Smith (1990) where existence, uniqueness and positivity are treated simultaneously. The idea is to view the system (9) as the abstract ordinary differential equation in $X^{+}$ and the so-called mild solutions can be obtained for any given initial data. More precisely,

$$\begin{aligned} \left( \begin{array}{c}\Phi _1(t)\\ \Phi _2(t)\end{array}\right)= & {} H_0(t)\cdot \left( \begin{array}{c}\Phi _1^0\\ \Phi _2^0\end{array}\right) +\int _0^tG_0(t-s)\cdot \left( \begin{array}{c}F_1(\Phi (s))\\ F_2(\Phi (s))\end{array}\right) \mathrm {d}s,\\ \Phi _3(t)= & {} G_1(t)\Phi _3^0+\int _0^t G_1(t-s)F_3(\Phi (s))\mathrm {d}s, \end{aligned}$$

where $G_0(t)$ is the positive, non-expansive, analytic semigroup on $C([0,1],{\mathbb {R}}^2)$ (see, e.g., Chapter 7 in the book by Smith 1995) such that $(u,v)^\intercal =G_0(t)\cdot (\Phi _1^0,\Phi _2^0)^\intercal$ satisfies the linear initial value problem

$$\begin{aligned}&\left( \begin{array}{c}u\\ v\end{array}\right) _t={\mathcal {L}}_0\left( \begin{array}{c}u\\ v\end{array}\right) ,\quad 0<x<1, t>0,\\&-u_x(0,t)+\alpha u(0,t)= 0,\quad u(1,t)=0,\quad v_x(0,t)= 0,\ v(1,t)=0,\quad t>0,\\&u(x,0)=\Phi _{1}^0(x),\quad v(x, 0)=\Phi _{2}^0(x),\quad 0\le x\le 1, \end{aligned}$$

and $G_1(t)$ is the positive, non-expansive, analytic semigroup on C[0, 1] (see, e.g., Chapter 7 in the book by Smith 1995) such that $u=G_1(t)\Phi _{3}^0$ satisfies the linear initial value problem

$$\begin{aligned} \begin{array}{ll} u_t=Du_{xx}+\nu u_x-(r+m)u,&{}\quad 0<x<1, t>0, \\ u_x(0,t)=0,\quad u_x(1,t)= 0,&{}\quad t>0,\\ u(x,0)=e^{-\frac{\nu }{D}x}\Phi _{3}^0(x), &{}\quad 0\le x\le 1. \end{array} \end{aligned}$$

$H_0(t), t>0$, is the family of affine operators on $C([0,1],{\mathbb {R}}^2)$ (see, e.g., Chapter 5 in the book by Pazy 1983) such that $(u,v)^\intercal =H_0(t)\cdot (\Phi _1^0,\Phi _2^0)^\intercal$ satisfies the linear systems with nonhomogeneous boundary condition given by

$$\begin{aligned}&\left( \begin{array}{c}u\\ v\end{array}\right) _t={\mathcal {L}}_0\left( \begin{array}{c}u\\ v\end{array}\right) ,\quad t>0, 0<x<1,\\&-u_x(0,t)+\alpha u(0, t)=\alpha \hat{R},\quad u(1,t)=R^0,\quad v_x(0,t)=0, \quad v(1,t)=S^0,\quad t>0,\\&u(x,0)=\Phi _1^0(x),\quad v(x,0)=\Phi _2^0(x),\quad 0\le x\le 1. \end{aligned}$$

The nonlinear operators $F_{i}:C([0,1],{\mathbb {R}}_{+})\rightarrow C[0,1]$ are defined by

$$\begin{aligned} F_{1}(\Phi )= & {} -(f(\Phi _1)-r)\Phi _3,\\ F_{2}(\Phi )= & {} -g(\Phi _2)\Phi _3,\\ F_{3}(\Phi )= & {} [f(\Phi _1)+g(\Phi _2)]\Phi _3. \end{aligned}$$

By the maximum principle arguments, it follows that

$$\begin{aligned} G_{0}(t)\cdot C\left( [0,1],{\mathbb {R}}_{+}^2\right)&\subset C\left( [0,1],{\mathbb {R}}_{+}^2\right) ,\quad \forall \ t>0,\\ H_0(t)\cdot C\left( [0,1],{\mathbb {R}}_{+}^2\right)&\subset C\left( [0,1],{\mathbb {R}}_{+}^2\right) ,\quad \forall \ t>0,\\ G_{1}(t)C([0,1],{\mathbb {R}}_{+})&\subset C([0,1],{\mathbb {R}}_{+}),\quad \forall \ t>0. \end{aligned}$$

Since $f(0)=0,g(0)=0$, it follows that $F_{i}(\Phi )\ge 0$ whenever $\Phi _{i}\equiv 0, \forall \ 1\le i \le 3$, and hence, $\mathbf {F}:=(F_1,F_2,F_3)$ is quasipositive (see, e.g., Remark 1.1 of Martin and Smith 1990). By Theorem 1 and Remark 1.1 of Martin and Smith (1990), it follows that the system (9) has a unique solution and the solutions to (9) remain non-negative on their interval of existence if they are non-negative initially. In other words, the following results hold:

Lemma 5.1

For every initial value function $\Phi ^0=(\Phi _1^0, \Phi _2^0, \Phi _3^0)\in X^{+}$, the system (9) has a unique mild solution $\Phi (x,t,\Phi ^0)$ on $(0,\delta _{\Phi ^0})$ with $\Phi (\cdot ,0,\Phi ^0)=\Phi ^0$, where $\delta _{\Phi ^0}\le \infty$. Furthermore, $\Phi (\cdot ,t,\Phi ^0)\in X^{+}, \forall \ t\in (0,\delta _{\Phi ^0})$ and $\Phi (x,t,\Phi ^0)$ is a classical solution of (9) for $t>0.$

Next, we are ready to show the solutions of (9) exist globally on $(0, +\infty )$ and converge to a compact attractor in $X^{+}$. At first, we show solutions are ultimately bounded and uniformly bounded in $X^{+}$.

Lemma 5.2

Suppose $f(R^*)>r$ and $m>0$. Then for every initial value function $\Phi ^0=(\Phi _1^0, \Phi _2^0, \Phi _3^0)\in X^{+}$, the system (9) has a unique solution $\Phi (x,t,\Phi ^0)$ on $[0,\infty )$ with $\Phi (\cdot ,0,\Phi ^0)=\Phi ^0$, and the solutions of (9) are ultimately bounded and uniformly bounded in $X^{+}$.

Proof

Let $W=e^{-\frac{\nu }{D}x}B$. Then (9) becomes

$$\begin{aligned}&R_t=DR_{xx}+\omega _sS-\omega _rR-(f(R)-r)e^{\frac{\nu }{D}x}W,\quad x\in (0, 1),t>0,\nonumber \\&S_t=DS_{xx}-\omega _sS+\omega _rR-g(S)e^{\frac{\nu }{D}x}W,\quad x\in (0, 1),t>0,\nonumber \\&e^{\frac{\nu }{D}x}W_t=D\left( e^{\frac{\nu }{D}x}W_x\right) _x+(f(R)+g(S)-r-m)e^{\frac{\nu }{D}x}W, \quad x\in (0, 1),t>0,\nonumber \\&R_x(0, t)=\alpha (R(0, t)-\hat{R}),\quad R(1, t)=R^0,\quad t>0,\nonumber \\&S_x(0, t)=0,\quad S(1, t)=S^0,\quad W_x(0, t)=W_x(1, t)=0,\quad t>0,\nonumber \\&R(x, 0)=R_0(x)\ge 0,\quad S(x, 0)=S_0(x)\ge 0,\quad W(x, 0)=e^{-\frac{\nu }{D}x}B_0(x)\ge 0,\not \equiv 0. \end{aligned}$$

(16)

By Lemma 5.1, any solution (R, S, W) to (16) satisfies $R(x, t)>0, S(x, t)>0, W(x, t)>0$. Note that there exists a constant $\rho >1$ large enough such that $R_0(x)\le \rho R^*, S_0(x)\le \rho S^*$. For given $W(x,t)\ge 0$, consider the following system

$$\begin{aligned}&R_t=DR_{xx}+\omega _sS-\omega _rR-(f(R)-r)e^{\frac{\nu }{D}x}W, \quad x\in (0, 1),t>0,\nonumber \\&S_t=DS_{xx}-\omega _sS+\omega _rR-g(S)e^{\frac{\nu }{D}x}W, \quad x\in (0, 1),t>0,\nonumber \\&R_x(0, t)=\alpha (R(0, t)-\hat{R}),\quad R(1, t)=R^0,\quad t>0,\nonumber \\&S_x(0, t)=0, \quad S(1, t)=S^0, \quad t>0,\nonumber \\&R(x, 0)=R_0(x)\ge 0,\quad S(x, 0)=S_0(x)\ge 0. \end{aligned}$$

(17)

Clearly, (0, 0) and $\rho (R^*, S^*)$ are the ordered lower and upper solutions of (17) by Definition 8.1.2 in the book by Pao (1992). It follows from the iteration process of Chapter 8.2 in the book by Pao (1992) and Theorem 8.3.1 in the book by Pao (1992) that (17) has a unique solution (R(x, t), S(x, t)) satisfies $0\le R(x,t)\le \rho R^*, 0\le S(x,t)\le \rho S^*$ for all $x\in [0,1],t>0.$ Namely, $\Lambda =\{(R, S):0\le R\le \rho R^*, 0\le S\le \rho S^*\}$ is an invariant set (cf. Definition 5.4.1 in the book by Pao 1992) of the system (17), which implies R(x, t), S(x, t) are ultimately bounded and uniformly bounded in $X^{+}$.

Next, we claim W(1, t) is bounded in $t\in (0, +\infty )$. If not, we can find $t_n\rightarrow \infty$ such that $W(1, t_n)\rightarrow \infty$ as $t_n\rightarrow \infty$. Let $\widehat{W}_n(x, t)=\frac{W(x, t+t_n)}{W(1, t_n)}$. Then $\widehat{W}_n(x, t)$ satisfies

$$\begin{aligned} \left( e^{\frac{\nu }{D}x}\widehat{W}_n\right) _t&=D\left( e^{\frac{\nu }{D}x}\left( \widehat{W}_n\right) _x\right) _x+(f(R(x, t+t_n))+g(S(x, t+t_n))-r-m)e^{\frac{\nu }{D}x}\widehat{W}_n,\\ \left( \widehat{W}_n\right) _x(0, t)&=\left( \widehat{W}_n\right) _x(1, t)=0, \\ \widehat{W}_n(x, 0)&\ge 0, \quad \widehat{W}_n(1, 0)=1. \end{aligned}$$

Note that $|f(R(x, t+t_n))+g(S(x, t+t_n))-r-m|\le |f(\rho R^*)+g(\rho S^*)-r-m|$ is bounded. It follows from Lemma 5.12 that $\widehat{W}_n(x, t)>\delta >0$ for all $x\in [0,1]$ and $t>0$, which implies $W(x, t+t_n)>\delta W(1, t_n)$. Thus, $W(x, t)\rightarrow \infty$ as $t\rightarrow \infty$ uniformly in [0, 1]. Hence, for any $M>0$, there exists $t_0>0$ large enough such that $W(x, t)>M$ on [0, 1] for $t\ge t_0,$ which implies

$$\begin{aligned} S_t\le DS_{xx}-\omega _sS-g(S)e^{\frac{\nu }{D}x}M+\omega _r\rho R^*. \end{aligned}$$

Namely, S(x, t) is a lower solution of the parabolic problem

$$\begin{aligned} \begin{array}{ll} \tilde{S}_t=D\tilde{S}_{xx}-\omega _s\tilde{S}-g(\tilde{S})e^{\frac{\nu }{D}x}M+\omega _r\rho R^*, &{}\quad x\in (0, 1), t>t_0,\\ \tilde{S}_x(0, t)=0,\quad \tilde{S}(1, t)=S^0, &{}\quad t>t_0,\\ \tilde{S}(x, t_0)=S(x, t_0), &{}\quad x\in [0, 1]. \end{array} \end{aligned}$$

(18)

It follows from the comparison principle for parabolic equation that $S(x, t)\le \tilde{S}(x, t)$ for $t\ge t_0$. Note that the steady state system of (18) satisfies

$$\begin{aligned}&-D\tilde{S}_{xx}+\left( \omega _s+\int _0^1g'(\tau \tilde{S})d\tau e^{\frac{\nu }{D}x}M\right) \tilde{S}=\omega _r\rho R^*>0,\quad x\in (0, 1),\nonumber \\&\quad \tilde{S}_x(0)=0,\quad \tilde{S}(1)=S^0. \end{aligned}$$

(19)

It follows from the maximum principle that the steady state solution $\tilde{S}>0$ on [0, 1]. Let $\chi =S^0-\tilde{S}$. Then

$$\begin{aligned}&D\chi _{xx}-\omega _s\chi +g\left( S^0-\chi \right) e^{\frac{\nu }{D}x}M+\omega _sS^0-\omega _r\rho R^*=0,\quad x\in (0, 1),\nonumber \\&\chi _x(0)=0,\quad \chi (1)=0. \end{aligned}$$

(20)

Suppose $\inf _{[0, 1]}\chi =\chi (x_0)<0$. Then $-\omega _s\chi (x_0)+g(S^0-\chi (x_0))e^{\frac{\nu }{D}x_0}M+\omega _sS^0-\omega _r\rho R^*(x_0)\le 0$, which implies $g(S^0)M+\omega _sS^0<\omega _r\rho \max _{[0,1]}R^*$. Choosing M such that $\omega _sS^0+g(S^0)M\ge \omega _r\rho \max _{[0,1]}R^*$, we get a contradiction. Hence, $\chi \ge 0$ on [0, 1], which means $0<\tilde{S}\le S^0$ on [0, 1]. Moreover, in view of $\omega _sS^0+g(S^0)M\ge \omega _r\rho \max _{[0,1]}R^*$, it is easy to see that 0 is a strictly lower solution to the steady state system of (18), and $S^0$ is a strictly upper solution to the steady state system of (18). It follows from monotone iteration process that there exists a pair $\tilde{S}^+$ and $\tilde{S}^-$, which are the maximal and minimal solutions to the steady state system of (18), and satisfy the relation $0<\tilde{S}^-\le \tilde{S}^+\le S^0.$ Next, we show $\tilde{S}^-\equiv \tilde{S}^+.$ Obviously,

$$\begin{aligned} D(\tilde{S}^+-\tilde{S}^-)_{xx}-\omega _s (\tilde{S}^+-\tilde{S}^-)-(g(\tilde{S}^+)-g(\tilde{S}^-))e^{\frac{\nu }{D}x}M=0. \end{aligned}$$

Integrating over [0, x], and integrating over [0, 1] again, we have

$$\begin{aligned}&-D(\tilde{S}^+(0)-\tilde{S}^-(0))-\omega _s\int _0^1\int _0^x(\tilde{S}^+(\xi )-\tilde{S}^-(\xi ))\mathrm {d}\xi \mathrm {d}x -M\int _0^1\int _0^x(g(\tilde{S}^+(\xi ))\\&\quad -g(\tilde{S}^-(\xi )))e^{\frac{\nu }{D}\xi }M\mathrm {d}\xi \mathrm {d}x=0. \end{aligned}$$

Noting that $\tilde{S}^+\ge \tilde{S}^-$ on [0, 1], and g(S) is strictly increasing with respect to S, we must have $\tilde{S}^+\equiv \tilde{S}^-$. Hence, if $\omega _sS^0+g(S^0)M\ge \omega _r\rho \max _{[0,1]}R^*$, (18) has a unique positive steady state solution, denoted by $\tilde{S}^*_M(x)$, which satisfies $0<\tilde{S}^*_M(x)\le S^0$.

Meanwhile, it follows from (20) that $\chi =a\int _0^1G(x, \xi )(g(\tilde{S}(\xi ))e^{\frac{\nu }{D}\xi }M+\omega _sS^0-\omega _r\rho R^*(\xi ))\mathrm {d}\xi$, where $G(x,\xi )$ is the Green’s function corresponding to

$$\begin{aligned} -DG_{xx}+\omega _sG=\delta (x-\xi ),\quad x\in (0, 1),\quad G_x(0)=0,\quad G(1)=0. \end{aligned}$$

Letting $M\rightarrow \infty$, we obtain $g(\tilde{S}(\xi ))\rightarrow 0$ in (0, 1), that is, $\lim _{M\rightarrow \infty }g(\tilde{S}^*_M(x))=0$ in (0, 1).

On the other hand, it is easy to see that the solutions of (18) generate a monotone semi-dynamical system on $C([0,1],{\mathbb {R}}_{+})$. Hence, it follows from Theorem 2.2.6 of Zhao (2003) that the solution $\tilde{S}(x,t)$ of (18) converges to the unique positive steady-state solution $\tilde{S}^*_M(x)$ of (19) as $t\rightarrow \infty$, and $g(\tilde{S}^*_M(x))\rightarrow 0$ in (0, 1) as $M\rightarrow \infty$. Hence, for any $\epsilon >0$, there exist $M_1>0$ and $t_1>t_0$ large enough such that for $M\ge M_1$ and $t\ge t_1$, we have $0<g(S(x,t))\le g(\tilde{S}(x, t))<\epsilon$ in (0, 1). Similarly, we can show that for any $\epsilon >0$, there exist $M_2>0$ and $t_2>0$ large enough such that for $M\ge M_2$ and $t\ge t_2$, we have $f(R)-r<\epsilon$ in (0, 1). Take $M_0=\max \{M_1,M_2\}$ and $T_0=\max \{t_1, t_2\}$. Thus for $M\ge M_0, t\ge T_0$ and $m\ge \delta _0,$ we have $f(S)+g(S)-r-m<2\epsilon -\delta _0<0$ in (0, 1) as long as $0<\epsilon <\delta _0/2$. Take $\epsilon =\frac{\delta _0}{4}$. Then $e^{\frac{\nu }{D}x}W_t\le D(e^{\frac{\nu }{D}x}W_x)_x-\frac{\delta _0}{2}e^{\frac{\nu _1}{D}x}W$ for $M\ge M_0, t\ge T_0$ and $m\ge \delta _0$, which implies $W(x,t)\rightarrow 0$ in (0,1) as $t\rightarrow \infty$, a contradiction. Hence, W(1, t) is bounded in $t\in (0, +\infty )$.

Let $\phi$ be the principal eigenfunction of

$$\begin{aligned} -\phi _{xx}=\mu \phi ,\quad x\in (0, 1),\quad \phi _x(0)=0, \phi (1)=0. \end{aligned}$$

Then the principal eigenvalue $\mu _1=\frac{\pi ^2}{4}$, and the associated eigenfunction $\phi =\cos (\frac{\pi }{2}x)$, and $\phi (0)=1, \phi _x(1)=-\frac{\pi }{2}$. Let $Q(x,t)=R+S+e^{\frac{\nu }{D}x}W$. By direct computation, we obtain

$$\begin{aligned}&\frac{\mathrm {d}}{\mathrm {d}t}\int _0^1Q(x,t)\phi \mathrm {d}x\\&\quad =D\int _0^1\left[ R_{xx}\phi +S_{xx}\phi +\left( e^{\frac{\nu }{D}x}W_x\right) _x\phi \mathrm {d}x\right] -m\int _0^1e^{\frac{\nu }{D}x}W\phi \mathrm {d}x\\&\quad =D\left( \alpha \left( \hat{R}-R(0, t)\right) +\frac{\pi }{2}R^0+\frac{\pi }{2}S^0\right) -\frac{\pi ^2}{4}D\int _0^1(R+S)\phi \mathrm {d}x\\&\qquad +D\int _0^1\left( e^{\frac{\nu }{D}x}W_x\right) _x\phi \mathrm {d}x-m\int _0^1e^{\frac{\nu }{D}x}W\phi \mathrm {d}x, \end{aligned}$$

where

$$\begin{aligned}&D\int _0^1\left( e^{\frac{\nu }{D}x}W_x\right) _x\phi \mathrm {d}x=-D\int _0^1\phi _x e^{\frac{\nu }{D}x}W_x\mathrm {d}x\\&\quad =-D\int _0^1\phi _x\mathrm {d}\left( e^{\frac{\nu }{D}x}W_x-\int _0^x\frac{\nu }{D}W(\xi ,t)e^{\frac{\nu }{D}\xi }\mathrm {d}\xi \right) \\&\quad =\frac{\pi }{2}De^{\frac{\nu }{D}}W(1,t)-\frac{\pi }{2}\nu \int _0^1We^{\frac{\nu }{D}x}\mathrm {d}x -\frac{\pi ^2}{4}D\int _0^1We^{\frac{\nu }{D}x}\phi \mathrm {d}x\\&\qquad +\frac{\pi ^2}{4}\nu \int _0^1\left( \int _0^xW(\xi ,t)e^{\frac{\nu }{D}\xi }\mathrm {d}\xi \right) \phi \mathrm {d}x. \end{aligned}$$

Note that $\int _0^1\phi \mathrm {d}x=\frac{2}{\pi }$, and

$$\begin{aligned}&\frac{\pi ^2}{4}\nu \int _0^1\left( \int _0^xW(\xi ,t)e^{\frac{\nu }{D}\xi }\mathrm {d}\xi \right) \phi \mathrm {d}x -\frac{\pi }{2}\nu \int _0^1We^{\frac{\nu }{D}x}\mathrm {d}x\\&\quad \le \frac{\pi ^2}{4}\nu \int _0^1W(\xi ,t)e^{\frac{\nu }{D}x}\mathrm {d}x\int _0^1\phi \mathrm {d}x -\frac{\pi }{2}\nu \int _0^1We^{\frac{\nu }{D}x}\mathrm {d}x\\&\quad =0. \end{aligned}$$

Hence,

$$\begin{aligned}&\frac{\mathrm {d}}{\mathrm {d}t}\int _0^1Q(x,t)\phi \mathrm {d}x+\frac{\pi ^2}{4}D\int _0^1Q\phi \mathrm {d}x \le D\left( \alpha \hat{R}+\frac{\pi }{2}R^0+\frac{\pi }{2}S^0\right) \\&\quad +\frac{\pi }{2}De^{\frac{\nu }{D}}W(1,t) -m\int _0^1e^{\frac{\nu }{D}x}W\phi \mathrm {d}x, \end{aligned}$$

that is,

$$\begin{aligned}&\frac{\mathrm {d}}{\mathrm {d}t}\left( e^{\frac{\pi ^2}{4}Dt}\int _0^1Q(x,t)\phi \mathrm {d}x\right) \le D\left( \alpha \hat{R}+\frac{\pi }{2}R^0+\frac{\pi }{2}S^0\right) e^{\frac{\pi ^2}{4}Dt}\\&\quad +\frac{\pi }{2}De^{\frac{\nu }{D}}W(1,t)e^{\frac{\pi ^2}{4}Dt}-me^{\frac{\pi ^2}{4}Dt}\int _0^1e^{\frac{\nu }{D}x}W\phi \mathrm {d}x. \end{aligned}$$

Since W(1, t) is bounded, by Gronwall inequality we get

$$\begin{aligned} \int _0^1Q(x,t)\phi \mathrm {d}x&\le e^{-\frac{\pi ^2}{4}Dt}\int _0^1Q(x,0)\phi \mathrm {d}x+D\left( \alpha \hat{R}+\frac{\pi }{2}R^0+\frac{\pi }{2}S^0\right) \nonumber \\&\quad \times \frac{4}{\pi ^2D}\left( 1-e^{-\frac{\pi ^2}{4}Dt}\right) \nonumber \\&\quad +\frac{\pi }{2}De^{\frac{\nu }{D}}\int _0^tW(1,\tau )e^{-\frac{\pi ^2}{4}D(t-\tau )}\mathrm {d}\tau \nonumber \\&\quad -m\int _0^t\left( \int _0^1e^{\frac{\nu }{D}x}W(x,\tau )\phi \mathrm {d}x\right) e^{-\frac{\pi ^2}{4}D(t-\tau )}\mathrm {d}\tau \nonumber \\&\le e^{-\frac{\pi ^2}{4}Dt}\int _0^1Q(x,0)\phi \mathrm {d}x+D\left( \alpha \hat{R}+\frac{\pi }{2}R^0+\frac{\pi }{2}S^0\right) \nonumber \\&\quad \times \frac{4}{\pi ^2D}\left( 1-e^{-\frac{\pi ^2}{4}Dt}\right) \nonumber \\&\quad +\frac{\pi }{2}De^{\frac{\nu }{D}}\int _0^tW(1,\tau )e^{-\frac{\pi ^2}{4}D(t-\tau )}\mathrm {d}\tau \nonumber \\&\le e^{-\frac{\pi ^2}{4}Dt}\int _0^1Q(x,0)\phi \mathrm {d}x+D\left( \alpha \hat{R}+\frac{\pi }{2}R^0+\frac{\pi }{2}S^0\right) \nonumber \\&\quad \times \frac{4}{\pi ^2D}\left( 1-e^{-\frac{\pi ^2}{4}Dt}\right) \nonumber \\&\quad +\frac{2}{\pi }De^{\frac{\nu }{D}}C\left( 1-e^{-\frac{\pi ^2}{4}Dt}\right) \end{aligned}$$

(21)

Next, we show W(x, t) is bounded for all $x\in [0, 1]$ and $t>0.$ Let $\mathbf {W}(t)=\max _{x\in [0, 1],\tau \in [0,t]}W(x,\tau ).$ Clearly, $\mathbf {W}(t)$ is nondecreasing. Suppose for contradiction that $\mathbf {W}(t)\rightarrow \infty$ as $t\rightarrow \infty$. Then we can find $t_n\rightarrow \infty$ such that $\mathbf {W}(t_n)=\max _{x\in [0, 1]}W(x,t_n).$ We may assume that $t_n>1$ for all $n\ge 1.$ Define $\widetilde{W}_n(x, t)=\frac{W(x, t+t_n-1)}{\mathbf {W}(t_n)}$. Then $\widetilde{W}_n(x, t)$ satisfies

$$\begin{aligned}&\left( e^{\frac{\nu }{D}x}\widetilde{W}_n\right) _t=D\left( e^{\frac{\nu }{D}x}\left( \widetilde{W}_n\right) _x\right) _x+(f(R(x, t+t_n-1))\\&\qquad +g(S(x, t+t_n-1))-r-m)e^{\frac{\nu }{D}x}\widetilde{W}_n,\\&\left( \widetilde{W}_n\right) _x(0, t)=\left( \widetilde{W}_n\right) _x(1, t)=0, \\&0\le \widetilde{W}_n(x, 0)\le 1. \end{aligned}$$

Noting that $|f(R(x, t+t_n-1))+g(S(x, t+t_n-1))-r-m|\le |f(\rho R^*)+g(\rho S^*)-r-m|:=\Lambda _0$, the comparison principle for parabolic system leads to $0\le \widetilde{W}_n(x, t)\le e^{\Lambda _0t}$ for $x\in [0,1]$ and $t\ge 0.$ Hence by the application of standard parabolic regularity, we can conclude that $\{\widetilde{W}_n\}$ is bounded in $C^{1+\gamma , \gamma }([0,1]\times \left[ \frac{1}{2}, 2\right] )$ for any $\gamma \in (0,1)$. Hence, by passing to a subsequence if necessary we get $\widetilde{W}_n(x, t)\rightarrow \widetilde{W}$ in $C^{1, 0}([0,1]\times \left[ \frac{1}{2}, 2\right] )$. Since $|f(R(x, t+t_n-1))+g(S(x, t+t_n-1))-r-m|\le \Lambda _0$, we may assume that $f(R(x, t+t_n-1))+g(S(x, t+t_n-1))-r-m\rightarrow h(x,t)$ weakly in $L^2([0,1]\times \left[ \frac{1}{2}, 2\right] )$ by passing to a further subsequence if necessary. Moreover, $|h(x,t)|\le \Lambda _0$, and $\widetilde{W}$ is a weak solution to

$$\begin{aligned}&\left( e^{\frac{\nu }{D}x}\widetilde{W}\right) _t=D\left( e^{\frac{\nu }{D}x}\widetilde{W}_x\right) _x+h(x,t)e^{\frac{\nu }{D}x}\widetilde{W}, \quad x\in (0, 1), t\in \left[ \frac{1}{2}, 2\right] ,\\&\widetilde{W}_x(0, t)=\widetilde{W}_x(1, t)=0,\quad t\in \left[ \frac{1}{2}, 2\right] ,\\&0\le \widetilde{W}(x, t)\le e^{\Lambda _0t},\quad x\in [0, 1], t\in \left[ \frac{1}{2}, 2\right] . \end{aligned}$$

It follows from $\max _{x\in [0,1]}\widetilde{W}_n(x, 1)=1$ that $\max _{x\in [0,1]}\widetilde{W}(x, 1)=1$, which implies $\widetilde{W}\not \equiv 0.$ By the strong maximum principle, we deduce that $\widetilde{W}(x, 1)\ge \delta _1>0$ in [0,1]. Hence, $\widetilde{W}_n(x, 1)\ge \frac{\delta _1}{2}$ for all large n and $x\in [0,1]$, which leads to

$$\begin{aligned} W(x, t_n)=\widetilde{W}_n(x, 1)\mathbf {W}(t_n)\ge \frac{\delta _1}{2}\mathbf {W}(t_n)\quad \text{ for } \text{ all } \text{ large } n \text{ and } x\in [0,1]. \end{aligned}$$

It follows that

$$\begin{aligned} \int _0^1Q(x,t_n)\phi \mathrm {d}x>\int _0^1W(x, t_n)\phi \mathrm {d}x\ge \int _0^1\frac{\delta _1}{2}\mathbf {W}(t_n)\phi \mathrm {d}x\rightarrow \infty \end{aligned}$$

as $n\rightarrow \infty$, a contradiction to (21). That is, W(x, t) is bounded for all $x\in [0, 1]$ and $t>0.$ In view of $W(x, t)>0$ for all $x\in [0,1]$ and $t>0$, we obtain that there exists a positive constant $C_1>0$ such that $0<B(x,t)\le C_1$ for all $x\in [0,1]$ and $t>0$. Namely, B(x, t) is ultimately bounded and uniformly bounded in $X^+$. The proof is completed. $\square$

Next, we derive a priori estimates for positive solutions of the steady-state system (10)–(11).

Lemma 5.3

Suppose $f(R^*)>r$ and (R, S, B) is a nonnegative solution of (10)–(11) with $B\not \equiv 0$. Then

• (i) $0<R<R^*,\, 0<S<S^*,\, B>0$ in (0, 1);

• (ii) $0<m<m^*$, where $m^*=-\lambda _1(-f(R^*)-g(S^*), \nu )-r$;

• (iii) for any given $\delta _0>0$, there exists a positive constant $M_0(\delta _0)$ such that $\Vert B\Vert _\infty \le M_0$ provided that $m\in [\delta _0, m^*)$.

Proof

(i) Note that

$$\begin{aligned}&-DS_{xx}+\left[ \omega _s+B\int _0^1g'(\tau S)d\tau \right] S=\omega _rR\ge 0,\quad x\in (0, 1),\\&-DR_{xx}+\left[ \omega _r+B\int _0^1f'(\tau R)d\tau \right] R=\omega _sS+rB\ge 0,\quad x\in (0, 1),\\&S_x(0)=0,\quad S(1)=S^0>0,\quad -R_x(0)+\alpha R(0)=\alpha \hat{R}>0,\quad R(1)=R^0>0. \end{aligned}$$

By the strong maximum principle, it is easy to see that $R>0,\ S>0$ on [0, 1]. Let $W=e^{-\frac{\nu }{D}x}B$. Then

$$\begin{aligned}&-DW_{xx}-\nu W_x+(r+m)W=[f(R)+g(S)]W\ge 0,\not \equiv 0,\quad x\in (0, 1),\\&\quad W_x(0)=0, \quad W_x(1)=0. \end{aligned}$$

It follows from the strong maximum principle that $W>0$ on [0, 1], and hence $B>0$ on [0, 1].

Now, we begin to prove $R<R^*, S<S^*$. To this end, let $U=R^*-R,V=S^*-S$. Then $U<R^*, V<S^*$, and

$$\begin{aligned}&-DU_{xx}+\omega _rU-\omega _sV=(f(R^*-U)-r)B,\quad x\in (0, 1),\nonumber \\&-DV_{xx}+\omega _sV-\omega _rU=g(S^*-V)B,\quad x\in (0, 1),\nonumber \\&-U_x(0)+\alpha U(0)=0,\quad U(1)=0,\quad V_x(0)=0,\quad V(1)=0. \end{aligned}$$

(22)

At first, it is easy to see that there exists some $x_0\in (0, 1)$ such that $U(x_0)>0$. Otherwise, $U(x)\le 0$ on [0, 1]. Then $-{\mathcal {L}}_0(U, V)^T>0$ based on $f(R^*)>r$ and $V<S^*, B>0.$ Hence, we have $(U, V)>0$ on [0, 1] by using Theorem 15 of Amann (2004), a contradiction. Noting that $U(1)=0, V(1)=0$ and $f(R^*)>r$, one can show that there exists $\epsilon >0$ small enough such that $U_{xx}<0$ and $V_{xx}<0$ for any $x\in (1-\epsilon , 1)$. Furthermore, we claim that $U_x(1)<0, V_x(1)<0.$ If not, we have $U_x(1)\ge 0$ or $V_x(1)\ge 0$. Thus we have three cases: (1) $U_x(1)\ge 0, V_x(1)\ge 0$; (2) $U_x(1)\ge 0, V_x(1)<0$; (3) $U_x(1)<0, V_x(1)\ge 0$.

Case (1): Suppose $U_x(1)\ge 0, V_x(1)\ge 0$. Since $U(1)=0$ and $U_{xx}<0$ for any $x\in (1-\epsilon , 1)$, we have $U(1-\epsilon )<0$. Let

$$\begin{aligned} x_1=\sup \{x\in (0, 1)|U_x(x)=0, U(x)<0\}. \end{aligned}$$

Then $0<x_1<1$ because there exists some $x_0\in (0, 1)$ such that $U(x_0)>0$. Moreover, $U(x)<0$ for $x\in [x_1, 1), U_x(x_1)=0$ and $U_{xx}(x_1)\ge 0$. It follows from the first equation of (22) that

$$\begin{aligned} \omega _sV(x_1)=-DU_{xx}(x_1)+\omega _rU(x_1)-(f(R^*(x_1)-U(x_1))-r)B(x_1)<0, \end{aligned}$$

which leads to $V(x_1)<0$. Adding the equations for U and V, we obtain

$$\begin{aligned} -D(U+V)_{xx}=(f(R^*-U)-r)B+g(S^*-V)B>0 \end{aligned}$$

for any $x\in [x_1, 1)$ since $f(R^*)>r, U<0$ in $[x_1, 1)$ and $V<S^*, B>0$ in [0, 1]. Hence, $V_{xx}(x_1)<0$. Integrating this equation over $[x_1, 1]$, we obtain $V_x(x_1)>U_x(1)+V_x(1)\ge 0.$ Define

$$\begin{aligned} y_1=\sup \{x\in (0, x_1)|V_x(x)=0, V(x)<0\}. \end{aligned}$$

Noting that $V(x_1)<0, V_x(x_1)>0$ and $V_{xx}(x_1)<0$, we can show that $0\le y_1<x_1, v(y)<0$ on $[y_1, x_1], V_x(y_1)=0, V_x\ge 0$ on $[y_1, x_1]$, and $V_{xx}(y_1)\ge 0$. It follows from the second equation of (22) that

$$\begin{aligned} \omega _rU(y_1)=-DV_{xx}(y_1)+\omega _sV(y_1)-g(S^*(y_1)-V(y_1))B(y_1)<0, \end{aligned}$$

which leads to $U(y_1)<0$. If $y_1=0$. Then $U(0)<0$ and $U_x(0)=\alpha U(0)<0$. Let $z_1=\inf \{x\in (0, x_1)|U(x)=0\}.$ Then $0<z_1<1$ since there exists some $x_0\in (0, 1)$ such that $U(x_0)>0$. Moreover, we have $U(x)<0$ in $(0, z_1)$ and $U_x(z_1)\ge 0$. Now, consider the following elliptic problem on $[0, z_1]$

$$\begin{aligned}&-DU_{xx}+\omega _rU-\omega _sV=(f(R^*-U)-r)B>0,\quad x\in (0, z_1),\nonumber \\&-DV_{xx}+\omega _sV-\omega _rU=g(S^*-V)B>0,\quad x\in (0, z_1),\nonumber \\&-U_x(0)+\alpha U(0)=0,\quad U_x(z_1)\ge 0,\quad V_x(0)=0,\ V_x(z_1)\ge 0. \end{aligned}$$

It follows from Theorem 15 of Amann (2004) that $(U, V)>0$ on $[0, z_1]$, a contradiction. Hence $0<y_1<x_1.$

Now, we have two cases: (a) $U(x)<0$ on $[y_1, 1]$; (b) $U(x_0)>0$ for some $x_0\in (y_1, x_1)$. If $U(x)<0$ on $[y_1, 1]$, by adding the equations for U and V, we obtain $-D(U+V)_{xx}=(f(R^*-U)-r)B+g(S^*-V)B>0$ for any $x\in (y_1, 1)$. Hence, $U_{xx}(y_1)<0$. Integrating this equation over $[y_1, 1]$, we obtain $U_x(y_1)>U_x(1)+V_x(1)\ge 0.$ Define

$$\begin{aligned} x_2=\sup \{x\in (0, y_1)|U_x(x)=0, U(x)<0\}. \end{aligned}$$

Then we can assert that $0<x_2<x_1, U(x_2)<0$ in $(x_2, 1), U_x(x_2)=0, U_x\ge 0$ on $[x_2, y_1]$, and $U_{xx}(x_2)\ge 0$ similarly. The same arguments lead to $V(x_2)<0, V_x(x_2)>0$ and $V_{xx}(x_2)<0$. Hence, similarly, we can define

$$\begin{aligned} y_2=\sup \{x\in (0, x_2)|V_x(x)=0, V(x)<0\}. \end{aligned}$$

Moreover, $0<y_2<x_2$ by the above arguments. Continuing the above process, we can show that there must exist positive integer i such that $U(x_0)>0$ for some $x_0\in (y_i, x_i)$. Moreover,

$$\begin{aligned} U(y_i)<0, U_{xx}(y_i)<0, U(x_i)<0, U_x(x_i)=0, U_{xx}(x_i)\ge 0, \end{aligned}$$

and

$$\begin{aligned} V(x_i)<0, V_{xx}(x_i)<0, V(y_i)<0, V_x(y_i)=0, V_{xx}(y_i)\ge 0, V_x\ge 0\quad \text{ on } [y_i, x_i]. \end{aligned}$$

Let $z_i=\inf \{x\in (y_i, x_i)|U(x)=0\}$. Then $y_i<z_i<x_i, U_x(z_i)\ge 0$ and $U<0$ in $(y_i, z_i)$. Adding the equations for U and V, we obtain $-D(U+V)_{xx}=(f(R^*-U)-r)B+g(S^*-V)B>0$ for any $x\in (y_i, z_i)$. Integrating this equation over $[y_i, z_i]$, we obtain $U_x(y_i)>U_x(z_i)+V_x(z_i)\ge 0.$ Hence, we can define

$$\begin{aligned} x_{i+1}= & {} \sup \{x\in (0, y_i)|U_x(x)=0, U(x)<0\}\quad \text{ and }\\ y_{i+1}= & {} \sup \{x\in (0, x_{i+1})|V_x(x)=0, V(x)<0\}. \end{aligned}$$

This process will be terminated if $y_{i+1}=0$ or $V_x<0$ for any $x\in (0, y_{i+1})$. If $y_{i+1}=0$. Then $U(0)<0$. Noting that $U<0$ in $(x_{i+1}, z_i)$, we can prove that there exists some $\hat{x}_0\in (0, x_{i+1})$ such that $U(\hat{x}_0)>0$. Otherwise, $U\le 0$ on $[0, z_i]$. On the other hand, considering the equations for U and V on $[0, z_i]$, we have $(U, V)>0$ on $[0, z_i]$, a contradiction. Let $z_{i+1}=\inf \{x\in (0, x_{i+1})|U(x)=0\}$. Considering the equations for U and V on $[0, z_{i+1}]$, we have $(U, V)>0$ on $[0, z_{i+1}]$, a contradiction. If $V_x<0$ for any $x\in (0, y_{i+1})$, we define $x_{i+2}=\sup \{x\in (0, y_{i+1})|U_x(x)=0, U(x)<0\}$. Then $0<x_{i+2}<y_{i+1}$. Considering the equations for U and V on $[x_{i+2}, y_{i+1}]$, we get

$$\begin{aligned}&-DU_{xx}+\omega _rU-\omega _sV=(f(R^*-U)-r)B>0,\quad x\in (x_{i+2}, y_{i+1}),\\&-DV_{xx}+\omega _sV-\omega _rU=g(S^*-V)B>0,\quad x\in (x_{i+2}, y_{i+1}),\\&U_x(x_{i+2})=0,\quad U_x(y_{i+1})\ge 0,\quad -V_x(x_{i+2})\ge 0,\quad V_x(y_{i+1})=0. \end{aligned}$$

It follows from Theorem 15 of Amann (2004) that $(U, V)>0$ on $[x_{i+2}, y_{i+1}]$, a contradiction.

Case (2): $U_x(1)\ge 0, V_x(1)<0$. Noting that $U(1)=0, V(1)=0$ and $U_{xx}<0, V_{xx}<0$ for any $x\in (1-\epsilon , 1)$, we have $U(1-\epsilon )<0$ and $V(1-\epsilon )>0$. Let $x_1=\sup \{x\in (0, 1)|U_x=0, U(x)<0\}$. Then $U(x)<0$ for $x\in [x_1, 1), U_x(x_1)=0, U_x\ge 0$ on $[x_1, 1]$, and $U_{xx}(x_1)\ge 0$. It follows from the first equation of (22) that $\omega _sV(x_1)=-DU_{xx}(x_1)+\omega _rU(x_1)-(f(R^*(x_1)-U(x_1))-r)B(x_1)<0$ since $f(R^*)>r$, which leads to $V(x_1)<0$. In view of $V(x_1)<0, V(1-\epsilon )>0$, we define $z_1=\inf \{x\in (x_1, 1)|V(x)=0\}$. Then $x_1<z_1<1$ and $V_x(z_1)\ge 0$. On the other hand, we have $U_x(z_1)\ge 0$. Hence, we obtain a contradiction as Case (1) on $[0, z_1]$ by the same arguments.

Case (3): $U_x(1)<0, V_x(1)\ge 0$. Noting that $U(1)=0, V(1)=0$ and $U_{xx}<0, V_{xx}<0$ for any $x\in (1-\epsilon , 1)$, we have $U(1-\epsilon )>0$ and $V(1-\epsilon )<0$. Let $y_1=\sup \{x\in (0, 1)|V_x=0, V(x)<0\}$. Then $V(x)<0$ for $x\in [y_1, 1), V_x(y_1)=0, V_{xx}(y_1)\ge 0$ and $V_x\ge 0$ for any $x\in (y_1, 1)$. It follows from the second equation of (22) that $\omega _rU(y_1)=-DV_{xx}(y_1)+\omega _sV(y_1)-g(S^*(y_1)-V(y_1))B(y_1)<0$, which leads to $U(y_1)<0$. In view of $U(y_1)<0, U(1-\epsilon )>0$, we define $z_1=\inf \{x\in (y_1, 1)|U(x)=0\}$. Then $y_1<z_1<1$ and $U_x(z_1)\ge 0$. On the other hand, we have $V_x(z_1)\ge 0$. Similarly, we obtain a contradiction as Case (1) on $[0, z_1]$.

Thus, $U_x(1)<0, V_x(1)<0.$ Hence, there exists $\epsilon >0$ small such that $U(1-\epsilon )>0, V(1-\epsilon )>0.$ Next, we show $U>0, V>0$ in [0, 1). Suppose $U(x_0)<0$ for some point $x_0\in [0, 1)$. Define

$$\begin{aligned} x_1=\sup \{x\in (0, 1)|U_x=0, U(x)<0\}\quad \text{ and }\quad z_1=\inf \{x\in (x_1, 1)|U(x)=0\}. \end{aligned}$$

Then $U\ge 0$ on $[z_1, 1]$. By virtue of $U(1-\epsilon )>0$, it is easy to check that $U(x)<0$ in $[x_1, z_1), U_x(x_1)=0, U_x\ge 0$ on $[x_1, z_1], U_{xx}(x_1)\ge 0$. It follows from the first equation of (22) that

$$\begin{aligned} \omega _sV(x_1)=-DU_{xx}(x_1)+\omega _rU(x_1)-(f(R^*(x_1)-U(x_1))-r)B(x_1)<0 \end{aligned}$$

based on $f(R^*)>r$, which leads to $V(x_1)<0$. Define

$$\begin{aligned} y_1=\sup \{x\in (0, 1)|V_x=0, V(x)<0\}\quad \text{ and }z_2=\inf \{x\in (y_1, 1)|V(x)=0\}. \end{aligned}$$

In view of $V(1-\epsilon )>0$, it is easy to check that $V(x)<0$ in $[y_1, z_2), V_x(y_1)=0, V_x\ge 0$ on $[y_1, z_2], V_{xx}(y_1)\ge 0$. It follows from the second equation of (22) that

$$\begin{aligned} \omega _rU(y_1)=-DV_{xx}(y_1)+\omega _sV(y_1)-g(S^*(y_1)-V(y_1))B(y_1)<0, \end{aligned}$$

which leads to $U(y_1)<0$. Hence $y_1<z_1$ based on $U\ge 0$ on $[z_1, 1]$. Let $z_0=\min \{z_1, z_2\}$. Then $U_x(z_0)\ge 0$ and $V_x(z_0)\ge 0$. Hence, we can derive a contradiction as Case (1) on $[0, z_0]$ by similar arguments. Thus we have $U\ge 0$ on [0, 1]. By the application of strong maximum principle to the Eq. (22), we can find immediately that $U, V>0$ in (0, 1). That is, $0<R<R^*, 0<S<S^*$ in (0, 1).

(ii) It follows from the equation for B that $m=-\lambda _1(-f(R)-g(S), \nu )-r.$ Noting that $0<R<R^*,\, 0<S<S^*$ and the properties of eigenvalue $\lambda _1(q(x),\nu )$, it is easy to see that $0<m<m^*=-\lambda _1(-f(R^*)-g(S^*), \nu )-r$.

(iii) We argue by contradiction. Suppose there exists a sequence $m_n\in (\delta , m^*)(n=1, 2, \ldots )$, and positive solution $(R_n, S_n, B_n)$ of (10)–(11) with $m=m_n$ such that $\Vert B_n\Vert _\infty \rightarrow \infty$ as $n\rightarrow \infty$. Passing to a subsequence if necessary we may assume that $m_n\rightarrow m_0\in [\delta , m^*]$. Set $\hat{B}_n=\frac{B_n}{\Vert B_n\Vert _\infty }.$ Then

$$\begin{aligned}&D\left( \hat{B}_n\right) _{xx}-\nu \left( \hat{B}_n\right) _{x}+(f(R_n)+g(S_n)-r-m_n)\hat{B}_n=0,\quad x\in (0, 1),\\&D\left( \hat{B}_n\right) _{x}(0)-\nu \left( \hat{B}_n\right) (0)=0,\quad D\left( \hat{B}_n\right) _{x}(1)-\nu \left( \hat{B}_n\right) (1)=0. \end{aligned}$$

Integrating the above equation from 0 to x, we obtain

$$\begin{aligned} D\left( \hat{B}_n\right) _{x}(x)-\nu \left( \hat{B}_n\right) (x)+\int _0^x(f(R_n)+g(S_n)-r-m_n)\hat{B}_ndx=0, \end{aligned}$$

which indicates $\big (\hat{B}_n\big )_{x}(x)$ is uniformly bounded since $0<R_n<R^*,\, 0<S_n<S^*$ and $\Vert \hat{B}_n\Vert _\infty =1.$ Hence, $\big (\hat{B}_n\big )_{xx}$ is uniformly bounded. Passing to a subsequence if necessary, we may assume $\hat{B}_n\rightarrow \hat{B}$ in $C^1[0, 1]$, and $\hat{B}\ge 0, \Vert \hat{B}\Vert _\infty =1.$ Let $F_n(x)=f(R_n)+g(S_n)-r.$ Then $-r\le F_n(x)\le f(R^*)+g(S^*)-r$ on [0, 1], and hence we may assume $F_n(x)\rightarrow F_0(x)$ weakly in $L^2(0, 1)$ for some function $F_0(x)$ satisfy $-r\le F_0(x)\le f(R^*)+g(S^*)-r.$ Hence, $\hat{B}$ is a weak solution to

$$\begin{aligned}&D\hat{B}_{xx}-\nu \hat{B}_{x}+(F_0(x)-m_0)\hat{B}=0, x\in (0, 1),\nonumber \\&D\hat{B}_{x}(0)-\nu \hat{B}(0)=0,\quad D\hat{B}(1)-\nu \hat{B}(1)=0. \end{aligned}$$

(23)

It follows from the strong maximum principle that $\hat{B}>0$ on [0, 1]. Let $U_n=R^*-R_n, V_n=S^*-S_n.$ Then $0<U_n<R^*, 0<V_n<S^*$, and $(U_n, V_n)$ satisfies

$$\begin{aligned}&D(U_n)_{xx}-\omega _rU_n+\omega _sV_n+(f(R_n)-r)\Vert B_n\Vert _\infty \hat{B}_n=0, \quad x\in (0, 1),\\&D(V_n)_{xx}-\omega _sV_n+\omega _rU_n+g(S_n)\Vert B_n\Vert _\infty \hat{B}_n=0,\quad x\in (0, 1),\\&{-}(U_n)_x(0)+\alpha U_n(0)=0,\quad U_n(1)=0,\quad (V_n)_x(0)=0,\quad V_n(1)=0. \end{aligned}$$

Hence, we have

$$\begin{aligned} D(U_n)_{xx}+D(V_n)_{xx}+F_n(x)\Vert B_n\Vert _\infty \hat{B}_n=0. \end{aligned}$$

Multiplying this equation by any smooth function $\varphi \in C^\infty [0,1]$ whose support is in (0, 1), we obtain

$$\begin{aligned} D\frac{U_n(0)+V_n(0)}{\Vert B_n\Vert _\infty }\varphi _x(0)+D\int _0^1\frac{U_n+V_n}{\Vert B_n\Vert _\infty }\varphi _{xx}dx+\int _0^1F_n(x)\hat{B}_n\varphi dx=0. \end{aligned}$$

Taking the weak limits, we get $\int _0^1F_0(x)\hat{B}\varphi dx=0$. The arbitrariness of $\varphi$ leads to $F_0(x)\hat{B}=0$ a.e. in (0, 1). Integrating (23) over (0, 1), we get $m_0=0$, a contradiction to $m_0\in [\delta , m^*].$ $\square$

Remark 5.4

It is easy to check that $m^*=-\lambda _1(-f(R^*)-g(S^*), \nu )-r>0$ based on the hypothesis $f(R^*)>r$. Moreover, it follows from Lemma 5.3 that (10)–(11) has no positive solution when $m\ge m^*$.

Now we are ready to prove Theorem 2.3. Since the proof is complicated, we divided it into the following three lemmas.

Lemma 5.5

Assume $f(R^*)>r$. Then given $B\in X_1^+:=C([0,1], {\mathbb {R}}_+)$, the problem

$$\begin{aligned}&{\mathcal {L}}_0\left( \begin{array}{c} R \\ S \end{array}\right) -\left( \begin{array}{c} f(R)-r \\ g(S) \end{array}\right) B(x)=0,\nonumber \\&-R_x(0)+\alpha R(0)=\alpha \hat{R},\quad R(1)=R^0, \quad S_x(0)=0,\quad S(1)=S^0 \end{aligned}$$

(24)

has a unique solution $(R(\cdot ,B),S(\cdot ,B))$, which satisfies $0<R(\cdot ,B)\le R^*,0<S(\cdot ,B)\le S^*$ and $(R(\cdot ,0), S(\cdot ,0))=(R^*, S^*)$. Moreover, for any $0\le B\le M_0$, the maps $B\mapsto R(\cdot ,B)$ and $B\mapsto S(\cdot ,B)$ are Lipschitz continuous from $X_1^+\rightarrow X_1^+$ and $C^1$ continuous from $\dot{X}_1^+\rightarrow X_1^+$, where $M_0$ is given in Lemma 5.3 and $\dot{X}_1^+=\{u(x)\in X_1^+:u(x)>0 \text{ on } [0, 1]\}$.

Proof

From Lemma 5.3, one can find that any nonnegative solution to (24) satisfies $0<R\le R^*, 0<S<S^*$ provided $B(x)\ge 0$. Note that (24) is a cooperative system, and the reaction terms are $C^1$ continuous. According to Definition 8.4.1 of Pao (1992), it is easy to see that (0, 0) is a strictly lower solution to (24), and $(R^*, S^*)$ is a strictly upper solution to (24) provided $f(R^*)>r$. It follows from monotone iteration process in Chapter 8.4 of Pao (1992) that there exists a pair $(R^+, S^+)$ and $(R^-, S^-)$, which are the maximal and minimal solutions to (24), and satisfy the relation $0<R^-\le R^+\le R^*, \ 0<S^-\le S^+\le S^*.$ The detailed proof can be found in Lemma 8.4.1 and Theorem 8.4.1 of Pao (1992). Next, we show $R^-\equiv R^+, S^-\equiv S^+.$ Obviously,

$$\begin{aligned} {\mathcal {L}}_0\left( \begin{array}{l} R^+-R^- \\ S^+-S^- \end{array}\right) +\left( \begin{array}{l} f(R^-)-f(R^+) \\ g(S^-)-g(S^+) \end{array}\right) B(x)=0. \end{aligned}$$

Hence, we have

$$\begin{aligned} D(R^+-R^-)_{xx}+D(S^+-S^-)_{xx}+[f(R^-)-f(R^+)+g(S^-)-g(S^+)]B(x)=0. \end{aligned}$$

Integrating over [0, x], and integrating over [0, 1] again, we have

$$\begin{aligned}&{-}(D+\alpha D)(R^+(0)-R^-(0))-D(S^+(0)-S^-(0))\\&\quad +\int _0^1\int _0^x\left[ f(R^-)-f(R^+)+g(S^-)-g(S^+)\right] B(\xi )d\xi dx=0. \end{aligned}$$

Noting that $R^+\ge R^-,S^+\ge S^-$ on [0, 1], and f(R), g(S) are strictly increasing, we must have $R^+\equiv R^-,S^+\equiv S^-$. Hence, (24) has a unique solution, denoted by $(R(\cdot ,B),S(\cdot ,B))$, satisfying $0<R(\cdot ,B)\le R^*,0<S(\cdot ,B)\le S^*$. It follows from Theorem 2.1 that $(R(\cdot ,0), S(\cdot ,0))=(R^*, S^*)$.

Next, we prove the Lipschitz continuity of the maps $R(\cdot , B)$ and $S(\cdot , B)$ with respect to B. To this end, let $(R_1, S_1)=(R(\cdot ,B_1), S(\cdot ,B_1))$ and $(R_2, S_2)=(R(\cdot ,B_2), S(\cdot ,B_2))$ be the unique solution to (24) with $B=B_1$ and $B=B_2$ respectively. Here $0\le B_1(x)\le B_2(x)\le M_0$ with $B_1(x), B_2(x)\in C[0,1]$ and $B_1(x)\not \equiv B_2(x)$. Then

$$\begin{aligned} {\mathcal {L}}_0\left( \begin{array}{c} R_1-R_2 \\ S_1-S_2 \end{array}\right) - \left( \begin{array}{c} f(R_1)B_1-f(R_2)B_2-r(B_1-B_2) \\ g(S_1)B_1-g(S_2)B_2 \end{array}\right) =0. \end{aligned}$$

It follows from mean value theorem that

$$\begin{aligned} \left[ {\mathcal {L}}_0-\left( \begin{array}{c@{\quad }c} f'(\xi _1)B_1 &{} 0\\ 0&{} g'(\xi _2)B_1 \end{array}\right) \right] \left( \begin{array}{c} R_1-R_2 \\ S_1-S_2 \end{array}\right) = \left( \begin{array}{c} (f(R_2)-r)(B_1-B_2) \\ g(S_2)(B_1-B_2) \end{array}\right) , \end{aligned}$$

where $\xi _1$ lies between $R_1$ and $R_2$, and $\xi _2$ lies between $S_1$ and $S_2$. Hence, $0<\xi _1\le R^*, \ 0<\xi _2\le S^*.$ Recalling that $f'(R), g'(S)>0$, we can find that the operator

$$\begin{aligned} \hat{{\mathcal {L}}}_0={\mathcal {L}}_0-\left( \begin{array}{c@{\quad }c} f'(\xi _1)B_1 &{} 0\\ 0&{} g'(\xi _2)B_1 \end{array}\right) \end{aligned}$$

is invertible, and its inverse operator is a bounded negative operator by Theorem 2.6 or Remark 2.4 of López-Gómez and Molina-Meyer (1994). Therefore,

$$\begin{aligned} \left( \begin{array}{c} R_1-R_2 \\ S_1-S_2 \end{array}\right) =(\hat{{\mathcal {L}}}_0)^{-1} \left( \begin{array}{c} (f(R_2)-r)(B_1-B_2) \\ g(S_2)(B_1-B_2) \end{array}\right) . \end{aligned}$$

The boundedness of the operator $(\hat{{\mathcal {L}}}_0)^{-1}$ and $(R_i, S_i)(i=1, 2)$ leads to the Lipschitz continuity of the maps $B\mapsto R(\cdot ,B)$ and $B\mapsto S(\cdot ,B)$.

At last, we show the $C^1$ continuity of the maps $B\mapsto R(\cdot ,B)$ and $B\mapsto S(\cdot ,B)$ by the implicit function theorem. Define $\mathbf {H}:\dot{X}_1^+\times C^{2+\gamma }[0, 1]\times C^{2+\gamma }[0, 1]\rightarrow C^\gamma [0, 1]$ by

$$\begin{aligned} \mathbf {H}(B,R,S)={\mathcal {L}}_0\left( \begin{array}{c} R \\ S \end{array}\right) -\left( \begin{array}{c} f(R)-r \\ g(S) \end{array}\right) B(x), \end{aligned}$$

subject to the boundary conditions $-R_x(0)+\alpha R(0)=\alpha \hat{R},\ R(1)=R^0,\ \ S_x(0)=0,\ S(1)=S^0$. Clearly, $\mathbf {H}$ is a $C^1$ function. Given $B_0(x)\in \dot{X}_1^+, \mathbf {H}(B_0, R(\cdot ,B_0), S(\cdot ,B_0))\equiv 0$, and the Fréchet derivative

$$\begin{aligned} \mathrm {D}_{(R, S)}\mathbf {H}(B_0, R(\cdot ,B_0), S(\cdot ,B_0))={\mathcal {L}}_0-\left( \begin{array}{cc} f'(R(\cdot ,B_0))B_1 &{} 0\\ 0&{}\quad g'(S(\cdot ,B_0))B_1 \end{array}\right) \end{aligned}$$

is a non-degenerate negative operator subject to the boundary conditions $-R_x(0)+\alpha R(0)=0,\ R(1)=0,\ \ S_x(0)=0,\ S(1)=0$. It follows from the implicit function theorem that there exists a $C^1$ map $(R(\cdot ,B), S(\cdot ,B)):\dot{X}_1^+\rightarrow C^{2+\gamma }[0, 1]\times C^{2+\gamma }[0, 1]$ defined in a neighborhood of $B_0$ such that $(R(\cdot ,B), S(\cdot ,B))|_{B=B_0}= (R(\cdot ,B_0), S(\cdot ,B_0))$, and $\mathbf {H}(B, R(\cdot ,B), S(\cdot ,B))=0.$ It follows from the uniqueness of the solution $(B, R(\cdot ,B), S(\cdot ,B))$ close to $(B_0, R(\cdot ,B_0), S(\cdot ,B_0))$ that $R(\cdot ,B), S(\cdot ,B)$ are continuously differentiable with respect to B respectively. $\square$

Lemma 5.6

Suppose $f(R^*)>r$. Then for any given $\delta _0>0$, the following problem has a unique positive solution provided $m\in [\delta _0, m^*)$

$$\begin{aligned}&DB_{xx}-\nu B_x+(f(R(\cdot ,B))+g(S(\cdot ,B))-r-m)B=0,\nonumber \\&DB_x(0)=\nu B(0),\quad DB_x(1)=\nu B(1). \end{aligned}$$

(25)

Proof

At first, by Lemma 5.3, if B is a nonnegative solution, we must have $0<B<M_0$ provided $m\in [\delta _0, m^*)$. Next, we show (25) has exactly only one positive solution $B\in (0,M_0)$. Let $W=e^{-\frac{\nu }{D}x}B(x)$. Then

$$\begin{aligned}&D\left( e^{\frac{\nu }{D}x}W_x\right) _x+\left( f\left( R\left( \cdot ,e^{\frac{\nu }{D}x}W\right) \right) + g\left( S\left( \cdot ,e^{\frac{\nu }{D}x}W\right) \right) -r-m\right) e^{\frac{\nu }{D}x}W=0,\nonumber \\&\quad W_x(0)=W_x(1)=0. \end{aligned}$$

(26)

Let $\Omega =\{W\in X_1^+:W<M_0+1\}$, and define a differential operator $T_\tau :[0, 1]\times \Omega \rightarrow X_1^+$ by

$$\begin{aligned} T_\tau (W)=K_P\left( \left( \tau f\left( R\left( \cdot ,e^{\frac{\nu }{D}x}W\right) \right) +\tau g\left( S\left( \cdot ,e^{\frac{\nu }{D}x}W\right) \right) -r-m\right) e^{\frac{\nu }{D}x}W+PW\right) \end{aligned}$$

where P is large enough such that $(\tau f(R(\cdot ,e^{\frac{\nu }{D}x}W))+\tau g(S(\cdot ,e^{\frac{\nu }{D}x}W))-r-m)e^{\frac{\nu }{D}x}+P>0$ for all $W\in \Omega$ and $\tau \in [0, 1]$, and $K_P$ is the solution operator $W=K_P(h(x))$ for the problem

$$\begin{aligned} -D\left( e^{\frac{\nu }{D}x}W_x\right) _x+PW=h(x),\quad x\in (0, 1), \quad W_x(0)=W_x(1)=0. \end{aligned}$$

Let $T=T_1$. Then $T:\Omega \rightarrow X_1^+$ is compact and continuously differentiable, and (26) has nonnegative solutions if and only if the operator T has a fixed point in $\Omega .$ Moreover, $T_\tau$ has no fixed point on $\partial \Omega$. By the homotopic invariance of the degree, it is easy to see that

$$\begin{aligned} \mathrm {index}\left( T,\Omega ,X_1^+\right) =\mathrm {index}\left( T_\tau ,\Omega ,X_1^+\right) =\mathrm {index}\left( T_0,\Omega ,X_1^+\right) =\mathrm {index}\left( T_0,0,X_1^+\right) =1. \end{aligned}$$

By Lemma 5.13, it is easy to check that $\mathrm {index}(T,0,X_1^+)=0$ provided that $m<m_*.$ The additivity of index implies that T has at least one positive fixed point in $\Omega$.

It remains to prove the uniqueness of positive fixed points. To this end, we first claim that any positive fixed point $W_0$ of T is non-degenerative, and $\mathrm {index}(T,W_0,X_1^+)=1.$ It follows from Leray–Schauder degree theory that $\mathrm {index}(T,W_0,X_1^+)=(-1)^p$, where p is the sum of the multiplicities of all the eigenvalue of T which are greater than one. Hence it suffices to show T has no eigenvalue greater than or equal to 1. Suppose $\lambda \ge 1$ is an eigenvalue of the Fréchet derivative operator of T at $W_0$ with the associated eigenfunction $\psi$. Then

$$\begin{aligned}&-\lambda D\left( e^{\frac{\nu }{D}x}\psi _x\right) _x+(\lambda -1)P\psi -(f(R(\cdot ,B_0))+ g(S(\cdot ,B_0))-r-m)e^{\frac{\nu }{D}x}\psi \nonumber \\&\quad -B_0f'(R(\cdot ,B_0))\cdot \partial _BR(\cdot ,B_0)e^{\frac{\nu }{D}x}\psi -B_0 g'(S(\cdot ,B_0))\cdot \partial _BS(\cdot ,B_0)e^{\frac{\nu }{D}x}\psi =0,\nonumber \\&\psi _x(0)=\psi _x(1)=0, \end{aligned}$$

(27)

where $B_0=e^{\frac{\nu }{D}x}W_0.$ Let $\phi _1=\partial _BR(\cdot ,B_0)e^{\frac{\nu }{D}x}\psi , \phi _2=\partial _BS(\cdot ,B_0)e^{\frac{\nu }{D}x}\psi$. It follows from $(R(\cdot ,B_0),S(\cdot ,B_0))$ is the unique solution to (24) with $B=B_0$ that

$$\begin{aligned}&{\mathcal {L}}_0\left( \begin{array}{c} \phi _1 \\ \phi _2 \end{array}\right) -\left( \begin{array}{c} B_0f'(R(\cdot ,B_0))\phi _1 \\ B_0g'(S(\cdot ,B_0))\phi _2 \end{array}\right) =\left( \begin{array}{c} f(R(\cdot ,B_0))-r \\ g(S(\cdot ,B_0)) \end{array}\right) e^{\frac{\nu }{D}x}\psi ,\\&L_\lambda \psi =B_0f'(R(\cdot ,B_0))\phi _1+B_0g'(S(\cdot ,B_0))\phi _2, \end{aligned}$$

where $L_\lambda \psi =-\lambda D(e^{\frac{\nu }{D}x}\psi _x)_x+(\lambda -1)P\psi -(f(R(\cdot ,B_0))+ g(S(\cdot ,B_0))-r-m)e^{\frac{\nu }{D}x}\psi .$ It follows from Theorem 13 of Amann (2004) that

$$\begin{aligned} \tilde{{\mathcal {L}}}_0={\mathcal {L}}_0-\left( \begin{array}{cc} B_0f'(R(\cdot ,B_0)) &{} 0\quad \\ 0 &{}\quad B_0g'(S(\cdot ,B_0)) \end{array}\right) \end{aligned}$$

is invertible subject to the boundary conditions: $-(\phi _1)_x(0)+\alpha \phi _1(0)=0, \phi _1(1)=0,\ (\phi _2)_x(0)=0, \phi _2(1)=0$, and all eigenvalues of $\tilde{{\mathcal {L}}}_0$ are negative.

Let $L_1$ be the linear operator $L_\lambda$ with $\lambda =1$. Then $L_1=-D(e^{\frac{\nu }{D}x}\psi _x)_x-(f(R(\cdot ,B_0))+g(S(\cdot ,B_0))-r-m)e^{\frac{\nu }{D}x}\psi .$ Noting that $W_0$ is a positive solution to (26), that is $L_1W_0=0$ in (0, 1), we can find that for $\lambda >1, L_\lambda$ is invertible subject to the boundary conditions: $\psi _x(0)=\psi _x(1)=0$, and all eigenvalues of $L_\lambda$ are positive, which implies the strong maximum principle can be applied to the operator $L_\lambda .$ Meanwhile, since $L_1W_0=0$ in (0, 1), we conclude that $\lambda _1(L_1)=0$ and all of other eigenvalue of $L_1$ are positive. Hence, the general maximum principle can be applied to the operator $L_1$ for the function $\psi /W_0.$ By the similar arguments as in Lemma 3.3 of Nie et al. (2015) and Theorem 3.1 of López-Gómez and Pardo (1994), one can deduce that $\psi \equiv 0$. That is, the Fréchet derivative operator of T at $W_0$ has no eigenvalue greater than or equal to 1. Hence, $\mathrm {index}(T,W_0,X_1^+)=(-1)^0=1.$

Since T is compact and any positive fixed point of T is non-degenerative, and the only trivial non-negative fixed point 0 is also non-degenerative, we see that T has finitely many positive fixed points in $\Omega$. Let them be $W_i(i=1, 2, \ldots , l)$. By the additivity of the fixed-point index, we obtain

$$\begin{aligned} 1=\mathrm {index}\left( T,\Omega ,X_1^+\right) =\mathrm {index}\left( T,0,X_1^+\right) +\sum _{i=1}^l\mathrm {index}\left( T,W_i,X_1^+\right) =l. \end{aligned}$$

Hence $l=1$ and T has a unique positive fixed point. Namely, (25) has a unique positive solution provided $m\in [\delta _0, m^*)$. $\square$

It follows from Lemmas 5.5–5.6 that for $m\in (0, m^*)$, (10)–(11) has a unique positive solution $(R_m(x),S_m(x),B_m(x))$ provided $f(R^*)>r$. Next, we turn to show the continuity of the unique positive solution $(R_m(x),S_m(x),B_m(x))$ with respect to m.

Lemma 5.7

Suppose $f(R^*)>r$, and let $(R_m(x),S_m(x),B_m(x))$ be the unique positive solution to (10)–(11) when $m\in (0, m^*)$. Then $(R_m(x),S_m(x),B_m(x))$ is continuous from $(0, m^*)$ to $(C^1[0,1])^3$.

Proof

The continuity of the unique positive solution $(R_m(x),S_m(x),B_m(x))$ with respect to m follows from a standard compactness and uniqueness consideration. Indeed, if $m_n\rightarrow m_0\in (0, m^*)$, then there exists a subsequence of $(R_{m_n}(x), S_{m_n}(x), B_{m_n}(x))$ converges in $C^1([0,1], {\mathbb {R}}^3)$ to a positive solution of (10)–(11) with $m=m_0$. By the uniqueness, this positive solution must be $(R_{m_0}(x), S_{m_0}(x), B_{m_0}(x))$. Therefore the entire sequence converges to $(R_{m_0}(x), S_{m_0}(x), B_{m_0}(x))$. $\square$

Remark 5.8

It follows from Remark 5.4 and Lemmas 5.5–5.7 that Theorem 2.3 holds.

Remark 5.9

By application of a standard bifurcation argument, $(m^*; R^*, S^*, 0)$ is a simple bifurcation point, and (10)–(11) has an unbounded connected branch of positive solution bifurcating from $(m^*; R^*, S^*, 0)$. Moreover, we can show the branch of positive solution can only become unbounded through $(m; R_m, S_m, B_m)$ belongs to the branch and satisfies $m\rightarrow 0$, and $\Vert B_m\Vert _\infty \rightarrow \infty$, which leads to $f(R_n)+g(S_n)-r\rightarrow 0$ a.e. in (0, 1).

Proof of Theorem 2.4

(i) By Lemma 5.1, any solution (R, S, B) to (9) satisfies $R(x, t)>0, S(x, t)>0, B(x, t)>0$. In order to show (R(x, t), S(x, t), B(x, t)) converges to $(R^*,S^*,0)$, we first consider the following system

$$\begin{aligned}&R_t=DR_{xx}+\omega _sS-\omega _rR+rB-f(R)B,\quad x\in (0, 1),\nonumber \\&S_t=DS_{xx}-\omega _sS+\omega _rR-g(S)B,\quad x\in (0, 1),\nonumber \\&-R_x(0,t)+\alpha R(0,t)=\alpha \hat{R},\quad R(1,t)=R^0,\nonumber \\&S_x(0,t)=0,\quad S(1,t)=S^0,\nonumber \\&R(x, 0)=R_0(x)\ge 0,\quad S(x, 0)=S_0(x)\ge 0, \end{aligned}$$

(28)

where $B(x, t)>0$ fixed. Clearly, there exists $\rho >1$ large enough such that $\rho (R^*, S^*)\ge (R_0(x), S_0(x))$. Hence (0, 0) and $\rho (R^*, S^*)$ are the ordered lower and upper solutions of (28) by Definition 8.1.2 of Pao (1992). It follows from the iteration process of Chapter 8.2 in the book by Pao (1992) and Theorem 8.3.1 of Pao (1992) that (28) has a unique solution (R(x, t, B), S(x, t, B)) satisfies $0<R(x,t, B)<\rho R^*, 0<S(x,t, B)<\rho S^*$. Let $\Lambda =\{(R, S):0\le R\le \rho R^*, 0\le S\le \rho S^*\}$. Then $\Lambda$ is an invariant set of the semi-dynamical system generated by the solutions of (28). Since (28) is a cooperative system, the semi-dynamical system generated by the solutions of (28) is strictly monotone. By Lemma 5.5, the corresponding steady state system (24) has a unique solution (R(x, B), S(x, B)), which satisfies $0<R(x,B)\le R^*,0<S(x,B)\le S^*$. Hence, $\limsup _{t\rightarrow \infty } R\le R^*, \limsup _{t\rightarrow \infty } S\le S^*$ by Theorem 2.2.6 of Zhao (2003). This implies there exists $\epsilon >0$ small such that $R\le R^*+\epsilon ,\ S\le S^*+\epsilon$. Let $W=e^{-\frac{\nu }{D}x}B$. Then

$$\begin{aligned} e^{\frac{\nu }{D}x}W_t= & {} D\left( e^{\frac{\nu }{D}x}W_x\right) _x+(f(R)+g(S)-r-m)W e^{\frac{\nu }{D}x}\\\le & {} D\left( e^{\frac{\nu }{D}x}W_x\right) _x+(f(R^*+\epsilon )+g(S^*+\epsilon )-r-m)We^{\frac{\nu }{D}x}. \end{aligned}$$

Noting that $m>m^*=-\lambda _1(-(f(R^*)+g(S^*)), \nu )-r$, there is $\epsilon$ small enough such that $r+m>-\lambda _1(-(f(R^*+\epsilon )+g(S^*+\epsilon )), \nu )$. Hence the comparison principle leads to $W(x,t)\rightarrow 0$ as $t\rightarrow \infty$ uniformly in $x\in [0, 1]$. Thus $\lim _{t\rightarrow \infty }B(x,t)=0$ uniformly for $x\in [0, 1]$ provided $m>m_*$, which leads to $0<B(x, t)\le \epsilon$ for some $\epsilon >0$. Therefore,

$$\begin{aligned} \left( \begin{array}{c} R \\ S \end{array} \right) _t\ge {\mathcal {L}}_0\left( \begin{array}{c} R \\ S \end{array} \right) -\left( \begin{array}{c} f(R)-r \\ g(S) \end{array} \right) \epsilon \ge {\mathcal {L}}_0\left( \begin{array}{c} R \\ S \end{array} \right) -\left( \begin{array}{c} f(R^*+\epsilon )-r \\ g(S^*+\epsilon ) \end{array} \right) \epsilon . \end{aligned}$$

The comparison principle implies $(R, S)\ge (R_\epsilon , S_\epsilon )$, where $(R_\epsilon , S_\epsilon )$ is the solution of

$$\begin{aligned} \left( \begin{array}{c} R_\epsilon \\ S_\epsilon \end{array}\right) _t={\mathcal {L}}_0\left( \begin{array}{c} R_\epsilon \\ S_\epsilon \end{array}\right) -\left( \begin{array}{c} f(R^*+\epsilon )-r \\ g(S^*+\epsilon ) \end{array} \right) \epsilon . \end{aligned}$$

Obviously, $(R_\epsilon , S_\epsilon )\rightarrow (R^*, S^*)$. Hence, (R(x, t), S(x, t), B(x, t)) converges to $(R^*,S^*,0)$.

We prove (ii) by making use of the abstract persistence theory (Smith and Zhao 2001). Let $\Psi (t)$ be the solution semiflow generated by the system (9) on the state space $X^+$. Set $X_0:=\{(R, S, B)\in X^+:B(x)\not \equiv 0\}$ and $\partial X_0:=X^+\backslash X_0$. Let $M_\partial :=\{\Phi \in \partial X_0:\Psi (t)\Phi \in \partial X_0, \forall t\ge 0\}$ and $\omega (\Phi )$ be the omega limit set of the forward orbit $\gamma ^+(\Phi ):=\{\Psi (t)\Phi :t\ge 0\}$. Then $X_0$ is open in $X^+$ and forward invariant under the dynamics generated by (9) and $\partial X_0$ contains the washout equilibrium $(R^*,S^*,0)$.

We first claim that $\cup _{\Phi \in M_\partial }\omega (\Phi )\subset \{(R^*, S^*, 0)\}$. For any given $\Phi \in M_\partial$, we have $\Psi (t)\Phi \in M_\partial , \forall t\ge 0$, which implies for each $t\ge 0$, we have $B(\cdot , t, \Phi )\equiv 0$. Thus (R, S) satisfies (8). It follows from Theorem 2.1 that $\lim _{t\rightarrow \infty }(R,S)=(R^*, S^*)$ uniformly for $x\in [0, 1].$ Hence, the claim is proved.

Next, we claim that $(R^*, S^*, 0)$ is uniform weak repeller in the sense that $\limsup _{t\rightarrow \infty }\Vert \Psi (t)\Phi -(R^*, S^*, 0)\Vert \ge \delta$ for all $\Phi \in X_0.$ Assume to the contrary that $(R^*, S^*, 0)$ is not a weak repeller. Then there exists such a solution satisfying $(R(x,t), S(x,t), B(x,t))\rightarrow (R^*, S^*, 0)$ uniformly in $x\in [0, 1]$ as $t\rightarrow \infty$. Note that for $(R(x,0),S(x,0),B(x,0))\in X_0$, we have $B(x,t)>0$ for all $t>0$ by change of variable. Since $m<m^*=-\lambda _1(-f(R^*)-g(S^*),\nu )-r$, there is an $\epsilon >0$ small such that $r+m<-\lambda _1(-f(R^*-\epsilon )-g(S^*-\epsilon ),\nu )$. Recalling the hypothesis $(R,S,B)\rightarrow (R^*,S^*,0)$, there exists $t_0>0$ such that $R^*-\epsilon <R(x,t)<R^*+\epsilon ,S^*-\epsilon <S(x,t)<S^*+\epsilon ,0<B<\epsilon$ for $t\ge t_0$. Consequently, for $t\ge t_0, B_t\ge DB_{xx}-\nu B_x+[f(R^*-\epsilon )+g(S^*-\epsilon )-r-m]B.$ Let $W=e^{-\frac{\nu }{D}x}B$. Then for $t\ge t_0$,

$$\begin{aligned}&e^{\frac{\nu }{D}x}W_t\ge D\left( e^{\frac{\nu }{D}x}W_x\right) _x+[f(R^*-\epsilon )+g(S^*-\epsilon )-r-m]We^{\frac{\nu }{D}x},\\&\qquad W_x(0,t)=W_x(1,t)=0. \end{aligned}$$

Choosing $W(x, t_0)\ge \delta _1\psi ^*(x, \epsilon )$, by comparison principle, $W\ge \delta _1\psi ^*(x, \epsilon )e^{\lambda _\epsilon (t-t_0)}$ for $t>t_0$, where $\lambda _\epsilon =-\lambda _1(-f(R^*-\epsilon )-g(S^*-\epsilon ),\nu )-r-m>0$, and $\psi ^*(x, \epsilon )$ is the associate positive eigenfunction to the eigenvalue problem (39) with $q(x)=-f(R^*-\epsilon )-g(S^*-\epsilon ).$ This is a contradiction to $e^{\frac{\nu }{D}x}W(x,t)<\epsilon$. Hence, we conclude that $(R^*, S^*, 0)$ is a uniform weak repeller and $\{(R^*, S^*, 0)\}$ is an isolated invariant set in $X^+$.

Define a continuous function $\mathbf {p}:X^+\rightarrow [0, \infty )$ by $\mathbf {p}(\Phi ):=\min _{x\in [0,1]}\Phi _3(x)$ for any $\Phi =(\Phi _1,\Phi _2,\Phi _3)\in X^+.$ It follows from the standard comparison principle that $\mathbf {p}^{-1}(0,\infty )\subseteq X_0$ and $\mathbf {p}$ satisfies that if $\mathbf {p}(\Phi )>0$ or $\Phi \in X_0$ with $\mathbf {p}(\Phi )=0$, then $\mathbf {p}(\Psi (t)\Phi )>0$ for all $t>0.$ That is, $\mathbf {p}$ is a generalized distance function for the semiflow $\Psi (t):X^+\rightarrow X^+$ (Smith and Zhao 2001). It follows from $\cup _{\Phi \in M_\partial }\omega (\Phi )\subset \{(R^*, S^*, 0)\}$ that any forward orbit of $\Psi (t)$ in $M_\partial$ converges to $(R^*, S^*, 0)$. Note that $\{(R^*, S^*, 0)\}$ is an isolated invariant set in $X^+$, and the stable set $W^s(\{(R^*, S^*, 0)\})\cap X_0=\emptyset$. Hence,there is no subsets of $\{(R^*, S^*, 0)\}$ forms a cycle in $M_\partial$. Meanwhile, it follows from Lemma 5.2 that $\Psi (t)$ is point dissipative on $X^+$, and forward orbits of bounded subsets of $X^+$ for $\Psi (t)$ are bounded. By Theorem 2.6 of Magal and Zhao (2005), $\Psi (t)$ has a global attractor that attracts each bounded set in $X^+$. It follows from Theorem 3 of Smith and Zhao (2001) that there exists a $\epsilon _0$ such that for any $\Phi \in X_0,\min _{\Phi ^0\in \omega (\Phi )}\mathbf {p}(\Phi ^0)>\epsilon _0$, which implies that for any $\Phi \in X_0, \liminf _{t\rightarrow \infty }B(\cdot ,t)\ge \epsilon _0$. The proof is completed.

1.3 Coexistence results

The aim of this subsection is devoted to study coexistence solutions of the two species system (6)–(7), and to establish Theorem 2.6 by the global bifurcation theory (Crandall and Rabinowitz 1971; Du 1996). Let

$$\begin{aligned} m_1^*=-\lambda _1(-f_1(R^*)-g_1(S^*), \nu _1)-r_1,\quad m_2^*=-\lambda _1(-f_2(R^*)-g_2(S^*), \nu _2)-r_2, \end{aligned}$$

where $\lambda _1(-f_i(R^*)-g_i(S^*), \nu _i)(i=1,2)$ is the smallest eigenvalue corresponding to the linear eigenvalue problem (39) (or (38) equivalently) with $q(x)=-f_i(R^*)-g_i(S^*)$ and $\nu =\nu _i.$ It follows from Theorem 2.3 that there are three types of nonnegative steady-state solutions to (6)–(7):

• (i) washout solution $(R^*, S^*, 0, 0)$;

• (ii) semi-trivial solutions: $(\bar{R}_1, \bar{S}_1, \bar{B}_1, 0)$ provided $0<m_1<m_1^*; (\bar{R}_2, \bar{S}_2, 0, \bar{B}_2)$ provided $0<m_2<m_2^*$;

• (iii) positive solutions $(R, S, B_1, B_2)$ with $B_1(x)>0$ and $B_2(x)>0$ on [0, 1].

Repeating the same arguments in Lemma 5.3, we obtain a priori estimates for positive solutions of (6)–(7).

Lemma 5.10

Assume $f_i(R^*)>r_i (i=1, 2)$ and $(R, S, B_1, B_2)$ is a nonnegative solution of (6)–(7) with $B_1\not \equiv 0$ and $B_2\not \equiv 0.$ Then

• (i) $0<R<R^*, 0<S<S^*, B_1>0, B_2>0$ in $\mathrm {(0, 1)}$;

• (ii) $0<m_1<m_1^*, 0<m_2<m_2^*$;

• (iii) for any given $\delta >0$, there exists a positive constant $M(\delta )$ such that every positive solution $(R, S, B_1, B_2)$ of (6)–(7) with $m_1\in [\delta , m_1^*), m_2\in [\delta , m_2^*)$ satisfies $\Vert B_1\Vert _\infty +\Vert B_2\Vert _\infty \le M(\delta )$.

It follows from Lemma 5.10 that the necessary conditions for the existence of a positive solution of (6)–(7) are

$$\begin{aligned} 0<m_1<m_1^*,0<m<m_2^*. \end{aligned}$$

Next, we assume $0<m_1<m_1^*,0<m_2<m_2^*$, and construct a positive solution of (6)–(7) by the global bifurcation theorem. Thus we need to rewrite (6)–(7) as an abstract equation related to a completely continuous operator. Let ${\mathbb {X}}=C([0, 1], {\mathbb {R}}^4)$, and ${\mathbb {X}}^+=C([0, 1], {\mathbb {R}}^4_+)$ be the positive cone of the ordered Banach space ${\mathbb {X}}$.

Let $u=R^*-R, v=S^*-S, w_1=e^{-\frac{\nu _1}{D}x}B_1, w_2=e^{-\frac{\nu _2}{D}x}B_2.$ Then the steady state system (6)–(7) is equivalent to

$$\begin{aligned} {-}(Du_{xx}+\omega _s v-\omega _r u)= & {} (f_1(R^*-u)-r_1)e^{\frac{\nu _1}{D}x}w_1+(f_2(R^*-u)-r_2)e^{\frac{\nu _2}{D}x}w_2,\nonumber \\ -(Dv_{xx}-\omega _s v+\omega _r u)= & {} g_1(S^*-v)e^{\frac{\nu _1}{D}x}w_1+g_2(S^*-v)e^{\frac{\nu _2}{D}x}w_2,\nonumber \\ -D\left( e^{\frac{\nu _1}{D}x}(w_1)_{x}\right) _x= & {} [f_1(R^*-u)+g_1(S^*-v)-r_1-m_1]e^{\frac{\nu _1}{D}x}w_1, \nonumber \\ -D\left( e^{\frac{\nu _2}{D}x}(w_2)_{x}\right) _x= & {} [f_2(R^*-u)+g_2(S^*-v)-r_2-m_2]e^{\frac{\nu _2}{D}x}w_2, \end{aligned}$$

(29)

with the boundary conditions

$$\begin{aligned}&-u_x(0)+\alpha u(0)=0,\quad u(1)=0,\quad v_x(0)=0,\quad v(1)=0,\nonumber \\&\quad (w_i)_x(0)=(w_i)_x(1)=0,\quad i=1, 2, \end{aligned}$$

(30)

We define ${\mathbb {A}}:(0, +\infty )\times {\mathbb {X}}\rightarrow {\mathbb {X}}$ by

$$\begin{aligned} {\mathbb {A}}(m_2; u,v,w_1,w_2)=({\mathbb {A}}_0(u,v,w_1,w_2), {\mathbb {A}}_1(u,v,w_1,w_2), {\mathbb {A}}_2(u,v,w_1,w_2)), \end{aligned}$$

where

$$\begin{aligned} {\mathbb {A}}_0(u,v,w_1,w_2)= & {} \mathbb {K}_0\left( \begin{array}{c} (f_1(R^*-u)-r_1)e^{\frac{\nu _1}{D}x}w_1+(f_2(R^*-u)-r_2)e^{\frac{\nu _2}{D}x}w_2\\ g_1(S^*-v)e^{\frac{\nu _1}{D}x}w_1+g_2(S^*-v)e^{\frac{\nu _2}{D}x}w_2 \end{array} \right) \\ {\mathbb {A}}_1(u,v,w_1,w_2)= & {} \mathbb {K}_1\left( [f_1(R^*-u)+g_1(S^*-v)-r_1-m_1]e^{\frac{\nu _1}{D}x}w_1+M_1w_1 \right) \\ {\mathbb {A}}_2(u,v,w_1,w_2)= & {} \mathbb {K}_2\left( [f_2(R^*-u)+g_2(S^*-v)-r_2-m_2]e^{\frac{\nu _1}{D}x}w_2+M_2w_2 \right) \end{aligned}$$

and $\mathbb {K}_0, \mathbb {K}_i(i=1, 2)$ are the solution operators for the problems, respectively,

$$\begin{aligned}&-{\mathcal {L}}_0(\phi _1, \phi _2)^\intercal =(h_1(x), h_2(x))^\intercal ,\quad x\in (0, 1),\nonumber \\&-(\phi _1)_x(0)+\alpha \phi _1(0)=0, \phi _1(1)=0,\quad (\phi _2)_x(0)=0, \phi _2(1)=0;\end{aligned}$$

(31)

$$\begin{aligned}&-D\left( e^{\frac{\nu _i}{D}x}w_x\right) _x+M_iw =h(x),\quad x\in (0, 1),\quad w_x(0)=w_x(1)=0. \end{aligned}$$

(32)

That is, for any given $h_1(x), h_2(x)\in C[0, 1]$ and $h(x)\in C[0, 1], (\phi _1, \phi _2)^\intercal =\mathbb {K}_0(h_1(x), h_2(x))^\intercal$ and $w=\mathbb {K}_i(h(x))$. Here, $M_i (i=1, 2)$ is large enough such that $(f_i(R^*-u)+g_i(S^*-v)-r_i-m_i)e^{\frac{\nu _i}{D}x}+M_i>0$. By the application of Theorem 2.6 of López-Gómez and Molina-Meyer (1994), we obtain that $\mathbb {K}_0$ is strongly positive compact operator when seen as an operator from $C^1([0, 1], {\mathbb {R}}^2)$ to $C^1([0, 1], {\mathbb {R}}^2)$ and from $L^2((0, 1), {\mathbb {R}}^2)$ to $L^2((0, 1), {\mathbb {R}}^2)$. Similarly, $\mathbb {K}_i(i=1, 2)$ is strongly positive compact operator when seen as an operator from $C^1[0, 1]$ to $C^1[0, 1]$ and from $L^2(0, 1)$ to $L^2(0, 1)$. By standard elliptic regularity theory we know that ${\mathbb {A}}:(0, +\infty )\times {\mathbb {X}}\rightarrow {\mathbb {X}}$ is completely continuous. Let $\mathbf {U}=(u,v,w_1,w_2)^\intercal$ and $\mathbb {G}(m_2; \mathbf {U})=\mathbf {U}-{\mathbb {A}}(m_2, \mathbf {U})$. Then the zeros of $\mathbb {G}(m_2; \mathbf {U})=0$ with $0\le u\le R^*, 0\le v\le S^*, w_1\ge 0, w_2\ge 0$ correspond to the nonnegative solutions of (29)–(30).

It follows from Theorem 2.3 that (29)–(30) have two semi-trivial solutions

$$\begin{aligned} \mathbf {U}_1=\left( R^*-\bar{R}_1, S^*-\bar{S}_1, e^{-\frac{\nu _1}{D}x}\bar{B}_1,0\right) \quad \text{ and }\quad \mathbf {U}_2=\left( R^*-\bar{R}_2, S^*-\bar{S}_2, 0, e^{-\frac{\nu _2}{D}x}\bar{B}_2\right) \end{aligned}$$

when $0<m_1<m_1^*,0<m_2<m_2^*$. Next, we construct a positive solution branch $\Gamma '=\{m_2; \mathbf {U}\}\subset (0, +\infty )\times {\mathbb {X}}^+$ bifurcating from the semi-trivial solution branches $\Gamma '_1=\{(m_2; \mathbf {U}_1)\subset (0, +\infty )\times {\mathbb {X}}^+:m_2\in (0, +\infty )\}$ and $\Gamma '_2=\{(m_2; \mathbf {U}_2)\subset (0, +\infty )\times {\mathbb {X}}^+:m_2\in (0, +\infty )\}$. To this end, we fix $m_1\in (0,m_1^*)$ and take $m_2$ as the bifurcation parameter. Introduce

$$\begin{aligned} \hat{m}_1(m_2)= & {} -\lambda _1(-f_1(\bar{R}_2(\cdot ,m_2))-g_1(\bar{S}_2(\cdot ,m_2)), \nu _1)-r_1,\\ \hat{m}_2(m_1)= & {} -\lambda _1(-f_2(\bar{R}_1(\cdot ,m_1))-g_2(\bar{S}_1(\cdot ,m_1)), \nu _2)-r_2, \end{aligned}$$

where $\lambda _1(-f_1(\bar{R}_2(\cdot ,m_2))-g_1(\bar{S}_2(\cdot ,m_2)), \nu _1)$ and $\lambda _1(-f_2(\bar{R}_1(\cdot ,m_1))-g_2(\bar{S}_1(\cdot ,m_1)), \nu _2)$ are the smallest eigenvalues corresponding to the linear eigenvalue problem (39) (or (38) equivalently) with $q(x)=-f_1(\bar{R}_2(\cdot ,m_2))-g_1(\bar{S}_2(\cdot ,m_2)), \nu =\nu _1$ and $q(x)=-f_2(\bar{R}_1(\cdot ,m_1))-g_2(\bar{S}_1(\cdot ,m_1)), \nu =\nu _2$. In view of $0<\bar{R}_1<R^*,0<\bar{S}_1<S^*$, it follows from Lemma 5.11 that $0<\hat{m}_1(m_2)<m_1^*,\ 0<\hat{m}_2(m_1)<m_2^*$.

Proof of Theorem 2.6

For any $\delta >0$ and $m_1\in [\delta , m_1^*)$ fixed, we construct the global bifurcation which corresponds to positive solutions by treating $m_2$ as a bifurcation parameter. The Fréchet derivative of $\mathbb {G}(m_2; \mathbf {U})$ with respect to $\mathbf {U}$ at $\mathbf {U}_1$ is denoted by $\mathrm {D}_\mathbf {U}G(m_2; \mathbf {U}_1)$. In order to apply Crandall–Rabinowitz theorem of bifurcation from simple eigenvalue (Crandall and Rabinowitz 1971), we first show that the dimension of the null space of $\mathrm {D}_\mathbf {U}G(m_2; \mathbf {U}_1)$ is 1. Let $\mathrm {D}_\mathbf {U}G(m_2; \mathbf {U}_1)(\phi _1, \phi _2, \psi _1, \psi _2)=0$. Then direct computation gives

$$\begin{aligned}&{\mathcal {L}}_0\left( \begin{array}{c} \phi _1 \\ \phi _2 \end{array}\right) - \left( \begin{array}{c} f'_1\left( \bar{R}_1\right) \bar{B}_1\phi _1 \\ g'_1\left( \bar{S}_1\right) \bar{B}_1\phi _2 \end{array}\right) \nonumber \\&\quad + \left( \begin{array}{c} (f_1\left( \bar{R}_1\right) -r_1)e^{\frac{\nu _1}{D}x}\psi _1+(f_2\left( \bar{R}_1\right) -r_2)e^{\frac{\nu _2}{D}x}\psi _2 \\ g_1\left( \bar{S}_1\right) e^{\frac{\nu _1}{D}x}\psi _1+g_2\left( \bar{S}_1\right) e^{\frac{\nu _2}{D}x}\psi _2 \end{array}\right) =0\\&D\left( e^{\frac{\nu _1}{D}x}\psi _{1x}\right) _x+\left[ f_1\left( \bar{R}_1\right) +g_1\left( \bar{S}_1\right) -r_1-m_1\right] e^{\frac{\nu _1}{D}x}\psi _1\nonumber \\&\quad -f'_1\left( \bar{R}_1\right) \bar{B}_1\phi _1- g'_1\left( \bar{S}_1\right) \bar{B}_1\phi _2=0\\&D\left( e^{\frac{\nu _2}{D}x}\psi _{2x}\right) _x+[f_2\left( \bar{R}_1\right) +g_2\left( \bar{S}_1\right) -r_2-m_2]e^{\frac{\nu _2}{D}x}\psi _2=0 \end{aligned}$$

with the corresponding boundary conditions. Take $m_2=\hat{m}_2,\ \psi _2=\hat{\psi }_2$, which is the associated positive eigenfunction to the eigenvalue $\lambda _1(-f_2\left( \bar{R}_1\right) -g_2\left( \bar{S}_1\right) , \nu _2)$. It follows from Theorem 13 of Amann (2004) that

$$\begin{aligned} \bar{{\mathcal {L}}}_0={\mathcal {L}}_0-\left( \begin{array}{c@{\quad }c} f'_1\left( \bar{R}_1\right) \bar{B}_1 &{} 0 \\ 0 &{} g'_1\left( \bar{S}_1\right) \bar{B}_1 \end{array} \right) \end{aligned}$$

is invertible subject to the boundary conditions: $-(\phi _1)_x(0)+\alpha \phi _1(0)=0, \phi _1(1)=0,\ (\phi _2)_x(0)=0, \phi _2(1)=0$, and all eigenvalues of $\bar{{\mathcal {L}}}_0$ are negative. Hence

$$\begin{aligned} \left( \begin{array}{c} \phi _1 \\ \phi _2 \end{array} \right) =-\bar{{\mathcal {L}}}_0^{-1}\left[ \left( \begin{array}{c} f_1\left( \bar{R}_1\right) -r_1 \\ g_1\left( \bar{S}_1\right) \end{array} \right) e^{\frac{\nu _1}{D}x}\psi _1\right] -\bar{{\mathcal {L}}}_0^{-1}\left[ \left( \begin{array}{c} f_2\left( \bar{R}_1\right) -r_2 \\ g_2\left( \bar{S}_1\right) \end{array} \right) e^{\frac{\nu _2}{D}x}\hat{\psi }_2\right] . \end{aligned}$$

Here $\bar{{\mathcal {L}}}_0^{-1}$ is the inverse operator of $\bar{{\mathcal {L}}}_0$ subject to the boundary conditions $-(\phi _1)_x(0)+\alpha \phi _1(0)=0, \phi _1(1)=0,\ \ (\phi _2)_x(0)=0, \phi _2(1)=0.$ Let

$$\begin{aligned} \left( \begin{array}{c} \bar{\phi }_1(\psi _1) \\ \bar{\phi }_2(\psi _1) \end{array}\right) =\bar{{\mathcal {L}}}_0^{-1}\left[ \left( \begin{array}{c} f_1\left( \bar{R}_1\right) -r_1 \\ g_1\left( \bar{S}_1\right) \end{array} \right) e^{\frac{\nu _1}{D}x}\psi _1\right] \end{aligned}$$

and

$$\begin{aligned} \left( \begin{array}{c} \tilde{\phi }_1(\hat{\psi }_2) \\ \tilde{\phi }_2(\hat{\psi }_2) \end{array}\right) =\bar{{\mathcal {L}}}_0^{-1}\left[ \left( \begin{array}{c} f_2\left( \bar{R}_1\right) -r_2 \\ g_2\left( \bar{S}_1\right) \end{array} \right) e^{\frac{\nu _2}{D}x}\hat{\psi }_2\right] \end{aligned}$$

Putting them into the equation for $\psi _1$, we have

$$\begin{aligned}&D\left( e^{\frac{\nu _1}{D}x}\psi _{1x}\right) _x+\left[ f_1\left( \bar{R}_1\right) +g_1\left( \bar{S}_1\right) -r_1-m_1\right] e^{\frac{\nu _1}{D}x}\psi _1\nonumber \\&\quad +\left( f'_1\left( \bar{R}_1\right) \bar{\phi }_1(\psi _1)+ g'_1\left( \bar{S}_1\right) \bar{\phi }_2(\psi _1)\right) \bar{B}_1 +\left( f'_1\left( \bar{R}_1\right) \tilde{\phi }_1(\hat{\psi }_2)\right. \nonumber \\&\quad + \left. g'_1\left( \bar{S}_1\right) \tilde{\phi }_2(\hat{\psi }_2) \right) \bar{B}_1=0. \end{aligned}$$

(33)

Clearly, $\bar{\phi }_1(\psi _1),\bar{\phi }_2(\psi _1)$ are differentiable with respect to $\psi$. Note that $\bar{\phi }_1(0)=\bar{\phi }_2(0)=0$. Take $\Vert \psi _1\Vert =\varepsilon$ by re-scaling. Then $\bar{B}_1(f'_1\left( \bar{R}_1\right) \bar{\phi }_1(\psi _1)+ g'_1\left( \bar{S}_1\right) \bar{\phi }_2(\psi _1))= \bar{B}_1[f'_1\left( \bar{R}_1\right) ((\partial _{\psi _1}\bar{\phi }_1)\psi _1+o(\varepsilon )\psi _1)+ g'_1\left( \bar{S}_1\right) ((\partial _{\psi _1}\bar{\phi }_2)\psi _1+o(\varepsilon )\psi _1)]= \bar{B}_1[f'_1\left( \bar{R}_1\right) \partial _{\psi _1} \bar{\phi }_1+g'_1\left( \bar{S}_1\right) \partial _{\psi _1}\bar{\phi }_2+o(\epsilon )]\psi _1$. Note that ${\mathcal {L}}_1=D\frac{d}{dx}(e^{\frac{\nu _1}{D}x}\frac{d}{dx})+\left[ f_1\left( \bar{R}_1\right) +g_1\left( \bar{S}_1\right) -r_1-m_1\right] e^{\frac{\nu _1}{D}x}+ \bar{B}_1f'_1\left( \bar{R}_1\right) \partial _{\psi _1} \bar{\phi }_1+g'_1\left( \bar{S}_1\right) \partial _{\psi _1}\bar{\phi }_2)$ is invertible. Hence $\psi _1=\hat{\psi }_1$ can be solved by (33) uniquely, which implies the null space of $\mathrm {D}_\mathbf {U}G(\hat{m}_2; \mathbf {U}_1)$ is spanned by $(\hat{\phi }_1,\hat{\phi }_2,\hat{\psi }_1,\hat{\psi }_2)$. Here

$$\begin{aligned} \left( \begin{array}{c} \hat{\phi }_1 \\ \hat{\phi }_2 \end{array}\right) =-\bar{{\mathcal {L}}}_0^{-1}\left[ \left( \begin{array}{c} f_1\left( \bar{R}_1\right) -r_1 \\ g_1\left( \bar{S}_1\right) \end{array} \right) e^{\frac{\nu _1}{D}x}\hat{\psi }_1\right] -\bar{{\mathcal {L}}}_0^{-1}\left[ \left( \begin{array}{c} f_2\left( \bar{R}_1\right) -r_2 \\ g_2\left( \bar{S}_1\right) \end{array} \right) e^{\frac{\nu _2}{D}x}\hat{\psi }_2\right] . \end{aligned}$$

Direct computation leads to that the range of $\mathrm {D}_\mathbf {U}G(\hat{m}_2; \mathbf {U}_1)$ is

$$\begin{aligned} \left\{ \mathbf {U}=(u, v, w_1, w_2)\in {\mathbb {X}}: \int _0^1\left[ \left( f_2\left( \bar{R}_1\right) +g_2\left( \bar{S}_1\right) -r_2-\hat{m}_2\right) e^{\frac{\nu _2}{D}x}+M_2\right] \hat{\psi }_2w_2dx=0\right\} . \end{aligned}$$

By virtue of $\mathbb {K}_2(e^{\frac{\nu _2}{D}x}\hat{\psi }_2)>0$, we have

$$\begin{aligned} \int _0^1\mathbb {K}_2\left( e^{\frac{\nu _2}{D}x}\hat{\psi }_2\right) \left[ \left( f_2\left( \bar{R}_1\right) +g_2\left( \bar{S}_1\right) -r_2-\hat{m}_2\right) e^{\frac{\nu _2}{D}x}+M_2\right] \hat{\psi }_2dx>0 \end{aligned}$$

Hence, $\mathrm {D}^2_{m_2\mathbf {U}}G(\hat{m}_2; \mathbf {U}_1)(\hat{\phi }_1,\hat{\phi }_2,\hat{\psi }_1,\hat{\psi }_2)= (0,0,0,\mathbb {K}_2(e^{\frac{\nu _2}{D}x}\hat{\psi }_2))$ does not belong to the range of $\mathrm {D}_\mathbf {U}G(\hat{m}_2; \mathbf {U}_1).$ By application of the bifurcation theorem from a simple eigenvalue (Crandall and Rabinowitz 1971), there exists a $\tau _0>0$ and $C^1$ function $(m_2(\tau ),R(\tau ),S(\tau ),B_1(\tau ),B_2(\tau )):(-\tau _0,\tau _0)\mapsto (-\infty , +\infty )\times {\mathbb {X}}$ such that $m(0)=\hat{m}_2, R(0)=\bar{R}_1, S(0)=\bar{S}_1,B_1(0)=\bar{B}_1,B_2(0)=0$ and $(m_2,R(\tau ),S(\tau ),B_1(\tau ),B_2(\tau ))=(m_2(\tau ),\bar{R}_1+\tau (\hat{\phi }_1+U(\tau )),\bar{S}_1+\tau (\hat{\phi }_2+V(\tau )), \bar{B}_1+\tau (\hat{\psi }_1+\omega _1(\tau )),\tau (\hat{\psi }_2+\omega _2(\tau ))) (|\tau |<\tau _0)$, which is the solution of the steady state system (6)–(7). If we take $0<\tau <\tau _0$, this bifurcation branch is just the positive solution of the steady state system (6)–(7).

Next, we extend the local bifurcation to the global one. Suppose $\lambda \ge 1$ is an eigenvalue of $\mathrm {D}_\mathbf {U}{\mathbb {A}}(m_2; \mathbf {U}_1)$ with the corresponding eigenfunction $(\phi _1,\phi _2,\psi _1,\psi _2)$. Then

$$\begin{aligned}&\lambda {\mathcal {L}}_0\left( \begin{array}{c} \phi _1 \\ \phi _2 \end{array} \right) \nonumber \\&\quad +\left( \begin{array}{c} -f'_1\left( \bar{R}_1\right) \bar{B}_1\phi _1+(f_1\left( \bar{R}_1\right) -r_1)e^{\frac{\nu _1}{D}x}\psi _1+(f_2\left( \bar{R}_1\right) -r_2)e^{\frac{\nu _2}{D}x}\psi _2 \\ -g'_1\left( \bar{S}_1\right) \bar{B}_1\phi _2+g_1\left( \bar{S}_1\right) e^{\frac{\nu _1}{D}x}\psi _1+g_2\left( \bar{S}_1\right) e^{\frac{\nu _2}{D}x} \psi _2 \end{array} \right) =0,\nonumber \\&\lambda D\left( e^{\frac{\nu _1}{D}x}\psi _{1x}\right) _x+(1-\lambda )M_1\psi _1+\left[ f_1\left( \bar{R}_1\right) +g_1\left( \bar{S}_1\right) -r_1-m_1\right] e^{\frac{\nu _1}{D}x}\psi _1\nonumber \\&\quad -f'_1\left( \bar{R}_1\right) \bar{B}_1\phi _1-g'_1\left( \bar{S}_1\right) \bar{B}_1\phi _2=0,\nonumber \\&D\left( e^{\frac{\nu _2}{D}x}\psi _{2x}\right) _x+M_2\left( -1+\frac{1}{\lambda }\right) \psi _2+\frac{1}{\lambda }\left( f_2\left( \bar{R}_1\right) +g_2\left( \bar{S}_1\right) -r_2-m_2\right) e^{\frac{\nu _2}{D}x}\psi _2=0 \end{aligned}$$

(34)

with the boundary conditions (30). Claim that $\psi _2\not \equiv 0.$ If not, then $\psi _2\equiv 0$, similar arguments lead to

$$\begin{aligned} \bar{{\mathcal {L}}}_\lambda ={\mathcal {L}}_0-\frac{1}{\lambda }\left( \begin{array}{cc} f'_1\left( \bar{R}_1\right) \bar{B}_1 &{} 0 \\ 0 &{} \quad g'_1\left( \bar{S}_1\right) \bar{B}_1 \end{array} \right) \end{aligned}$$

is invertible subject to the boundary conditions: $-(\phi _1)_x(0)+\alpha \phi _1(0)=0, \phi _1(1)=0,\ (\phi _2)_x(0)=0, \phi _2(1)=0$, and all eigenvalues of $\bar{{\mathcal {L}}}_\lambda$ are negative. Hence

$$\begin{aligned} \left( \begin{array}{c} \phi _1 \\ \phi _2 \end{array} \right) =-\bar{{\mathcal {L}}}_\lambda ^{-1}\left[ \frac{1}{\lambda }\left( \begin{array}{c} f_1\left( \bar{R}_1\right) -r_1 \\ g_1\left( \bar{S}_1\right) \end{array} \right) e^{\frac{\nu _1}{D}x}\psi _1\right] . \end{aligned}$$

Substituting $(\phi _1, \phi _2)$ into the equation for $\psi _1$ in (34), we have $\psi _1\equiv 0$ by similar arguments as above, which leads to $\phi _1=\phi _2\equiv 0$. This is a contradiction. Hence $\psi _2\not \equiv 0$. Noting that $\bar{{\mathcal {L}}}_\lambda$ is invertible, similar arguments as above deduce that $\phi _1, \phi _2$ and $\psi _1$ can be determined by (34) uniquely. Hence, $\lambda \ge 1$ is an eigenvalue of $\mathrm {D}_\mathbf {U}{\mathbb {A}}(m_2; \mathbf {U}_1)$ if and only if $\lambda \ge 1$ satisfies

$$\begin{aligned}&D\left( e^{\frac{\nu _2}{D}x}\psi _{2x}\right) _x+M_2\left( -1+\frac{1}{\lambda }\right) \psi _2+\frac{1}{\lambda }\left( f_2\left( \bar{R}_1\right) +g_2\left( \bar{S}_1\right) -r_2-m_2\right) e^{\frac{\nu _2}{D}x}\psi _2=0,\\&\psi _{2x}(0)=\psi _{2x}(1)=0. \end{aligned}$$

That is, $\lambda \ge 1$ is an eigenvalue of $\mathrm {D}_\mathbf {U}{\mathbb {A}}(m_2; \mathbf {U}_1)$ if and only if $\frac{1}{\lambda } (0<\frac{1}{\lambda }\le 1)$ is an eigenvalue of

$$\begin{aligned}&-D\left( e^{\frac{\nu _2}{D}x}\psi _{2x}\right) _x+M_2\psi _2=\sigma \left[ \left( f_2\left( \bar{R}_1\right) +g_2\left( \bar{S}_1\right) -r_2-m_2\right) e^{\frac{\nu _2}{D}x}+M_2\right] \psi _2,\nonumber \\&\psi _{2x}(0)=\psi _{2x}(1)=0. \end{aligned}$$

(35)

If $m_2>\hat{m}_2$, then $\lambda _1(-f_2\left( \bar{R}_1\right) -g_2\left( \bar{S}_1\right) +r_2+m_2, \nu _2)>0$. It follows from Lemma 5.14 that the eigenvalue problem (35) has no eigenvalue less than or equal to 1, which leads to $\mathrm {D}_\mathbf {U}{\mathbb {A}}(m_2; \mathbf {U}_1)$ has no eigenvalue $\lambda >1$. Thus $\text{ index }({\mathbb {A}}(m_2; \mathbf {U}), \mathbf {U}_1)=1.$

On the other hand, in view of $f_2\left( \bar{R}_1\right) +g_2\left( \bar{S}_1\right) -r_2-m_2+M_2>0$, it is easy to check that all eigenvalues $\sigma _i(m_2)$ of (35) are real and strictly increasing with respect to $m_2$, and can be ordered as $0<\sigma _1(m_2)<\sigma _2(m_2)\le \sigma _3(m_2)\le \cdots$ with $\sigma _1(\hat{m}_2)=1$ (cf. Courant and Hilbert 1953). Hence, for $\hat{m}_2-\epsilon <m_2<\hat{m}_2$, one can find that $0<\sigma _1(m_2)<\sigma _1(\hat{m}_2)=1$ and $\sigma _2(\hat{m}_2)>\sigma _1(\hat{m}_2)=1$. The continuity of $\sigma _2(m_2)$ leads to $\sigma _2(m_2)>\sigma _2(\hat{m}_2-\epsilon )>1$ as long as $\epsilon$ is small enough. Thus $\sigma _1(m_2)$ is the unique eigenvalue of (35), which is less than 1, and (35) has exactly one nontrivial solution (up to a multiplicative constant), denoted by $\tilde{\psi }_2$, whenever $m_2<\hat{m}_2$ is close enough to $\hat{m}_2$. This establishes that $\mathrm {D}_\mathbf {U}{\mathbb {A}}(m_2; \mathbf {U}_1)$ has a unique eigenvalue $\lambda _0=\frac{1}{\sigma _1(m_2)}>1$.

Next, we can show that

$$\begin{aligned} N(\lambda _0I-\mathrm {D}_\mathbf {U}{\mathbb {A}}(m_2; \mathbf {U}_1))\cap R(\lambda _0I-\mathrm {D}_\mathbf {U}{\mathbb {A}}(m_2; \mathbf {U}_1))=\{0\}, \end{aligned}$$

which implies the algebraic multiplicity of the eigenvalue $\lambda _0$ is one. If not, without loss of generality, we may assume that $(\phi _1, \phi _2, \psi _1, \tilde{\psi }_2)\in R(\lambda _0I-\mathrm {D}_\mathbf {U}{\mathbb {A}}(m_2; \mathbf {U}_1))$, where $(\phi _1, \phi _2, \psi _1, \tilde{\psi }_2)$ satisfies (34) with $\lambda =\lambda _0.$ Then there exists $(\Phi _1, \Phi _2, \Psi _1, \Psi _2)\in {\mathbb {X}}$ such that

$$\begin{aligned}&\lambda _0D\left( e^{\frac{\nu _2}{D}x}\Psi _{2x}\right) _x-M_2(\lambda _0-1)\Psi _2+(f_2\left( \bar{R}_1\right) +g_2\left( \bar{S}_1\right) -r_2-m_2)e^{\frac{\nu _2}{D}x}\Psi _2\nonumber \\&\quad =D(e^{\frac{\nu _2}{D}x}\tilde{\psi }_{2x})_x-M_2\tilde{\psi }_2,\nonumber \\&\Psi _{2x}(0)=\Psi _{2x}(1)=0. \end{aligned}$$

(36)

Meanwhile, note that $\lambda _0=\frac{1}{\sigma _1(m_2)}, \tilde{\psi }_2\not =0$ and

$$\begin{aligned}&-D\left( e^{\frac{\nu _2}{D}x}\tilde{\psi }_{2x}\right) _x+M_2\tilde{\psi }_2=\sigma _1(m_2)\left[ \left( f_2\left( \bar{R}_1\right) +g_2\left( \bar{S}_1\right) -r_2-m_2\right) e^{\frac{\nu _2}{D}x}+M_2\right] \tilde{\psi }_2,\nonumber \\&\tilde{\psi }_{2x}(0)=\tilde{\psi }_{2x}(1)=0. \end{aligned}$$

(37)

Multiplying (36) by $\tilde{\psi }_2$ and (37) by $\Psi _2$, and integrating over (0, 1) by parts, we obtain

$$\begin{aligned} -D\int _0^1e^{\frac{\nu _2}{D}x}\tilde{\psi }_{2x}^2dx-M_2\int _0^1\tilde{\psi }_{2}^2dx=0, \end{aligned}$$

a contradiction. Hence, $\lambda _0$ is the unique eigenvalue of $\mathrm {D}_\mathbf {U}{\mathbb {A}}(m_2; \mathbf {U}_1)$ greater than 1. Moreover, its algebraic multiplicity is one. This gives $\text{ index }({\mathbb {A}}(m_2; \mathbf {U}), \mathbf {U}_1)=-1.$

By the global bifurcation theorem (see Theorem 2.1 of Du 1996), the local bifurcation given as above can be extended to a continuum $\Gamma$, satisfying one of the alternative: (i) meets $(\bar{m}_2,\bar{R}_1,\bar{S}_1,\bar{B}_1,0)$ at $\bar{m}_2\ne \hat{m}_2$; (ii) joins $(\hat{m}_2, \bar{R}_1, \bar{S}_1, \bar{B}_1, 0)$ to $\infty$ in $(-\infty , +\infty )\times {\mathbb {X}}.$

Suppose (i) holds. Then we can find a sequence of points $(m_2^{(n)}, R_n, S_n, B_1^{(n)}, B_2^{(n)})\in (0, m_2^*)\times {\mathbb {X}}^+$ with $R_n, S_n, B_1^{(n)}, B_2^{(n)}>0\ \ \mathrm{on}\ \ [0,1]$, which converges to $(\bar{m}_2,\bar{R}_1,\bar{S}_1,\bar{B}_1,0)$ in $(0, +\infty )\times {\mathbb {X}}$. It follows from the equation for $B_2^{(n)}$, we have

$$\begin{aligned} -m_2^{(n)}= & {} \lambda _1(-f_2(R_n)-g_2(S_n), \nu _2)+r_2\\&\rightarrow \lambda _1(-f_2\left( \bar{R}_1\right) -g_2\left( \bar{S}_1\right) , \nu _2)+r_2=-\hat{m}_2. \end{aligned}$$

Hence, $\bar{m}_2=\hat{m}_2$, a contradiction. Thus (i) can not occur.

It follows from Lemma 5.3 that $0<R<R^*, 0<S<S^*, B_1>0, B_2>0$, and $\Vert B_1\Vert _\infty +\Vert B_2\Vert _\infty \le M$ for $m_1\in [\delta , m_1^*), m_2\in [\delta , m_2^*)$ and any $\delta >0$. By $L^p$ estimate and Sobolev embedding theorem, we can claim that $\Vert R\Vert , \Vert S\Vert , \Vert B_1\Vert , \Vert B_2\Vert$ are bounded. So $\Gamma$ is bounded in $[\delta , m_2^*)\times {\mathbb {X}}^+$. Since (ii) holds, one can claim that the global bifurcation branch $\Gamma$ must meet the boundary of $[\delta , m_2^*)\times {\mathbb {X}}^+$. Thus $\Gamma -\{(\hat{m}_2, \bar{R}_1, \bar{S}_1, \bar{B}_1, 0)\}\not \subseteq {\mathbb {X}}^+$ or $\Gamma$ contains a point $(m_2, R, S, B_1, B_2)\in [\delta , m_2^*)\times {\mathbb {X}}^+$ with $m_2=\delta , \text{ or } m_2=m_2^*$.

Suppose there exist $m_2^{(n)}\rightarrow m_2^*-$ and positive solution $(R_n,S_n,B_1^{(n)},B_2^{(n)})$ of (6)–(7) with $m_2=m_2^{(n)}$. Let $\hat{B}_i^{(n)}=\frac{B_i^{n}}{\Vert B_i^{n}\Vert _\infty }$. Since $0\le f_i(R_n)+g_i(S_n)\le f_i(R^*)+g_i(S^*) (i=1, 2)$, we can assume $f_i(R_n)+g_i(S_n)\rightarrow F_i(x)$ weakly in $L^2(0,1)$. Here $0\le F_i(x)\le f_i(R^*)+g_i(S^*).$ Then

$$\begin{aligned}&D(\hat{B}_2^{(n)})_{xx}-\nu _2(\hat{B}_2^{(n)})_x+\left[ f_2(R_n)+g_2(S_n)-r_2-m_2^{(n)}\right] \hat{B}_2^{(n)}=0,\quad x\in (0,1),\\&D(\hat{B}_2^{(n)})_x(0)-\nu _2\hat{B}_2^{(n)}(0)=0,\quad D(\hat{B}_2^{(n)})_x(1)-\nu _2\hat{B}_2^{(n)}(1)=0. \end{aligned}$$

Integrating the above equation from 0 to x, we obtain

$$\begin{aligned} D(\hat{B}_2^{(n)})_x(x)-\nu _2\hat{B}_2^{(n)}(x)+\int _0^x (f_2(R_n)+g_2(S_n)-r_2-m_2^{(n)})\hat{B}_2^{(n)}dx=0, \end{aligned}$$

which indicates $(\hat{B}_2^{(n)})_x(x)$ is uniformly bounded since $0<R_n<R^*,0<S_n<S^*$ and $\Vert \hat{B}_2^{(n)}\Vert _\infty =1$. Hence, $(\hat{B}_2^{(n)})_{xx}$ is uniformly bounded. Passing to a sequence if necessary, we may assume $\hat{B}_2^{(n)}\rightarrow \hat{B}_2$ in $C^1[0,1]$, and $\hat{B}_2$ is a weak solution to

$$\begin{aligned}&D(\hat{B}_2)_{xx}-\nu _2(\hat{B}_2)_{x}+(F_2(x)-r_2-m_2^*)\hat{B}_2=0,\quad x\in (0, 1),\\&D(\hat{B}_2)_{x}(0)-\nu _2\hat{B}_2(0)=0,\quad D(\hat{B}_2)_{x}(1)-\nu _2\hat{B}_2(1)=0. \end{aligned}$$

Here $0\le F_2(x)\le f_2(R^*)+g_2(S^*)$. It follows from the strong maximum principle that $\hat{B}_2>0$. Moreover, $r_2+m_2^*=-\lambda _1(-F_2(x), \nu _2)\le -\lambda _1(-f_2(R^*)-g_2(S^*), \nu _2)=m_2^*+r_2.$ The equality holds if and only if $F_2(x)=f_2(R^*)+g_2(S^*)$. Similar arguments lead to $\hat{B}_1^{(n)}\rightarrow \hat{B}_1$ in $C^1[0,1]$, and $\hat{B}_1$ satisfies

$$\begin{aligned}&D(\hat{B}_1)_{xx}-\nu _1(\hat{B}_1)_{x}+\hat{B}_1(F_1(x)-r_1-m_1)=0,\quad x\in (0, 1),\\&D(\hat{B}_1)_{x}(0)-\nu _1\hat{B}_1(0)=0,\quad D(\hat{B}_1)_{x}(1)-\nu _1\hat{B}_1(1)=0, \end{aligned}$$

where $0\le F_1(x)\le f_1(R^*)+g_1(S^*)$. By the strong maximum principle, we have $\hat{B}_1>0$. Hence, $r_1+m_1=-\lambda _1(-F_1(x), \nu _1)\le -\lambda _1(-f_1(R^*)-g_1(S^*), \nu _1)=m_1^*+r_1.$ Note that $f_i(R_n)+g_i(S_n)\rightarrow F_i(x)$ weakly in $L^2(0,1), F_2(x)=f_2(R^*)+g_2(S^*)$, and a priori estimates $0\le R_n\le R^*,\ 0\le S_n\le S^*$. It follows from the monotonicity of $f_i(R), g_i(S)$ that $F_1(x)=f_1(R^*)+g_1(S^*)$, which deduce $m_1=m_1^*$, a contradiction.

Suppose there exist $m_2^{(n)}\rightarrow 0+$ and positive solution $(R_n,S_n,B_1^{(n)},B_2^{(n)})$ of (6)–(7) with $m_2=m_2^{(n)}$. At first, we show $\Vert B_2^{(n)}\Vert _\infty \rightarrow \infty$. If not, it follows from Lemma 5.10 and (6)–(7) with $(R, S, B_1, B_2)=(R_n,S_n,B_1^{(n)},B_2^{(n)})$ and $m_2=m_2^{(n)}$ that $(R_n)_{xx}, (S_n)_{xx}, (B_1^{(n)})_{xx}, (B_2^{(n)})_{xx}$ are uniformly bounded. By $L^p$ estimates and Sobolev embedding theorem, we may assume by passing to a subsequence that $R_n\rightarrow R, S_n\rightarrow S, B_1^{(n)}\rightarrow B_1, B_2^{(n)}\rightarrow B_2$ in $C^1[0,1]$, and $(R, S, B_1, B_2)$ is a weak solution to (6)–(7) with $m_2=0$. Let $\hat{B}_2^{(n)}=\frac{B_2^{n}}{\Vert B_2^{n}\Vert _\infty }$. Similar arguments lead to $\hat{B}_2^{(n)}\rightarrow \hat{B}_2$ in $C^1[0,1]$, and $\hat{B}_2$ satisfies

$$\begin{aligned}&D(\hat{B}_2)_{xx}-\nu _2(\hat{B}_2)_{x}+\hat{B}_2(f_2(R)+g_2(S)-r_2)=0,\\&D(\hat{B}_2)_{x}(0)-\nu _2\hat{B}_2(0)=0,\ D(\hat{B}_2)_{x}(1)-\nu _2\hat{B}_2(1)=0. \end{aligned}$$

respectively. It follows from the strong maximum principle that $\hat{B}_2>0$. Integrating the equation for $\hat{B}_2$ over [0, 1], we deduce $\int _0^1(f_2(R)-r_2+g_2(S))\hat{B}_2dx=0$, which implies $f_2(R)-r_2+g_2(S)=0$ a.e in (0, 1) by Lemma 5.10. It follows that $S\equiv 0$ in [0, 1], a contradiction. Hence, $\Vert B_2^{(n)}\Vert _\infty \rightarrow \infty$. By the same reasoning as in the proof of Lemma 5.3, for given $B_1^{(n)}, B_2^{(n)}>0$, we can show that the following problem

$$\begin{aligned}&D(R_n)_{xx}+\omega _sS_n-\omega _rR_n-(f_1(R_n)-r_1)B_1^{(n)}-(f_2(R_n)-r_2)B_2^{(n)}=0,\\&D(S_n)_{xx}-\omega _sS_n+\omega _rR_n-g_1(S_n)B_1^{(n)}-g_2(S_n)B_2^{(n)}=0,\\&-(R_n)_x(0)+\alpha R_n(0)=\alpha \hat{R},\, R_n(1)=R^0,\quad (S_n)_x(0)=0,\, S_n(1)=S^0. \end{aligned}$$

has a unique solution $(R_n(x, B_1, B_2), S_n(x, B_1, B_2))$, which satisfies $f_i(R_n)-r_i\rightarrow 0$ and $g_i(S_n)\rightarrow 0$ a.e. in (0, 1) by similar arguments as in Lemma 5.3. Noting that the equation for $B_1^{(n)}$, we have $-m_1=\lambda _1(-f_1(R_n)+r_1-g_1(S_n))$. Letting $n\rightarrow \infty$, we get $m_1=0$, a contradiction.

Suppose $\Gamma -\{(\hat{m}_2, \bar{R}_1, \bar{S}_1, \bar{B}_1, 0)\}\not \subseteq {\mathbb {X}}^+.$ Then we can find a sequence of points

$$\begin{aligned} \left( m_2^{(n)}, R_n, S_n, B_1^{(n)}, B_2^{(n)}\right) \in \Gamma \cap {\mathbb {X}}^+\quad \text{ with }\quad R_n, S_n, B_1^{(n)}, B_2^{(n)}>0\quad \mathrm{on}\quad [0,1], \end{aligned}$$

which converges to $(\underline{m}_2, \underline{R}, \underline{S}, \underline{B}_1, \underline{B}_2)\in (\Gamma -\{(\hat{m}_2, \bar{R}_1, \bar{S}_1, \bar{B}_1, 0)\})\cap \partial {\mathbb {X}}^+$ in $(0, +\infty )\times {\mathbb {X}}$. Since $(\underline{R}, \underline{S}, \underline{B}_1, \underline{B}_2)\in \partial {\mathbb {X}}^+$ and $\underline{R}, \underline{S}>0$, we obtain that either $\underline{B}_1\ge 0, \underline{B}_1(x_0)=0$ for some point $x_0\in [0, 1]$ or $\underline{B}_2\ge 0, \underline{B}_2(x_0)=0$ for some point $x_0\in [0, 1].$ By the maximum principle, we have $\underline{B}_1\equiv 0$ if $\underline{B}_1(x_0)=0$ for some point $x_0\in [0, 1]$. Similarly, we can show $\underline{B}_2\equiv 0$ for the other case. Therefore, we have the following alternatives: (a) $(\underline{R}, \underline{S}, \underline{B}_1, \underline{B}_2)\equiv (R^*, S^*, 0, 0)$; (b) $(\underline{R}, \underline{S}, \underline{B}_1, \underline{B}_2)\equiv (\bar{R}_1,\bar{S}_1,\bar{B}_1,0)$; (c) $(\underline{R}, \underline{S}, \underline{B}_1, \underline{B}_2)\equiv (\bar{R}_2, \bar{S}_2, 0, \bar{B}_2).$

If $(m_2^{(n)},R_n,S_n,B_1^{(n)},B_2^{(n)})\rightarrow (\bar{m}_2, R^*, S^*, 0, 0)$, then $m_1=-\lambda _1(-f_1(R_n)-g_1(S_n))-r_1\rightarrow m_1^*$, contradicting $m_1\in [\delta , m_1^*)$. If $(m_2^{(n)},R_n,S_n,B_1^{(n)},B_2^{(n)})\rightarrow (\bar{m}_2,\bar{R}_1,\bar{S}_1,\bar{B}_1,0), -m_2^{(n)}=\lambda _1(-f_2(R_n)-g_2(S_n), \nu _2)+r_2\rightarrow \lambda _1(-f_2\left( \bar{R}_1\right) -g_2\left( \bar{S}_1\right) , \nu _2)+r_2=-\hat{m}_2.$ Hence, $\bar{m}_2=\hat{m}_2$, a contradiction. Therefore, (c) necessarily happens and the global bifurcation $\Gamma$ must meet the semi-trivial branch $\Gamma _2$ at the point $(\tilde{m}_2,\bar{R}_2,\bar{S}_2,0,\bar{B}_2)$, that is, $\Gamma \cap \Gamma _2=\{(\tilde{m}_2,\bar{R}_2,\bar{S}_2,0,\bar{B}_2)\}$. Hence there exists a sequence $(m_2^{(n)}, R_n, S_n, B_1^{(n)}, B_2^{(n)})\rightarrow (\tilde{m}_2, \bar{R}_2,\bar{S}_2,0,\bar{B}_2)$. By the equation for $B_1^{(n)}$, we have $m_1=-\lambda _1(-f_1(R_n)-g_1(S_n), \nu _1)-r_1$. Taking the limit, we get $m_1=-\lambda _1(-f_1(\bar{R}_2(\tilde{m}_2))-g_1(\bar{S}_2(\tilde{m}_2)), \nu _1)-r_1$. Namely, $\tilde{m}_2$ is determined by $m_1=-\lambda _1(-f_1(\bar{R}_2(\tilde{m}_2))-g_1(\bar{S}_2(\tilde{m}_2)), \nu _1)-r_1$. The proof is completed.

1.4 Some well-known lemmas

Finally, we state some well-known lemmas as appendix without proof, which is useful for obtaining the main results in this paper.

Consider the linear eigenvalue problem

$$\begin{aligned}&-D\varphi _{xx}+\nu \varphi _x+q(x)\varphi =\lambda \varphi ,\quad 0<x<1\nonumber \\&D\varphi _x(0)-\nu \varphi (0)=0,\quad D\varphi _x(1)-\nu \varphi (1)=0, \end{aligned}$$

(38)

where q(x) is a continuous function in [0, 1], $D,\nu$ are positive constants. Let $\psi =e^{-\frac{\nu }{D}x}\varphi (x)$. Then $\psi$ satisfies

$$\begin{aligned}&-D\left( e^{\frac{\nu }{D}x}\psi _{x}\right) _x+q(x)e^{\frac{\nu }{D}x}\psi =\lambda e^{\frac{\nu }{D}x}\psi ,\quad 0<x<1\nonumber \\&\psi _x(0)=\psi _x(1)=0. \end{aligned}$$

(39)

Lemma 5.11

(Courant and Hilbert 1953; Hsu and Lou 2010) All eigenvalues of (39) are real, and the smallest eigenvalue $\lambda _1(q(x), \nu )$ can be characterized as

$$\begin{aligned} \lambda _1(q(x), \nu )=\inf _{\psi \not =0, \psi \in H^1(0, 1)} \frac{\int _0^1e^{\frac{\nu }{D}x}(D\psi _x^2+q(x)\psi ^2)dx}{\int _0^1e^{\frac{\nu }{D}x}\psi ^2dx}, \end{aligned}$$

which corresponds to a positive eigenfunction $\psi _1$, and $\lambda _1(q(x),\nu )$ is the only eigenvalue whose corresponding eigenfunction does not change sign. Moreover,

• (i) $q_1(x)\ge q_2(x)$ implies $\lambda _1(q_1(x), \nu )\ge \lambda _1(q_2(x), \nu )$, and the equality holds only if $q_1(x)\equiv q_2(x)$;

• (ii) $q_n(x)\rightarrow q(x)$ in C[0, 1] implies $\lambda _1(q_n(x), \nu )\rightarrow \lambda _1(q(x), \nu )$.

Lemma 5.12

(Parabolic Harnack inequality) (Evans 1998) Let $\Omega \subset {\mathbb {R}}^n$ be an open set, $Q_T=\Omega \times (0,T]$ and

$$\begin{aligned} Lu=-\sum _{i,j=1}^na_{ij}(x)u_{x_ix_j}+\sum _{i=1}^nb_i(x)u_{x_i}+c(x)u, \end{aligned}$$

where the coefficients $a_{ij}(x), b_i(x), c(x)$ are continuous, and $-L$ is uniformly elliptic in $\Omega$. Assume $u\in C^{2,1}(Q_T)$ solves $u_t+Lu=0$ in $Q_T$, and $u\ge 0$ in $Q_T$. Suppose $K\subset \subset \Omega$ is connected. Then for each $0<t_1<t_2\le T$, there exists a constants C such that

$$\begin{aligned} \sup _Ku(\cdot ,t_1)\le C\inf _Ku(\cdot ,t_2). \end{aligned}$$

The constant C depends only on $K, t_1, t_2$, and the coefficients $a_{ij}(x), b_i(x), c(x)$.

Lemma 5.13

(Dancer 1983, 1984) Let $F:W\rightarrow W$ be a compact, continuously differentiable operator, W be a cone in the Banach space E with zero $\Theta$. Suppose that $W-W$ is dense in E and that $\Theta \in W$ is a fixed point of F and $A_0=F'(\Theta )$. Then the following results hold:

• (i) $\mathrm {index}_W(F, \Theta )=1$ if $r(A_0)< 1$;

• (ii) $\mathrm {index}_W(F, \Theta )=0$ if $A_0$ has eigenvalue greater than 1 and $\Theta$ is an isolated solution of $x=F(x)$, that is $h\not =A_0h$ if $h\in \overline{W}-\Theta$.

Lemma 5.14

(Wang 2010) Let $\Omega$ is a bounded domain in ${\mathbb {R}}^n$ with boundary surface $\partial \Omega \in C^{2+\gamma }, q(x)\in C(\overline{\Omega })$ and P be a positive constant such that $P-q(x)>0$ on $\overline{\Omega }$. Let $\lambda _1(q(x))$ be the principal eigenvalue of the eigenvalue problem

$$\begin{aligned}&-\sum _{i,j=1}^n\mathrm {D}_j(a_{ij}(x)\mathrm {D}_i\varphi )+q(x)\varphi =\lambda \varphi , x\in \Omega ,\\&\quad \sum _{i,j=1}^na_{ij}(x)\mathrm {D}_i\varphi \cos (\varpi ,x_j)+b(x)\varphi =0, x\in \partial \Omega , \end{aligned}$$

where $a_{i,j}(x), b(x)\in C(\partial \Omega ), b(x)\ge 0$, and $\varpi$ is the outward unit normal vector on $\partial \Omega$. Then the following conclusions hold

• (i) if $\lambda _1(q(x))<0$ then the spectral radius $r[(P-\mathrm {D}_j(a_{ij}(x)\mathrm {D}_i))^{-1}(P-q(x))]>1$;

• (ii) if $\lambda _1(q(x))>0$ then the spectral radius $r[(P-\mathrm {D}_j(a_{ij}(x)\mathrm {D}_i))^{-1}(P-q(x))]<1$;

• (iii) if $\lambda _1(q(x))=0$ then the spectral radius $r[(P-\mathrm {D}_j(a_{ij}(x)\mathrm {D}_i))^{-1}(P-q(x))]=1$.