The effects of random and seasonal environmental fluctuations on optimal harvesting and stocking

Abstract

We analyze the harvesting and stocking of a population that is affected by random and seasonal environmental fluctuations. The main novelty comes from having three layers of environmental fluctuations. The first layer is due to the environment switching at random times between different environmental states. This is similar to having sudden environmental changes or catastrophes. The second layer is due to seasonal variation, where there is a significant change in the dynamics between seasons. Finally, the third layer is due to the constant presence of environmental stochasticity—between the seasonal or random regime switches, the species is affected by fluctuations which can be modelled by white noise. This framework is more realistic because it can capture both significant random and deterministic environmental shifts as well as small and frequent fluctuations in abiotic factors. Our framework also allows for the price or cost of harvesting to change deterministically and stochastically, something that is more realistic from an economic point of view. The combined effects of seasonal and random fluctuations make it impossible to find the optimal harvesting-stocking strategy analytically. We get around this roadblock by developing rigorous numerical approximations and proving that they converge to the optimal harvesting-stocking strategy. We apply our methods to multiple population models and explore how prices, or costs, and environmental fluctuations influence the optimal harvesting-stocking strategy. We show that in many situations the optimal way of harvesting and stocking is not of threshold type.

Appendices

Appendix A: Transition probabilities

1.1 A.1. The formulation from Sect. 2

We first look at the details we need for the setting from Sect. 2. With the notation defined in Sect. 2.1, let $(x, \alpha , u)\in S_h\times {\mathcal {M}}\times \mathcal {U}$ and denote by ${\mathbb E}^{h, u}_{x, \alpha , n}$, ${\mathbb Cov}^{h, u}_{x, \alpha , n}$ the conditional expectation and covariance given by

$$\begin{aligned} \{X_m^h, \alpha ^h_m, U_m^h, m\le n, X_n^h=x, \alpha ^h_n=\alpha , U^h_n=u\}, \end{aligned}$$

respectively. Define $\Delta X^h_n = X^h_{n+1}-X^h_n$. Our objective in this subsection is to define transition probabilities $q^h ((x,k), (y, l) | u)$ so that the controlled Markov chain $\{(X^h_n, \alpha ^h_n)\}$ is locally consistent with respect to the controlled diffusion (2.3). By this we mean that the following conditions hold:

$$\begin{aligned} \begin{array}{ll}&{}\displaystyle {\mathbb E}^{h, u}_{x, k, n}\Delta X_n^h = ({b}(x, k) - u) \Delta t^h(x, k, u) + o(\Delta t^h(x, k, u)),\\ &{}\displaystyle {\mathbb Var}^{h, u}_{x, k, n}\Delta X_n^h = \sigma ^2(x, k, u)\Delta t^h(x, k, u) + o(\Delta t^h(x, k, u)),\\ &{}\displaystyle {\mathbb P}^{h, u}_{x, k, n}(\alpha ^h_{n+1}=l)= q_{kl}\Delta t^h(x, k, u)+o(\Delta t^h(x, k, u)) \quad \text {for} \quad l\ne k,\\ &{}\displaystyle {\mathbb P}^{h, u}_{x, k, n}(\alpha ^h_{n+1}=k)=1+ q_{kk}\Delta t^h(x, k, u)+o(\Delta t^h(x, k, u)),\\ &{}\displaystyle \sup \limits _{n, \ \omega } |\Delta X_n^h| \rightarrow 0 \quad \text {as}\quad h \rightarrow 0. \end{array}\end{aligned}$$

(A.1)

Using the procedure developed by Kushner (1990), for $(x, \alpha )\in S_h\times {\mathcal {M}}$ and $u\in \mathcal {U}$, we define

$$\begin{aligned} \begin{array}{ll}&{}\displaystyle Q_h (x, k, u)= \sigma ^2(x, k)+h |b(x, k)-u| -h^2q_{kk}+h,\\ &{}\displaystyle q^h \left( (x, k), (x+h, k) |u\right) =\dfrac{ \sigma ^2(x, k)/2+\big (b(x, k)-u\big )^+ h }{Q_h (x, k, u)}, \\ &{}\displaystyle q^h \left( (x, k), (x-h, k) | u\right) =\dfrac{ \sigma ^2(x, k)/2+\left( b(x, k)-u\right) ^- h}{Q_h (x, k, u)}, \\ &{}\displaystyle q^h \left( (x, k), (x, l) | u\right) =\dfrac{h^2 q_{kl} }{ Q_h (x, k, u)} \quad \text { for }\quad k\ne l, \\ &{}\displaystyle q^h \left( (x, k), (x, k) | u\right) =\dfrac{h }{ Q_h (x, k, u)},\quad \Delta t^h (x, k, u)=\dfrac{h^2}{Q_h(x, k, u)}, \end{array}\end{aligned}$$

(A.2)

where for a real number r, $r^+=\max \{r, 0\}$, $r^-=-\min \{0, r\}$. Set $q^h \left( (x, k), (y, l)|u\right) =0$ for all unlisted values of $(y, l)\in S_h\times {\mathcal {M}}$. Note that $\sup _{x, k, u} \Delta t^h (x, k, u)\rightarrow 0$ as $h\rightarrow 0$. Using the above transition probabilities, we can check that the locally consistent conditions of $\{(X^h_n, \alpha ^h_n)\}$ are satisfied.

Lemma A.1

The Markov chain $\{(X^h_n, \alpha ^h_n)\}$ with transition probabilities $\{q^h(\cdot )\}$ defined in (A.2) satisfies the local consistence in (A.1).

1.2 A.2. Variable effort harvesting-stocking strategies

For $(x, \alpha , u)\in S_h\times {\mathcal {M}}\times \mathcal {U}$, let ${\mathbb E}^{h, u}_{x, \alpha , n}$, ${\mathbb Cov}^{h, u}_{x, \alpha , n}$ denote the conditional expectation and covariance given by

$$\begin{aligned} \{X_m^h, \alpha ^h_m, U_m^{ h}, m\le n, X_n^h=x, \alpha ^h_n=\alpha , U^{h}_n=u\}, \end{aligned}$$

respectively. Define $\Delta X^h_n = X^h_{n+1}-X^h_n$. In order to approximate the process $(X(\cdot ), \alpha (\cdot ))$ given in (3.1), the controlled Markov chain $\{(X^h_n, \alpha ^h_n)\}$ must be locally consistent with respect to $(X(\cdot ), \alpha (\cdot ))$ in the sense that the following conditions hold

$$\begin{aligned}&{\mathbb E}^{h, u}_{x, k, n}\Delta X_n^h = ({b}(x, k) - ux) \Delta t^h(x, k, u) + o(\Delta t^h(x, k, u)), \nonumber \\&{\mathbb Var}^{h, u}_{x, k, n}\Delta X_n^h = \sigma ^2(x, k, u)\Delta t^h(x, k, u) + o(\Delta t^h(x, k, u)),\nonumber \\&{\mathbb P}^{h, u}_{x, k, n}(\alpha ^h_{n+1}=l)= q_{kl}\Delta t^h(x, k, u)+o(\Delta t^h(x, k, u)) \quad \text {for} \, l\ne k,\nonumber \\&{\mathbb P}^{h, u}_{x, k, n}(\alpha ^h_{n+1}=k)=1+ q_{kk}\Delta t^h(x, k, u)+o(\Delta t^h(x, k, u)),\\&\sup \limits _{n, \ \omega } |\Delta X_n^h| \rightarrow 0 \quad \text {as}\quad h \rightarrow 0. \nonumber \end{aligned}$$

(A.3)

To this end, we define the transition probabilities $q^h ((x,k), (y, l) | u)$ as follows. For $(x, k)\in S_h\times {\mathcal {M}}$ and $u\in \mathcal {U}$, let

$$\begin{aligned} \begin{array}{ll}&{}\displaystyle Q_h (x, k, u)= \sigma ^2(x, k)+h |b(x, k)-ux| -h^2q_{kk}+h,\\ &{}\displaystyle q^h \left( (x, k), (x+h, k) |u\right) =\dfrac{ \sigma ^2(x, k)/2+\big (b(x, k)-ux)^+ h }{Q_h (x, k, u)}, \\ &{}\displaystyle q^h \left( (x, k), (x-h, k) | u\right) =\dfrac{ \sigma ^2(x, k)/2+\left( b(x, k)-ux\right) ^- h}{Q_h (x, k, u)}, \\ &{}\displaystyle q^h \left( (x, k), (x, l) | u\right) =\dfrac{h^2 q_{kl} }{ Q_h (x, k, u)} \quad \text { for }\quad k\ne l, \\ &{}\displaystyle q^h \left( (x, k), (x, k) | u\right) =\dfrac{h }{ Q_h (x, k, u)},\quad \Delta t^h (x, k, u)=\dfrac{h^2}{Q_h(x, k, u)}. \end{array}\end{aligned}$$

(A.4)

Set $q^h \left( (x, k), (y, l)|u\right) =0$ for all unlisted values of $(y, l)\in S_h\times {\mathcal {M}}$.

1.3 A.3. Uncertain price functions

Let $(\phi , x, \alpha , u)\in \widehat{S}_h\times {\mathcal {M}}\times \mathcal {U}$ and denote by ${\mathbb E}^{h, u}_{\phi , x, \alpha , n}$, ${\mathbb Cov}^{h, u}_{\phi , x, \alpha , n}$ the conditional expectation and covariance given by

$$\begin{aligned} \{\Phi ^h_m, X_m^h, \alpha ^h_m, U_m^{h}, m\le n, \Phi ^h_n=\phi , X_n^h=x, \alpha ^h_n=\alpha , U^{h}_n=u\}, \end{aligned}$$

respectively. Define $\Delta X^h_n = X^h_{n+1}-X^h_n$ and $\Delta \Phi ^h_n = \Phi ^h_{n+1}-\Phi ^h_n$. In order to approximate $(\Phi (\cdot ), X(\cdot ), \alpha (\cdot ))$ given by (2.3)-and-(3.2), the controlled Markov chain $\{(\Phi ^h_n, X^h_n, \alpha ^h_n)\}$ must be locally consistent with respect to $(\Phi (\cdot ), X(\cdot ), \alpha (\cdot ))$ in the sense that the following conditions hold:

$$\begin{aligned} \begin{array}{ll}&{}\displaystyle {\mathbb E}^{h, u}_{\phi , x, k, n}\Delta X_n^h = ({b}(x, k) - u) \Delta t^h(\phi , x, k, u) + o(\Delta t^h(\phi , x, k, u)),\\ &{}\displaystyle {\mathbb Var}^{h, u}_{\phi , x, k, n}\Delta X_n^h = \sigma ^2(x, k, u)\Delta t^h(\phi , x, k, u) + o(\Delta t^h(\phi , x, k, u)),\\ &{}\displaystyle {\mathbb E}^{h, u}_{\phi , x, k, n}\Delta \Phi _n^h = {b}_0(x, k) \Delta t^h(\phi , x, k, u) + o(\Delta t^h(\phi , x, k, u)),\\ &{}\displaystyle {\mathbb Var}^{h, u}_{\phi , x, k, n}\Delta \Phi _n^h = \sigma ^2_0(x, k)\Delta t^h(\phi , x, k, u) + o(\Delta t^h(\phi , x, k, u)),\\ &{}\displaystyle {\mathbb P}^{h, u}_{\phi , x, k, n}(\alpha ^h_{n+1}=l)= q_{kl}\Delta t^h(\phi , x, k, u)+o(\Delta t^h(\phi , x, k, u)) \quad \text {for} \quad l\ne k,\\ &{}\displaystyle {\mathbb P}^{h, u}_{\phi , x, k, n}(\alpha ^h_{n+1}=k)=1+ q_{kk}\Delta t^h(\phi , x, k, u)+o(\Delta t^h(\phi , x, k, u)),\\ &{}\displaystyle \sup \limits _{n, \ \omega } \big ( |\Delta X_n^h| + |\Delta \Phi ^h_n|\big )\rightarrow 0 \quad \text {as}\quad h \rightarrow 0. \end{array}\end{aligned}$$

(A.5)

To this end, we define the transition probabilities $q^h ((\phi , x,k), (\psi , y, l) | u)$ as follows. For $(\phi , x, k)\in \widehat{S}_h\times {\mathcal {M}}$ and $u\in \mathcal {U}$, let

$$\begin{aligned} \begin{array}{ll}&{}\displaystyle Q_h (\phi , x, k, u)=\sigma ^2(x, k) +h |b(x, k)-u| +\sigma _0^2 (x, k) +h |b_0 (x, k)|-h^2q_{kk}+h,\\ &{}\displaystyle q^h \left( (\phi , x, k), (\phi , x+h, k) |u\right) = \dfrac{\sigma ^2(x, k)/2+\big (b(x, k)-u\big )^+ h }{Q_h (\phi , x, k, u)}, \\ &{}\displaystyle q^h \left( (\phi , x, k), (\phi , x-h, k) | u\right) = \dfrac{\sigma ^2(x, k)/2+\left( b(x, k)-u\right) ^- h}{Q_h (\phi , x, k, u)}, \\ &{}\displaystyle q^h \left( (\phi , x, k), (\phi , x, l) | u\right) =\dfrac{h^2 q_{kl} }{ Q_h (\phi , x, k, u)} \quad \text {for} \quad k\ne l,\\ &{}\displaystyle q^h \left( (\phi , x, k), (\phi , x, k) | u\right) =\dfrac{h }{ Q_h (\phi , x, k, u)}, \quad \Delta t^h (\phi , x, k, u)=\dfrac{h^2}{Q_h(\phi , x, k, u)},\\ &{}\displaystyle q^h \left( (\phi , x, k), (\phi +h, x, k) |u\right) = \dfrac{ \sigma ^2_0 (x, k)/2 +h b^+_0 (x, k)}{Q_h (\phi , x, k, u)},\\ &{}\displaystyle q^h \left( (\phi , x, k), (\phi -h, x, k) |u\right) = \dfrac{\sigma _0^2 (x, k)/2 +h b^-_0 (x, k) }{Q_h (\phi , x, k, u)}. \end{array}\end{aligned}$$

(A.6)

Set $q^h \left( (\phi , x, k), (\psi , y, l)|u\right) =0$ for all unlisted values of $(\psi , y, l)\in \widehat{S}_h\times {\mathcal {M}}$.

1.4 A.4. The combined effects of seasonality and Markovian switching

Recall that $\widetilde{S}_{h}: = \{(\gamma , x)=(k_1 h_1, k_2 h_2)'\in \mathbb R^2: k_i\in \mathbb {Z}_{\ge 0}, k_1\le T/h_1\}.$ Let $(\gamma , x, \alpha , u)\in \widetilde{S}_h\times {\mathcal {M}}\times \mathcal {U}$ and denote by ${\mathbb E}^{h, u}_{\gamma , x, \alpha , n}$, ${\mathbb Cov}^{h, u}_{\gamma , x, \alpha , n}$ the conditional expectation and covariance given by

$$\begin{aligned} \{\Gamma ^h_m, X_m^h, \alpha ^h_m, U_m^{h}, m\le n, \Gamma ^h_n=\gamma , X_n^h=x, \alpha ^h_n=\alpha , U^{h}_n=u\}, \end{aligned}$$

respectively. Define $\Delta X^h_n = X^h_{n+1}-X^h_n$ and $\Delta \Gamma ^h_n = \Gamma ^h_{n+1}-\Gamma ^h_n$. In order to approximate $(\Gamma (\cdot ), X(\cdot ), \alpha (\cdot ))$ given by (3.4), the controlled Markov chain $\{(\Gamma ^h_n, X^h_n, \alpha ^h_n)\}$ must be locally consistent with respect to $(\Gamma (\cdot ), X(\cdot ), \alpha (\cdot ))$ in the sense that the following conditions hold:

$$\begin{aligned} \begin{array}{ll}&{}\displaystyle {\mathbb E}^{h, u}_{\gamma , x, k, n}\Delta X_n^h = ({b}(\gamma , x, k) - u) \Delta t^h(\gamma , x, k, u) + o(\Delta t^h(\gamma , x, k, u)),\\ &{}\displaystyle {\mathbb Var}^{h, u}_{\gamma , x, k, n}\Delta X_n^h = \sigma ^2(\gamma , x, k, u)\Delta t^h(\gamma , x, k, u) + o(\Delta t^h(\gamma , x, k, u)),\\ &{}\displaystyle \Delta \Gamma _n^h = \Delta t^h(\gamma , x, k, u),\quad \text {if} \quad \gamma + \Delta t^h(\phi , x, k, u)<T,\\ &{}\displaystyle \Delta \Gamma _n^h = \gamma + \Delta t^h(\gamma , x, k, u)-T \quad \text {if} \quad \gamma + \Delta t^h(\gamma , x, k, u)\ge T,\\ &{}\displaystyle {\mathbb P}^{h, u}_{\gamma , x, k, n}(\alpha ^h_{n+1}=l)= q_{kl}\Delta t^h(\gamma , x, k, u)+o(\Delta t^h(\gamma , x, k, u)) \quad \text {for } \, l\ne k,\\ &{}\displaystyle {\mathbb P}^{h, u}_{\gamma , x, k, n}(\alpha ^h_{n+1}=k)=1+ q_{kk}\Delta t^h(\gamma , x, k, u)+o(\Delta t^h(\gamma , x, k, u)),\\ &{}\displaystyle \sup \limits _{n, \ \omega } \big ( |\Delta X_n^h| + \Delta \Gamma ^h_n \big ) \rightarrow 0 \quad \text {as}\quad h \rightarrow 0. \end{array}\end{aligned}$$

(A.7)

To this end, we define the transition probabilities $q^h ((\gamma , x,k), (\lambda , y, l) | u)$ as follows. Let $(\gamma , x, k)\in \widetilde{S}_h\times {\mathcal {M}}$ and $u\in \mathcal {U}$. If $\gamma +h_1=T$, $\gamma +h_1$ in the following definition is understood as 0. Let

$$\begin{aligned} \begin{array}{ll}&{}\!\!\!\displaystyle \Delta t^h(\gamma , x, k, u)=h_1,\\ &{}\!\!\!\displaystyle q^h \left( (\gamma , x, k), (\gamma + h_1, x+ h_2, k) | u\right) = \dfrac{\Big ( \sigma ^2(\gamma , x, k)/2+\big (b(\gamma , x, k)-u\big )^+ h_2 \Big )h_1}{h_2^2}, \\ &{}\!\!\!\displaystyle q^h \left( (\gamma , x, k), (\gamma + h_1, x-h_2, k) | u\right) = \dfrac{\Big ( \sigma ^2(\gamma , x, k)/2+\big (b(\gamma , x, k)-u\big )^- h_2 \Big )h_1}{h_2^2}, \\ &{}\!\!\!\displaystyle q^h\big ((\gamma , x, k), (\gamma +h_1, x, l)|u\big )=h_1q_{kl}, \quad \text {for} \quad l\ne k,\\ &{}\!\!\!\displaystyle q^h\big ( (\gamma , x, k), (\gamma +h_1, x, k)|u \big ) = 1 - q^h\big ( (\gamma , x,k), (\gamma +h_1, x+ h_2, k)| u \big ) \\ &{}\!\!\!\displaystyle \qquad - q^h\big ( (\gamma , x,k), (\gamma +h_1, x- h_2, k)| u \big ) - \sum \limits _{l\ne k} q^h\big ( (\gamma , x, k), (\gamma +h_1, x, l)|u \big ).\end{array}\end{aligned}$$

(A.8)

Set $q^h \left( (\gamma , x, k), (\lambda , y, l)|u\right) =0$ for all unlisted values of $(\lambda , y, l)\in \widetilde{S}_h\times {\mathcal {M}}$.

Appendix B: Continuous–time interpolation

We will present the convergence analysis for the formulation in Sect. 2. The other formulas can be handled in a similar way. Our procedure and methods are similar to those in Kushner (1990), Kushner and Dupuis (1992), Song et al. (2006). The convergence result is based on a continuous-time interpolation of the controlled Markov chain, which will be constructed to be piecewise constant on the time interval $[t^h_n, t^h_{n+1}), n\ge 0$. To this end, we define $n^h(t)=\max \{n: t^h_n\le t\}, t\ge 0.$ The piecewise constant interpolation of $\{(X^h_n,\alpha ^h_n, U^h_n)\}$, denoted by $\big (X^h(t),\alpha ^h(t), U^h(t)\big )$ is naturally defined as

$$\begin{aligned} \begin{array}{ll}&\displaystyle X^h(t) = X^h_{n^h(t)}, \quad \alpha ^h(t) = \alpha ^h_{n^h(t)}, \quad U^h(t) = U^h_{n^h(t)}, \quad t\ge 0. \end{array}\end{aligned}$$

(B.1)

Define $\mathcal {F}^h(t)=\sigma \{X^h(s), \alpha ^h(s), U^h(s): s\le t\}=\mathcal {F}^h_{n^h(t)}$. Also define

$$M^h(0)=0, \quad M^h(t) = \sum \limits _{m=0}^{n^h(t)-1} (\Delta X^h_m - {\mathbb E}^{h}_m \Delta X_m) \quad \text {for}\quad t\ge 0.$$

It is obvious that

$$\begin{aligned} X^h(t) = x + \sum \limits _{m=0}^{n^h(t)-1} {\mathbb E}^{h}_m \Delta X^h_m + M^h(t). \end{aligned}$$

(B.2)

Recall that $\Delta t^h_m = h^2/Q_h(X^h_m, \alpha ^h_m, U^h_m)$. It follows that

$$\begin{aligned} \sum \limits _{m=0}^{n^h(t)-1} {\mathbb E}^{h}_m \Delta X^h_m= & {} \sum \limits _{m=0}^{n^h(t)-1} \left[ b (X^h_m, \alpha ^h_m)+U^h_m\right] \Delta t^h_m \nonumber \\&=\int _0^t \left[ b (X^h(s), \alpha ^h(s)) + U^h(s)\right] ds \nonumber \\&-\int _{t^h_{n^h(t)}}^t \left[ b (X^h(s), \alpha ^h(s))+U^h(s)\right] ds\nonumber \\&= \int _0^t \left[ b (X^h(s), \alpha ^h(s))+U^h(s)\right] ds + \varepsilon ^h_1(t), \end{aligned}$$

(B.3)

with $\{\varepsilon _1^h(\cdot )\}$ being an $\mathcal {F}^h(t)$-adapted process satisfying

$$\begin{aligned}\lim \limits _{h\rightarrow 0} \sup \limits _{t\in [0, T_0]}{\mathbb E}|\varepsilon _1^h(t)|=0 \quad \text {for any }0<T_0<\infty .\end{aligned}$$

For simplicity, we suppose that $\inf \limits _{(x, k)}1/|\sigma (x, k)|>0$ [if this is not the case, we can use the trick from Kushner and Dupuis (1992, pp. 288–289)]. Define $w^h(\cdot )$ by

$$\begin{aligned} w^h(t) = \sum \limits _{m=0}^{n^h(t)-1} \big [ 1/\sigma (X^h_m, \alpha ^h_m)\big ] (\Delta X^h_m -{\mathbb E}^{h}_m \Delta X^h_m). \end{aligned}$$

(B.4)

Then we can write

$$\begin{aligned} M^h(t) =\int _0^t \sigma (X^h(s), \alpha ^h(s)) dw^h(s) + \varepsilon _2^h(t), \end{aligned}$$

(B.5)

with $\{\varepsilon _2^h(\cdot )\}$ being an $\mathcal {F}^h(t)$-adapted process satisfying

$$\begin{aligned}\lim \limits _{h\rightarrow 0} \sup \limits _{t\in [0, T_0]}{\mathbb E}|\varepsilon _2^h(t)|=0 \quad \text {for any }0<T_0<\infty .\end{aligned}$$

Using (B.3) and (B.5), we can write (B.2) as

$$\begin{aligned} X^h(t) = x + \int _0^t \left[ b (X^h(s), \alpha ^h(s))+U^h(s)\right] ds + \int _0^t \sigma (X^h(s), \alpha ^h(s)) dw^h(s) +\varepsilon ^h(t), \end{aligned}$$

(B.6)

where $\varepsilon ^h(\cdot )$ is an $\mathcal {F}^h(t)$-adapted process satisfying

$$\begin{aligned}\lim \limits _{h\rightarrow 0} \sup \limits _{t\in [0, T_0]}{\mathbb E}|\varepsilon ^h(t)|=0 \quad \text {for any }0<T_0<\infty .\end{aligned}$$

The performance function from (2.9) can be rewritten as

$$\begin{aligned} J^h(x, \alpha , U^h(\cdot ))= {\mathbb E}\int _0^{\infty } e^{-\delta s} p\big (X^h(s), \alpha ^h(s), U^h(s)\big )ds. \end{aligned}$$

(B.7)

Appendix C: Convergence

The convergence of the algorithms is established via the weak convergence method. To proceed, let $D[0, \infty )$ denote the space of functions that are right continuous and have left-hand limits endowed with the Skorokhod topology. All the weak analysis will be on this space or its k-fold products $D^k[0, \infty )$ for appropriate k. We follow Kushner and Dupuis (1992, Section 4.6) in order to introduce relaxed control representations, which we need in order to prove the weak convergence.

Definition C.1

Let $\mathcal {B}(\mathcal {U}\times [0, \infty ))$ be the $\sigma$-algebra of Borel subsets of $\mathcal {U}\times [0, \infty )$. An admissible relaxed control, which we will call a relaxed control, $m(\cdot )$ is a measure on $\mathcal {B}(\mathcal {U}\times [0, \infty ))$ such that

$$m(\mathcal {U}\times [0, t]) = t \quad \text {for all}\quad t\ge 0.$$

Given a relaxed control $m(\cdot )$, there is a probability measure $m_t(\cdot )$ defined on the $\sigma$-algebra $\mathcal {B} (\mathcal {U})$ such that $m(du dt)=m_t(du)dt$. Let $\mathcal {R}(\mathcal {U}\times [0, \infty ))$ denote the set of all relaxed controls on $\mathcal {U}\times [0, \infty )$.

With the given probability space, we say that $m(\cdot )$ is an admissible relaxed (stochastic) control if (i) for each fixed $t\ge 0$, $m(t, \cdot )$ is a random variable taking values in $\mathcal {R}(\mathcal {U}\times [0, \infty ))$, and for each fixed $\omega$, $m(\cdot , \omega )$ is a deterministic relaxed control; (ii) the function defined by $m(A\times [0, t])$ is $\mathcal {F}(t)$-adapted for any $A\in \mathcal {B}(\mathcal {U})$. As a result, with probability one, there is a measure $m_t(\cdot , \omega )$ on the Borel $\sigma$-algebra $\mathcal {B}(\mathcal {U})$ such that $m(dcdt) = m_t(dc)dt$.

Remark C.2

For a sequence of controls $U^h=\{U^h_n: n\in \mathbb {Z}_{\ge 0}\}$, we define a sequence of relaxed control equivalence as follows. First, we set $m_{t}^h(du)=\delta _{U^h(t)}(du)$ for $t\ge 0$, where $\delta _{U^h(t)}(\cdot )$ is the probability measure concentrated at $U^h(t)$. Then $m^h(\cdot )$ is defined by $m^h(dudt)=m_t(du)dt$; that is,

$$\begin{aligned} m^h(B\times [0, t])=\int _0^t \Big (\int _B\delta _{U^h(s)}(du)\Big )ds, \quad B\in \mathcal {B}({\mathcal {U}})\quad \text {and}\quad t\ge 0. \end{aligned}$$

Recall that $\mathcal {R}(\mathcal {U}\times [0, \infty ))$ is the space of all relaxed controls on $\mathcal {U}\times [0, \infty )$. Then $\mathcal {R}(\mathcal {U}\times [0, \infty ))$ can be metrized using the Prokhorov metric in the usual way as in Kushner and Dupuis (1992, pp. 263–264). With the Prokhorov metric, $\mathcal {R}(\mathcal {U}\times [0, \infty ))$ is a compact space. It follows that any sequence of relaxed controls has a convergent subsequence. Moreover, a sequence $(\eta _n)_{n\in \mathbb {N}}$ with $\eta _n\in \mathcal {R}(\mathcal {U}\times [0, \infty ))$ converges to $\eta \in \mathcal {R}(\mathcal {U}\times [0, \infty ))$ if and only if for any continuous functions with compact support $\Psi (\cdot )$ on $\mathcal {U}\times [0, \infty )$ one has

$$\begin{aligned} \int _{\mathcal {U}\times [0, \infty )} \Psi (u, s)\eta _n(du, ds) \rightarrow \int _{\mathcal {U}\times [0, \infty )} \Psi (u,s)\eta (du, ds) \end{aligned}$$

as $n\rightarrow \infty$. Note that for a sequence of ordinary controls $U^h=\{U^h_n: n\in \mathbb {Z}_{\ge 0}\}$, the associated relaxed control ${m}^h(dcdt)$ belongs to $\mathcal {R}(\mathcal {U}\times [0, \infty ))$. Note also that the limits of the “relaxed control representations” of the ordinary controls might not be ordinary controls, but only relaxed controls.

With the notion of relaxed control given above, we can write (B.6) and (B.7) as

$$\begin{aligned} X^h(t)= & {} x + \int _0^t \left[ b (X^h(s), \alpha ^h(s))+U(s)\right] m^h_s(du)ds\nonumber \\&\quad + \int _0^t \sigma (X^h(s), \alpha ^h(s)) dw^h(s) +\varepsilon ^h(t), \end{aligned}$$

(C.1)

$$\begin{aligned} J^h(x, \alpha , m^h(\cdot ))= & {} {\mathbb E}\int _0^{\infty } e^{-\delta s} p\big (X^h(s), \alpha ^h(s), u\big )m_s^h(du)ds. \end{aligned}$$

(C.2)

The value function defined in (2.6) can be rewritten as

$$\begin{aligned} V(x, \alpha )=\sup \{ J(x, \alpha , m(\cdot )): m(\cdot )\,\,\, \text { is an admissible relaxed control}\}, \end{aligned}$$

where

$$\begin{aligned} J(x, \alpha , m(\cdot )):= {\mathbb E}_{x, \alpha } \int _0^{\infty } e^{-\delta s} p\big (X(s), \alpha (s), u\big ) m_s(du)ds. \end{aligned}$$

Lemma C.3

The process $\{\alpha ^h(\cdot )\}$ converges weakly to $\alpha (\cdot )$, which is a Markov chain with generator $Q=(q_{kl})$.

Proof

The proof is similar to that of Yin et al. (2003, Theorem 3.1) and is therefore omitted.

Theorem C.4

Suppose Assumption 2.1 holds. Let the chain $\{(X^h_n, \alpha ^h_n) \}$ be constructed using the transition probabilities defined in (A.2), $\big (X^h(\cdot ), \alpha ^h(\cdot ), w^h(\cdot )\big )$ be the continuous-time interpolation defined in (B.1) and (B.4), $\{U^h_n\}$ be an admissible strategy and $m^h(\cdot )$ be the relaxed control representation of $\{U^h_n\}$. Then the following assertions hold.

• (a) The family of processes $H^h(\cdot )=\big ({X}^h(\cdot ), {\alpha }^h(\cdot ), m^h(\cdot ), {w}^h(\cdot )\big )$ is tight. As a result, it has a weakly convergent subsequence with limit $H(\cdot )= \big ({X}(\cdot ), {\alpha }(\cdot ), m(\cdot ), {w}(\cdot )\big ).$

• (b) Let $\mathcal { F}(t)$ be the $\sigma$-algebra generated by $\big \{H(s): s\le t\big \}$. Then $w(\cdot )$ is a standard $\mathcal { F}(t)$ adapted Brownian motion, $m(\cdot )$ is an admissible control, and

  $$\begin{aligned} X(t) = x + \int _0^t \left[ b (X(s), \alpha (s))+u\right] m_s(du)ds + \int _0^t \sigma (X(s), \alpha (s)) dw(s), \quad t\ge 0. \end{aligned}$$

  (C.3)

Proof

(a) We use the tightness criteria in Kushner (1984, p. 47). Specifically, a sufficient condition for tightness of a sequence of processes $\zeta ^h(\cdot )$ with paths in $D^k[0, \infty )$ is that for any $T_0, \rho \in (0, \infty )$,

$$\begin{aligned} \begin{array}{ll} &{}\!\!\!\displaystyle {\mathbb E}_t^h\big |\zeta ^h(t+s)-\zeta ^h(t)\big |^2\le {\mathbb E}^h_t \gamma (h, \rho ) \quad \text {for all}\quad s\in [0, \rho ], \quad t\le T_0,\\ &{}\!\!\!\displaystyle \lim \limits _{\rho \rightarrow 0}\limsup \limits _{h\rightarrow 0} {\mathbb E}\gamma (h, \rho ) =0. \end{array} \end{aligned}$$

The tightness of $\{\alpha ^h(\cdot )\}$ is obvious by the preceding lemma. The process $\{m^h(\cdot )\}$ is tight since its range space is compact. It is standard to show the tightness of $\{w^h(\cdot )\}$ and $\{X^h(\cdot )\}$—see Song et al. (2006) for details. As a result, a subsequence of $H^h(\cdot )=\big (X^h(\cdot ), \alpha ^h(\cdot ), m^h(\cdot ), w^h(\cdot )\big )$ converges weakly to the limit $H(\cdot )=\big (X(\cdot ), \alpha (\cdot ), m(\cdot ), w(\cdot )\big )$.

(b) For the rest of the proof, we assume the probability space is chosen as required by Skorokhod representation. Thus, with a slight abuse of notation, we assume that $H^h(\cdot )$ converges to the limit $H(\cdot )$ with probability one via Skorokhod representation.

To characterize ${w}(\cdot )$, let $\widetilde{k}, \widetilde{j}$ be arbitrary positive integers. Pick $t>0$, $\rho >0$ and $\{t_k: k\le \widetilde{k}\}$ such that $t_k\le t\le t+\rho$ for each k. Let $\phi _j(\cdot )$ be real-valued continuous functions that are compactly supported on $\mathcal {U}\times [0, \infty )$ for any $j\le \widetilde{j}$. Define $(\phi _j, m)_t := \int _0^t \int _{\mathcal {U}} \phi _j (u, s)m(duds)$.

Let $\Psi (\cdot )$ be a real-valued and continuous function of its arguments with compact support. By the definition of $w^h(\cdot )$ in (B.4), $w^h(\cdot )$ is an $\mathcal {F}^h(t)$-martingale. Thus, we have

$$\begin{aligned} {\mathbb E}\Psi \big (X^h(t_k), \alpha ^h(t_k), w^h(t_k), (\phi _j, m^h)_{t_k}, j\le \widetilde{j}, k\le \widetilde{k}\big )\big [ {w}^h(t+\rho )-{w}^h(t)\big ]=0, \end{aligned}$$

(C.4)

and

$$\begin{aligned}&{\mathbb E}\Psi \big (X^h(t_k), \alpha ^h(t_k), w^h(t_k), (\phi _j, m^h)_{t_k},\nonumber \\&j\le \widetilde{j}, k\le \widetilde{k}\big )\big [ \big ({w}^h(t+\rho )\big )^2-\big ({w}^h(t)\big )^2-\rho -{\varepsilon }^h(\rho )\big ]=0. \end{aligned}$$

(C.5)

By using the Skorokhod representation and the dominated convergence theorem, letting $h\rightarrow 0$ in (C.4), we obtain

$$\begin{aligned} {\mathbb E}\Psi \big (X(t_k), \alpha (t_k), w(t_k), (\phi _j, m)_{t_k}, j\le \widetilde{j}, k\le \widetilde{k}\big )\big [ {w}(t+\rho )-{w}(t)\big ]=0. \end{aligned}$$

(C.6)

Since ${w}(\cdot )$ has continuous paths with probability one, (C.6) implies that ${w}(\cdot )$ is a continuous ${\mathcal {F}}(\cdot )$-martingale. Moreover, (C.5) gives us that

$$\begin{aligned} {\mathbb E}\Psi \big (X(t_k), \alpha (t_k), w(t_k), (\phi _j, m)_{t_k}, j\le \widetilde{j}, k\le \widetilde{k}\big )\big [ \big ({w}(t+\rho )\big )^2-\big ({w}(t)\big )^2-\rho \big ]=0. \end{aligned}$$

(C.7)

Thus, the quadratic variation of w(t) is t, which implies that $w(\cdot )$ is a standard $\mathcal { F}(t)$ adapted Brownian motion.

By the convergence with probability one via Skorokhod representation, we have

$$\begin{aligned}&{\mathbb E}\left| \int _0^t \int _{\mathcal {U}} \left[ b(X^h(s), \alpha ^h(s)) + u\right] m_s^h(du)ds \right. \\&\left. - \int _0^t \int _{\mathcal {U}} \left[ b(X(s), \alpha (s)) + u\right] m_s^h(du)ds \right| \rightarrow 0 \end{aligned}$$

uniformly in t as $h\rightarrow 0$.

Also, by the weak convergence of $\{m^h(\cdot )\}$, for any bounded and continuous function $\phi (\cdot )$ with compact support, $(\phi , m^h)_\infty \rightarrow (\phi , m)_\infty$; see also Remark C.2. The weak convergence and the Skorokhod representation imply that

$$\begin{aligned} \int _0^t \int _{\mathcal {U}} \big [b(X(s), \alpha (s)) + u\big ]m_s^h(du)ds - \int _0^t \int _{\mathcal {U}}\big [b(X(s), \alpha (s)) + u\big ]m_s(du)ds\rightarrow 0 \end{aligned}$$

uniformly in t on any bounded interval with probability one.

For each positive constant $\rho$ and process ${\nu }(\cdot )$, define the piecewise constant process ${\nu }^\rho (\cdot )$ by ${\nu }^\rho (t)={\nu }(k\rho )$ for $t\in [k\rho , k\rho +\rho ), k\in \mathbb {Z}_{\ge 0}$. Then, by the tightness of $({X}^h(\cdot ), {\alpha }^h(\cdot ))$, (C.1) can be rewritten as

$$\begin{aligned} {X}^h(t)= & {} x + \int _0^t \int _{\mathcal {U}} \big [b ({X}^h(s), {\alpha }^h(s)) +u\big ]m_s^h(du)ds \\&+ \int _0^t \sigma ({X}^{h, \rho }(s), {\alpha }^{h, \rho }(s)) d {w}^h(s) + {\varepsilon }^{h, \rho }(t), \end{aligned}$$

where $\lim \limits _{\rho \rightarrow 0}\limsup \limits _{h\rightarrow 0} {\mathbb E}|{\varepsilon }^{h, \rho }(t)|=0.$ Noting that the processes ${X}^{h, \rho }(\cdot )$ and ${\alpha }^{h, \rho }(\cdot )$ take constant values on the intervals $[n\rho , n\rho +\rho )$, we have

$$\begin{aligned} \int _0^t \sigma ({X}^{h, \rho }(s), {\alpha }^{h, \rho }(s))d{w}^h(s)\rightarrow \int _0^t \sigma ({X}^\rho (s), {\alpha }^\rho (s))d{w}(s) \quad \text { as }\quad h\rightarrow 0.\end{aligned}$$

The integrals above are well defined with probability one since they can be written as finite sums. Combining the last results, we have

$$\begin{aligned} {X}(t)=x&+\int _0^t \int _{\mathcal {U}} \big [ b({X}(s), {\alpha }(s))+u\big ] m_s(du)ds \\&+\int _0^t \sigma ({X}^\rho (s), {\alpha }^\rho (t))d{w}(s) +{\varepsilon }^{\rho }(t), \end{aligned}$$

where $\lim \limits _{\rho \rightarrow 0}E|{\varepsilon }^{ \rho }(t)|=0.$ Taking the limit as $\rho \rightarrow 0$ finishes the proof.

Theorem C.5

Suppose Assumption 2.1 holds. Let $V^h(x, \alpha )$ and $V(x, \alpha )$ be the value functions defined in (2.6) and (2.9). Then $V^h(x, \alpha )\rightarrow V(x, \alpha )$ as $h\rightarrow 0$.

Proof

The proof is motivated by that of Theorem 7 in Song et al. (2006). Let $U^h(\cdot )$ be an admissible strategy for the chain $\{(X^h_n, \alpha ^h_n)\}$ and $m^h(\cdot )$ be the corresponding relaxed control representation. Without loss of generality (passing to an additional subsequence if needed), we assume that $\big ({X}^{{h}}(\cdot ), {\alpha }^{{h}}(\cdot ), {w}^{{h}}(\cdot ), {m}^{{h}}(\cdot )\big )$ converges weakly to $\big ({X}(\cdot ), {\alpha }(\cdot ), {w}(\cdot ), {m}(\cdot )\big )$. We show that as $h\rightarrow 0$ we have

$$\begin{aligned} J^h(x, \alpha , U^h(\cdot )) \rightarrow J(x, \alpha ,m(\cdot )). \end{aligned}$$

(C.8)

From (C.2) one has

$$\begin{aligned} \begin{array}{ll}J^h (x, \alpha , U^h(\cdot ))&\!\!\!\displaystyle = {\mathbb E}\int _0^{\infty } e^{-\delta s} p\big (X^h(s), \alpha ^h(s), u\big )m_s^h(du)ds. \end{array}\end{aligned}$$

(C.9)

By the weak convergence and the Skorokhod representation, as $h\rightarrow 0$,

$$\begin{aligned} J^h (x, \alpha , U^h(\cdot )) \rightarrow {\mathbb E}\int _0^{\infty } e^{-\delta s} p\big (X(s), \alpha (s), u)\big )m_s(du)ds. \end{aligned}$$

This yields that $J^h(x, \alpha , U^h(\cdot )) \rightarrow J(x, \alpha ,m(\cdot ))$ as $h\rightarrow 0$.

Next, we prove that

$$\begin{aligned} \limsup \limits _{h\rightarrow 0} V^h(x, \alpha ) \le V(x, \alpha ). \end{aligned}$$

(C.10)

For any small positive constant $\varepsilon$, let $\widetilde{U}^h(\cdot )$ be an $\varepsilon$-optimal harvesting strategy for the chain $\{(X^h_n, \alpha ^h_n)\}$; that is,

$$\begin{aligned} V^h(x, \alpha )=\sup \limits _{U^h(\cdot )} J^h(x, \alpha , U^h(\cdot ))\le J^h(x, \alpha , \widetilde{U}^h(\cdot )) + \varepsilon . \end{aligned}$$

Choose a subsequence $\{\widetilde{h}\}$ of $\{h\}$ such that

$$\begin{aligned} \limsup \limits _{{h}\rightarrow 0} V^{{h}}(x, \alpha )=\lim \limits _{\widetilde{h}\rightarrow 0}V^{\widetilde{h}} (x, \alpha )\le \limsup \limits _{\widetilde{h}\rightarrow 0} J^{\widetilde{h}}(x, \alpha , {\widetilde{U}}^{\widetilde{h}}(\cdot ))+\varepsilon .\end{aligned}$$

(C.11)

Let $\widetilde{m}^{\widetilde{h}}(\cdot )$ be the relaxed control representation of $\widetilde{U}^{\widetilde{h}}(\cdot )$. Without loss of generality (passing to an additional subsequence if needed), we may assume that $\big ({X}^{\widetilde{h}}(\cdot ), {\alpha }^{\widetilde{h}}(\cdot ), {w}^{\widetilde{h}}(\cdot ), {m}^{\widetilde{h}}(\cdot )\big )$ converges weakly to $\big ({X}(\cdot ), {\alpha }(\cdot ), {w}(\cdot ), {m}(\cdot )\big )$. It follows from our claim in the beginning of the proof that

$$\begin{aligned} \lim \limits _{\widetilde{h}\rightarrow 0} J^{\widetilde{h}}(x, \alpha , {\widetilde{U}}^{\widetilde{h}}(\cdot ))=\lim \limits _{\widetilde{h}\rightarrow 0} J^{\widetilde{h}}(x, \alpha , {\widetilde{m}}^{\widetilde{h}}(\cdot ))= J(x, \alpha , m(\cdot ))\le V(x, \alpha ), \end{aligned}$$

(C.12)

where $J(x, \alpha , m(\cdot ))\le V(x, \alpha )$ by the definition of $V(x, \alpha )$. Since $\varepsilon$ is arbitrarily small, (C.10) follows from (C.11) and (C.12).

To prove the reverse inequality $\liminf \limits _{h} V^h(x, \alpha )\ge V(x, \alpha )$, for any small positive constant $\varepsilon$, we choose a particular $\varepsilon$-optimal strategy $\overline{m}(\cdot )$ for (2.3)–(2.4) such that the approximation can be applied to the chain $\{(X^h_n, \alpha ^h_n)\}$ and the associated cost compared with $V^h(x, \alpha )$. By the chattering lemma [see for instance Kushner (1990, Theorem 3.1)], for any given $\varepsilon >0$, there is a constant $\lambda >0$ and an ordinary control $\overline{U}^\varepsilon (\cdot )$ for (2.3)–(2.4) with the following properties:

• (a) $\overline{U}^\varepsilon (\cdot )$ takes only finitely many values (denoted by $\mathcal {U}_\varepsilon$ the set of all such values);

• (b) $\overline{U}^\varepsilon (\cdot )$ is constant on the intervals $[k\lambda , k\lambda + \lambda )$ for $k\in \mathbb {Z}_{\ge 0};$

• (c) with $\overline{m}^\varepsilon (\cdot )$ denoting the relaxed control representation of $\overline{U}^\varepsilon (\cdot )$, we have that $(\overline{X}^\varepsilon (\cdot ), \overline{\alpha }^\varepsilon (\cdot ), \overline{w}^\varepsilon (\cdot ), \overline{m}^\varepsilon (\cdot ))$ converges weakly to $(\overline{X}(\cdot ), \overline{\alpha }(\cdot ), \overline{w}(\cdot ), \overline{m}(\cdot ))$ as $\varepsilon \rightarrow 0$;

• (d) $J(x, \alpha , \overline{m}^\varepsilon (\cdot ))\ge V(x, \alpha ) -\varepsilon$.

For $\varepsilon >0$ and the corresponding $\lambda$ in the chattering lemma, consider an optimal control problem for (2.3) subject to (2.4), but where the controls are constants over the interval $[k\lambda , k\lambda +\lambda )$ for $k\in \mathbb {Z}_{\ge 0}$ and take values in $\mathcal {U}_\varepsilon$ (the set of control values of $\overline{U}^\varepsilon (\cdot )$). This corresponds to controlling the discrete-time Markov process that is obtained by sampling $X(\cdot )$ and $\alpha (\cdot )$ at times $k\lambda$ for $k\in \mathbb {Z}_{\ge 0}$. Let $\widehat{U}^\varepsilon (\cdot )$ denote the $\varepsilon$-optimal control, $\widehat{m}^\varepsilon (\cdot )$ denote the relaxed control representation, and let $\widehat{X}^\varepsilon (\cdot )$ denote the associated state process. Since $\widehat{m}^\varepsilon (\cdot )$ is $\varepsilon$-optimal in the chosen class of controls, we have

$$\begin{aligned} J(x, \alpha , \widehat{m}^\varepsilon (\cdot ))\ge J(x, \alpha , \overline{m}^\varepsilon (\cdot ))-\varepsilon \ge V(x, \alpha )-2\varepsilon . \end{aligned}$$

We next approximate $\widehat{U}^\varepsilon (\cdot )$ by a suitable function of $w(\cdot )$ and $\alpha (\cdot )$. Using the same method as in Song et al. (2006), we can approximate $\widehat{U}^\varepsilon (\cdot )$ by the ordinary control $U^{\varepsilon , \theta }(\cdot )$ with the corresponding relaxed control $m^{\varepsilon , \theta }(\cdot )$ and the state process $X^{\varepsilon , \theta }(\cdot )$ such that

$$\begin{aligned} m^{\varepsilon , \theta }(\cdot )\rightarrow \widehat{m}^\varepsilon (\cdot )\end{aligned}$$

as $\theta \rightarrow 0$ and

$$\begin{aligned} J(x, \alpha , m^{\varepsilon , \theta }(\cdot ))\ge J(x, \alpha , \widehat{m}^\varepsilon (\cdot )) - \varepsilon \ge V(x, \alpha )-3\varepsilon . \end{aligned}$$

Then a sequence of ordinary controls $\{\overline{U}^h_n\}$ for the chain $\{(X^h(\cdot ), \alpha ^h(\cdot ))\}$ can be constructed with the relaxed control representation $\{\overline{m}^h_n\}$ such that as $h\rightarrow 0$, the $(X^h(\cdot ), \alpha ^h(\cdot ), \overline{m}^h(\cdot ), w^h(\cdot ))$ converges weakly to $(X^{\varepsilon , \theta }(\cdot ), \alpha (\cdot ), m^{\varepsilon , \theta }(\cdot ), w(\cdot ))$. By the optimality of $V^h(x, \alpha )$ and the weak convergence above, we have as $h\rightarrow 0$,

$$\begin{aligned} V^h(x, \alpha ) \ge J(x, \alpha , \overline{m}^h(\cdot ))\rightarrow J(x, \alpha , m^{\varepsilon , \theta }(\cdot )). \end{aligned}$$

It follows that $V^h(x, \alpha )\ge V(x, \alpha )-4\varepsilon$ for sufficiently small h. Since any subsequence of $H^h(\cdot )$ has a subsequence that converges weakly and $\varepsilon$ is arbitrary, we have $\liminf \limits _{h}V^h(x, \alpha )\ge V(x, \alpha )$. The conclusion follows. $\square$

Appendix D: Numerical experiments

1.1 D.1. Varying the cost dependency

We want to see what effect different specifications of the cost function have on the shape of the optimal harvesting rate, and in particular whether it is bang-bang. We suspect that the convexity of the cost function leads to bang-bang (all or nothing) optimal harvesting. In Fig. 9, we have as an example a cost functions of the form $C(u) = \sqrt{|u|})$. The rest of the parameters are kept the same as in Sect. 4.1. This example has concave costs, but there is a point of convexity at 0. Experiments with other partly concave cost functions show a similar pattern. Piecewise linear costs like $C(u)= |u|$, or $C(u) = \ln (1+|u|)$, lead to optimal controls that are step functions. However, when we use a purely concave cost function, like $C(u) = \ln (1+u/3)$, seen in Fig. 10, we again obtain bang-bang optimal control. Further experiments confirm the observation.

Fig. 9

Value function (left) and optimal harvesting-stocking rate (right) for a model with switching affecting $\mu (\alpha ) = 4 - \alpha$, and a cost function $C(u) = \sqrt{|u|}$. Other parameters described in Sect. 4.1

Fig. 10

Value function (left) and optimal harvesting-stocking rate (right) for a model with switching affecting $\mu (\alpha ) = 4 - \alpha$, and a cost function $C(u) = \ln (1+u/3)$. Other parameters described in Sect. 4.1

1.2 D.2. The Gompertz model of population growth

In this example, the dynamics of the population size without harvesting is given by a Gompertz model (Winsor 1932; Zeide 1993) of the form

$$\begin{aligned} dX(t)=\big [b(X(t), \alpha (t)) -U(t)X(t)\big ]dt + \sigma (X(t), \alpha (t))dw(t), \end{aligned}$$

where

$$\begin{aligned}&b(x, \alpha )=(4 - \alpha ) x \ln \dfrac{2}{x}, \quad \sigma (x, \alpha ) = x, \\&\mathcal {U}=\{u :u = k/500, k\in \mathbb {Z}, -1000\le k\le 1500\}, \quad (x, \alpha )\in \mathbb {R}_+ \times \{1, 2\}. \end{aligned}$$

The generator Q of the Markov chain $\alpha (\cdot )$ is given by

$$\begin{aligned} q_{11}=-0.1, \quad q_{12}=0.1, \quad q_{21}=0.1, \quad q_{22}=-0.1. \end{aligned}$$

Fig. 11

Value function (left) and optimal harvesting-stocking rate (right) for a Gompertz model with absolute harvesting, with switching affectin $b(x, \alpha )=(4 - \alpha ) x \ln \frac{2}{x}$, constant price $P(\cdot ) = 1$, and a cost function $C(\cdot ) = u^2/2$. Other parameters described in Sect. 4.1

Figure 11 shows the value function and the optimal stocking-harvesting rate as a function of population size X(t) and the environmental state $\alpha$. In the Gompertz model, the deterministic rate of growth near extinction goes to $\infty$, unlike in the logistic model where it is linear. Comparing these results with the ones in Fig. 2, we can also see that low population values in the Gompertz model are much less unfavorable, both in terms of future value and in terms of the benefit of extraction.

1.3 D.3. The Nisbet–Gurney model of population growth

In this model, the evolution of the population size without harvesting is given by a switched Nisbet–Gurney model; thus,

$$\begin{aligned} dX(t)=\big [b(X(t), \alpha (t)) -U(t)\big ]dt + \sigma (X(t), \alpha (t))dw(t), \end{aligned}$$

where

$$\begin{aligned}&b(x, \alpha ) = (4-\alpha ) x e^{-x} - x, \quad \sigma (x, \alpha )= x, \\&\mathcal {U}=\{u :u = k/500, k\in \mathbb {Z}, -1000\le k\le 1500\}, \quad (x, \alpha )\in \mathbb {R}_+ \times \{1, 2\}. \end{aligned}$$

The generator Q of the Markov chain $\alpha (\cdot )$ is given by

$$\begin{aligned} q_{11}=-0.1, \quad q_{12}=0.1, \quad q_{21}=0.1, \quad q_{22}=-0.1. \end{aligned}$$

Fig. 12

Value function (left) and optimal harvesting-stocking rate (right) for a Nisbet–Gurney model with absolute harvesting, with switching affecting the growth rate $b(x, \alpha ) = (4-\alpha ) x e^{-x} - x$, constant price $P(\cdot ) = 1$, and a cost function $C(\cdot ) = u^2/2$. Other parameters described in Sect. 4.1

Figure 12 shows a numerical estimation of this model. The value function has the usual features, being increasing and concave. The harvesting rate is monotonic, which is not a surprise considering the cost choice and our discussion in Sect. 4.6. Again, the control in state $\alpha = 1$ shows higher harvesting and seeding, which is consistent with this state being more favourable for growth.