The Theory of Individual Based Discrete-Time Processes

Abstract

A general theory is developed to study individual based models which are discrete in time. We begin by constructing a Markov chain model that converges to a one-dimensional map in the infinite population limit. Stochastic fluctuations are hence intrinsic to the system and can induce qualitative changes to the dynamics predicted from the deterministic map. From the Chapman–Kolmogorov equation for the discrete-time Markov process, we derive the analogues of the Fokker–Planck equation and the Langevin equation, which are routinely employed for continuous time processes. In particular, a stochastic difference equation is derived which accurately reproduces the results found from the Markov chain model. Stochastic corrections to the deterministic map can be quantified by linearizing the fluctuations around the attractor of the map. The proposed scheme is tested on stochastic models which have the logistic and Ricker maps as their deterministic limits.

Appendices

Appendix A: Derivation of the Kramers–Moyal expansion

In this appendix we discuss the derivation of the Kramers–Moyal expansion for discrete-time stochastic processes of the kind that we are interested in and which are described in Sect. 2 of the main text.

The form that we ultimately use, given by Eq. (22), is of a different type to that found in textbooks [12, 31]. However let us begin by briefly reviewing the derivation of the conventional form as given by Eq. (8). To derive this equation, we start from the Chapman–Kolmogorov equation which defines a Markov process. We shall suppress the dependence on the initial conditions $z_0$ at $t_0$, since they play no part in the derivation. Letting $z' = z - \Delta z$, the integrand of the integral appearing in Eq. (7) may be written as

$$\begin{aligned} P( \left[ z-\Delta z \right] + \Delta z,t+1 |z-\Delta z,t)P(z-\Delta z,t). \end{aligned}$$

Taylor expanding this, the integrand equals

$$\begin{aligned} \sum ^{\infty }_{\ell =0} \left( -1 \right) ^{\ell } \frac{\left( \Delta z \right) ^{\ell }}{\ell !}\,\frac{\partial ^{\ell } }{\partial z^{\ell }} \left\{ P(z + \Delta z,t+1 |z,t)P(z,t) \right\} . \end{aligned}$$

In the Chapman–Kolmogorov equation we had to integrate over $z'$. For fixed $z$, this is equivalent to integrating over $\Delta z$. We integrate $\Delta z$ from minus infinity to plus infinity, even though it is “small”. This assumes that the integrand is very highly peaked and essentially gives no contribution to the integral apart from at very small $\Delta z$.

Introducing the jump moments defined by

$$\begin{aligned} M_{\ell }(z) \equiv \int d(\Delta z) (\Delta z)^{\ell } P(z+\Delta z,t+1|z,t), \end{aligned}$$

we find that the Chapman–Kolmogorov equation reduces to Eq. (8). Another form for the $M_{\ell }(z)$ is

$$\begin{aligned} M_{\ell }(z)=\int dw\,(w-z)^{\ell } P(w,t+1|z,t), \end{aligned}$$

from which the form (9) given in the main text can be found.

However, as described in the main text, the Kramers–Moyal expansion (8) is not so useful. Essentially this stems from the fact that $\Delta z$ is not “small”, and so the expansion cannot usefully be truncated after a finite number (that is, two) terms. We showed, in Eqs. (16) and (17), that the first term in the expansion could be considerably simplified by taking the Fourier transform. We can also show this for the next term (the diffusion-like term). Its Fourier transform is

$$\begin{aligned}&\frac{1}{2N}\,\sum _{\ell =0}^{\infty } \frac{(-1)^{\ell }}{\ell !}\,\int \limits ^{\infty }_{-\infty } dz\,e^{ikz}\, \frac{\partial ^{\ell + 2}}{\partial z^{\ell + 2}} \left[ (p-z)^{\ell }\,p(1-p)\, P_t(z)\right] \nonumber \\&\quad = \frac{1}{2N}\,\sum _{\ell =0}^{\infty } \frac{1}{\ell !}\, \int \limits ^{\infty }_{-\infty } dz\,e^{ikz}\, \left( ik \right) ^{\ell + 2} \left( p - z \right) ^{\ell }\,p(1-p)\,P_t(z) \nonumber \\&\quad = \frac{1}{2N}\,\int \limits ^{\infty }_{-\infty } dz\,e^{ikz}\,e^{ik(p-z)}\, \left( ik \right) ^{2}\,p(1-p)\,P_t(z) \nonumber \\&\quad = \frac{1}{2N}\,\int \limits ^{\infty }_{-\infty } dz\,e^{ikp}\, \left( ik \right) ^{2}\,p(1-p)\,P_t(z) \nonumber \\&\quad = \frac{1}{2N}\,\int \limits ^{\infty }_{-\infty } dp\,e^{ikp}\, \left( ik \right) ^{2}\,p(1-p)\,\mathcal {P}_t(p) \nonumber \\&\quad = \frac{1}{2N}\,\int \limits ^{\infty }_{-\infty } dp\,e^{ikp}\, \frac{\partial ^2}{\partial p^2}\left[ p(1-p) \mathcal {P}_t(p) \right] . \end{aligned}$$

Taking the inverse Fourier transform of this expression gives the diffusion-like term in Eq. (19).

In order to be relatively concise, there are several aspects of the above derivation that we have treated in a rather informal manner. In the first three lines of the derivation $p$ is shorthand for $p(z)$. On the fourth line the integral takes the form $\int ^{\infty }_{-\infty } f(p)\,P_{t}(z)\,dz$, where again $p=p(z)$, and $f(p)$ in this case is $p(1-p) e^{ikp}$. The change of variable used to obtain the next line may not be $1-1$ and one may have to split up the range of integration, and carry out different transformations in different ranges. For example, in the case of the logistic map, the range is split up so that for $0 \le z \le 1/2$, the transformation is $z=z_{-} \equiv [1 - \sqrt{1 - (4p/\lambda )]}/2$, whereas for $1/2 \le z \le 1$ it is $z=z_{+} \equiv [1 + \sqrt{1 - (4p/\lambda )]}/2$. The key point is that this has no effect on $f(p)$, since $p=\lambda z(1-z)$ takes on the same value whether $z=z_{-}$ or $z=z_{+}$. Thus the only relevant part of the transformation is $P_t(z) dz = \mathcal {P}_t(p) dp$, where

$$\begin{aligned} \mathcal {P}_t(p) = \frac{1}{\lambda }\, \left[ 1 - \frac{4p}{\lambda } \right] ^{-1/2} \left\{ P_t(z=z_{-}) + P_t(z=z_{+}) \right\} . \end{aligned}$$

Having checked that the simplification occurs for $r=2$ (the diffusion-like term), we now show that it occurs to all orders in $r$. To do, this let us eliminate the jump moments $M_{\ell }$ in favor of the jump moments $J_r$ by substituting Eq. (11) into Eq. (8) to find

$$\begin{aligned} P_{t+1}(z)&= \sum _{\ell =0}^{\infty } \frac{(-1)^{\ell }}{\ell !} \frac{\partial ^{\ell }}{\partial z^{\ell }} \left[ \sum ^{\ell }_{r=0}\, {\ell \atopwithdelims ()r} \left( p - z \right) ^{\ell - r} J_r(p) P_t(z)\right] \nonumber \\&= \sum _{\ell =0}^{\infty } \sum ^{\ell }_{r=0}\,\frac{(-1)^{\ell }}{(\ell - r)!\,r!} \frac{\partial ^{\ell }}{\partial z^{\ell }} \left[ \left( p - z \right) ^{\ell - r} J_r(p) P_t(z)\right] . \end{aligned}$$

(48)

But now for any series $\sum _{\ell =0}^{\infty } \sum ^{\ell }_{r=0}\,a_{r \ell } = \sum ^{\infty }_{r=0}\,\sum _{\ell = r}^{\infty } a_{r \ell }$, and so

$$\begin{aligned} P_{t+1}(z)&= \sum _{r=0}^{\infty } \sum ^{\infty }_{\ell =r}\, \frac{(-1)^{\ell }}{(\ell - r)!\,r!} \frac{\partial ^{\ell }}{\partial z^{\ell }} \left[ \left( p - z \right) ^{\ell - r} J_r(p) P_t(z)\right] \nonumber \\&= \sum _{r=0}^{\infty }\,\frac{(-1)^{r}}{r!}\,\sum ^{\infty }_{m=0}\, \frac{(-1)^{m}}{m!} \frac{\partial ^{r+m}}{\partial z^{r+m}} \left[ \left( p - z \right) ^{m} J_r(p) P_t(z)\right] . \end{aligned}$$

(49)

We now take the Fourier transform of Eq. (49) to find:

$$\begin{aligned} \tilde{P}_{t+1}(k)&= \int \limits ^{\infty }_{-\infty } dz\,e^{ikz} P_{t+1}(z) \nonumber \\&= \sum _{r=0}^{\infty }\,\frac{(-1)^{r}}{r!}\,\int \limits ^{\infty }_{-\infty } dz\,e^{ikz}\, \sum ^{\infty }_{m=0}\,\frac{(-1)^{m}}{m!} \frac{\partial ^{r+m}}{\partial z^{r+m}} \left[ \left( p - z \right) ^{m} J_r(p) P_t(z)\right] \nonumber \\&= \sum _{r=0}^{\infty }\,\frac{1}{r!}\,\int \limits ^{\infty }_{-\infty } dz\,e^{ikz}\, \sum ^{\infty }_{m=0}\,\frac{1}{m!} \left( ik \right) ^{r+m}\, \left[ \left( p - z \right) ^{m} J_r(p) P_t(z)\right] \nonumber \\&= \sum _{r=0}^{\infty }\,\frac{1}{r!}\,\int \limits ^{\infty }_{-\infty } dz\,e^{ikz}\, e^{ik(p-z)}\,\left( ik \right) ^{r}\,J_r(p) P_t(z) \nonumber \\&= \sum _{r=0}^{\infty }\,\frac{1}{r!}\,\int \limits ^{\infty }_{-\infty } dz\,e^{ikp}\, \left( ik \right) ^{r}\,J_r(p) P_t(z) \nonumber \\&= \sum _{r=0}^{\infty }\,\frac{1}{r!}\,\int \limits ^{\infty }_{-\infty } dp\,e^{ikp}\, \left( ik \right) ^{r}\,J_r(p) \mathcal {P}_t(p) \nonumber \\&= \sum _{r=0}^{\infty }\,\frac{(-1)^r}{r!}\,\int \limits ^{\infty }_{-\infty } dp\,e^{ikp}\, \frac{\partial ^{r}}{\partial p^{r}}\,\left[ J_r(p) \mathcal {P}_t(p) \right] . \end{aligned}$$

(50)

Taking the inverse Fourier transform of this equation we find Eq. (22).

Appendix B: The Form of the Jump Moments $J_r$ for large $N$.

The truncation of the Kramers–Moyal expansion (22) at the $r=2$ term, relies on showing that the $J_r$ fall off like $N^{-2}$ for $r > 2$. In Sect. 2, we have already shown that $J_1 = 0$ (and that $J_0 = 1$). For orientation, let us determine $J_2$:

$$\begin{aligned} \left\langle \left[ z_{t+1} - p_t \right] ^2 \right\rangle _{z_{t}=z}&= N^{-2}\,\left\langle n^2_{t+1} \right\rangle _{n_{t}=m} - 2N^{-1}p\,\left\langle n_{t+1} \right\rangle _{n_{t}=m} + p^2 \nonumber \\&= N^{-2}\,\sum _{n} n^2 P(n,t+1|m,t) - 2N^{-1}p\,\sum _{n} n P(n,t+1|m,t) + p^2 \nonumber \\&= N^{-2}\,\sum _{n} n^2 Q_{n m} - 2N^{-1}p\,\sum _{n} n Q_{n m} + p^2 \nonumber \\&= N^{-2}\left[ Np(1-p) + N^{2}p^{2} \right] - 2N^{-1}p\,Np + p^{2} \nonumber \\&= N^{-1}p(1-p). \end{aligned}$$

(51)

More generally in this appendix we want to investigate the form of $J_r$ for large $N$ and any value of $r$. Now,

$$\begin{aligned} J_{r}(p)&= \left\langle \left[ z_{t+1}-p_t \right] ^r \right\rangle _{z_{t}=z} \nonumber \\&= \frac{1}{N^{r}}\,\left\langle \left[ n_{t+1} - (Np_t) \right] ^r \right\rangle _{n_{t}=m} \nonumber \\&= \frac{1}{N^{r}}\,\sum ^{r}_{s=0}\,{r \atopwithdelims ()s}\, \left( - 1 \right) ^{s}\,\left( Np \right) ^{r - s}\,\left\{ \sum _n n^s Q_{n m} \right\} . \end{aligned}$$

(52)

If we denote the $s$th moment of the binomial distribution by $\mu _{s}(p)$, then Eq. (52) becomes

$$\begin{aligned} J_{r}(p) = \frac{1}{N^{r}}\,\sum ^{r}_{s=0}\,{r \atopwithdelims ()s}\, \left( - 1 \right) ^{s}\,\left( Np \right) ^{r - s}\,\mu _{s}(p). \end{aligned}$$

(53)

So to calculate the $J_{r}(p)$, the moments of the binomial distribution need to be known. We have seen that $J_1(p)=0$ and $J_{2}(p)=N^{-1}p(1-p)$. We can find $J_{3}(p)$ from

$$\begin{aligned} J_{3}(p)&= \frac{1}{N^{3}}\,\left[ (Np)^3 - 3 (Np)^2 \mu _1 + 3 (Np) \mu _2 - \mu _3 \right] \nonumber \\&= - \frac{p(1-p)(1-2p)}{N^2}, \end{aligned}$$

(54)

which goes like $N^{-2}$ and so does not contribute to the generalization of the Fokker–Planck equation. We now show that all $J_{r}(p)$ for $r \ge 3$ fall off this fast. We use properties of the factorial moments to prove this result. Recall that the factorial moments are defined by

$$\begin{aligned} \nu _r(p) \equiv \langle n(n-1)\ldots (n-r+1) \rangle = \sum ^{N}_{n=0}\, n(n-1)\ldots (n-r+1)\,{N \atopwithdelims ()n} p^n (1-p)^{N-n}, \end{aligned}$$

(55)

and can be shown to be given by

$$\begin{aligned} \nu _r(p) = N(N-1)\ldots (N-r+1) p^r. \end{aligned}$$

(56)

The proof is straightforward. Define

$$\begin{aligned} \phi (p,w) = \sum ^{N}_{n=0}\,{N \atopwithdelims ()n} p^n (1-p)^{N-n} w^n = \left[ pw + (1-p) \right] ^N. \end{aligned}$$

Then differentiating $r$ times with respect to $w$ and setting $w=1$ gives $\nu _r(p)$, and so the result.

Now,

$$\begin{aligned} \nu _r(p) =\langle n(n-1)\ldots (n-r+1) \rangle = \langle n \left[ n^{r-1} - A n^{r-2} + C_1 n^{r-3} + \cdots \right] \rangle , \end{aligned}$$

where $A = 1 + \cdots + (r-1) = r(r-1)/2$ and where $C_1$ (like the constants $C_2$ and $C_3$ appearing below) is constant which need not concern us. Here we assume that $r \ge 3$. So we have,

$$\begin{aligned} \nu _r(p)&= \langle n^r \rangle - A \langle n^{r-1} \rangle + C_1 \langle n^{r-2} \rangle + \cdots \nonumber \\ \mathrm {and} \ \ \nu _{r-1}(p)&= \langle n^{r-1} \rangle - C_2 \langle n^{r-2} \rangle + \cdots . \end{aligned}$$

These results imply that

$$\begin{aligned} \langle n^r \rangle = \nu _r + A \langle n^{r-1} \rangle + \cdots = \nu _r + A \nu _{r-1} + C_3 \nu _{r-2} + \cdots . \end{aligned}$$

We can now use the result (56): $\nu _r(p) = (Np)^r -Ap (Np)^{r-1} + \mathcal {O}(N)^{r-2}$ to see that

$$\begin{aligned} \mu _r(p) \equiv \langle n^r \rangle = (Np)^r + A(1-p) (Np)^{r-1} + \mathcal {O}(N)^{r-2}. \end{aligned}$$

(57)

for $r \ge 3$. Since $A=r(r-1)/2$ and $\mu _2(p) = (Np)^2 + Np(1-p)$, the result is also true for $r=0,1,2$ with the correction term being exactly zero.

Substituting Eq. (57) into Eq. (53) gives

$$\begin{aligned} J_{r}(p)&= \frac{1}{N^{r}}\,\sum ^{r}_{s=0}\,{r \atopwithdelims ()s}\, \left( - 1 \right) ^{s}\,\left[ \left( Np \right) ^{r} + A(1-p) \left( Np \right) ^{r - 1} + \mathcal {O}\left( N ^{r - 2} \right) \right] \nonumber \\&= p^{r}\,\sum ^{r}_{s=0}\,{r \atopwithdelims ()s}\,\left( - 1 \right) ^{s}\, \left[ 1 + \frac{1}{2} s(s-1) (1-p) \left( Np \right) ^{-1} + \mathcal {O}\left( N^{-2} \right) \right] . \end{aligned}$$

(58)

But

$$\begin{aligned} \sum ^r_{s=0}\,{r \atopwithdelims ()s}\,\left( - 1 \right) ^{s} = 0, \ \ \mathrm {for\ } r \ne 0, \end{aligned}$$

and if $r > 2$

$$\begin{aligned} \sum ^r_{s=0}\,{r \atopwithdelims ()s}\,\left( - 1 \right) ^{s}\,s(s-1) = 0. \end{aligned}$$

Therefore Eq. (58) proves that $J_r(p)=\mathcal {O}(N^{-2})$ for $r > 2$.