Abstract
We propose an optimal control framework to describe intraseasonal predator–prey interactions, which are characterized by a continuoustime dynamical model comprising predator and prey density, as well as the energy budget of the prey over the length of a season. The model includes a timedependent decision variable for the prey, representing the portion of the prey population in time that is active, as opposed to diapausing (a state of physiological rest). The predator follows autonomous dynamics and accordingly it remains active during the season. The proposed model is a generalization of the classical Lotka–Volterra predator–prey model towards nonautonomous dynamics that furthermore includes the effect of an energy variable. The model has been inspired by a specific biological system of predatory mites (Acari: Phytoseiidae) and prey mites (socalled fruittree red spider mites) (Acari: Tetranychidae) that feed on leaves of apple trees—its parameters have been instantiated based on laboratory and field studies. The goal of the work is to understand the decisions of the prey mites to enter diapause (a state of physiological rest) given the dynamics of the predatory mites: this is achieved by solving an optimization problem hinging on the maximization of the prey population contribution to the next season. The main features of the optimal strategy for the prey are shown to be that (1) once in diapause, the prey does not become active again within the same season and hence diapause is an irreversible process; (2) for the vast majority of parameter space, the portion of prey individuals entering diapause within the season does not decrease in time; (3) with an increased number of predators, the optimal population strategy for the prey is to start diapause earlier and to enter diapause more gradually. This optimal population strategy will be studied for its ESS properties in a sequel to the work presented in this article.
Similar content being viewed by others
Notes
Mixed strategies are often referred to as singular or intermediate strategies.
References
Akhmetzhanov A, Grognard F, Mailleret L (2011) Optimal lifehistory strategies in seasonal consumerresource dynamics. Evolution 65(11):3113–3125
Başar T, Olsder GJ (1999) Dynamic noncooperative game theory. SIAM, Philadelphia
Bellman R (1957) Dynamic programming. Princeton University Press, Princeton
Bertsekas DP (2003) Nonlinear programming. Athena Scientific, Belmont
Danks H (1987) Insect dormancy: an ecological perspective. Biological Survey of Canada, Ottawa
DeLeenheer P, Aeyels D (2000) Accessibility properties of controlled Lotka–Volterra systems. In: Proceedings of the 39th IEEE conference on decision and control, Sydney, USA, pp 3977–3981
Diekmann O, Gyllenberg M, Metz JAJ (2007) Physiologically structured population models: towards a general mathematical theory. In: Takeuchi Y, Iwasa Y, Sato K (eds) Biological and medical physics, biomedical engineering. Springer, Berlin, pp 5–20
Dreyfus S (2002) Richard Bellman on the birth of dynamic programming. Oper Res 50(1):48–51
Fitzgerald J, Solomon M (1991) Diapause induction and duration in the phytoseiid mite Typhlodromus pyri. Exp Appl Acarol 12:135–145
Goh B, Leitmann G, Vincent T (1974) Optimal control of a preypredator system. Math Biosci 19:263–286
Gouzé J (1993) Global behavior of ndimensional Lotka–Volterra systems. Math Biosci 113:231–242
Gouzé J (1994) Global stabilization of ndimensional population models by a positive control. In: Proceedings of the 33rd IEEE conference on decision and control, Orlando, USA, pp 1335–1336
Gyllenberg M, Hanski I, Lindström T (1996) A predator–prey model with optimal suppression of reproduction in the prey. Math Biosci 134(2):119–152
Gyllenberg M, Service R (2011) Necessary and sufficient conditions for the existence of an optimisation principle in evolution. J Math Biol 62(3):359–369
Helle W, Sabelis MW (eds) (1985a) Spider mites: their biology, natural enemies and control. World crop pests, vol 1A. Elsevier, Amsterdam, p 403
Helle W, Sabelis MW (eds) (1985b) Spider mites: their biology, natural enemies and control. World crop pests, vol 1B. Elsevier, Amsterdam, p 458
Hopper JL (1987) Opportunities and handicaps of antipodean scientists: A. J. Nicholson and V. A. Bailey on the balance of animal populations. Hist Rec Aust Sci 7(2):179–188
Kirk DE (1970) Optimal control theory, an introduction. Prentice Hall, Englewood Cliffs
Kolmanovskii V, Koroleva N (1991) Control of a predator–prey system with intraspecies competition. J Appl Math Mech 55:588–595
Kroon A, Veenendaal R, Bruin J, Egas M, Sabelis MW (2004) Predation risk affects diapause induction in the spider mite Tetranychus urticae. Exp Appl Acarol 34:307–314
Kroon A, Veenendaal RL (2005) Tradeoff between diapause and other lifehistory traits in the spider mite Tetranychus urticae. Ecol Entomol 23(3):298–304
Kroon A, Veenendaal RL, Bruin J, Egas M, Sabelis MW (2008) Sleeping with the enemy—predatorinduced diapause in mite. Naturwissenschaften 95:1195–1198
Kroon A, Veenendaal RL, Egas M, Bruin J, Sabelis MW (2005) Diapause incidence in the twospotted spider mite increases due to predator presence, not due to selective predation. Exp Appl Acarol 35(1–2):73–81
Lika K, Kooijman SALM (2003) Life history implications of allocation to growth versus reproduction in dynamic energy budgets. Bull Math Biol 65:809–834
Lotka AJ (1920) Undamped oscillations derived from the law of mass action. J Am Chem Soc 42:1595–1599
McNamara J, Houston AI (1996) Statedependent life histories. Nature 380(6571):215–221
Melikyan A (1994) Necessary optimality conditions for a singular surface in the form of synthesis. J Optim Theory Appl 82(2):203–217
Melikyan A (1998) Generalized characteristics of first order PDEs: applications in optimal control and differential games. Birkhäuser, Boston
Melikyan A, Olsder G (2010) Boundary singularities and characteristics of Hamilton–Jacobi equation. J Dyn Control Syst 16(1):77–99
Metz JAJ, Mylius S, Diekmann O (2008) Even in the odd cases when evolution optimizes, unrelated population dynamical details may shine through in the ESS. Evol Ecol Res 10:655–666
Metz JAJ, Mylius S, Diekmann O (2008) When does evolution optimize? Evol Ecol Res 10:629–654
Pachepsky E, Nisbet R, Murdoch W (2008) Between discrete and continuous: consumerresource dynamics with synchronized reproduction. Ecology 89(1):280–288
Perrin N, Sibly R (1993) Dynamicmodels of energy allocation and investment. Ann Rev Ecol Syst 24:379–410
Perrin N, Sibly R, Nichols N (1993) Optimalgrowth strategies when mortality and productionrates are sizedependent. Evol Ecol 7(6):576–592
Persson L, Roos AD (2003) Adaptive habitat use in size structured populations: linking individual behavior to population processes. Ecology 84(5):1129–1139
Stanková K (2009) On Stackelberg and inverse Stackelberg games& their applications in the optimal toll design problem, the energy market liberalization problem, and in the theory of incentives. Ph.D. thesis, Delft University of Technology, Delft, The Netherlands
Tauber M, Tauber C, Masaki S (1986) Seasonal adaptations of insects. Oxford University Press, New York
Veerman A (1992) Diapause in phytoseiid mites: a review. Exp Appl Acarol 14:1–60
Volterra V (1926) Variations and fluctuations of the number of the individuals in animal species living together. In: Chapman R (ed) Animal Ecology. McGrawHill, New York
Volterra V (1927) Variations and fluctuations in the number of coexisting animal species, (1927). In: Scudo F, Ziegler J (eds) The golden age of theoretical ecology: 1923–1940. Springer, Berlin, pp 65–234
Wright S (1932) The roles of mutation, inbreeding, crossbreeding, and selection in evolution. In: Proceedings of the sixth international congress on genetics, pp 355–366
Author information
Authors and Affiliations
Corresponding author
Appendices
Appendix A: Why energy has to be included in the model (quantitative argument from Sect. 2.3)
Let us consider the instantiated model in (3.2)–(3.4), and assume that the populationdependent environmental feedback is not explicit, namely \(E(t)=1, t\in [0,T]\). Then the optimization problem (3.1)–(3.4) simplifies to
Introducing a value function \(W(p, r, t,u)=\int _{Tt}^T (1u ) r \mathrm {d}t^{\prime }\) and the new variables \(b \stackrel{\mathrm{def}}{=}\frac{\partial W}{\partial p}\) and \(c \stackrel{\mathrm{def}}{=}\frac{\partial W}{\partial r}\), it is possible to show that the optimal control takes the form \(u^{*}= \mathrm{Heav}\, \fancyscript{C}\), where
which, as in the more general case, implies that the optimal behavior of the prey is again fully independent of the prey population level, and that \(u^{*}= 0\) at the end of the season.
The characteristic system can be expressed as follows:
With reference to the reverse time \(\tau \), selecting a \(u=0\) and transversal conditions \(b(0)=c(0)=0\) yields:
Hence, the condition \(\fancyscript{C} = 0\), related to mixed optimal strategies and \(u^{s}\in (0,1)\), can take place at time \(\tau _1\) if the following condition holds:
Assuming that \(r(0)>0\), the equality in (6.1) is satisfied if \({p(0)} = {\frac{3+4\,{\mathrm{e}^{\frac{\tau _1}{20}}}}{{20\,\mathrm{e}^{\frac{\tau _1}{20}}} ( 1+{\mathrm{e}^{\frac{\tau }{20}}}) }}\). Assuming that \(p(0) \ge 0\), a time \(\tau _1\) such that \(u(\tau _1)\ne 0\) exists if \(\tau _1 \ge 20 \ln \frac{3}{4} \approx 5.75\). Furthermore, pairs \((\tau _1,p(0))\) related to possible nonzero optimal strategies for the prey are those corresponding to the curve depicted in Fig. 7. For all the values of \((\tau _1,p(0))\), such that \({p(0)} < {\frac{3+4\,{\mathrm{e}^{\frac{\tau _1}{20}}}}{{20\,\mathrm{e}^{\frac{\tau _1}{20}}}( 1+{\mathrm{e}^{\frac{\tau }{20}}}) }}\) (below the curve in Fig. 7) the optimal strategy of the prey is to switch to an active state, namely \(u^{*}(\tau ) = 1\,\forall \tau \ge \tau _1\). For all the remaining values of \((\tau _1,p(0))\), which in practice means for \(p(0) > 0.05\) and any \(\tau _1\), the optimal strategy of the prey is to remain in diapause for the entire summer season: this in practice would deplete the energy of the prey and therefore would lead to its death. This leads to the conclusion that modeling the interactions of the considered system without including the energy variable leads to inconsistent outcomes.
The argument can be generalized to models that are parameterized as in (3.3), (3.4), that is where \(\alpha \) and \(\gamma \) have not been fixed to the values \(1/20\) and \(1/5\), respectively. It can be algebraically shown that the maximal value of he curve \(p(0)\) is upper bounded by the quantity \((\gamma  \alpha )\) which, given the ranges of interest, is again a very small quantity.
Appendix B: Proof of Proposition 3.1
As portrayed in Fig. 8, let us assume that there exists a time \(\tau _{3}: u^{*}(\tau _3) = 1\) and there exists a \(\delta >0: \forall \tau \in (\tau _3, \tau _3 +\delta ]: u^{*}(\tau ) \in [0,1)\). Let variable \(E_2\stackrel{\mathrm{def}}{=}E(\tau _2)\). Setting \(u^{*}= 1\) for \(\tau >\tau _2\), the energy \(E(\tau )\) satisfies
therefore \(E_3\stackrel{\mathrm{def}}{=}E(\tau _3)=1+ (E_21)\mathrm{e}^{d\,(\tau _3\tau _{2})}\). Moreover, let us introduce the following variables: \(a_3 \stackrel{\mathrm{def}}{=}a(\tau _3), c_3 \stackrel{\mathrm{def}}{=}a(\tau _3), p_3 \stackrel{\mathrm{def}}{=}p(\tau _3)\), and \(r_3\stackrel{\mathrm{def}}{=}r(\tau _3)\).
At time \(\tau =\tau _3\), the condition \(\fancyscript{S}=0\) has to be satisfied. Substituting expression for \(E_3\) into equation \(\fancyscript{S}=0\) leads to
From this equation we can express \(E_2\) as:
where we have set \(\Delta =\tau _3\tau _2>0\).
Since \(E_2\in (0,1]\), the inequality (7.1) is satisfied only if
Recall that if there is a second event at time \(\tau _3\), then \(\fancyscript{S}(\tau _3)=0,\) but also \(\fancyscript{S}^{\prime }(\tau _3)=\fancyscript{S}^{\prime \prime }(\tau _3)= \cdots =0\). From the equation \(\fancyscript{S}^{\prime }=0\) the parameter \(a_3\) can be expressed in terms of the other variables:
Furthermore, \(b_3\) can be expressed from the equation \(\fancyscript{S}^{\prime \prime }=0\) (after substituting (7.3) into this same equation), and likewise \(c_3\) can be expressed from the condition \(\fancyscript{S}^{\prime \prime \prime }=0\) (after substituting expressions for \(a_3\) and \(b_3\) into this equation)—we omit reporting the expressions for \(b_3\) and \(c_3\), as their computation is straightforward.
Denoting the nominator and denominator of the fraction in Eq. (7.2) as “\(\mathrm{Nom}\)” and “\(\mathrm{Den}\)”, respectively, there are two cases characterizing the necessary conditions in (7.2), for the existence of a \(\tau _3 > \tau _2\) such that \(u(\tau )=1\) for \(\tau \in [\tau _2,\tau _3]\) and of a \(\delta >0: \forall \tau \in (\tau _3,\tau _3+\delta ], u(\tau )\in (0,1)\):

Case 1
$$\begin{aligned}&0 \le \mathrm{Nom} ,\end{aligned}$$(7.4)$$\begin{aligned}&0 < \mathrm{Den} ,\end{aligned}$$(7.5)$$\begin{aligned}&0 < \mathrm{Den}\mathrm{Nom}. \end{aligned}$$(7.6) 
Case 2
$$\begin{aligned}&0 \ge \mathrm{Nom}, \end{aligned}$$(7.7)$$\begin{aligned}&0 > \mathrm{Den}, \end{aligned}$$(7.8)$$\begin{aligned}&0 > \mathrm{Den} \mathrm{Nom}. \end{aligned}$$(7.9)
The quantities \(\mathrm{Den}\) and \(\mathrm{Nom}\) can be then written with \(E_2\) expressed by (7.1).
In the following, the two cases are considered in detail. Case 1: Condition (7.4) implies
whereas condition (7.5) implies
and condition (7.6) implies
Note that inequalities (7.10) and (7.11) imply either
and
or
and
We have substituted the expressions for \(a_3, b_3,\) and \(c_3\) into inequalities (7.13), (7.14), (7.15) and (7.16), respectively.
Assuming that \(p_3 > 0, r_3 > 0, E^{f}\in (0,1]\), and \(d>\frac{1}{250}\), in both cases it can be shown that
which contradicts Eq. (7.12) and therefore also Eq. (7.6).
Case 2: Condition (7.7) implies
whereas condition (7.8) implies
and condition (7.9) implies
Note that the inequalities in (7.17) and (7.18) imply either
and
or
and
We have substituted expressions for \(a_3, b_3,\) and \(c_3\) into inequalities (7.20), (7.21), (7.22) and (7.23), respectively. Assuming that \(p_3 > 0, r_3 > 0, E^{f}\in (0,1]\), and \(d>\frac{1}{250}\), in both cases it can be shown that
which contradicts Eq. (7.19) and therefore also Eq. (7.9). \(\square \)
Appendix C: Proof of Proposition 3.2
If \(p=0,\) the characteristic system takes the following form in reverse time:
Furthermore, the surface \(\fancyscript{S}\) can be expressed as
It is again easy to check that \(u^{s}(0) = 0\) in reverse time, and that \(\tau _1=500 \ln w\), with \(w\) being the smallest root of the following polynomial
From (8.1), the energy level \(E^1\) of the prey entering diapause is \(E(\tau _1)=E^{f}e^{\frac{\tau _1}{250}}\). Figure 2 represents the values of \(\tau _1\) as functions of \(d\) and \(E^{f}\). Notice that \(E^1\ge 1\) for \(E^{f}\ge 0.9775\), therefore in the following we will assume that \(E^{f}\in (0,0.9775)\). (Moreover, recall that \(d>1/250\).)
Equations \(\fancyscript{S}=0, \fancyscript{S}^{\prime }=0,\) and \(\fancyscript{S}^{\prime \prime }=0\) allow expressing \(a(\tau ), c(\tau ),\) and \(u^{s}(\tau )\) in terms of \(r(\tau )\) and \(E(\tau ),\) respectively. Of interest to this proof, the expression for the mixed strategy \(u^{s}(\tau )\) reads as the ratio of two polynomials:
\((E ( 25\, E^{2}29\cdot 10^3 \,d E^{3}+125\cdot 10^3 \,{d}^{2} E^{2}+116\, E^{3 }13\cdot 10^3 \,d\,E +230\cdot 10^3 \,{d}^{2}+67{,}750\,d E^{2}375\cdot 10^3 \,{d}^{2}E))/( 4\,(125\,d \,E^{2} 5\cdot 10^3 \,{d}^{2}E 21\, E^{4}1{,}250\cdot 10^3 \,{d}^{3}+100\,E^{5}9\cdot 10^3 \,d E^ {3}+54{,}250\,d E^{4}+531{,}250\,{d}^{2 } E^{2}875\cdot 10^3 \,{d}^{2} E^{3}+1{,}250\cdot 10^3 \,{d}^{3}E +250\cdot 10^3 \,{d}^{2} E^{4} 25\cdot 10^3 \, E^{5}d))\).
Remarkably, the expression is independent of \(r\), which aligns to earlier outcomes on the independence of the prey population density.
Since \(u^{s}\) cannot by definition take values that are lower than \(0\) or greater than \(1\), we denote the values for \((d,E)\) for which \(u^{s}\in [0,1]\) as “feasible” and we will call those for which \(u^{s}\ne [0,1]\) “unfeasible.” Figure 9 represents the feasibility regions for \((d,E)\), assuming \(d\in (\frac{1}{250},1]\) and \(E\in (0,0.9775]\). The feasible region for the given parameters corresponds to possible trajectories that have mixed strategies, whereas the unfeasible region relates to trajectories that stay always in diapause mode (\(u^{s}= 0\)), or discontinuously switch to \(u^{s}= 1\). In either case, trajectories for the optimal strategy will be nondecreasing in reverse time.
From \(\fancyscript{S}^{\prime \prime \prime }=0\) we can derive the expression of \(\frac{\mathrm{d}u^{s}}{\mathrm{d}\tau }\). Figure 10 plots the parts of the feasibility region for which \(u^{s}\) is increasing and decreasing, respectively.
With focus on the feasible region, the following observations can be made:

For \(d>0.15\), no mixed strategy takes place, since the parameter space corresponds to the unfeasible region.

For \(\frac{1}{250}<d<0.15\), either

no mixed strategy takes place if \(E\) is small, else

a mixed strategy \(u^{s}\) takes place.

Let us further elaborate on the latter case (presence of mixed strategies) with the help of Fig. 10. Recall that
and assume the dynamics at time \(\tau _1\) land in a region where \(u^{s}\) has negative derivative. It can be numerically shown that the values of \(u^{s}\) are quite small in this region. Because of the values of \(E>0.5\) and \(1/250 < d < 0.15, E^{\prime } > 0\) persistently, given that \(d u^{s}/dt < 0\). This regime will be sustained until \(E = 1\), which will force \(u^{s}\) to switch to \(u^{*}= 1\). Given the values of the quantities of interest, this will happen for a short interval—if \(d = \frac{1}{250}\), the interval will be approximately less than \(0.5\), whereas if \(d \sim 0.15\), the interval will be even smaller. \(\square \)
Rights and permissions
About this article
Cite this article
Staňková, K., Abate, A. & Sabelis, M.W. Irreversible prey diapause as an optimal strategy of a physiologically extended Lotka–Volterra model. J. Math. Biol. 66, 767–794 (2013). https://doi.org/10.1007/s0028501205995
Received:
Revised:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s0028501205995
Keywords
 Predator–prey problems
 Fruittree red spider mites
 Game theory
 Optimal control
 Singular characteristics