Abstract
Expected waiting times for equilibrium selection are exponentially increasing as the noise level goes to zero in evolutionary models with time-constant noise, raising questions about whether the history independent prediction of equilibrium selection is relevant in economic and social studies. However, by using the theoretical results on simulated annealing, we show that expected waiting times in models with time-decreasing noise need not tend toward infinity in the small noise limit. Our model thus describes conditions under which the waiting-time critique of the predictions of stochastic stability theory may have less force.
Similar content being viewed by others
Notes
To handle the case of multiple global minimizers, we would have to introduce more tedious definitions. Since this is not typical situation in the evolutionary model, we assume a unique global minimizer throughout this paper.
One such procedure is the steepest decent procedure: from point \(x\), one moves to \(y\) if \(y\) is the minimizer of \(U\) for which \(q(x,y)>0\) and \(U(x)\ge U(y)\), and one stays at \(x\) if there is no such \(y\).
But we should be careful about the speed of the noise level decreasing because the algorithm might be trapped at local optima even in later periods if the noise level decreases to zero too rapidly. See Proposition 3.3.
Note that we take the opposite sign to the standard definition of a potential function in order to make it correspond to the definition of an energy function.
But \(U\) is an ordinal transformation of the potential function.
These are usually called cycles in the literature of simulated annealing.
The reason \([\cdot ]_+\) is needed here is that \(\min _{y\in B(C)}\{U(y)-U(C)\}\) might be negative.
Actually we can obtain the same bound for any set \(D \subset E\) but, then we would have to introduce more definitions in order to define a depth \(H(D)\) on the general set \(D\). So here we focus on a subclass.
In the optimization problem the designer is allowed to choose the optimal (possibly variable) step sizes. But in our evolutionary model the state space \(E\) is a finite subset of \(n\)-dimensional grid. Moreover, it is standard to assume a single mutation in each period: we consider a period as time interval that is small enough that the probability of more than one individual changing his action in the same period is negligible as in the construction of Poisson processes. Hence, in our model the step size is exogenously fixed.
Hajek (1988) derives the above necessary and sufficient conditions for more restrictive setup.
Specifically, \(-1/H^*\) is equal to \(U(M,x^*)\) in their model.
The motivation of introducing the definition of a subclass is that at low noise level \(\eta \) the probability of escaping from a subclass \(C\) is approximately given by \(\mathrm {e}^{-H(C)/\eta }\) irrespective of the starting point within \(C\).
Letting \(\eta _k=\eta \) for all \(k\) and then letting \(x=1-b\mathrm {e}^{-H(C)/\eta }\), we have \(\sum _{t=0}^\infty \prod _{k=1}^t\left( 1-b\mathrm {e}^{-H(C)/\eta }\right) = \sum _{t=0}^\infty x^t=1/(1-x)=b\mathrm {e}^{H(C)/\eta }\).
The definition of \(G_{y,y}(W)\) is used in Eq. (8). Note also \(G_{x,y}(W)=\varnothing \) for \(x \in W{\setminus }\{y\}\) since no arrow starts from \(x\).
References
Beggs A (2005) Waiting times and equilibrium selection. Econ Theory 25:599–628
Bergin J, Lipman B (1996) Evolution with state-dependent mutations. Econometrica 64:943–956
Blume A (1993) The statistical mechanics of strategic interaction. Games Econ Behav 5:387–424
Catoni O (1991) Sharp large deviation estimates for simulated annealing algorithms. Ann lnst Henri Poincaré 27:291–383
Catoni O (1992) Rough large deviation estimates for simulated annealing: aplication to exponential schedules. Ann Probab 20:1109–1146
Catoni O (1995) Simulated annealing algorithm and Markov chains wtih rare transitions”, Lecture Note
Chen H, Chow Y (2001) On the convergence of evolution processes with time-varying mutations and local interaction. J Appl Probab 38:301–323
Ellison G (1993) Learning local interaction, and coordination. Econometrica 61:1047–1071
Ellison G (2000) Basin of attraction long-run stochastic stability, and the speed of step-by step evolution. Rev Econ Stud 67:17–45
Freidlin M, Wentzell A (1998) Random perturbations of dynamical systems, 2nd edn. Springer-Verlag, New York
Hajek B (1988) Cooling schedules for optimal annealing. Math Oper Res 13:311–329
Kandori M, Mailath GJ, Rob R (1993) Learning mutation, and long run equilibria in games. Econometrica 61:29–56
Monderer D, Shapley LS (1996) Potential games. Games Econ Behav 14:124–143
Pak M (2008) Stochastic stability and time-dependent mutations. Games Econ Behav 64:650–665
Robles J (1998) Evolution with changing mutation rates. J Econ Theory 79:207–223
Sandholm W (1998) Simple and clever decision rules in a model of evolution. Econ Lett 61:165–170
Sandholm W, Pauzner A (1998) Evolution popultion growth, and history independence. Games Econ Behav 22:84–120
Trouvé A (1996) Rough large deviation esitimates for the optimal convergence speed exponent of generalized simulated annealing algorithm. Ann lnst Henri Poincaré 32:299–348
Young P (1993) The evolution of convention. Econometirca 61:57–84
Acknowledgments
We are grateful to Bill Sandholm, an Associate Editor, and anonymous referees for their comments and suggestions.
Author information
Authors and Affiliations
Corresponding author
Appendix: Proof of Proposition 3.2
Appendix: Proof of Proposition 3.2
We denote \(x\mapsto y\) if there is a directed link from state \(x\) to \(y\). For any subset \(W \subset E\), we let \(G(W)\) be the set of oriented graphs \(g=\{ (x,y) \in E \times E\,\vert \, x\mapsto y\}\) such that \((i)\) every point \(x \in \overline{W}=E{\setminus }W\) is the initial point of exactly one arrow (no arrow starts from \(W\)), and \((ii)\) there are no closed cycles in the graph. Let \(G_{x,y}(W)\) denote the set of graphs \(g \in G(W)\) which leads from \(x \in E\) to \(y \in W\), where, by convention, \(G_{y,y}(W)=G(W)\) for \(y \in W\).Footnote 16 Catoni (1995, Lemma 3.4), based on Freidlin and Wenztel (1998, Lemma 6.3.4), gives a formula
where \(p(g):=\prod _{(x,y)\in g}p_\eta (x,y)\) is the product of transition probabilities over the graph \(g\). The transition probability (1) has a form
where \(V(x,y):=[U(y)-U(x)]_+\) if \(q(x,y)>0\), and \(V(x,y):=\infty \) otherwise. Let \(V(g):=\sum _{(x,y)\in g}V(x,y)\) and \(g^*\) the graph solving \(\min _{y \in C}\min _{g \in G_{x,y}(\overline{C}\cup \{y\})}V(g)\). Then the numerator of (8) is bounded as follows:
for some constant \(B_1>0\), where \(\left| g\right| \) is the number of elements in the graph \(g\). The last inequality follows because the summation is over a finite number of terms, each of which is positive and bounded. Similarly, letting \(g^{**} \in \mathrm{argmin}_{g\in G(\overline{C})}V(g)\), the denominator of (8) is bounded as follows:
Therefore, we conclude that there exists \(b>0\) such that \(b\mathrm {e}^{H_C(x)/\eta }\!\le \!E\left( \tau (\overline{C})\,\vert \,X_0\!=\!x\right) \le b^{-1}\mathrm {e}^{H_C(x)/\eta }\), where \(H_C(x):=\min _{g\in G(\overline{C})}V(g)-\min _{y \in C}\min _{g \in G_{x,y}(\overline{C}\cup \{y\})}V(g)\). Now we are done if we show that \(\max _{x \in C}H_C(x)=H(C)\). But this comes from Proposition 4.15 of Catoni (1995).\(\square \)
Rights and permissions
About this article
Cite this article
Aiba, K. Waiting times in evolutionary dynamics with time-decreasing noise. Int J Game Theory 44, 499–514 (2015). https://doi.org/10.1007/s00182-014-0442-x
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00182-014-0442-x