Skip to main content

On a Class of Random Walks with Reinforced Memory

Abstract

This paper deals with different models of random walks with a reinforced memory of preferential attachment type. We consider extensions of the Elephant Random Walk introduced by Schütz and Trimper (Phys Rev E 70:044510(R), 2004) with stronger reinforcement mechanisms, where, roughly speaking, a step from the past is remembered proportional to some weight and then repeated with probability p. With probability \(1-p\), the random walk performs a step independent of the past. The weight of the remembered step is increased by an additive factor \(b\ge 0\), making it likelier to repeat the step again in the future. A combination of techniques from the theory of urns, branching processes and \(\alpha \)-stable processes enables us to discuss the limit behavior of reinforced versions of both the Elephant Random Walk and its \(\alpha \)-stable counterpart, the so-called Shark Random Swim introduced by Businger (J Stat Phys 172(3):701–717, 2004). We establish phase transitions, separating subcritical from supercritical regimes.

This is a preview of subscription content, access via your institution.

Fig. 1

References

  1. Alves, G. A., de Araújo, Cressoni, J. C., da Silva, L. R., da Silva, M. A. A., Viswanathan, G.M.: Superdiffusion driven by exponentially decaying memory. J. Stat. Mech. 2014 (2014)

  2. Athreya, K.B., Ney, P.E.: Branching Processes. Dover Books on Mathematics (2004)

  3. Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. Science 286, 509–512 (1999)

    ADS  MathSciNet  Article  Google Scholar 

  4. Baur, E., Bertoin, J.: Elephant random walks and their connection to Pólya-type urns. Phys. Rev. E 49, 052134 (2016)

    ADS  Article  Google Scholar 

  5. Bercu, B.: A martingale approach for the elephant random walk. J. Phys. A 51, 015201 (2017)

    ADS  MathSciNet  Article  Google Scholar 

  6. Bercu, B., Laulin, L.: On the multi-dimensional elephant random walk. J. Stat. Phys. 175(6), 1146–1163 (2019)

    ADS  MathSciNet  Article  Google Scholar 

  7. Bertoin, J.: Noise reinforcement for Lévy processes. Preprint, arXiv:1810.08364 : To appear in Ann. Inst, Henri Poincaré B (2018)

  8. Bertoin, J.: A version of Herbert A. Simon’s model with slowly fading memory and its connections to branching processes. J. Stat. Phys. 176, 679 (2019)

    ADS  MathSciNet  Article  Google Scholar 

  9. Bertoin, J.: Universality of Noise Reinforced Brownian Motions. Preprint (2019)

  10. Bertoin, J., Goldschmidt, C.: Dual random fragmentation and coagulation and an application to the genealogy of yule processes. Mathematics and Computer Science III (2012)

  11. Bertoin, J., Uribe Bravo, G.: Supercritical percolation on large scale-free random trees. Ann. Appl. Probab. 25–1, 81–103 (2015)

    MathSciNet  Article  Google Scholar 

  12. Businger, S.: The shark random swim (Lévy flight with memory). J. Stat. Phys. 172(3), 701–717 (2018)

    ADS  MathSciNet  Article  Google Scholar 

  13. Coletti, C.F., Gava, R., Schütz, G.M.: Central limit theorem for the elephant random walk. J. Math. Phys. 58(5), 053003 (2017)

    MathSciNet  Article  Google Scholar 

  14. Coletti, C.F., Gava, R., Schütz, G.M.: A strong invariance principle for the elephant random walk. J. Stat. Mech. Theory Exp. 12, 123207 (2017)

    MathSciNet  Article  Google Scholar 

  15. Cotar, C., Thacker, D.: Edge- and vertex-reinforced random walks with super-linear reinforcement on infinite graphs. Ann. Probab. 45(4), 2655–2706 (2017)

    MathSciNet  Article  Google Scholar 

  16. Diaconis, P., Rolles, S.W.W.: Bayesian analysis for reversible Markov chains. Ann. Stat. 34(3), 1270–1292 (2006)

    MathSciNet  Article  Google Scholar 

  17. Gut, A., Stadtmüller, U.: Variations of the elephant random walk. Preprint, arXiv:1812.01915 (2018)

  18. Janson, S.: Functional limit theorems for multitype branching processes and generalized Pólya urns. Stoch. Proc. Appl. 110(2), 177–245 (2004)

    Article  Google Scholar 

  19. Kürsten, R.: Random recursive trees and the elephant random walk. Phys. Rev. E 93, 032111 (2016)

    ADS  MathSciNet  Article  Google Scholar 

  20. Mahmoud, H.: Pólya Urn Models. CRC Press, Boca Raton (2009)

    MATH  Google Scholar 

  21. Mailler, C., Marckert, J.-F.: Measure-valued Pólya processes. Electron. J. Probab. 22(26), 33 (2017)

    MATH  Google Scholar 

  22. Mailler, C., Uribe Bravo, G.: Random walks with preferential relocations and fading memory: a study through random recursive trees. J. Stat. Mech. Theory Exp. 9, 093206 (2019)

    MathSciNet  Article  Google Scholar 

  23. Metzler, R., Klafter, J.: The random walk’s guide to anomalous diffusion: a fractional dynamics approach. Phys. Rep. 339, 1–77 (2000)

    ADS  MathSciNet  Article  Google Scholar 

  24. Oliveira, F.A., Ferreira, R.M.S., Lapas, L.C., Vainstein, M.H.: Anomalous diffusion: a basic mechanism for the evolution of inhomogeneous systems. Front. Phys. 19, 18 (2019)

    Article  Google Scholar 

  25. Paraan, F.N.C., Esguerra, J.P.: Exact moments in a continuous time random walk with complete memory of its history. Phys. Rev. E 74, 032101 (2006)

    ADS  Article  Google Scholar 

  26. Pemantle, R.: A survey of random processes with reinforcement. Prob. Surv. 4, 1–79 (2007)

    MathSciNet  Article  Google Scholar 

  27. Sabot, C., Tarrès, P.: Edge-reinforced random walk, vertex-reinforced jump process and the supersymmetric hyperbolic sigma model. JEMS 17(9), 2353–2378 (2015)

    MathSciNet  Article  Google Scholar 

  28. Sabot, C., Zeng, X.: A random Schrödinger operator associated with the vertex reinforced jump process on infinite graphs. J. Am. Math. Soc. 32, 311–349 (2019)

    Article  Google Scholar 

  29. Samorodnitsky, G., Taqqu, M.S.: Stable Non-Gaussian Random Processes: Stochastic Models with Infinite Variance. Chapman and Hall/CRC, Boca Raton (2000)

    MATH  Google Scholar 

  30. Schütz, G.M., Trimper, S.: Elephants can always remember: exact long-range memory effects in a non-Markovian random walk. Phys. Rev. E 70, 045101(R) (2004)

    ADS  Article  Google Scholar 

  31. Silver, D., Huang, A., et al.: Mastering the game of go with deep neural networks and tree search. Nature 529, 484–489 (2016)

    ADS  Article  Google Scholar 

Download references

Acknowledgements

I warmly thank Silvia Businger for explaining her work and for her help, and Jean Bertoin for valuable comments. I am also grateful to two anonymous referees for their careful reading of the manuscript and for their helpful remarks.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Erich Baur.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Communicated by Antti Knowles.

A Appendix

A Appendix

Proof of Lemma 4.1

The fact that \((Y(t),\,t\ge 0)\) is a pure birth process with the stated properties is a consequence of (17) and of the dynamics of \((T(t),\,t\ge 0)\). Standard properties of branching processes (see, e.g.,  [2]) show that \((\mathrm{e}^{-(b+1)t}Y(t),\,t\ge 0)\) is a square-integrable martingale, and it follows from Lemma 3 in  [10] that its (almost surely and \(L^2\)-)limit is Gamma\((1/(b+1),1/(b+1))\)-distributed. \(\square \)

Proof of Lemma 4.2

The i.i.d. property of the processes \((Y_i^{(p)}(b_i+\cdot ),\,t\ge 0)\), \(i\ge 1\), is obvious from the construction. We shall therefore prove everything for \(i=1\), in which case \(b_i=b_1=0\).

Clearly, the sum of degrees of vertices of \(T^{(p)}_1(t)\) is equal to

$$\begin{aligned} \left( 2(|T_1^{(p)}(t)|-1)+H^{(p)}_1(t)\right) . \end{aligned}$$

It now follows from the construction of the preferential attachment tree T(t) at the beginning of Sect. 4.2 (recall in particular the parameters of the exponential clocks) that \((Y_1^{(p)}(t),\,t\ge 0)\) is a pure birth process with the stated birth rate and reproduction law. It is then well-known (see again  [2]) that \((\mathrm{e}^{-(b+p)t} Y_1^{(p)}(t),\,t\ge 0)\) is a martingale, whose terminal value \(W_1\) is almost surely strictly positive. By Kolmogorov’s forward equation (see once more  [2]) we compute for \(t>0\)

$$\begin{aligned} \mathbb {E}\left[ Y_1^{(p)}(t)\right] =\mathrm{e}^{(b+p)t}\,,\quad \quad \mathbb {E}\left[ Y_1^{(p)}(t)^2\right] =\frac{(b+1)(b+2p)}{b+p}\left( \mathrm{e}^{2(b+p)t}-\mathrm{e}^{(b+p)(b_i+t)}\right) . \end{aligned}$$

This proves square-integrability of \((\mathrm{e}^{-(b+p)t} Y_1^{(p)}(t),\,t\ge 0)\), and the claim about the first and second moment of \(W_1\) follows from the last display.

It remains to show boundedness in \(L^k\) for \(k\ge 3\), that is, we have to show that there exists a constant \(c_k<\infty \) such that

$$\begin{aligned} \mathbb {E}\left[ Y_1^{(p)}(t)^k\right] \le c_k \mathrm{e}^{k(b+p)t}\quad for all t\ge 0. \end{aligned}$$
(47)

In order to prove this, we adapt  [8, Proof of Lemma 3] to our situation. First, we note that the generator \(\mathfrak {G}\) of \((Y_1^{(p)}(t),\,t\ge 0)\) is given for any smooth function \(f:(0,\infty )\rightarrow \mathbb {R}\) by

$$\begin{aligned} \mathfrak {G}f(x)=x(1-p)\left( f(x+b)-f(x)\right) +xp\left( f(x+b+1)-f(x)\right) . \end{aligned}$$

Specifying to \(f(x)=x^\ell \) for some integer \(\ell \ge 3,\)

$$\begin{aligned} \mathfrak {G}f(x)&=x(1-p)\sum _{j=0}^{\ell -1}{\ell \atopwithdelims ()j}x^jb^{\ell -j}+xp\sum _{j=0}^{\ell -1} {\ell \atopwithdelims ()j}x^j(b+1)^{\ell -j}\nonumber \\&=\ell (b+p)x^\ell +(1-p)\sum _{j=0}^{\ell -2}{\ell \atopwithdelims ()j}x^{j+1}b^{\ell -j}+p\sum _{j=0}^{\ell -2}{\ell \atopwithdelims ()j}x^{j+1}(b+1)^{\ell -j}. \end{aligned}$$
(48)

We prove now by induction that (47) holds for all \(k\in \mathbb {N}\). We already know it for \(k=1\) and \(k=2\), so let us assume that for some \(\ell \ge 3\), (47) holds for all \(k=1,\ldots ,\ell -1\). Kolmogorov’s forward equation reads

$$\begin{aligned} \frac{d }{d t}\mathbb {E}\left[ f(Y_1^{(p)}(t))\right] =\mathbb {E}\left[ \mathfrak {G}f(Y_1^{(p)}(t))\right] . \end{aligned}$$

In combination with (48), and using (47) for \(k=1,\ldots ,\ell -1\), we deduce that for some \(\gamma >0\) depending on b, we have

$$\begin{aligned} \frac{d }{d t}\ln \mathbb {E}\left[ Y_1^{(p)}(t)^\ell \right] \le (b+p)\ell +\gamma \frac{\mathrm{e}^{(\ell -1)(b+p)t}}{\mathbb {E}\left[ Y_1^{(p)}(t)^\ell \right] }. \end{aligned}$$
(49)

By Jensen’s inequality,

$$\begin{aligned} \mathbb {E}\left[ Y_1^{(p)}(t)^\ell \right] \ge \mathrm{e}^{\ell (b+p)t}\quad for all t\ge 0\,, \end{aligned}$$

so that

$$\begin{aligned} \int _0^\infty \frac{\mathrm{e}^{(\ell -1)(b+p)t}}{\mathbb {E}\left[ Y_1^{(p)}(t)^\ell \right] }d t\le \int _0^\infty \mathrm{e}^{-(b+p)t}d t=\frac{1}{b+p}. \end{aligned}$$

Going back to (49) and integrating, we obtain

$$\begin{aligned} \mathbb {E}\left[ Y_1^{(p)}(t)^\ell \right] \le \mathrm{e}^{\gamma \frac{1}{b+p}}\mathrm{e}^{\ell (b+p)t}\quad for all t\ge 0. \end{aligned}$$

Thus, (47) does hold for \(k=\ell \) as well, as wanted. \(\square \)

Proof of Lemma 4.3

We fix a small \(\varepsilon >0\) and a sequence \((x_n,\,n\in \mathbb {N})\) of positive integers with \(\lim _{n\rightarrow \infty }x_n=\infty \) and \(x_n\le n\). Recalling Lemma 4.1 and the notation from there, we define for each \(k\in \mathbb {N}\) the event

$$\begin{aligned} E^1_k:=\left\{ W(1-\varepsilon )\le \mathrm{e}^{-(b+1)\tau _k}\left( (b+1)k-b\right) \le W(1+\varepsilon )\right\} . \end{aligned}$$

Lemma 4.1 ensures that \(\lim _{n\rightarrow \infty }\mathbb {P}\left( \bigcap _{k=x_n}^\infty E^1_k\right) =1\). On \(E^1_k\), it holds for k sufficiently large that

$$\begin{aligned} \tau _k&\le \frac{1}{b+1}\left( \ln k - \ln W +\ln (b+1)-\ln \left( 1-\varepsilon \right) \right) \,,\nonumber \\ \tau _k&\ge \frac{1}{b+1}\left( \ln k - \ln W +\ln (b+1)-2\ln \left( 1+\varepsilon \right) \right) . \end{aligned}$$
(50)

Writing D(k) for the number of subtrees present at time \(\tau _k\), i.e.,

$$\begin{aligned} D(k)=\max \left\{ i\ge 1: T_i^{(p)}(\tau _k)\ne \emptyset \right\} \,, \end{aligned}$$

we deduce from the construction of \(T^{(p)}(t)\) that D(k) has the same law as \(1+\sum _{i=1}^{k-1}\epsilon _{i,1-p}\), where \(\epsilon _{i,1-p}\), \(i\ge 1\), are i.i.d. Bernoulli random variables with success probability \(1-p\). Consequently, an application of the law of large numbers shows that if we define

$$\begin{aligned} E^2_k:=\left\{ k(1-p)(1-\varepsilon )\le D(k) \le k(1-p)(1+\varepsilon )\right\} \,, \end{aligned}$$

then \(\lim _{n\rightarrow \infty }\mathbb {P}\left( \bigcap _{k=x_n}^\infty E_k^2\right) =1\). On \(E_k^2\) it holds by construction that

$$\begin{aligned} b_{\lceil k(1-p)(1+\varepsilon )\rceil }\ge \tau _k. \end{aligned}$$

Using (50), we find that on the event \(E^1_k\cap E^2_k\), for k large enough and provided \(\varepsilon \) is sufficiently small,

$$\begin{aligned} b_k \ge \tau _{\lfloor \frac{k}{(1-p)(1+\varepsilon )}\rfloor }\ge \frac{1}{b+1}\left( \ln (k-1)-\ln W+\ln (b+1)-\ln (1-p)-3\ln (1+\varepsilon )\right) . \end{aligned}$$

Letting

$$\begin{aligned} E_n:=\bigcap _{k=x_n}^\infty \left( E^1_k\cap E^2_k\right) \,, \end{aligned}$$

we have by the properties of \(E^1_k\) and \(E^2_k\) that \(\lim _{n\rightarrow \infty }\mathbb {P}\left( E_n\right) =1\).

On the event \(E_n\), it holds by construction that for all n large and i with \(x_n\le i\le n\),

$$\begin{aligned} \tau _n-b_i\le \frac{1}{b+1}\left( \ln n-\ln (i-1)+\ln (1-p)+3\ln (1+\varepsilon )-\ln (1-\varepsilon )\right) . \end{aligned}$$

Entirely similar, one sees that on \(E_n\)

$$\begin{aligned} \tau _n-b_i\ge \frac{1}{b+1}\left( \ln n-\ln (i+1)+\ln (1-p)+2\ln (1-\varepsilon )-2\ln (1+\varepsilon )\right) . \end{aligned}$$

Now notice that

$$\begin{aligned} \max \left\{ 3\ln (1+\varepsilon )-\ln (1-\varepsilon ),\,2\ln (1+\varepsilon )-2\ln (1-\varepsilon )\right\} \downarrow 0 \end{aligned}$$

if \(\varepsilon \downarrow 0\). Since \(\varepsilon >0\) can be chosen arbitrarily small, we can clearly construct a sequence \((\varepsilon _n)\) with \(\varepsilon _n\downarrow 0\) such that on \(E_n\), the stated bounds hold. \(\square \)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Baur, E. On a Class of Random Walks with Reinforced Memory. J Stat Phys 181, 772–802 (2020). https://doi.org/10.1007/s10955-020-02602-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10955-020-02602-3

Keywords

  • Reinforced random walks
  • Preferential attachment
  • Memory
  • Stable processes
  • Branching processes
  • Pólya urns

Mathematics Subject Classification

  • 60G50
  • 60G52
  • 60K35
  • 05C85