As its title indicates, the purpose of this section is to establish Theorem 1.2 in each of the three regimes.
Subcritical regime \(p<1/2\)
Throughout this section, we assume that the reinforcement parameter satisfies \(p<1/2\). Our approach in the subcritical regime relies on the following strengthening of Lemma 2.5 (recall the notation there).
Lemma 3.1
Define for every \(i\ge 1\)
$$\begin{aligned} c_i^{(p)}= \frac{(1-p) }{p} {\mathrm {B}}(i, 1+1/p). \end{aligned}$$
Then we have
$$\begin{aligned} \lim _{n\rightarrow \infty } \mathbb {E}\left( \sum _{i=1}^{\infty } i^2 \left| \frac{C_i(n)}{n} -c_i^{(p)}\right| \right) =0. \end{aligned}$$
Proof
For each \(n=1,2, \ldots \), write \({\mathbf{C}}(n)= (C_i(n))_{i\ge 1}\) and view \({\mathbf{C}}(n)\) as a function on the space \(\Omega \times \mathbb {N}\) endowed with the product measure \(\mathbb {P}\otimes \#^2\), where \(\#^2\) stands for the measure on \(\mathbb {N}\) which assigns mass \(i^2\) to every \(i\in \mathbb {N}\). Consider an arbitrary subsequence excerpt from \(({\mathbf{C}}(n))_{n\ge 1}\). From Lemma 2.5 and an argument of diagonal extraction, we can construct a further subsequence, say indexed by \(\ell (n)\) for \(n=1,2, \ldots \), such that
$$\begin{aligned} \lim _{n\rightarrow \infty } \frac{{\mathbf{C}}(\ell (n))}{\ell (n)} = {\mathbf{c}}^{(p)} \qquad (\mathbb {P}\otimes \#^2) \text {-almost everywhere,} \end{aligned}$$
(9)
where \({\mathbf{c}}^{(p)}=(c_i^{(p)})_{i\ge 1}\).
On the one hand, we observe that
$$\begin{aligned}\sum _{i=1}^{\infty } i^2{\mathrm {B}}(i, 1+1/p)&=\int _0^1 \left( \sum _{i=1}^{\infty } i^2 x^{i-1}\right) (1-x)^{1/p}\hbox {d}x\\&=\int _0^1(1+x)(1-x)^{-3+1/p}\hbox {d}x\\&=\frac{p}{(1-p)(1-2p)}, \end{aligned}$$
so that
$$\begin{aligned} \sum _{i=1}^{\infty } i^2 c_i^{(p)}= \frac{1}{1-2p}. \end{aligned}$$
(10)
On the other hand, we note the basic identity
$$\begin{aligned} {\mathcal {S}}^2(n) = \sum _{j=1}^{\infty } N_j(n)^2= \sum _{i=1}^{\infty } i^2 C_i(n). \end{aligned}$$
(11)
Since \(\Gamma (n+2p)/\Gamma (n)\sim n^{2p}\) and \(2p<1\), we see from Lemma 2.6 and (11) that
$$\begin{aligned} \lim _{n\rightarrow \infty } \mathbb {E}\left( \sum _{i=1}^{\infty } i^2 \frac{C_i(n)}{n}\right) = \frac{1}{1-2p}. \end{aligned}$$
Thanks to (10), we deduce from the Vitali-Scheffé theorem (see e.g. Theorem 2.8.9 in [9]) that the convergence (9) also holds in \(L^1(\mathbb {P}\otimes \#^2)\), that is
$$\begin{aligned}\lim _{n\rightarrow \infty } \mathbb {E}\left( \sum _{i=1}^{\infty } i^2 \left| \frac{C_i(\ell (n))}{\ell (n)} -c_i^{(p)}\right| \right) =0.\end{aligned}$$
Since the convergence above holds for any (initial) subsequence, our claim is proven. \(\square \)
We next point at the following consequence of Lemma 3.1.
Corollary 3.2
We have
$$\begin{aligned} \lim _{n\rightarrow \infty } \frac{{\mathcal {S}}^2(n)}{n}=\frac{1}{1-2p} \qquad \text {in }L^1(\mathbb {P}), \end{aligned}$$
and
$$\begin{aligned} \lim _{n\rightarrow \infty } \sup _{j\ge 1} \frac{N_j(n)}{\sqrt{n}} =0\qquad \text {in probability}. \end{aligned}$$
Proof
Observe from (10), (11), and the triangle inequality that
$$\begin{aligned} \mathbb {E}\left( \left| \frac{{\mathcal {S}}^2(n)}{n}-\frac{1}{1-2p}\right| \right)&= \mathbb {E}\left( \left| \sum _{i=1}^{\infty } i^2 \frac{C_i(n)}{n} -\sum _{i=1}^{\infty } i^2 c_i^{(p)}\right| \right) \\&\le \mathbb {E}\left( \sum _{i=1}^{\infty } i^2 \left| \frac{C_i(n)}{n} -c_i^{(p)}\right| \right) . \end{aligned}$$
Our first assertion thus follows from Lemma 3.1.
For the second assertion, observe that
$$\begin{aligned} \sup _{j\ge 1} N_j(n)=\sup \{i\ge 1: C_i(n)\ge 1\}. \end{aligned}$$
We then have for every \(\eta >0\) arbitrarily small
$$\begin{aligned} \mathbb {P}\left( \sup _{j\ge 1}N_j(n)>\eta \sqrt{n} \right)&=\mathbb {P}(\exists i\ge \eta \sqrt{n}: C_i(n)\ge 1)\\&\le \frac{1}{\eta ^2 n} \mathbb {E}\left( \sum _{i\ge \eta \sqrt{n}} i^2 C_i(n)\right) . \end{aligned}$$
It follows from Lemma 3.1 that the right-hand side converges to 0 as \(n\rightarrow \infty \), and the proof is now complete. \(\square \)
Theorem 1.2(i) now derives immediately from (4), Lemma 2.1(i) and Corollary 3.2 by setting \(\beta _{n,j}=N_j(n)/\sqrt{n}\) for every \(j\ge 1\).
Critical regime \(p=1/2\)
Throughout this section, we assume that the reinforcement parameter is \(p=1/2\). Recall from Lemma 2.6 that \(\mathbb {E}\left( {\mathcal {S}}^2(n) \right) \sim n \log n\) as \(n\rightarrow \infty .\) We establish now a stronger version of this estimate.
Lemma 3.3
One has
$$\begin{aligned} \lim _{n\rightarrow \infty } \frac{{\mathcal {S}}^2(n)}{n\log n} =1\qquad \text {almost surely}. \end{aligned}$$
Proof
It has been observed in [7] that, in the study of reinforcement induced by Simon’s algorithm, it may be convenient to perform a time-substitution based on a Yule process. We shall use this idea here again, and introduce a standard Yule process \(Y=(Y_t)_{t\ge 0}\), which we further assume to be independent of the preceding variables. Recall that Y is a pure birth process in continuous time started from \(Y_0=1\) and with birth rate n from any state \(n\ge 1\); in particular, for every function \(f: \mathbb {N}\rightarrow \mathbb {R}\), say such that \(f(n)=O(n^r)\) for some \(r >0\), the process
$$\begin{aligned} f(Y_t)-\int _0^t \left( f(Y_s+1)-f(Y_s)\right) Y_s \hbox {d}s \end{aligned}$$
is a martingale.
Consider the time changed process \({\mathcal {S}}^2\circ Y\). Applying the observation above to \(f= {\mathcal {S}}^2\) and then projecting on the natural filtration of \({\mathcal {S}}^2\circ Y\), the same calculation as in the proof of Lemma 2.6 shows that
$$\begin{aligned}&{\mathcal {S}}^2(Y_t) - \int _0^t \left( \frac{1}{2} + \sum _{i=1}^{\infty } \frac{N_i(Y_s)}{2Y_s} (2N_i(Y_s)+1) \right) Y_s \hbox {d}s \\&= {\mathcal {S}}^2(Y_t) -\int _0^t({\mathcal {S}}^2(Y_s)+Y_s)\hbox {d}s \end{aligned}$$
is a martingale. By elementary stochastic calculus, the same holds for
$$\begin{aligned} M_t= {\mathrm e}^{-t} {\mathcal {S}}^2(Y_t) -\int _0^t {\mathrm e}^{-s}Y_s \hbox {d}s. \end{aligned}$$
We shall now show that M is bounded in \(L^2(\mathbb {P})\) by checking that its quadratic variation \([M]_{\infty }\) has a finite expectation. Plainly, M is purely discontinuous; its jumps can arise either due to an innovation event (whose instantaneous rate at time t equals \(\frac{1}{2}Y_{t-}\)), and then \(\Delta M_t=M_t-M_{t-}={\mathrm e}^{-t}\), or by a repetition of the j-th item for some \(j\ge 1\) (whose instantaneous rate at time t equals \(\frac{1}{2}N_j(Y_{t-}))\), and then \(\Delta M_t={\mathrm e}^{-t}(2N_j(Y_{t-})+1)\). We thus find by a standard calculation of compensation that
$$\begin{aligned}\mathbb {E}([M]_{\infty })&=\mathbb {E}\left( \sum _{t>0} (\Delta M_t)^2\right) \\&= \mathbb {E}\left( \int _0^{\infty } {\mathrm e}^{-2t} \left( \frac{1}{2} Y_t + \frac{1}{2}\sum _{j\ge 1} N_j(Y_t) (2N_j(Y_t)+1)^2\right) \hbox {d}t \right) \\&= \int _0^{\infty } \mathbb {E}\left( Y_t + 2 \sum _{j\ge 1} (N_j(Y_t)^3 + N_j(Y_t)^2) \right) {\mathrm e}^{-2t} \hbox {d}t. \end{aligned}$$
First, recall that \(Y_t\) has the geometric distribution with parameter \({\mathrm e}^{-t}\), in particular \(\int _0^{\infty } \mathbb {E}( Y_t ){\mathrm e}^{-2t}\hbox {d}t = 1\). Second, \(\sum _{j\ge 1} N_j(Y_t)^2 = {\mathcal {S}}^2(Y_t)\), and since \(\mathbb {E}({\mathcal {S}}^2(n))\sim n\log n\) (see Lemma 2.6) and the processes S and Y are independent, we have also
$$\begin{aligned} \int _0^{\infty } \mathbb {E}\left( \sum _{j\ge 1} N_j(Y_t)^2\right) {\mathrm e}^{-2t} \hbox {d}t <\infty . \end{aligned}$$
Third, consider \(T(Y_t)=\sum _{j\ge 1} N_j(Y_t)^3\). By calculations similar to those for \(M_t\), one sees that the process
$$\begin{aligned} {\mathrm e}^{-3t/2} T(Y_t) - \int _0^t {\mathrm e}^{-3s/2} (Y_s + {\mathcal {S}}^2(Y_s)) \hbox {d}s, \qquad t\ge 0 \end{aligned}$$
is a local martingale. Just as above, one readily checks that
$$\begin{aligned} \int _0^{\infty } {\mathrm e}^{-3s/2} \mathbb {E}(Y_s + {\mathcal {S}}^2(Y_s)) \hbox {d}s<\infty , \end{aligned}$$
and hence \(\mathbb {E}(T(Y_t))=O({\mathrm e}^{3t/2})\). As a consequence,
$$\begin{aligned} \int _0^{\infty } \mathbb {E}\left( \sum _{j\ge 1} N_j(Y_t)^3\right) {\mathrm e}^{-2t} \hbox {d}t <\infty , \end{aligned}$$
and putting the pieces together, we have checked that \(\mathbb {E}([M]_{\infty })<\infty \).
We now know that \(\lim _{t\rightarrow \infty } M_t=M_{\infty }\) a.s. and in \(L^2(\mathbb {P})\), and recall the classical feature that \(\lim _{t\rightarrow \infty } {\mathrm e}^{-t}Y_t =W\) a.s., where W has the standard exponential distribution. In particular \(\int _0^t {\mathrm e}^{-s} Y_s \hbox {d}s \sim tW\) as \(t\rightarrow \infty \), so that
$$\begin{aligned} {\mathcal {S}}^2(Y_t) = t{\mathrm e}^{t} W + o({\mathrm e}^t),\qquad \text {a.s.} \end{aligned}$$
Using again \(Y_t= {\mathrm e}^t W+ o({\mathrm e}^t)\), we conclude that \({\mathcal {S}}^2(n) = n\log n + O(n)\) a.s., which implies our claim. \(\square \)
Remark 3.4
The first part of Corollary 3.2 and Lemma 3.3 seem to be of the same nature. Actually, one can also establish the former by adapting the proof of the latter, therefore circumventing the appeal to Lemma 2.5. There is nonetheless a fundamental difference between these two results: although the microscopic clusters (i.e. of size O(1)) determine the asymptotic behavior of \({\mathcal {S}}^2(n)\) in the sub-critical case, they have no impact in the critical case as it is seen from Lemma 2.5.
Thanks to Lemmas 2.1(ii) and 3.3, the following statement is the final piece of the proof of Theorem 1.2(ii).
Lemma 3.5
One has
$$\begin{aligned} \lim _{n\rightarrow \infty } \sup _{j\ge 1} \frac{N_j(n)}{\sqrt{n\log n}}=0\qquad \text {in probability}.\end{aligned}$$
Proof
We shall show that there is some numerical constant b such that
$$\begin{aligned} \mathbb {E}(N_j(n)^3)\le b(n/j)^{3/2} \qquad \text {for all }j,n\ge 1. \end{aligned}$$
(12)
Then, by Markov’s inequality, we have that for any \(\eta >0\)
$$\begin{aligned} \mathbb {P}\left( N_j(n)>\sqrt{\eta n\log n}\right) \le b(\eta n\log n)^{-3/2} (n/j)^{3/2}, \end{aligned}$$
and by the union bound
$$\begin{aligned} \mathbb {P}\left( \exists j\le n: N_j(n)>\sqrt{\eta n\log n}\right) \le b(\eta \log n)^{-3/2}\sum _{j\ge 1}j^{-3/2}, \end{aligned}$$
which proves our claim.
For \(i=1,2,3\), set \(a_i(n)=\Gamma (n+i/2)/\Gamma (n)\), so \(a_i(n) \sim n^{i/2}\) and actually \(a_2(n)= n\). Recall that \({\mathrm i}(n)\) denotes the number of innovations up to the n-step of Simon’s algorithm. Take any \(j\ge 1\) and, just as in the proof of Lemma 2.6, observe that on the event \({\mathrm i}(n)\ge j\), one has
$$\begin{aligned} \mathbb {E}\left( \frac{N_{j}(n+1)}{ a_1(n+1)} \mid {\mathcal {F}}_n\right)&= \frac{N_{j}(n)}{ a_1(n)},\\ \mathbb {E}\left( \frac{N_{j}(n+1)^2}{ a_2(n+1) }\mid {\mathcal {F}}_n\right)&= \frac{N_{j}(n)^2}{ a_2(n)} +\frac{N_j(n)}{2na_2(n+1)},\\ \mathbb {E}\left( \frac{N_{j}(n+1)^3}{ a_3(n+1)} \mid {\mathcal {F}}_n\right)&= \frac{N_{j}(n)^3}{ a_3(n)} + \frac{3N_{j}(n)^2}{ 2na_3(n+1)}+ \frac{N_{j}(n)}{ 2na_3(n+1)}.\\ \end{aligned}$$
The trivial bound \({\mathrm i}(j)\le j\) then yields for any \(n\ge j\)
$$\begin{aligned}\mathbb {E}(N_j(n))\le a_1(n)/a_1(j) \le b_1\sqrt{n/j}.\end{aligned}$$
Then we have
$$\begin{aligned}\mathbb {E}(N_j(n)^2)\le \frac{a_2(n)}{a_2(j)} + \frac{b_1}{2na_2(n)} \sum _{k=j}^n \sqrt{k/j}\le b_2 n/j,\end{aligned}$$
and finally also
$$\begin{aligned}\mathbb {E}(N_j(n)^3)\le \frac{a_3(n)}{a_3(j)} + \frac{3b_2}{2na_3(n)} \sum _{k=j}^n k/j+ \frac{b_1}{2na_3(n)} \sum _{k=j}^n \sqrt{k/j}\le b_3(n/j)^{3/2}, \end{aligned}$$
where \(b_1, b_2\) and \(b_3\) are numerical constants. This establishes (12) and completes the proof. \(\square \)
Supercritical regime \(p>1/2\)
Throughout this section, we assume that the reinforcement parameter satisfies \(p>1/2\). We first point at the following strengthening of Lemma 2.2 (in particular, recall the notation (5) there).
Corollary 3.6
We have
$$\begin{aligned} \lim _{n\rightarrow \infty } \mathbb {E}\left( \sum _{j=1}^{\infty } \left| \frac{ N_j(n)}{n^p} - X^{(p)}_j\right| ^2\right) =0. \end{aligned}$$
This result has been already observed by Businger, see Equation (6) in [10]. For the sake of completeness, we present here an alternative and shorter proof along the same line as for Lemma 3.1.
Proof
We view \({\mathbf{X}}^{(p)}=(X^{(p)}_j)_{j\ge 1}\) and \({\mathbf{N}}(n)=(N_j(n))_{j\ge 1}\) for each \(n\ge 1\) as functions on the space \(\Omega \times \mathbb {N}\) endowed with the product measure \(\mathbb {P}\otimes \#\), where \( \#\) denotes the counting measure on \(\mathbb {N}\). Since we already know from Lemma 2.2 that \(n^{-p} {\mathbf{N}}(n)\) converges as \(n\rightarrow \infty \) to \({\mathbf{X}}^{(p)}\) almost everywhere, in order to establish our claim, it suffices to verify that
$$\begin{aligned} \lim _{n\rightarrow \infty } \mathbb {E}\left( \sum _{j=1}^{\infty } \frac{ N_j(n)^2}{n^{2p}} \right) = \mathbb {E}\left( \sum _{j=1}^{\infty } (X^{(p)}_j)^2 \right) ; \end{aligned}$$
see e.g. Proposition 4.7.30 in [9].
Recall from Lemma 2.6 that
$$\begin{aligned} \mathbb {E}\left( \sum _{j=1}^{\infty } \frac{ N_j(n)^2}{n^{2p}} \right) = \frac{\Gamma (n+2p)}{n^{2p}\Gamma (n)}\sum _{i=1}^n \frac{\Gamma (i)}{\Gamma (i+2p)} . \end{aligned}$$
On the one hand, we know that
$$\begin{aligned} \lim _{n\rightarrow \infty } \frac{\Gamma (n+2p)}{n^{2p}\Gamma (n)}=1, \end{aligned}$$
and on the other hand, we recall from (7) that
$$\begin{aligned} \sum _{i=1}^{\infty } \frac{\Gamma (i)}{\Gamma (i+2p)}= \frac{1}{(2p-1)\Gamma (2p)}. \end{aligned}$$
We conclude from Lemma 2.2 that indeed
$$\begin{aligned} \lim _{n\rightarrow \infty } \mathbb {E}\left( \sum _{j=1}^{\infty } \frac{ N_j(n)^2}{n^{2p}} \right) = \frac{1}{(2p-1)\Gamma (2p)}= \mathbb {E}\left( \sum _{j=1}^{\infty } (X^{(p)}_j)^2 \right) \end{aligned}$$
and the proof is complete. \(\square \)
Theorem 1.2 can now be deduced from (4), Lemma 2.1(ii), and Corollary 3.6.