Asymptotically Normal Estimators for Zipf’s Law

Abstract

We study an infinite urn scheme with probabilities corresponding to a power function. Urns here represent words from an infinitely large vocabulary. We propose asymptotically normal estimators of the exponent of the power function. The estimators use the number of different elements and a few similar statistics. If we use only one of the statistics we need to know asymptotics of a normalizing constant (a function of a parameter). All the estimators are implicit in this case. If we use two statistics then the estimators are explicit, but their rates of convergence are lower than those for estimators with the known normalizing constant.

Appendix: Functional Central Limit Theorem

Let for t ∈ [0, 1],k ≥ 1

$${Y}_{n,k}^{*}(t) = \frac{{R}_{[nt],k}^{*} - \mathbf{E} {R}_{[nt],k}^{*}}{(\alpha(n))^{1/2}}, Y_{n,k}(t) = \frac{R_{[nt],k} - \mathbf{E} R_{[nt],k}}{(\alpha(n))^{1/2}}. $$

Theorem 4.

Let us assume that (1.2) holds,ν ≥ 1 is integer. Then random process$\left ((Y^{*}_{n,1}(t), Y_{n,1}(t),\ldots , Y_{n,\nu }(t)), 0 \leq t \leq 1 \right )$convergesweakly in the uniform metrics inD(0, 1) to (ν + 1)-dimensionalGaussian process with continuous sample paths, zero expectation and covariancefunction$(c_{ij}(\tau ,t))_{i,j = 0}^{\nu }$,

$$\begin{array}{@{}rcl@{}} c_{ij}(\tau,t) &=& \frac{\theta \tau^{i} (t-\tau)^{j-i} t^{\theta-j} {\Gamma}(j-\theta)}{i!(j-i)!} - \frac{\theta \tau^{i} t^{j} (t+\tau)^{\theta-i-j} {\Gamma}(i+j-\theta)}{i!j!}\\ && \text{for} 1 \leq i \le j, \tau\leq t,\\ c_{ij}(\tau,t) &=& - \frac{\theta \tau^{i} t^{j} (t+\tau)^{\theta-i-j} {\Gamma}(i+j-\theta)}{i!j!} \text{for} i> j\geq 1, \tau\leq t,\\ c_{00}(\tau,t) &=& \left( (t+\tau)^{\theta}-t^{\theta}\right) {\Gamma}(1-\theta) \text{for} \tau\leq t,\\ c_{i0}(\tau,t) &=& - \frac{\theta \tau^{i} (t+\tau)^{\theta-i} {\Gamma}(i-\theta)}{i!} \text{for} i> 0, \tau\leq t,\\ c_{0j}(\tau,t) &=& \frac{\theta ((t-\tau)^{j} t^{\theta-j} - t^{j} (t+\tau)^{\theta-j}) {\Gamma}(j-\theta)}{j!} \text{for} j>0, \tau\leq t, \end{array} $$

c_ji(t, τ) = c_ij(τ, t).

Proof.

Theorem 3 by Chebunin and Kovalevskii (2016) states weak convergence of vector random process $\left ((Y^{*}_{n,1}(t), \ldots , Y^{*}_{n,\nu }(t)), 0 \leq t \leq 1 \right )$ in the uniform metrics in D(0, 1) to (ν + 1)-dimensional Gaussian process with continuous sample paths, zero expectation and covariance function $(c^{*}_{ij}(\tau ,t))_{i,j = 0}^{\nu }$.

The main focus of this paper was to prove tightness of components $(Y^{*}_{n,i}(t), 0 \leq t \leq 1 )$ by Poissonization and construction of an appropriate inequality for covariances.

As $Y_{n,i}(t)=Y^{*}_{n_{i}}(t)-Y^{*}_{n,i-1}(t)$, we state tightness of components (Y_{n, i},0 ≤ t ≤ 1) and calculate c_ij(τ, t) by formulas

$$c_{ij}(\tau,t)=c^{*}_{ij}(\tau,t)-c^{*}_{i + 1,j}(\tau,t)-c^{*}_{i,j + 1}(\tau,t) +c^{*}_{i + 1,j + 1}(\tau,t), $$

$$c_{0j}(\tau,t)=c^{*}_{1j}(\tau,t)-c^{*}_{1,j + 1}(\tau,t), c_{i0}(\tau,t)=c^{*}_{i1}(\tau,t)-c^{*}_{i + 1,1}(\tau,t). $$

The proof is complete. □

The limiting (ν + 1)-dimensional Gaussian process is self-similar with Hurst parameter H = 𝜃/2 < 1/2. Its first component coincides in distribution with the first component of the limiting process in Theorem 1 in Durieu and Wang (2016).

We need some specific corollary to calculate limiting variance in Theorem 2.

Corollary 3.

In assumptions of Theorem 4, randomvector$((Y^{*}_{n,1}(1)$,Y_n,1(1)) convergesweakly to a normal one with zero mean and covariance matrix

$${\Gamma}(1-\theta) \left( \begin{array}{cc} 2^{\theta}-1 & -\theta 2^{\theta-1}\\ -\theta 2^{\theta-1} & \theta(1-2^{\theta-2}(1-\theta)) \end{array} \right). $$