Zipf law and the firm size distribution: a critical discussion of popular estimators

Abstract

The upper tail of the firm size distribution is often assumed to follow a Power Law. Several recent papers, using different estimators and different data sets, conclude that the Zipf Law, in particular, provides a good fit, implying that the fraction of firms with size above a given value is inversely proportional to the value itself. In this article we compare the asymptotic and small sample properties of different methods through which this conclusion has been reached. We find that the family of estimators most widely adopted, based on an OLS regression, is in fact unreliable and basically useless for appropriate inference. This finding raises doubts about previously identified Zipf behavior. Based on extensive numerical analysis, we recommend the adoption of the Hill estimator over any other method when individual observations are available.

Appendix

To keep comparability with Gabaix and Ibragimov (2011), in Tables 4 and 5 we extend their analysis of AR(1) and MA(1) data to all the estimators considered in this article, although these types of time dependence are more compelling for applications in finance. We design the Monte Carlo exactly as in Gabaix and Ibragimov (2011).

Table 4 AR(1) data with sub-asymptotic deviation from Zipf Law

Table 5 MA(1) data with sub-asymptotic deviation from Zipf Law

For the case of AR(1) DGP, we generate R = 10,000 random samples of size N = 2,000 extracted from the AR(1) process

$$ Y_{i} =\rho Y_{i-1} + \epsilon_{i} \;\;, \text{\(i \geq 1\), \(Y_{0} = 0\)} \quad, $$

(6.1)

with 𝜖 extracted from Eq. 4.1, for a given combination of the values of c and ρ. On each sample we apply all the estimators for two different tail widths, i.e. including either the top–50 or the top–500 observations in the tails. We then repeat the Monte Carlo tests for different values of the parameters.

In Table 4, we report the average of point estimates across the 10,000 runs, together with asymptotic (theoretical) and sampled standard errors, as well as rejection rates of a t-test (at 5 % level) of the true null of unitary tail index performed at each run. First, consider the sensitivity to the AR(1) structure, setting aside the impact of the sub-asymptotic correction (i.e., set c = 0 and vary ρ), and take the case when the top–50 observations are considered. Although all the rejection rates are above the theoretical 5 %, the results provide a clear ranking. First, the CDF and PDF estimators both severely over-reject. Second, among the other three estimators, the Rank −1/2 is over-performing the others. However, if we take the top–500 observations in the tail, then the frequency at which the true null of unitary tail index is mistakenly rejected rapidly grows to above 50 % for all the estimators. Similar conclusions emerge when we let both c and ρ vary at the same time.

Table 5 replicates the analysis to study the properties under the MA(1) process

$$ Y_{i} = \epsilon_{i} + \theta \epsilon_{i-1} \;\;\;, \text{\(i \geq 1\)} \quad, $$

(6.2)

with 𝜖 ∼ (4.1). As before, we simulate R = 10,000 random samples of size N = 2,000 with varying c and θ, and again compare the behavior of the estimators for different tail width (top–50 and top–500 observations). The findings for θ = 0 obviously replicate the analysis on AR(1) with ρ = 0. Further, if we switch off the sub-asymptotic correction (i.e. set c = 0, and vary θ), we observe that, first, the CDF and PDF estimators are once again unreliable, with very high rejection rates. Second, although rejection rates are above the theoretical 5 % for all the methods, the Rank and Rank −1/2 estimators perform better (smaller rejection rates) than the other methods. The Rank performs slightly better if the tail includes the top-50 observations, while the Rank −1/2 is slightly better for the top–500 observations. Third, the patterns are similar when we let c and θ vary together. If anything, we notice that the rejection rates associated with all the estimators rapidly increase to above 20 % if we take the top-500 observations in the tail. Conversely, they are less dependent from the parameters in the top-50 exercise.