Numerical tools for obtaining power-law representations of heavy-tailed datasets

Abstract

Many empirical datasets have highly skewed, non-Gaussian, heavy-tailed distributions, dominated by a relatively small number of data points at the high end of the distribution. Consistent with their role as stable distributions, power laws have frequently been proposed to model such datasets. However there are physical situations that require distributions with finite means. Such situations may call for power laws with high-end cutoffs. Here, I present a maximum-likelihood technique for determining an optimal cut-off power law to represent a given dataset. I also develop a new statistical test of the quality of fit. Results are demonstrated for a number of benchmark datasets. Non-power-law datasets can frequently be represented by power laws, but this is a trivial result unless the dataset spans a broad domain. Nevertheless, I demonstrate that there are non-power-law distributions, including broad log-normal distributions, whose tails can be fit to power laws over many orders of magnitude. Therefore, caution is called for whenever power laws are invoked to represent empirical data.