Guessing probability distributions from small samples

Abstract

We propose a new method for the calculation of the statistical properties, e.g., the entropy, of unknown generators of symbolic sequences. The probability distributionp(k) of the elementsk of a population can be approximated by the frequenciesf(k) of a sample provided the sample is long enough so that each elementk occurs many times. Our method yields an approximation if this precondition does not hold. For a givenf(k) we recalculate the Zipf-ordered probability distribution by optimization of the parameters of a guessed distribution. We demonstrate that our method yields reliable results.