Abstract
In this chapter, we present the results of Krichevsky [6]. We consider the following estimation problem, which arises in the context of data compression, is discussed: For a given discrete memoryless source, we want to estimate the unknown underlying source probabilities by means of a former source output, assuming that the estimated probabilities are used to encode the letters of the source alphabet.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
We identify \(\varSigma \) with the set
$$ \left\{ x = (x_1, x_2, \ldots , x_m) \in {\mathbb R}^m: x_i \ge 0 \quad \hbox {for} \quad i \in \{1, \ldots , m\} \quad \hbox {and} \quad \sum _{i=1}^m x_i = 1\right\} . $$\(\varSigma \) is compact in \({\mathbb R}^m\) and both \(\overline{L}_S^T(n)\) and \(H_S\) are continuous in \(\varSigma \) relative to \(S\). Therefore \(R_S^n(T)\) is continuous and, understood as a function of \(S\), assumes its maximum and minimum in \(\varSigma \).
- 2.
Instead of using the multinomial distribution for resolving the sum, the calculations show that it is sufficient to distinguish only between elements \(x\) and (not \(x\)) and apply the binomial distribution.
References
T.M. Cover, Admissibility of Gilbert’s encoding for unknown source probabilities. IEEE Trans. Inf. Theory 18(1), 216–217 (1972)
T.M. Cover, J.A. Thomas, Elements of Information Theory, Wiley Series in Telecommunications (Wiley, New York, 1991)
W. Feller, An Introduction to Probability Theory and Its Applications, 3rd edn, vol. I (Wiley, New York, 1968)
E.N. Gilbert, Codes based on inaccurate source probabilities. IEEE Trans. Inf. Theory 3, 304–314 (1971)
R. Krichevsky, Universal Compression and Retrieval (Kluwer Academic Publisher, Dordrecht, 1994)
R. Krichevsky, Laplace’s law of succession and universal encoding. IEEE Trans. Inf. Theory 44(1), 296–303 (1998)
F. Willems, Y. Shtarkov, T. Tjalkens, The context tree weighting method: basic properties. IEEE Trans. Inf. Theory IT–41, 653–664 (1995)
Further Reading
G. Box, G. Tiao, Bayesian Inference in Statistical Analysis (Wiley, New York, 1992)
H. Cramér, Mathematical Methods of Statistics (Princeton University Press, Princeton, 1974)
R. Krichevsky, Laplace’s Law of Succession and Universal Coding, Preprint, Institute of Mathematics of the Siberian Branch of the Russian Academy of Sciences, No. 21, Novosibirsk (1995)
K. Stange, Bayes-Verfahren (Springer, Berlin, 1977)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this chapter
Cite this chapter
Ahlswede, R., Ahlswede, A., Althöfer, I., Deppe, C., Tamm, U. (2015). \(\beta \)-Biased Estimators in Data Compression. In: Ahlswede, A., Althöfer, I., Deppe, C., Tamm, U. (eds) Transmitting and Gaining Data. Foundations in Signal Processing, Communications and Networking, vol 11. Springer, Cham. https://doi.org/10.1007/978-3-319-12523-7_8
Download citation
DOI: https://doi.org/10.1007/978-3-319-12523-7_8
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-12522-0
Online ISBN: 978-3-319-12523-7
eBook Packages: EngineeringEngineering (R0)