Skip to main content
SpringerLink
Log in

Universal coding for memoryless sources with countably infinite alphabets

  • Source Coding
  • Published:
Problems of Information Transmission Aims and scope Submit manuscript

Abstract

We present an asymptotically efficient coding strategy for a stationary countably infinite source determined over a set of nonnegative integers. If the kth moment µ k of the source data is finite, then asymptotic average coding redundancy for length-n blocks, n → ∞, is upper bounded by C (log n/n)k/(k+1), where C is a nonnegative constant. The coding efficiency is demonstrated via an example of scalar quantization of random variables with generalized Gaussian distribution.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Bocharova, I., Compression for Multimedia, Cambridge: Cambridge Univ. Press, 2010.

    MATH  Google Scholar 

  2. Ziv, J. and Lempel, A., Compression of Individual Sequences via Variable-Rate Coding, IEEE Trans. Inform. Theory, 1978, vol. 24, no. 5, pp. 530–536.

    Article  MATH  MathSciNet  Google Scholar 

  3. Cover, T.M. and Thomas, J.A., Elements of Information Theory, New York: Wiley, 1991.

    Book  MATH  Google Scholar 

  4. Golomb, S.W., Run-Length Encodings, IEEE Trans. Inform. Theory, 1966, vol. 12, no. 3, pp. 399–401.

    Article  MATH  MathSciNet  Google Scholar 

  5. Rice, R.F., Some Practical Universal Noiseless Coding Techniques, Tech. Rep. of Jet Propulsion Lab., California Inst. of Technology, Pasadena, CA, Mar. 15, 1979, no. JPL-79-22.

    Google Scholar 

  6. Gallager, R.G. and Van Voorhis, D.C., Optimal Source Codes for Geometrically Distributed Integer Alphabets, IEEE Trans. Inform. Theory, 1975, vol. 21, no. 2, pp. 228–230.

    Article  MATH  Google Scholar 

  7. Levenshtein, V.I., On the Redundancy and Delay of Uniquely Decodable Codes for Natural Numbers, Probl. Kibern., 1968, vol. 20, pp. 173–179.

    Google Scholar 

  8. Elias, P., Universal Codeword Sets and Representations of the Integers, IEEE Trans. Inform. Theory, 1975, vol. 21, no. 2, pp. 194–203.

    Article  MATH  MathSciNet  Google Scholar 

  9. Elias, P., Interval and Recency Rank Source Coding: Two On-line Adaptive Variable-Length Schemes, IEEE Trans. Inform. Theory, 1987, vol. 33, no. 1, pp. 3–10.

    Article  MATH  MathSciNet  Google Scholar 

  10. Shtarkov, Yu.M., Universal’noe kodirovanie. Teoriya i algoritmy (Universal Coding: Theory and Algorithms), Moscow: Fizmatlit, 2013.

    Google Scholar 

  11. Boucheron, S., Garivier, A., and Gassiat, E., Coding on Countably Infinite Alphabets, IEEE Trans. Inform. Theory, 2009, vol. 55, no. 1, pp. 358–373.

    Article  MathSciNet  Google Scholar 

  12. Fitingof, B.M., Optimal Coding in the Case of Unknown and Changing Message Statistics, Probl. Peredachi Inf., 1966, vol. 2, no. 2, pp. 3–11 [Probl. Inf. Trans. (Engl. Transl.), 1966, vol. 2, no. 2, pp. 1–7].

    MATH  MathSciNet  Google Scholar 

  13. Babkin, V.F., A Universal Encoding Method with Nonexponential Work Expenditure for a Source of Independent Messages, Probl. Peredachi Inf., 1971, vol. 7, no. 4, pp. 13–21 [Probl. Inf. Trans. (Engl. Transl.), 1971, vol. 7, no. 4, pp. 288–294].

    MATH  MathSciNet  Google Scholar 

  14. Krichevsky, R.E. and Trofimov, V.K., The Performance of Universal Coding, IEEE Trans. Inform. Theory, 1981, vol. 27, no. 2, pp. 199–207.

    Article  MATH  MathSciNet  Google Scholar 

  15. Gyorfi, L., Pali, I., and van der Meulen, E.C., There Is No Universal Source Code for an Infinite Source Alphabet, IEEE Trans. Inform. Theory, 1994, vol. 40, no. 1, pp. 267–271.

    Article  MathSciNet  Google Scholar 

  16. Acharya, J., Das, H., Jafarpour, A., Orlitsky, A., and Suresh, A.T., Tight Bounds for Universal Compression of Large Alphabets, in Proc. 2013 IEEE Int. Sympos. on Information Theory (ISIT’2013), Istanbul, Turkey, July 7–12, 2013, pp. 2875–2879.

  17. Shtarkov, Yu.M., Universal Sequential Coding of Single Messages, Probl. Peredachi Inf., 1987, vol. 23, no. 3, pp. 3–17 [Probl. Inf. Trans. (Engl. Transl.), 1987, vol. 23, no. 3, pp. 175–186].

    MathSciNet  Google Scholar 

  18. Gallager, R.G., Information Theory and Reliable Communication, New York: Wiley, 1968. Translated under the title Teoriya informatsii i nadezhnaya svyaz’, Moscow: Sov. Radio, 1974.

    MATH  Google Scholar 

  19. Sharifi, K. and Leon-Garcia, A., Estimation of Shape Parameter for Generalized Gaussian Distributions in Subband Decompositions of Video, IEEE Trans. Circ. Syst. Video Techn., 1995, vol. 5, no. 1, pp. 52–56.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. St. Petersburg National Research University of Information Technologies, Mechanics and Optics, St. Petersburg, Russia

    B. D. Kudryashov & A. V. Porov

Authors
  1. B. D. Kudryashov
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. A. V. Porov
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to B. D. Kudryashov.

Additional information

Original Russian Text © B.D. Kudryashov, A.V. Porov, 2014, published in Problemy Peredachi Informatsii, 2014, Vol. 50, No. 4, pp. 100–109.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kudryashov, B.D., Porov, A.V. Universal coding for memoryless sources with countably infinite alphabets. Probl Inf Transm 50, 390–399 (2014). https://doi.org/10.1134/S0032946014040085

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1134/S0032946014040085

Keywords

Search

Navigation