Advertisement

Designs, Codes and Cryptography

, Volume 15, Issue 1, pp 29–45 | Cite as

On Lower Bounds for the Redundancy of Optimal Codes

  • Roberto De Prisco
  • Alfredo De Santis
Article

Abstract

The problem of providing bounds on the redundancy of an optimal code for a discrete memoryless source in terms of the probability distribution of the source, has been extensively studied in the literature. The attention has mainly focused on binary codes for the case when the most or the least likely source letter probabilities are known. In this paper we analyze the relationships among tight lower bounds on the redundancy r. Let r ≥ φD,i(x) be the tight lower bound on r for D-ary codes in terms of the value x of the i-th most likely source letter probability. We prove that φD,i-1(x) ≤ φD,i(x) for all possible x and i. As a consequence, we can bound the redundancy when only the value of a probability (but not its rank) is known. Another consequence is a shorter and simpler proof of a known bound. We also provide some other properties of tight lower bounds. Finally, we determine an achievable lower bound on r in terms of the least likely source letter probability for D ≥ 3, generalizing the known bound for the case D = 2.

lower bounds huffman codes source coding redundancy of optimal codes 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    R. M. Capocelli and A. De Santis, Tight upper bounds on the redundancy of Huffman codes, IEEE Trans. Inform. Theory, Vol. IT-35, No.5 (1989) pp. 1084–1091.Google Scholar
  2. 2.
    R. M. Capocelli and A. De Santis, A note on D-ary Huffman codes, IEEE Trans. Inform. Theory, Vol. IT-37, No.1 (1991) pp. 174–179.Google Scholar
  3. 3.
    R. M. Capocelli and A. De Santis, New bounds on the redundancy of Huffman codes, IEEE Trans. Inform. Theory, Vol. IT-37, No.4 (1991) pp. 1095–1104.Google Scholar
  4. 4.
    R. M. Capocelli, R. Giancarlo, and I. J. Taneja, Bounds on the redundancy of Huffman codes, IEEE Trans. Inform. Theory, Vol. IT-32, No.6 (1986) pp. 854–857.Google Scholar
  5. 5.
    R. De Prisco and A. De Santis, On the redundancy achieved by Huffman codes, Information Sciences, Vol. 88, Nos.1-4 (1996) pp. 131–148.Google Scholar
  6. 6.
    R. G. Gallager, Variation on a theme by Huffman, IEEE Trans. Inform. Theory, Vol. IT-24, No.6 (1978) pp. 668–674.Google Scholar
  7. 7.
    Y. Horibe, An improved bound for weight-balanced trees, Inform. Contr., Vol. 34, No.2 (1977) pp. 148–151.Google Scholar
  8. 8.
    D. A. Huffman, A method for the construction of minimum-redundancy codes, Proc. IRE, Vol. 40, No.2 (1952) pp. 1098–1101.Google Scholar
  9. 9.
    O. Johnsen, On the redundancy of Huffman codes, IEEE Trans. Inform. Theory, Vol. IT-26, No.2 (1980) pp. 220–222.Google Scholar
  10. 10.
    G. O. H. Katona and T. O. H. Nemetz, Huffman codes and self-information, IEEE Trans. Inform. Theory, Vol. IT-22, No.3 (1976) pp. 337–340.Google Scholar
  11. 11.
    D. Manstetten, Tight bounds on the redundancy of Huffman codes, IEEE Trans. Inform. Theory, Vol. IT-38, No.1 (1992) pp. 144–151.Google Scholar
  12. 12.
    B. L. Montgomery and J. Abrahams, On the redundancy of optimal prefix-condition codes for finite and infinite sources, IEEE Trans. Inform. Theory, Vol. IT-33, No.1 (1987) pp. 156–160.Google Scholar
  13. 13.
    B. L. Montgomery and B. V. K. V. Kumar, On the average codeword length of optimal binary codes for extended sources, IEEE Trans. Inform. Theory, Vol. IT-33, No.2 (1987) pp. 293–296.Google Scholar
  14. 14.
    F. M. Reza, Introduction to Information Theory, McGraw-Hill, New York (1951).Google Scholar
  15. 15.
    J. Rissanen, Bounds for weight balanced tree, IBM J. Research and Development, Vol. 17 (1973) pp. 101–105.Google Scholar
  16. 16.
    R. Yeung, The redundancy theorem and new bounds of the expected length of the Huffman code, IEEE Trans. Inform. Theory, Vol. IT-37, No.3 (1991) pp. 687–691.Google Scholar

Copyright information

© Kluwer Academic Publishers 1998

Authors and Affiliations

  • Roberto De Prisco
    • 1
  • Alfredo De Santis
    • 2
  1. 1.MIT Laboratory for Computer Science545 Technology SquareCambridgeUSA
  2. 2.Dipartimento di Informatica ed ApplicazioniUniversità di SalernoBaronissi (SA)Italy

Personalised recommendations