# On Lower Bounds for the Redundancy of Optimal Codes

- 46 Downloads
- 1 Citations

## Abstract

The problem of providing bounds on the redundancy of an optimal code for a discrete memoryless source in terms of the probability distribution of the source, has been extensively studied in the literature. The attention has mainly focused on binary codes for the case when the most or the least likely source letter probabilities are known. In this paper we analyze the relationships among tight lower bounds on the redundancy r. Let r ≥ φ_{D,i}(x) be the tight lower bound on r for D-ary codes in terms of the value x of the i-th most likely source letter probability. We prove that φ_{D,i-1}(x) ≤ φ_{D,i}(x) for all possible x and i. As a consequence, we can bound the redundancy when only the value of a probability (but not its rank) is known. Another consequence is a shorter and simpler proof of a known bound. We also provide some other properties of tight lower bounds. Finally, we determine an achievable lower bound on r in terms of the least likely source letter probability for D ≥ 3, generalizing the known bound for the case D = 2.

## Preview

Unable to display preview. Download preview PDF.

## References

- 1.R. M. Capocelli and A. De Santis, Tight upper bounds on the redundancy of Huffman codes,
*IEEE Trans. Inform. Theory*, Vol. IT-35, No.5 (1989) pp. 1084–1091.Google Scholar - 2.R. M. Capocelli and A. De Santis, A note on
*D*-ary Huffman codes,*IEEE Trans. Inform. Theory*, Vol. IT-37, No.1 (1991) pp. 174–179.Google Scholar - 3.R. M. Capocelli and A. De Santis, New bounds on the redundancy of Huffman codes,
*IEEE Trans. Inform. Theory*, Vol. IT-37, No.4 (1991) pp. 1095–1104.Google Scholar - 4.R. M. Capocelli, R. Giancarlo, and I. J. Taneja, Bounds on the redundancy of Huffman codes,
*IEEE Trans. Inform. Theory*, Vol. IT-32, No.6 (1986) pp. 854–857.Google Scholar - 5.R. De Prisco and A. De Santis, On the redundancy achieved by Huffman codes,
*Information Sciences*, Vol. 88, Nos.1-4 (1996) pp. 131–148.Google Scholar - 6.R. G. Gallager, Variation on a theme by Huffman,
*IEEE Trans. Inform. Theory*, Vol. IT-24, No.6 (1978) pp. 668–674.Google Scholar - 7.Y. Horibe, An improved bound for weight-balanced trees,
*Inform. Contr.*, Vol. 34, No.2 (1977) pp. 148–151.Google Scholar - 8.D. A. Huffman, A method for the construction of minimum-redundancy codes,
*Proc. IRE*, Vol. 40, No.2 (1952) pp. 1098–1101.Google Scholar - 9.O. Johnsen, On the redundancy of Huffman codes,
*IEEE Trans. Inform. Theory*, Vol. IT-26, No.2 (1980) pp. 220–222.Google Scholar - 10.G. O. H. Katona and T. O. H. Nemetz, Huffman codes and self-information,
*IEEE Trans. Inform. Theory*, Vol. IT-22, No.3 (1976) pp. 337–340.Google Scholar - 11.D. Manstetten, Tight bounds on the redundancy of Huffman codes,
*IEEE Trans. Inform. Theory*, Vol. IT-38, No.1 (1992) pp. 144–151.Google Scholar - 12.B. L. Montgomery and J. Abrahams, On the redundancy of optimal prefix-condition codes for finite and infinite sources,
*IEEE Trans. Inform. Theory*, Vol. IT-33, No.1 (1987) pp. 156–160.Google Scholar - 13.B. L. Montgomery and B. V. K. V. Kumar, On the average codeword length of optimal binary codes for extended sources,
*IEEE Trans. Inform. Theory*, Vol. IT-33, No.2 (1987) pp. 293–296.Google Scholar - 14.F. M. Reza,
*Introduction to Information Theory*, McGraw-Hill, New York (1951).Google Scholar - 15.J. Rissanen, Bounds for weight balanced tree,
*IBM J. Research and Development*, Vol. 17 (1973) pp. 101–105.Google Scholar - 16.R. Yeung, The redundancy theorem and new bounds of the expected length of the Huffman code,
*IEEE Trans. Inform. Theory*, Vol. IT-37, No.3 (1991) pp. 687–691.Google Scholar