Skip to main content
SpringerLink
Log in

Simple one-shot bounds for various source coding problems using smooth Rényi quantities

  • Source Coding
  • Published:
Problems of Information Transmission Aims and scope Submit manuscript

Abstract

We consider the problem of source compression under three different scenarios in the one-shot (nonasymptotic) regime. To be specific, we prove one-shot achievability and converse bounds on the coding rates for distributed source coding, source coding with coded side information available at the decoder, and source coding under maximum distortion criterion. The one-shot bounds obtained are in terms of smooth max Rényi entropy and smooth max Rényi divergence. Our results are powerful enough to yield the results that are known for these problems in the asymptotic regime both in the i.i.d. (independent and identically distributed) and non-i.i.d. settings.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Uyematsu, T. and Matsuta, T. Revisiting the Slepian–Wolf Coding Problem for General Sources: A Direct Approach, in Proc. 2014 IEEE Int. Sympos. on Information Theory (ISIT’2014), Honolulu, HI,USA, June 29–July 4, 2014, pp. 1336–1340.

    Chapter  Google Scholar 

  2. Slepian, D. and Wolf, J.K., Noiseless Coding of Correlated Information Sources, IEEE Trans. Inform. Theory, 1973, vol. 19, no. 4, pp. 471–480.

    Article  MathSciNet  MATH  Google Scholar 

  3. Wyner, A.D., On Source Coding with Side Information at the Decoder, IEEE Trans. Inform. Theory, 1975, vol. 21, no. 3, pp. 294–300.

    Article  MathSciNet  MATH  Google Scholar 

  4. Han, T.S., Information-Spectrum Methods in Information Theory, Berlin: Springer, 2003.

    Book  MATH  Google Scholar 

  5. Renner, R. and Wolf, S., Smooth Rényi Entropy and Applications, in Proc. 2004 IEEE Int. Sympos. on Information Theory (ISIT’2004), Chicago, IL,USA, June 27–July 2, 2004, p. 233.

  6. Cover, T.M. and Thomas, J.A., Elements of Information Theory, Hoboken, NJ: Wiley, 2006, 2nd ed.

    MATH  Google Scholar 

  7. Gray, R.M., Entropy and Information Theory, New York: Springer, 1990.

    Book  MATH  Google Scholar 

  8. Renes, J.M. and Renner, R., Noisy Channel Coding via Privacy Amplification and Information Reconciliation, IEEE Trans. Inform. Theory, 2011, vol. 57, no. 11, pp. 7377–7385.

    Article  MathSciNet  Google Scholar 

  9. Wang, L., Colbeck, R., and Renner, R., Simple Channel Coding Bounds, in Proc. 2009 IEEE Int. Sympos. on Information Theory (ISIT’2009), Seoul, Korea, June 28–July 3, 2009, pp. 1804–1808.

    Chapter  Google Scholar 

  10. Verdú, S., Non-asymptotic Achievability Bounds in Multiuser Information Theory, in Proc. 50th Annual Allerton Conf. on Communication, Control, and Computing, Monticello, IL,USA, Oct. 1–5, 2012, pp. 1–8.

    Google Scholar 

  11. Polyanskiy, Y., Poor, H.V., and Verdú, S., Channel Coding Rate in the Finite Blocklength Regime, IEEE Trans. Inform. Theory, 2010, vol. 56, no. 5, pp. 2307–2359.

    Article  MathSciNet  Google Scholar 

  12. Datta, N. and Renner, R., Smooth Entropies and the Quantum Information Spectrum, IEEE Trans. Inform. Theory, 2009, vol. 55, no. 6, pp. 2807–2815.

    Article  MathSciNet  Google Scholar 

  13. König, R., Renner, R., and Schaffner, C., The Operational Meaning of Min- and Max-Entropy, IEEE Trans. Inform. Theory, 2009, vol. 55, no. 9, pp. 4337–4347.

    Article  MathSciNet  Google Scholar 

  14. Dupuis, F., Hayden, P., and Li, K., A Father Protocol for Quantum Broadcast Channels, IEEE Trans. Inform. Theory, 2010, vol. 56, no. 6, pp. 2946–2956.

    Article  MathSciNet  Google Scholar 

  15. Berta, M., Christandl, M., and Renner, R., The Quantum Reverse Shannon Theorem Based on One-Shot Information Theory, Commun. Math. Phys., 2011, vol. 306, no. 3, pp. 579–615.

    Article  MathSciNet  MATH  Google Scholar 

  16. Datta, N. and Hsieh, M.-H., The Apex of the Family Tree of Protocols: Optimal Rates and Resource Inequalities, New J. Phys., 2011, vol. 13, pp. 093042.

    Article  Google Scholar 

  17. Wang, L. and Renner, R., One-Shot Classical-Quantum Capacity and Hypothesis Testing, Phys. Rev. Lett., 2012, vol. 108, no. 20, pp. 200501.

    Article  Google Scholar 

  18. Renes, J.M. and Renner, R., One-Shot Classical Data Compression with Quantum Side Information and the Distillation of Common Randomness or Secret Keys, IEEE Trans. Inform. Theory, 2012, vol. 58, no. 3, pp. 1985–1991.

    Article  MathSciNet  Google Scholar 

  19. Datta, N., Renes, J.M., Renner, R., and Wilde, M.M., One-Shot Lossy Quantum Data Compression, IEEE Trans. Inform. Theory, 2013, vol. 59, no. 12, pp. 8057–8076.

    Article  MathSciNet  Google Scholar 

  20. Rényi, A., On Measures of Entropy and Information, Proc. 4th Berkeley Sympos. on Mathematical Statistics and Probability, Berkely, CA,USA, June 20–July 30, 1960, Neyman, J., Ed., Berkely: Univ. of California Press, 1961, vol. 1, pp. 547–561.

  21. Renner, R. andWolf, S., Simple and Tight Bounds for Information Reconciliation and Privacy Amplification, Advances in Cryptology—ASIACRYPT’2005 (Proc. 11th Int. Conf. on the Theory and Application of Cryptology and Information Security, Chennai, India, Dec. 4–8, 2005), Roy, B.K., Ed., Lect. Notes Comp. Sci., vol. 3788, Berlin: Springer, 2005, pp. 199–216.

    Google Scholar 

  22. Miyake, S. and Kanaya, F., Coding Theorems on Correlated General Sources, IEICE Trans. Fund. Electr. Commun. Comput. Sci., 1995, vol. E78-A, no. 9, pp. 1063–1070.

    Google Scholar 

  23. Kuzuoka, S., A Simple Technique for Bounding the Redundancy of Source Coding with Side Information, in Proc. 2012 IEEE Int. Sympos. on Information Theory (ISIT’2012), Cambridge, MA,USA, July 1–6, 2012, pp. 910–914.

    Chapter  Google Scholar 

  24. Jain, R., Radhakrishnan, J., and Sen, P., Prior Entanglement, Message Compression and Privacy in Quantum Channels, in Proc. 20th Annual IEEE Conf. on Computational Complexity, San Jose, CA,USA, June 11–15, 2005, pp. 285–296.

    Google Scholar 

  25. Jain, R., Sen, P., and Radhakrishnan, J., Optimal Direct Sum and Privacy Trade-off Results for Quantum and Classical Communication Complexity, arXiv:0807.1267 [cs.DC], 2008.

    Google Scholar 

  26. Schoenmakers, B., Tjoelker, J., Tuyls, P., and Verbitskiy, E., Smooth Rényi Entropy of Ergodic Quantum Information Sources, in Proc. 2007 IEEE Int. Sympos. on Information Theory (ISIT’2007), Nice, France, June 24–29, 2007, pp. 256–260.

    Chapter  Google Scholar 

  27. Datta, N., Min- and Max-Relative Entropies and a New Entangelement Monotone, IEEE Trans. Inform. Theory, 2009, vol. 55, no. 6, pp. 2816–2826.

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Technology and Computer Science, Tata Institute of Fundamental Research, Mumbai, India

    N. A. Warsi

Authors
  1. N. A. Warsi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to N. A. Warsi.

Additional information

Original Russian Text © N.A. Warsi, 2016, published in Problemy Peredachi Informatsii, 2016, Vol. 52, No. 1, pp. 43–71.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Warsi, N.A. Simple one-shot bounds for various source coding problems using smooth Rényi quantities. Probl Inf Transm 52, 39–65 (2016). https://doi.org/10.1134/S0032946016010051

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1134/S0032946016010051

Keywords

Search

Navigation