Problems of Information Transmission

, Volume 53, Issue 3, pp 215–221 | Cite as

On coupling of probability distributions and estimating the divergence through variation

Information Theory
  • 24 Downloads

Abstract

Let X be a discrete random variable with a given probability distribution. For any α, 0 ≤ α ≤ 1, we obtain precise values for both the maximum and minimum variational distance between X and another random variable Y under which an α-coupling of these random variables is possible. We also give the maximum and minimum values for couplings of X and Y provided that the variational distance between these random variables is fixed. As a consequence, we obtain a new lower bound on the divergence through variational distance.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Zhang, Z., Estimating Mutual Information via Kolmogorov Distance, IEEE Trans. Inform. Theory, 2007, vol. 53, no. 9, pp. 3280–3282.CrossRefMATHMathSciNetGoogle Scholar
  2. 2.
    Sason, I., Entropy Bounds for Discrete Random Variables via Maximal Coupling, IEEE Trans. Inform. Theory, 2013, vol. 59, no. 11, pp. 7118–7131.CrossRefMATHMathSciNetGoogle Scholar
  3. 3.
    Prelov, V.V., Coupling of Probability Distributions and an Extremal Problem for the Divergence, Probl. Peredachi Inf., 2015, vol. 51, no. 2, pp. 114–121 [Probl. Inf. Trans. (Engl. Transl.), 2015, vol. 51, no. 2, pp. 192–199].MATHMathSciNetGoogle Scholar
  4. 4.
    Fedotov, A.A., Harremöes, P., and Topsøe, F., Refinements of Pinsker’s Inequality, IEEE Trans. Inform. Theory, 2003, vol. 49, no. 6, pp. 1491–1498.CrossRefMATHMathSciNetGoogle Scholar
  5. 5.
    Csiszár, I. and Körner, J., Information Theory: Coding Theorems for Discrete Memoryless Systems, New York: Academic; Budapest: Akad. Kiadó, 1981. Translated under the title Teoriya informatsii: teoremy kodirovaniya dlya diskretnykh sistem bez pamyati, Moscow: Mir, 1985.MATHGoogle Scholar
  6. 6.
    Vajda, I., Note on Discrimination Information and Variation, IEEE Trans. Inform. Theory, 1970, vol. 16, no. 6, pp. 771–773.CrossRefMATHMathSciNetGoogle Scholar
  7. 7.
    Ordentlich, E. and Weinberger, M.J., A Distribution Dependent Refinement of Pinsker’s Inequality, IEEE Trans. Inform. Theory, 2005, vol. 51, no. 5, pp. 1836–1840.CrossRefMATHMathSciNetGoogle Scholar

Copyright information

© Pleiades Publishing, Inc. 2017

Authors and Affiliations

  1. 1.Kharkevich Institute for Information Transmission ProblemsRussian Academy of SciencesMoscowRussia

Personalised recommendations