On Generalized Csiszár-Kullback Inequalitieys

Abstract.

 Classical Csiszár-Kullback inequalities bound the L ¹-distance of two probability densities in terms of their relative (convex) entropies. Here we generalize such inequalities to not necessarily normalized and possibly non-positive L ¹ functions. Also, we analyse the optimality of the derived Csiszár-Kullback type inequalities and show that they are in many important cases significantly sharper than the classical ones (in terms of the functional dependence of the L ¹ bound on the relative entropy). Moreover our construction of these bounds is rather elementary.