The proper formula for relative entropy and its asymptotics in quantum probability


Umegaki's relative entropyS(ω,ϕ)=TrD ω(logD ω−logD ϕ) (of states ω and ϕ with density operatorsD ω andD ϕ, respectively) is shown to be an asymptotic exponent considered from the quantum hypothesis testing viewpoint. It is also proved that some other versions of the relative entropy give rise to the same asymptotics as Umegaki's one. As a byproduct, the inequality TrA logAB ≧TrA(logA+logB) is obtained for positive definite matricesA andB.

This is a preview of subscription content, access via your institution.


  1. 1.

    Araki, H.: Relative entropy of states of von Neumann algebras. Publ. RIMS, Kyoto Univ.11, 809–833 (1976).

    Google Scholar 

  2. 2.

    Araki, H.: Relative entropy for states of von Neumann algebras II. Publ. RIMS, Kyoto Univ.13, 173–192 (1977)

    Google Scholar 

  3. 3.

    Billingsley, P.: Ergodic theory and information. New York: Wiley 1965

    Google Scholar 

  4. 4.

    Blahut, R.E.: Principles and practice of information theory Reading, MA: Addison-Wesley 1987

    Google Scholar 

  5. 5.

    Belavkin, V.P., Staszewski, P.:C *-algebraic generalization of relative entropy and entropy. Ann. Inst. H. Poincaré Sect. A37, 51–58 (1982)

    Google Scholar 

  6. 6.

    Connes, A., Størmer, E.: Entropy for automorphisms of II1 von Neumann algebras. Acta Math.134, 289–306 (1975)

    Google Scholar 

  7. 7.

    Csiszár, I., Körner, J.: Information theory, coding theorems for discrete memorlyess systems. Budapest: Akadémiai Kiadó Orlando: Academic Press 1981

    Google Scholar 

  8. 8.

    Donald, M.J.: On the relative entropy.Commun. Math. Phys.105, 13–34 (1986)

    Google Scholar 

  9. 9.

    Donald, M.J.: Continuity and relative hamiltonians. Commun. Math. Phys.136, 625–632 (1991)

    Google Scholar 

  10. 10.

    Doplicher, S., Kasteler, D.: Ergodic states in a non-commutative ergodic theory. Commun. Math. Phys.7, 1–20 (1968)

    Google Scholar 

  11. 11.

    Ellis, R.S.: Entropy, large deviations, and statistical mechanics. Berlin, heidelberg, new York: Springer 1985

    Google Scholar 

  12. 12.

    Fujii, J.I., Kamei, E.: Relative operator entropy in noncommutative information theory. Math. Japon.34, 341–348 (1989)

    Google Scholar 

  13. 13.

    Helstrom, C.W.: Quantum detection and estimation theory. New York: Academic Press 1976

    Google Scholar 

  14. 14.

    Hiai, F., Ohya, M., Tsukada, M.: Sufficiency, KMS condition and relative entropy in von Neumann algebras. Pacific J. Math.96, 99–109 (1981)

    Google Scholar 

  15. 15.

    Hiai, F., Ohya, M., Tsukada, M.: Sufficiency and relative entropy in *-algebras with applications in quantum systems. Pacific J. Math.107, 117–140 (1983)

    Google Scholar 

  16. 16.

    Kosaki, H.: Relative entropy of states: a variational expression J. Operator Theory16, 335–348 (1986)

    Google Scholar 

  17. 17.

    Kullback, S., Leibler, R.A.: On information and sufficiency. Ann. Math. Statist.22, 79–86 (1951)

    Google Scholar 

  18. 18.

    Kovács, I., Szücs, J.: Ergodic type theorems in von Neumann algebras. Acta Sci. Math.27, 233–246 (1966)

    Google Scholar 

  19. 19.

    Ohya, M., Petz, D.: Notes on quantum entropy. Preprint

  20. 20.

    Ohya, M., Tsukada, M., Umegaki, H.: A formulation of noncommutative McMillan theorem. Proc. Jpn. Acad.63A, 50–53 (1987)

    Google Scholar 

  21. 21.

    Parry, W.: Properties of quantum entropy. In: Accaridi, L., von Waldenfels, W. (eds.) Quantum probability and applications. II. Lect.Notes Math., Vol. 1136, pp. 428–441. Berlin, Heidelberg, New York: Springer 1985

    Google Scholar 

  22. 23.

    Petz, D.: Properties of the relative entropy of states of von neumann algebras. ActaMath. Hungar.47, 65–72 (1986)

    Google Scholar 

  23. 24.

    Petz, D.: Sufficient subalgebras and the relative entropy of states of a von Neuman algebra. Commun. Math. Phys.105, 123–131 (1986)

    Google Scholar 

  24. 25.

    Petz, D.: First steps towards a Donsker and Varadhan theory in operator algebras. Lect. Notes Math., Vol. 1442, pp. 311–319. Berlin, heidelberg, New York: Springer 1990

    Google Scholar 

  25. 26.

    Petz, D.: On certain properties of the relative entropy of states of operator algebras. Math. Z. (to appear)

  26. 27.

    Petz. D.: Characterization of the relative entropy of states of matrix algebras. Preprint

  27. 28.

    Petz, D., Raggio, G.A., Verbeure, A.: Asymptotics of Varadhan-type and Gibbs variational principle. Commun. Math. Phys.121, 271–282 (1989)

    Google Scholar 

  28. 29.

    Raggio, G.A., Werner, R.F.: Quantum statistical mechanics of general mean field systems. Helv. Phys. Acta62, 980–1003 (1989)

    Google Scholar 

  29. 30.

    Størmer, E.: Large grups of automorphisms ofC *-algebras. Commun. Math. Phys.5, 1–22 (1967)

    Google Scholar 

  30. 31.

    Takesaki, M.: Conditional expectations in von Neumann algegras. J. Funct. Anal.9, 306–321 (1972)

    Google Scholar 

  31. 32.

    Uhlmann, A.: Relative entropy and the Wigner-Yanase-Dyson-Lieb concavity in an interpolation theory. Commun. Math. Phys.54, 21–32 (1977)

    Google Scholar 

  32. 33.

    Umegaki, H.: Conditional expectation in an operator algebra, IV (entropy and information). Kōdai Math. Sem. Rep.14, 59–85 (1962)

    Google Scholar 

Download references

Author information



Additional information

Communicated by H. Araki

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Hiai, F., Petz, D. The proper formula for relative entropy and its asymptotics in quantum probability. Commun.Math. Phys. 143, 99–114 (1991).

Download citation


  • Entropy
  • Neural Network
  • Statistical Physic
  • Complex System
  • Nonlinear Dynamics