Skip to main content
Log in

The quantum relative entropy as a rate function and information criteria

  • Published:
Quantum Information Processing Aims and scope Submit manuscript

Abstract

We prove that the quantum relative entropy is a rate function in large deviation principle. Next, we define information criteria for quantum states and estimate the accuracy of the use of them. Most of the results in this paper are essentially based on Hiai-Ohya-Tsukada theorem.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

Notes

  1. In probability theory and classical information theory, theory of generalized entropies is started by Rényi [44, 45].

  2. A positive linear functional \(\rho \) on a von Neumann algebra \(\mathcal M \) on \(\mathcal H \) is said to be normal if there exists a positive trace class operator \(\sigma \in {\varvec{T}}(\mathcal H )_{+}\) such that \(\rho (A)=\mathrm Tr [\sigma A]\) for each \(A\in \mathcal M \).

  3. See in [7], Sect. 4.1].

  4. A measure \(\mu \) on a Borel space \(K\) is said to be pseudosupported by an arbitrary set \(A\subseteq K\) if \(\mu (B)=0\) for all Baire sets \(B\) such that \(B\cap A=\emptyset \).

  5. The set of factor states \(F_\mathfrak{A }\) on \(\mathfrak A \) is not always Borel measurable. If a \(\text{ C}^*\)-algebra \(\mathfrak A \) is separable, \(F_\mathfrak{A }\) is Borel measurable and the central measure \(\mu _\omega \) of \(\omega \in E_\mathfrak A \) is supported by \(F_\mathfrak{A }\).

  6. The injective \(\text{ C}^{*}\)-tensor product \(\mathfrak A _{1}\otimes \mathfrak A _{2}=\mathfrak A _{1}\otimes _{min} \mathfrak A _{2}\) of two \(\text{ C}^{*}\)-algebras \(\mathfrak A _{1}\) and \(\mathfrak A _{2}\) is the completion of the algebraic tensor product \(\mathfrak A _{1}\otimes _{alg} \mathfrak A _{2}\) by the norm \(\Vert \cdot \Vert _{min}\) defined by \(\Vert C\Vert _{min}=\sup \Vert (\pi _{1}\otimes \pi _{2})(C)\Vert , C\in \mathfrak A _{1}\otimes _{alg} \mathfrak A _{2}\), where \(\pi _{1}\) and \(\pi _{2}\) run over all representations of \(\mathfrak A _{1}\) and \(\mathfrak A _{2}\), respectively. The tensor product \(\omega _1\otimes \omega _2\in E_\mathfrak{A _{1}\otimes \mathfrak A _{2}}\) of states \(\omega _1\in E_\mathfrak{A _1}\) and \(\omega _2\in E_\mathfrak{A _2}\) is defined as the state \(\omega \) satisfying \(\omega (A_1\otimes A_2)=\omega _1(A_1)\omega _2(A_2)\) for \(A\in \mathfrak A _1, A\in \mathfrak A _2\). See [46], in detail.

  7. A standard form of a von Neumann algebra \(\mathcal M \) on a Hilbert space \(\mathcal H \) is a quadtuple \((\mathcal M ,\mathcal H ,J,\mathcal P )\) constituting of \(\mathcal M , \mathcal H \), a unitary involution \(J\), a self-dual cone \(\mathcal P \) in \(\mathcal H \), which satisfy the followings: (i) \(J\mathcal M J=\mathcal M ^\prime \); (ii) \(JAJ=A^*, A\in \mathfrak Z (\mathcal M )\); (iii) \(J\xi =\xi , \xi \in \mathcal P \); (iv) \(AJAJ\mathcal P \subset \mathcal P , A\in \mathcal M \). See [47].

  8. See [10] and [11, p. 261].

  9. See [11, p. 263].

References

  1. Aitchison, J.: Goodness of prediction fit. Biometrika 62, 547–554 (1975)

    Article  MathSciNet  MATH  Google Scholar 

  2. Akaike, H.: A new look at the statistical model identification. IEEE Trans. Autom. Control 19, 716–723 (1974)

    Article  MathSciNet  ADS  MATH  Google Scholar 

  3. Amari, S., Nagaoka, H.: Methods of Information Geometry, Translations of mathematical monographs, vol. 191. Amer. Math. Soc. & Oxford Univ.Press (2000)

  4. Araki, H.: Relative Entropy for States of Von Neumann Algebras II. Publ. RIMS, Kyoto Univ. vol. 13, pp. 173–192 (1977)

  5. Audenaert, K.M.R., Calsamiglia, J., Munoz-Tapia, R., Bagan, E., Masanes, L., Acin, A., Verstraete, F.: Discriminating states: the quantum Chernoff bound. Phys. Rev. Lett. 98, 160501 (2007)

    Article  ADS  Google Scholar 

  6. Bjelakovic, I., Deuschel, J.-D., Kruger, T., Seiler, R., Siegmund-Schultze, R., Szkola, A.: A quantum version of Sanov’s theorem. Commun. Math. Phys. 260, 659–671 (2006)

    Article  MathSciNet  ADS  Google Scholar 

  7. Bratteli, O., Robinson, D.W.: Operator Algebras and Quantum Statistical Mechanics, 2nd printing of 2nd edn, vol. 1. Springer (2002)

  8. Corcuera, J.M., Giummolè, F.: A generalized Bayes rule for prediction. Scand. J. Stat. 26, 265 (1999)

    Article  MATH  Google Scholar 

  9. Cover, T.M., Thomas, J.A.: Elements of Information Theory. Wiley, New York (1991)

    Book  MATH  Google Scholar 

  10. Csiszár, I.: A simple proof of Sanov’s theorem. Bull. Brazilian Math. Soc. 37, 453–459 (2006)

    Article  MATH  Google Scholar 

  11. Dembo, A., Zeitouni, O.: Large Deviations Techniques and Applications, 2nd edn. Springer, Berlin (2002)

    Google Scholar 

  12. Drton, M., Sturmfels, B., Sullivant, S.: Lectures on Algebraic Statistics. Birkhäuser (2008)

  13. Harada, R., Ojima, I.: A unified scheme of measurement and amplification processes based on Micro-Macro duality-Stern-Gerlach experiment as a typical example. Open Syst. Inf. Dyn. 16, 55–74 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  14. Hayashi, M.: Error exponent in asymmetric quantum hypothesis testing and its application to classical-quantum channel coding. Phys. Rev. A 76, 062301 (2007)

    Article  ADS  Google Scholar 

  15. Helstrom, C.W.: Quantum Detection and Estimation Theory. Academic Press, New York (1976)

    Google Scholar 

  16. Hiai, F., Mosonyi, M., Petz, D., Bény, C.: Quantum f-Divergence and error Correction. Rev. Math. Phys. 23, 691–747 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  17. Hiai, F., Ohya, M., Tsukada, M.: Sufficiency and relative entropy in *-algebras with applications in quantum systems. Pac. J. Math. 107, 117–140 (1983)

    Article  MathSciNet  MATH  Google Scholar 

  18. Hiai, F., Petz, D.: The proper formula for relative entropy and its asymptotics in quantum probability. Commun. Math. Phys. 143, 99–114 (1991)

    Article  MathSciNet  ADS  MATH  Google Scholar 

  19. Holevo, A.S.: Probabilistic and Statistical Aspects of Quantum Theory. North-Holland, Amsterdam (1982)

    MATH  Google Scholar 

  20. Jaksic, V., Ogata, Y., Pautrat, Y., Pillet, C.-A.: Entropic fluctuations in quantum statistical mechanics. An introduction. In: Quantum Theory from Small to Large Scales, Lecture Notes of the Les Houches Summer School, vol. 95, August 2010. Oxford University Press, USA (2012)

  21. Jaksic, V., Ogata, Y., Pillet, C.-A., Seiringer, R.: Quantum hypothesis testing and non-equilibrium statistical mechanics. Rev. Math. Phys. 24, 1230002 (2012)

    Article  MathSciNet  Google Scholar 

  22. Jenova, A., Petz, D.: Sufficiency in quantum statistical inference. Commun. Math. Phys. 263, 259–276 (2006)

    Article  ADS  Google Scholar 

  23. Lehmann, E.L., Romano, J.P.: Testing Statistical Hypotheses, 3rd edn. Springer, New York (2005)

    MATH  Google Scholar 

  24. Nagaoka, H.: The Converse Part of the Theorem for Quantum Hoeffding Bound. arXiv:quant-ph/0611289 (2006)

  25. Nagaoka, H., Hayashi, M.: An information-spectrum approach to classical and quantum hypothesis testing for simple hypotheses. IEEE Trans. Inf. Theory 53, 534–549 (2007)

    Article  MathSciNet  Google Scholar 

  26. Nielsen, M., Chuang, I.: Quantum Computation and Quantum Information. Cambridge Univ. Press, Cambridge (2000)

    MATH  Google Scholar 

  27. Nussbaum, M., Szkola, A.: The Chernoff lower bound for symmetric quantum hypothesis testing. Ann. Stat. 37, 1040–1057 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  28. Ogata, Y.: Large deviations in quantum spin chains. Commun. Math. Phys. 296, 35–68 (2010)

    Article  MathSciNet  ADS  MATH  Google Scholar 

  29. Ogawa, T., Hayashi, M.: On error exponents in quantum hypothesis testing. IEEE Trans. Inf. Theory 50, 1368–1372 (2004)

    Article  MathSciNet  Google Scholar 

  30. Ogawa, T., Nagaoka, H.: Strong converse and Stein’s lemma in quantum hypothesis testing. IEEE Trans. Inf. Theory 46, 2428–2433 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  31. Ojima, I.: A unified scheme for generalized sectors based on selection criteria-order parameters of symmetries and of thermality and physical meanings of adjunctions. Open Syst. Inf. Dyn. 10, 235–279 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  32. Ojima, I.: Micro-macro duality in quantum physics. In: Proceedings of International Conference on Stochastic Analysis, Classical and Quantum, pp. 143–161. World Scientific, Singapore (2005). arXiv:math-ph/0502038

  33. Ojima, I.: Micro-macro duality and space-time emergence. In: Proceedings of International Conference, “Advances in Quantum Theory”, pp. 197–206 (2011)

  34. Ojima, I., Okamura, K.: Large devation strategy for inverse problem I. Open Syst. Inf. Dyn. 19, 1250021 (2012)

    Article  MathSciNet  Google Scholar 

  35. Ojima, I., Okamura, K.: Large devation strategy for inverse problem II. Open Syst. Inf. Dyn. 19, 1250022 (2012)

    Article  MathSciNet  Google Scholar 

  36. Ojima, I., Okamura, K., Saigo, H. (in preparation)

  37. Ozawa, M.: Quantum measuring processes of continuous obsevables. J. Math. Phys. 25, 79–87 (1984)

    Article  MathSciNet  ADS  Google Scholar 

  38. Ozawa, M.: Quantum perfect correlations, Ann. Phys. (N.Y.), 321, 744–769 (2006)

    Google Scholar 

  39. Ohya, M., Petz, D.: Qunatum Entropy and Its Use. Springer, Berlin (1993)

    Book  Google Scholar 

  40. Petz, D.: Quasi-entropies for states of a von Neumann algebra. Publ. RIMS Kyoto Univ. 21, 787–800 (1985)

    Article  MathSciNet  MATH  Google Scholar 

  41. Petz, D.: A variational expression for the relative entropy. Commun. Math. Phys. 114, 345–349 (1988)

    Article  MathSciNet  ADS  MATH  Google Scholar 

  42. Petz, D.: First steps towards a Donsker and Varadhan theory in operator algebras. In: Quantum Probability and Applications, V. Lec. Notes in Math. vol. 1442, pp. 311–319 (1990)

  43. Petz, D., Raggio, G.A., Verbeure, A.: Asymptotics of Varadhan-type and the Gibbs variational principle. Commun. Math. Phys. 121, 271–282 (1989)

    Article  MathSciNet  ADS  MATH  Google Scholar 

  44. Rényi, A.: Wahrscheinlichkeitsrechnung. VED Deutsch, Ver der Wiss, Berlin (1962)

  45. Rényi, A.: On the foundations of information theory. Rev. Int. Stat. Inst. 33, 1–14 (1965)

    Article  MATH  Google Scholar 

  46. Takesaki, M.: Theory of Operator Algebras I. Springer, Berlin (1979)

    Book  MATH  Google Scholar 

  47. Takesaki, M.: Theory of Operator Algebras II. Springer, Berlin (2002)

    Google Scholar 

  48. Tanaka, F.: Generalized Bayesian predictive density operator. In: Proceedings of 14th Quantum Information Technology Symposium (QIT14), Tokyo, Japan, May 29–30, pp. 107–110 (2006)

  49. Tanaka, F., Komaki, F.: Bayesian predictive density operators for exchangeable quantum-statistical models. Phys. Rev. A 71, 052323 (2005)

    Article  ADS  Google Scholar 

  50. Uhlmann, A.: Relative entropy and the Wigner-Yanase-Dyson-Lieb concavity in an interpolation theory. Commun. Math. Phys. 54, 21–32 (1977)

    Article  MathSciNet  ADS  MATH  Google Scholar 

  51. Usami, K., Nambu, Y., Tsuda, Y., Matsumoto, K., Nakamura, K.: Accuracy of quantum-state estimation utilizing Akaike’s information criterion. Phys. Rev. A 68, 022314 (2003)

    Article  ADS  Google Scholar 

  52. Watanabe, S.: Algebraic Geometry and Statistical Learning Theory. Cambridge University Press, Cambridge (2009)

  53. Watanabe, S.: Asymptotic learning curve and renormalizable condition in statistical learning theory. J. Phys. Conf. Ser. 233, 012014 (2010)

    Article  ADS  Google Scholar 

  54. Watanabe, S.: Asymptotic equivalence of Bayes cross validation and widely applicable information criterion in singular learning theory. J. Mach. Lear. Res. 11, 3571–3591 (2010)

    MATH  Google Scholar 

  55. Yin, J.O.S., van Enk, S.J.: Information criteria for efficient quantum state estimation. Phys. Rev. A 83, 062110 (2011)

    Article  ADS  Google Scholar 

Download references

Acknowledgments

The author would like to thank Prof. Izumi Ojima, Dr. Hayato Saigo, Dr. Takahiro Hasebe, Dr. Hiroshi Ando for their warm encouragement and useful comments. He also thanks anonymous referees for their valuable comments to improve the paper.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Kazuya Okamura.

Proofs of Theorems

Proofs of Theorems

Proof of Theorem 7

It suffices to prove for \(|\alpha |<1\). It is proved by discussion in [17, p. 129] that

$$\begin{aligned} QF_t\left(\varphi ^R,\psi ^L\right)(1,1)=\left\langle \Psi ,\Delta _{\Phi ,\Psi }^{1-t} \Psi \right\rangle . \end{aligned}$$

Therefore, we have

$$\begin{aligned} S^{(\alpha )}(\varphi \Vert \psi )_\mathrm{Uhlmann }&= \frac{4}{1-\alpha ^2}\left(1-QF_{\frac{1+\alpha }{2}}\left(\varphi ^R,\psi ^L\right)(1,1)\right) \\&= \frac{4}{1-\alpha ^2}\left(1-\langle \Psi ,\Delta _{\Phi ,\Psi }^{1-\frac{1+\alpha }{2}} \Psi \rangle \right) \\&= S^{(\alpha )}(\varphi \Vert \psi )_\mathrm{Araki }. \end{aligned}$$

\(\square \)

Proof of Theorem 8

It suffices to prove for \(|\alpha |<1\). By [17, Lemma 3.1], it is proved that

$$\begin{aligned} QF_t(\varphi ^R,\psi ^L)(A,A) =QF_t(\tilde{\varphi }^R,\tilde{\psi }^L)(\pi (A),\pi (A)), \end{aligned}$$

where \(\pi =\pi _{\varphi +\psi }\). Thus we have

$$\begin{aligned} S^{(\alpha )}(\varphi \Vert \psi )&= \frac{4}{1-\alpha ^2}\Bigg (1-QF_{\frac{1+\alpha }{2}}(\varphi ^R,\psi ^L)(1,1)\Bigg ) \\&= \frac{4}{1-\alpha ^2}\Bigg (1-QF_{\frac{1+\alpha }{2}} \Bigg (\tilde{\varphi }^R,\tilde{\psi }^{L}\Bigg )(\pi (1),\pi (1)\Bigg )\Bigg ) \\&= S^{(\alpha )}\left(\tilde{\varphi }\Vert \tilde{\psi }\right). \end{aligned}$$

\(\square \)

Lemma 4

Let \(\varphi |_\mathcal{B }, \psi |_\mathcal{B }\) be the restrictions of states \(\varphi ,\psi \) on a von Neumann algebra \(\mathcal M \) to a \(*\)-subalgebra \(\mathcal B \) to \(\mathcal M \). Then,

$$\begin{aligned} S^{(\alpha )}_\mathcal{B }(\varphi \Vert \psi ):=S^{(\alpha )}(\varphi |_\mathcal{B }\Vert \psi |_\mathcal{B }) \le S^{(\alpha )}(\varphi \Vert \psi ). \end{aligned}$$

Proof

It suffices to prove it for \(|\alpha |<1\). \(|_\mathcal{B }:E_\mathcal M \rightarrow E_\mathcal B \) is the dual of the embedding \(\mathcal B \ni B \mapsto B\in \mathcal M \). Using [50, Proposition 9] and Theorem 8,

$$\begin{aligned} QF_t\Bigg (\varphi ^R\Vert \psi ^L\Bigg )(1,1)&= QI_t\Bigg (\varphi ^R\Vert \psi ^L\Bigg )(1)^2 \\&\le QI_t\Bigg ((\varphi |_\mathcal{B })^R\Vert (\psi |_\mathcal{B })^L\Bigg )(1)^2 \\&= QF_t\Bigg ((\varphi |_\mathcal{B })^R\Vert (\psi |_\mathcal{B })^L\Bigg )(1,1). \end{aligned}$$

Thus we have

$$\begin{aligned} S^{(\alpha )}(\varphi \Vert \psi )&= \frac{4}{1-\alpha ^2}\left(1-QF_{\frac{1+\alpha }{2}}\left(\varphi ^R,\psi ^L\right)(1,1)\right) \\&\ge \frac{4}{1-\alpha ^2}\left(1-QF_{\frac{1+\alpha }{2}} \left((\varphi |_\mathcal{B })^R\Vert (\psi |_\mathcal{B })^L\right)(1,1)\right) \\&= S^{(\alpha )}(\varphi |_\mathcal{B }\Vert \psi |_\mathcal{B })=S^{(\alpha )}_\mathcal{B }(\varphi \Vert \psi ). \end{aligned}$$

\(\square \)

Proof of Theorem 9

It suffices to prove for \(|\alpha |<1\).

We define \(\Gamma : \mathfrak A \rightarrow C(E_\mathfrak A )\) by \(\Gamma (A)(\rho )=\rho (A)\), and its dual map \(\Gamma ^*:M_1(E_\mathfrak A )\rightarrow E_\mathfrak A \) by

$$\begin{aligned} (\Gamma ^*\lambda )(A)=\int \rho (A)\;d\lambda (\rho )=\lambda (\Gamma (A)), \end{aligned}$$

for \(\lambda \in M_1(E_\mathfrak A )\). \(\Gamma \) satisfies

$$\begin{aligned} \Gamma (1)=1,\;\Gamma (A^*) =\Gamma (A)^*,\; \Gamma (A^*)\Gamma (A) \le \Gamma (A^*A), \end{aligned}$$

for any \(A\in \mathfrak A \). Thus,

$$\begin{aligned} S^{(\alpha )}(\varphi \Vert \psi ) =S^{(\alpha )}(\Gamma ^*\mu \Vert \Gamma ^*\nu ) \le D^{(\alpha )}(\mu \Vert \nu ). \end{aligned}$$

By assumption, \(m\) is a subcentral measure \(\mu _\mathcal{C }\) associated with a von Neumann subalgebra \(\mathcal C \) of the center \(\mathfrak Z _\chi (\mathfrak A )=\pi _{\chi }(\mathfrak A )^{\prime \prime }\cap \pi _{\chi }(\mathfrak A )^{\prime }\) of a state \(\chi \in E_\mathfrak{A }\). We define a \(*\)-isomorphism \(\kappa _m:L^\infty (E_\mathfrak A ,m)\rightarrow \mathcal C \) by

$$\begin{aligned} \langle \Omega _\chi ,\pi _\chi (A)\kappa _m(f) \Omega _\chi \rangle =\int f(\rho )\;\rho (A)\;dm(\rho ). \end{aligned}$$

It is then proved in [17, Example 2.6 and Theorem 3.2] that

$$\begin{aligned} \tilde{\varphi }(\kappa _m(f))&= \int f(\rho )\;d\mu (\rho ),\\ \tilde{\psi }(\kappa _m(f))&= \int f(\rho )\;d\nu (\rho ), \end{aligned}$$

where \(\tilde{\varphi }, \tilde{\psi }\) are the normal extensions of \(\varphi , \psi \) to \(\pi _{\chi }(\mathfrak A )^{\prime \prime }\). By Lemma 4,

$$\begin{aligned} S^{(\alpha )}(\varphi \Vert \psi )&= S^{(\alpha )}(\tilde{\varphi }\Vert \tilde{\psi }) \\&\ge S^{(\alpha )}_\mathcal{C }(\tilde{\varphi }\Vert \tilde{\psi })=D^{(\alpha )}(\mu \Vert \nu ). \end{aligned}$$

\(\square \)

Rights and permissions

Reprints and permissions

About this article

Cite this article

Okamura, K. The quantum relative entropy as a rate function and information criteria. Quantum Inf Process 12, 2551–2575 (2013). https://doi.org/10.1007/s11128-013-0540-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11128-013-0540-x

Keywords

Navigation