Information Quantities and Parameter Estimation in Classical Systems

  • Masahito Hayashi
Part of the Graduate Texts in Physics book series (GTP)


For the study of quantum information theory, mathematical statistics, and information geometry, which are mainly examined in a nonquantum context. This chapter briefly summarizes the fundamentals of these topics from a unified viewpoint. Since these topics are usually treated individually, this chapter will be useful even for nonquantum applications.


Mutual Information Maximum Likelihood Estimator Fisher Information Relative Entropy Haar Measure 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    I. Csiszár, Information type measures of difference of probability distribution and indirect observations. Studia Scient. Math. Hungar. 2, 299–318 (1967)MathSciNetzbMATHGoogle Scholar
  2. 2.
    S. Amari, H. Nagaoka, Methods of Information Geometry (AMS & Oxford University Press, Oxford, 2000)zbMATHGoogle Scholar
  3. 3.
    A. Rényi, On measures of information and entropy, in Proceedings of the Fourth Berkeley Symposium on Mathematical Statistics and Probability (University of California Press, Berkeley, 1961), pp. 547–561Google Scholar
  4. 4.
    R.M. Fano, Transmission of Information: A Statistical Theory of Communication (Wiley, New York, 1961)Google Scholar
  5. 5.
    M. Hayashi, Security analysis of \(\varepsilon \)-almost dual universal\(_2\) hash functions: smoothing of min entropy vs. smoothing of Rényi entropy of order 2 (2013). arXiv:1309.1596
  6. 6.
    S. Amari, \(\alpha \)-divergence Is unique, belonging to both \(f\)-divergence and Bregman divergence classes. IEEE Trans. Inform. Theory 55(11), 4925–4931 (2009)MathSciNetCrossRefGoogle Scholar
  7. 7.
    A.W. van der Vaart, Asymptotic Statistics (Cambridge University Press, Cambridge, 1998)CrossRefzbMATHGoogle Scholar
  8. 8.
    I. Csiszár, J. Körner, Information Theory: Coding Theorems for Discrete Memoryless Systems (Academic, 1981)Google Scholar
  9. 9.
    I.N. Sanov, On the probability of large deviations of random variables. Mat. Sbornik 42, 11–44 (1957) (in Russian). English translation: Selected Translat. Math. Stat. 1, 213–244 (1961)Google Scholar
  10. 10.
    M. Keyl, R.F. Werner, Estimating the spectrum of a density operator. Phys. Rev. A 64, 052311 (2001)ADSMathSciNetCrossRefGoogle Scholar
  11. 11.
    K. Matsumoto, Seminar notes (1999)Google Scholar
  12. 12.
    M. Hayashi, Optimal sequence of POVMs in the sense of Stein’s lemma in quantum hypothesis. J. Phys. A Math. Gen. 35, 10759–10773 (2002)ADSCrossRefzbMATHGoogle Scholar
  13. 13.
    M. Hayashi, Exponents of quantum fixed-length pure state source coding. Phys. Rev. A 66, 032321 (2002)ADSCrossRefGoogle Scholar
  14. 14.
    M. Hayashi, K. Matsumoto, Variable length universal entanglement concentration by local operations and its application to teleportation and dense coding, quant-ph/0109028 (2001); K. Matsumoto, M. Hayashi, Universal entanglement concentration. Phys. Rev. A 75, 062338 (2007)Google Scholar
  15. 15.
    M. Hayashi, K. Matsumoto, Quantum universal variable-length source coding. Phys. Rev. A 66, 022311 (2002)ADSMathSciNetCrossRefzbMATHGoogle Scholar
  16. 16.
    M. Hayashi, K. Matsumoto, Simple construction of quantum universal variable-length source coding. Quant. Inf. Comput. 2, Special Issue, 519–529 (2002)Google Scholar
  17. 17.
    M. Hayashi, Asymptotics of quantum relative entropy from a representation theoretical viewpoint. J. Phys. A Math. Gen. 34, 3413–3419 (2001)ADSMathSciNetCrossRefzbMATHGoogle Scholar
  18. 18.
    H. Cramér, Sur un nouveaux theoorème-limite de la théorie des probabilités, in Actualités Scientifiques et Industrielles, no. 736, in Colloque consacré à la thèorie des probabilités (Hermann, Paris, 1938), pp. 5–23Google Scholar
  19. 19.
    J. Gärtner, On large deviations from the invariant measure. Theory Prob. Appl. 22, 24–39 (1977)MathSciNetCrossRefzbMATHGoogle Scholar
  20. 20.
    R. Ellis, Large deviations for a general class of random vectors, Ann. Probab. 12, 1, 1–12 (1984); Entropy, Large Deviations and Statistical Mechanics (Springer, Berlin, 1985)Google Scholar
  21. 21.
    R.R. Bahadur, On the asymptotic efficiency of tests and estimates. Sankhyā 22, 229 (1960)MathSciNetzbMATHGoogle Scholar
  22. 22.
    R.R. Bahadur, Rates of Convergence of Estimates and Test Statistics. Ann. Math. Stat. 38, 303 (1967)Google Scholar
  23. 23.
    R.R. Bahadur, Some limit theorems in statistics, in Regional Conference Series in Applied Mathematics, no. 4 (SIAM, Philadelphia, 1971)Google Scholar
  24. 24.
    J.C. Fu, On a theorem of Bahadur on the rate of convergence of point estimators. Ann. Stat. 1, 745 (1973)MathSciNetCrossRefzbMATHGoogle Scholar
  25. 25.
    A.I. Khinchin, Mathematical Foundations of Information Theory (Dover, New York, 1957)zbMATHGoogle Scholar
  26. 26.
    T.S. Han, Information-Spectrum Methods in Information Theory (Springer, Berlin, 2002) (originally appeared in Japanese in 1998)Google Scholar
  27. 27.
    V.D. Milman, G. Schechtman, Asymptotic theory of finite-dimensional normed spaces, vol. 1200, Lecture Notes in Mathematics (Springer, Berlin, 1986)Google Scholar
  28. 28.
    T. Cover, J. Thomas, Elements of Information Theory (Wiley, New York, 1991)Google Scholar
  29. 29.
    M. Hayashi, Exponential decreasing rate of leaked information in universal random privacy amplification. IEEE Trans. Inf. Theory 57, 3989–4001 (2011)MathSciNetCrossRefGoogle Scholar
  30. 30.
    M. Iwamoto, J. Shikata, Information theoretic security for encryption based on conditional Rényi entropies. Inform. Theor. Secur. Lect. Notes Comput. Sci. 8317(2014), 103–121 (2014)CrossRefzbMATHGoogle Scholar
  31. 31.
    M. Müller-Lennert, F. Dupuis, O. Szehr, S. Fehr, M. Tomamichel, On quantum Renyi entropies: a new generalization and some properties. J. Math. Phys. 54, 122203 (2013)ADSMathSciNetCrossRefzbMATHGoogle Scholar
  32. 32.
    E.L. Lehman, G. Casella, Theory of Point Estimation (Springer, Berlin Heidelberg New York, 1998)Google Scholar
  33. 33.
    A. Dembo, O. Zeitouni, Large Deviation Techniques and Applications (Springer, Berlin, 1997)zbMATHGoogle Scholar
  34. 34.
    J.A. Bucklew, Large Deviation Techniques in Decision, Simulation, and Estimation (Wiley, New York, 1990)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2017

Authors and Affiliations

  1. 1.Graduate School of MathematicsNagoya UniversityNagoyaJapan

Personalised recommendations