Advertisement

Probability Theory and Related Fields

, Volume 162, Issue 1–2, pp 327–364 | Cite as

Information geometry and sufficient statistics

  • Nihat Ay
  • Jürgen Jost
  • Hông Vân Lê
  • Lorenz Schwachhöfer
Article

Abstract

Information geometry provides a geometric approach to families of statistical models. The key geometric structures are the Fisher quadratic form and the Amari–Chentsov tensor. In statistics, the notion of sufficient statistic expresses the criterion for passing from one model to another without loss of information. This leads to the question how the geometric structures behave under such sufficient statistics. While this is well studied in the finite sample size case, in the infinite case, we encounter technical problems concerning the appropriate topologies. Here, we introduce notions of parametrized measure models and tensor fields on them that exhibit the right behavior under statistical transformations. Within this framework, we can then handle the topological issues and show that the Fisher metric and the Amari–Chentsov tensor on statistical models in the class of symmetric 2-tensor fields and 3-tensor fields can be uniquely (up to a constant) characterized by their invariance under sufficient statistics, thereby achieving a full generalization of the original result of Chentsov to infinite sample sizes. More generally, we decompose Markov morphisms between statistical models in terms of statistics. In particular, a monotonicity result for the Fisher information naturally follows.

Keywords

Fisher quadratic form Amari–Chentsov tensor Sufficient statistic Chentsov theorem 

Mathematics Subject Classification (2010)

53C99 62B05 

Notes

Acknowledgments

H.V.L. would like to thank Shun-ichi Amari for many fruitful discussions, and Giovanni Pistone for providing the articles [11, 18]. We thank Holger Bernigau for his critical helpful comments on an early version of this paper. We are grateful to the anonymous referees for their helpful remarks and suggestions. This work has been supported by the Max-Planck Institute for Mathematics in the Sciences in Leipzig, the BSI at RIKEN in Tokyo, the ASSMS, GCU in Lahore-Pakistan, the VNU for Sciences in Hanoi, the Mathematical Institute of the Academy of Sciences of the Czech Republic in Prague, and the Santa Fe Institute. We are grateful for excellent working conditions and financial support of these institutions during extended visits of some of us.

References

  1. 1.
    Amari, S.: Theory of information spaces a geometrical foundation of statistics. POST RAAG Report 106 (1980)Google Scholar
  2. 2.
    Amari, S.: Differential geometry of curved exponential families curvature and information loss. Ann. Stat. 10, 357–385 (1982)CrossRefzbMATHMathSciNetGoogle Scholar
  3. 3.
    Amari, S.: Differential geometrical theory of statistics. In: Differential Geometry in Statistical Inference. Lecture Note-Monograph Series, vol. 10. Institute of Mathematical Statistics, California (1987)Google Scholar
  4. 4.
    Amari, S., Nagaoka, H.: Methods of Information Geometry. Translations of Mathematical Monographs, vol. 191, American Mathematical Society, Providence (2000)Google Scholar
  5. 5.
    Ay, N.: An Information-geometric approach to a theory of pragmatic structuring. Ann. Probab. 30(1), 416–436 (2002)CrossRefzbMATHMathSciNetGoogle Scholar
  6. 6.
    Ay, N., Olbrich, E., Bertschinger, N., Jost, J.: A geometric approach to complexity. Chaos 21, 37–103 (2011)CrossRefGoogle Scholar
  7. 7.
    Ay, N., Jost, J., Lê, H.V., Schwachhöfer, L.: Information geometry (book in preparation)Google Scholar
  8. 8.
    Blackwell, D.: Equivalent comparisons of experiments. Ann. Math. Stat. 24, 265–272 (1953)CrossRefzbMATHMathSciNetGoogle Scholar
  9. 9.
    Borovkov, A.A.: Mathematical Statistics. Gordon and Breach Science Publishers, Amsterdam (1998)zbMATHGoogle Scholar
  10. 10.
    Campbell, L.L.: An extended Chentsov characterization of a Riemannian metric. Proc. AMS 98, 135–141 (1986)zbMATHGoogle Scholar
  11. 11.
    Cena, A., Pistone, G.: Exponential statistical model. AISM 59, 27–56 (2007)CrossRefzbMATHMathSciNetGoogle Scholar
  12. 12.
    Cramer, H.: Mathematical Methods of Statistics. Princeton University Press, Princeton (1946)zbMATHGoogle Scholar
  13. 13.
    Chentsov, N.: Category of mathematical statistics. Dokl. Acad. Nauk USSR 164, 511–514 (1965)Google Scholar
  14. 14.
    Chentsov, N.: Algebraic foundation of mathematical statistics. Math. Operationsforsch. Stat. Ser. Stat. 9, 267–276 (1978)zbMATHGoogle Scholar
  15. 15.
    Chentsov, N.: Statistical Decision Rules and Optimal Inference. Translation of Mathematical Monograph, vol. 53. AMS, Providence (1982)Google Scholar
  16. 16.
    Efron, B.: Defining the curvature of a statistical problem (with applications to second order efficiency), with a discussion by C. R. Rao, Don A. Pierce, D. R. Cox, D. V. Lindley, Lucien LeCam, J. K. Ghosh, J. Pfanzagl, Niels Keiding, A. P. Dawid, Jim Reeds and with a reply by the author. Ann. Stat. 3, 1189–1242 (1975)CrossRefzbMATHMathSciNetGoogle Scholar
  17. 17.
    Fisher, R.A.: On the mathematical foundations of theoretical statistics. Philos. Trans. R. Soc. Lond. Ser. A 222, 309–368 (1922)Google Scholar
  18. 18.
    Gibilisco, P., Pistone, G.: Connections on non-parametric statistical models by Orlicz space geometry. Infin. Dimens. Anal. Quantum Probab. Relat. Top. 1(2), 325–347 (1998)CrossRefzbMATHMathSciNetGoogle Scholar
  19. 19.
    Hamilton, R.: The inverse function theorem of Nash Moser. Bull. AMS. 7, 65–222 (1982)CrossRefzbMATHGoogle Scholar
  20. 20.
    Jeffreys, H.: An invariant form for the prior probability in estimation problems. Proc. R. Soc. Lond. Ser. A 186, 453–461 (1946)CrossRefzbMATHMathSciNetGoogle Scholar
  21. 21.
    Kallenberg, O.: Foundations of Modern Probability, 2nd edn. Springer, New York (2001)Google Scholar
  22. 22.
    Krasnoselskii, M.A., Rutickii, Ya.B.: Convex functions and Orlicz spaces. Fizmatgiz, Moskava (1958) [In Russian; English translation: P. Noordfoff Ltd., Groningen (1961)]Google Scholar
  23. 23.
    Kullback, S.: Information Theory and Statistics. Dover Publications, New York (1968)Google Scholar
  24. 24.
    Kass, R., Vos, P.W.: Geometrical foundations of asymptotic inference. Wiley, New York (1997)CrossRefzbMATHGoogle Scholar
  25. 25.
    Lauritzen, S.: Statistical manifolds. In: Differential Geometry in Statistical Inference. Lecture Note-Monograph Series, vol. 10. Institute of Mathematical Statistics, California (1987)Google Scholar
  26. 26.
    Lê, H.V.: Statistical manifolds are statistical models. J. Geom. 84, 83–93 (2005)CrossRefzbMATHMathSciNetGoogle Scholar
  27. 27.
    Lê, H.V.: Monotone invariants and embedding of statistical models. In: Advances in Deterministic and Stochastic Analysis. World Scientific, pp. 231–254 (2007). arXiv:math/0506163
  28. 28.
    Lê, H.V.: The uniqueness of the Fisher metric as information metric. arXiv:1306.1465
  29. 29.
    Lê, H.V., Somberg, P., Vanžura, J.: Smooth structures on pseudomanifolds with isolated conical singularities. Acta Math. Vietnam. 38, 33–54 (2013). ArXiv:1006.5707
  30. 30.
    Morse, N., Sacksteder, R.: Statistical isomorphism. Ann. Math. Stat. 37, 203–214 (1966)CrossRefzbMATHMathSciNetGoogle Scholar
  31. 31.
    Morozova, E., Chentsov, N.: Markov invariant geometry on manifolds of states. In: Itogi Nauki i Techniki, Current Problems of Mathematics, Newest Achievements, vol. 6. Moscow, pp. 69–102 (1990)Google Scholar
  32. 32.
    Morozova, E., Chentsov, N.: Natural geometry on families of probability laws. In: Itogi Nauki i Techniki, Current Problems of Mathematics. Fundamental Directions, vol. 83, Moscow, pp. 133–265 (1991)Google Scholar
  33. 33.
    Murray, M., Rice, J.: Differential Geometry and Statistics. Chapman and Hall, London (1993)CrossRefzbMATHGoogle Scholar
  34. 34.
    Olbrich, E., Kahle, T., Bertschinger, N., Ay, N., Jost, J.: Quantifying structure in networks. Eur. Phys. J. B Condens. Matter Complex Syst. 77, 239–247 (2010)Google Scholar
  35. 35.
    Pistone, G., Sempi, C.: An infinite-dimensional structure on the space of all the probability measures equivalent to a given one. Ann. Stat. 5, 1543–1561 (1995)CrossRefMathSciNetGoogle Scholar
  36. 36.
    Rao, C.R.: Information and the accuracy attainable in the estimation of statistical parameters. Bull. Calcutta Math. Soc. 37, 81–89 (1945)zbMATHMathSciNetGoogle Scholar
  37. 37.
    Shahshahani, S.: A new mathematical framework for the study of linkage and selection. Memoirs of the American Mathematical Society, vol. 17, no. 211. American Mathematical Society (1979)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2014

Authors and Affiliations

  • Nihat Ay
    • 1
    • 3
  • Jürgen Jost
    • 1
    • 2
    • 3
  • Hông Vân Lê
    • 4
  • Lorenz Schwachhöfer
    • 5
  1. 1.Max-Planck-Institut für Mathematik in den NaturwissenschaftenLeipzigGermany
  2. 2.Mathematisches InstitutUniversität LeipzigLeipzigGermany
  3. 3.Santa Fe InstituteSanta FeUSA
  4. 4.Institute of Mathematics of ASCRPrague 1Czech Republic
  5. 5.Fakultät für MathematikTechnische Universität DortmundDortmundGermany

Personalised recommendations