Probability Theory and Related Fields

, Volume 162, Issue 1–2, pp 327–364 | Cite as

Information geometry and sufficient statistics

  • Nihat Ay
  • Jürgen Jost
  • Hông Vân Lê
  • Lorenz Schwachhöfer
Article

Abstract

Information geometry provides a geometric approach to families of statistical models. The key geometric structures are the Fisher quadratic form and the Amari–Chentsov tensor. In statistics, the notion of sufficient statistic expresses the criterion for passing from one model to another without loss of information. This leads to the question how the geometric structures behave under such sufficient statistics. While this is well studied in the finite sample size case, in the infinite case, we encounter technical problems concerning the appropriate topologies. Here, we introduce notions of parametrized measure models and tensor fields on them that exhibit the right behavior under statistical transformations. Within this framework, we can then handle the topological issues and show that the Fisher metric and the Amari–Chentsov tensor on statistical models in the class of symmetric 2-tensor fields and 3-tensor fields can be uniquely (up to a constant) characterized by their invariance under sufficient statistics, thereby achieving a full generalization of the original result of Chentsov to infinite sample sizes. More generally, we decompose Markov morphisms between statistical models in terms of statistics. In particular, a monotonicity result for the Fisher information naturally follows.

Keywords

Fisher quadratic form Amari–Chentsov tensor Sufficient statistic Chentsov theorem 

Mathematics Subject Classification (2010)

53C99 62B05 

References

  1. 1.
    Amari, S.: Theory of information spaces a geometrical foundation of statistics. POST RAAG Report 106 (1980)Google Scholar
  2. 2.
    Amari, S.: Differential geometry of curved exponential families curvature and information loss. Ann. Stat. 10, 357–385 (1982)CrossRefMATHMathSciNetGoogle Scholar
  3. 3.
    Amari, S.: Differential geometrical theory of statistics. In: Differential Geometry in Statistical Inference. Lecture Note-Monograph Series, vol. 10. Institute of Mathematical Statistics, California (1987)Google Scholar
  4. 4.
    Amari, S., Nagaoka, H.: Methods of Information Geometry. Translations of Mathematical Monographs, vol. 191, American Mathematical Society, Providence (2000)Google Scholar
  5. 5.
    Ay, N.: An Information-geometric approach to a theory of pragmatic structuring. Ann. Probab. 30(1), 416–436 (2002)CrossRefMATHMathSciNetGoogle Scholar
  6. 6.
    Ay, N., Olbrich, E., Bertschinger, N., Jost, J.: A geometric approach to complexity. Chaos 21, 37–103 (2011)CrossRefGoogle Scholar
  7. 7.
    Ay, N., Jost, J., Lê, H.V., Schwachhöfer, L.: Information geometry (book in preparation)Google Scholar
  8. 8.
    Blackwell, D.: Equivalent comparisons of experiments. Ann. Math. Stat. 24, 265–272 (1953)CrossRefMATHMathSciNetGoogle Scholar
  9. 9.
    Borovkov, A.A.: Mathematical Statistics. Gordon and Breach Science Publishers, Amsterdam (1998)MATHGoogle Scholar
  10. 10.
    Campbell, L.L.: An extended Chentsov characterization of a Riemannian metric. Proc. AMS 98, 135–141 (1986)MATHGoogle Scholar
  11. 11.
    Cena, A., Pistone, G.: Exponential statistical model. AISM 59, 27–56 (2007)CrossRefMATHMathSciNetGoogle Scholar
  12. 12.
    Cramer, H.: Mathematical Methods of Statistics. Princeton University Press, Princeton (1946)MATHGoogle Scholar
  13. 13.
    Chentsov, N.: Category of mathematical statistics. Dokl. Acad. Nauk USSR 164, 511–514 (1965)Google Scholar
  14. 14.
    Chentsov, N.: Algebraic foundation of mathematical statistics. Math. Operationsforsch. Stat. Ser. Stat. 9, 267–276 (1978)MATHGoogle Scholar
  15. 15.
    Chentsov, N.: Statistical Decision Rules and Optimal Inference. Translation of Mathematical Monograph, vol. 53. AMS, Providence (1982)Google Scholar
  16. 16.
    Efron, B.: Defining the curvature of a statistical problem (with applications to second order efficiency), with a discussion by C. R. Rao, Don A. Pierce, D. R. Cox, D. V. Lindley, Lucien LeCam, J. K. Ghosh, J. Pfanzagl, Niels Keiding, A. P. Dawid, Jim Reeds and with a reply by the author. Ann. Stat. 3, 1189–1242 (1975)CrossRefMATHMathSciNetGoogle Scholar
  17. 17.
    Fisher, R.A.: On the mathematical foundations of theoretical statistics. Philos. Trans. R. Soc. Lond. Ser. A 222, 309–368 (1922)Google Scholar
  18. 18.
    Gibilisco, P., Pistone, G.: Connections on non-parametric statistical models by Orlicz space geometry. Infin. Dimens. Anal. Quantum Probab. Relat. Top. 1(2), 325–347 (1998)CrossRefMATHMathSciNetGoogle Scholar
  19. 19.
    Hamilton, R.: The inverse function theorem of Nash Moser. Bull. AMS. 7, 65–222 (1982)CrossRefMATHGoogle Scholar
  20. 20.
    Jeffreys, H.: An invariant form for the prior probability in estimation problems. Proc. R. Soc. Lond. Ser. A 186, 453–461 (1946)CrossRefMATHMathSciNetGoogle Scholar
  21. 21.
    Kallenberg, O.: Foundations of Modern Probability, 2nd edn. Springer, New York (2001)Google Scholar
  22. 22.
    Krasnoselskii, M.A., Rutickii, Ya.B.: Convex functions and Orlicz spaces. Fizmatgiz, Moskava (1958) [In Russian; English translation: P. Noordfoff Ltd., Groningen (1961)]Google Scholar
  23. 23.
    Kullback, S.: Information Theory and Statistics. Dover Publications, New York (1968)Google Scholar
  24. 24.
    Kass, R., Vos, P.W.: Geometrical foundations of asymptotic inference. Wiley, New York (1997)CrossRefMATHGoogle Scholar
  25. 25.
    Lauritzen, S.: Statistical manifolds. In: Differential Geometry in Statistical Inference. Lecture Note-Monograph Series, vol. 10. Institute of Mathematical Statistics, California (1987)Google Scholar
  26. 26.
    Lê, H.V.: Statistical manifolds are statistical models. J. Geom. 84, 83–93 (2005)CrossRefMATHMathSciNetGoogle Scholar
  27. 27.
    Lê, H.V.: Monotone invariants and embedding of statistical models. In: Advances in Deterministic and Stochastic Analysis. World Scientific, pp. 231–254 (2007). arXiv:math/0506163
  28. 28.
    Lê, H.V.: The uniqueness of the Fisher metric as information metric. arXiv:1306.1465
  29. 29.
    Lê, H.V., Somberg, P., Vanžura, J.: Smooth structures on pseudomanifolds with isolated conical singularities. Acta Math. Vietnam. 38, 33–54 (2013). ArXiv:1006.5707
  30. 30.
    Morse, N., Sacksteder, R.: Statistical isomorphism. Ann. Math. Stat. 37, 203–214 (1966)CrossRefMATHMathSciNetGoogle Scholar
  31. 31.
    Morozova, E., Chentsov, N.: Markov invariant geometry on manifolds of states. In: Itogi Nauki i Techniki, Current Problems of Mathematics, Newest Achievements, vol. 6. Moscow, pp. 69–102 (1990)Google Scholar
  32. 32.
    Morozova, E., Chentsov, N.: Natural geometry on families of probability laws. In: Itogi Nauki i Techniki, Current Problems of Mathematics. Fundamental Directions, vol. 83, Moscow, pp. 133–265 (1991)Google Scholar
  33. 33.
    Murray, M., Rice, J.: Differential Geometry and Statistics. Chapman and Hall, London (1993)CrossRefMATHGoogle Scholar
  34. 34.
    Olbrich, E., Kahle, T., Bertschinger, N., Ay, N., Jost, J.: Quantifying structure in networks. Eur. Phys. J. B Condens. Matter Complex Syst. 77, 239–247 (2010)Google Scholar
  35. 35.
    Pistone, G., Sempi, C.: An infinite-dimensional structure on the space of all the probability measures equivalent to a given one. Ann. Stat. 5, 1543–1561 (1995)CrossRefMathSciNetGoogle Scholar
  36. 36.
    Rao, C.R.: Information and the accuracy attainable in the estimation of statistical parameters. Bull. Calcutta Math. Soc. 37, 81–89 (1945)MATHMathSciNetGoogle Scholar
  37. 37.
    Shahshahani, S.: A new mathematical framework for the study of linkage and selection. Memoirs of the American Mathematical Society, vol. 17, no. 211. American Mathematical Society (1979)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2014

Authors and Affiliations

  • Nihat Ay
    • 1
    • 3
  • Jürgen Jost
    • 1
    • 2
    • 3
  • Hông Vân Lê
    • 4
  • Lorenz Schwachhöfer
    • 5
  1. 1.Max-Planck-Institut für Mathematik in den NaturwissenschaftenLeipzigGermany
  2. 2.Mathematisches InstitutUniversität LeipzigLeipzigGermany
  3. 3.Santa Fe InstituteSanta FeUSA
  4. 4.Institute of Mathematics of ASCRPrague 1Czech Republic
  5. 5.Fakultät für MathematikTechnische Universität DortmundDortmundGermany

Personalised recommendations