Abstract
Chentsov’s theorem characterizes the Fisher information metric on statistical models as the only Riemannian metric (up to rescaling) that is invariant under sufficient statistics. This implies that each statistical model is equipped with a natural geometry, so Chentsov’s theorem explains why many statistical properties can be described in geometric terms. However, despite being one of the foundational theorems of statistics, Chentsov’s theorem has only been proved previously in very restricted settings or under relatively strong invariance assumptions. We therefore prove a version of this theorem for the important case of exponential families. In particular, we characterise the Fisher information metric as the only Riemannian metric (up to rescaling) on an exponential family and its derived families that is invariant under independent and identically distributed extensions and canonical sufficient statistics. We then extend this result to curved exponential families. Our approach is based on the central limit theorem, so it gives a unified proof for discrete and continuous exponential families, and it is less technical than previous approaches.
Similar content being viewed by others
References
Amari, S., Nagaoka, H.: Methods of Information Geometry, Translations of mathematical monographs, vol. 191. American Mathematical Society, Providence (2000)
Ay, N., Jost, J., Lê, H.V., Schwachhöfer, L.: Information geometry and sufficient statistics. Probab. Theory Relat. Fields 162, 327–364 (2015)
Barndorff-Nielsen, O.: Information and exponential families. Wiley, New York (1978)
Barndorff-Nielsen, O., Cox, D.R.: Edgeworth and saddle-point approximations with statistical applications. J. R. Stat. Soc. Ser. B (Methodological) 41(3), 279–312 (1979)
Barndorff-Nielsen, O., Pedersen, K.: Sufficient data reduction and exponential families. Math. Scand. 22, 197–202 (1968)
Barron, A.R., Rissanen, J., Yu, B.: The minimum description length principle in coding and modeling. IEEE Trans. Inf. Theory 44(6), 2743–2760 (1998)
Bauer, M., Bruveris, M., Michor, P.W.: Uniqueness of the Fisher-Rao metric on the space of smooth densities. Bull. Lond. Math. Soc. 48, 499–506 (2016)
Bogachev, V.I.: Measure theory, vol. I. Springer, Berlin (2007)
Campbell, L.L.: An extended Cencov characterization of the information metric. Proc. Am. Math. Soc. 98, 135–141 (1986)
Chentsov, N.N.: Algebraic foundation of mathematical statistics. Math. Operationsforsch. Stat. 9, 267–276 (1978)
Dowty, J.G.: Chentsov’s theorem for exponential families. arXiv:1701.08895 (2017)
Friedlander, F.G., Joshi, M.: Introduction to the theory of distributions, 2nd edn. Cambridge University Press, Cambridge (1998)
Kass, R.E., Vos, P.W.: Geometrical foundations of asymptotic inference. Wiley, New York (1997)
Lang, S.: Fundamentals of differential geometry, graduate texts in mathematics, vol. 191. Springer, New York (1999)
Lê, H.V.: The uniqueness of the fisher metric as information metric. arXiv:1306.1465 (2013)
Meerschaert, M.M., Scheffler, H.-P.: Limit distributions for sums of independent random vectors: heavy tails in theory and practice, Wiley series in probability and statistics. Wiley, New York (2001)
Pistone, G., Sempri, C.: An infinite-dimensional geometric structure on the space of all the probability measures equivalent to a given one. Ann. Stat. 23(5), 1543–1561 (1995)
Rissanen, J.: Fisher information and stochastic complexity. IEEE Trans. Inf. Theory 42(1), 40–47 (1996)
Schwachhöfer, L., Ay, N., Jost, J., Lê, H.V.: Congruent families and invariant tensors. arXiv: 1705.11014 (2017)
Čencov, N.N.: Statistical Decision Rules and Optimal Inference, Translations of Mathematical Monographs, vol. 53. American Mathematical Society, Providence (1982)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Dowty, J.G. Chentsov’s theorem for exponential families. Info. Geo. 1, 117–135 (2018). https://doi.org/10.1007/s41884-018-0006-4
Received:
Revised:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s41884-018-0006-4