The uniqueness of the Fisher metric as information metric

Abstract

We define a mixed topology on the fiber space $\cup _\mu \oplus ^n L^n(\mu )$ over the space ${\mathcal M}({\Omega })$ of all finite non-negative measures $\mu$ on a separable metric space ${\Omega }$ provided with Borel $\sigma$-algebra. We define a notion of strong continuity of a covariant n-tensor field on ${\mathcal M}({\Omega })$. Under the assumption of strong continuity of an information metric, we prove the uniqueness of the Fisher metric as information metric on statistical models associated with ${\Omega }$. Our proof realizes a suggestion due to Amari and Nagaoka to derive the uniqueness of the Fisher metric from the special case proved by Chentsov by using a special kind of limiting procedure. The obtained result extends the monotonicity characterization of the Fisher metric on statistical models associated with finite sample spaces and complement the uniqueness theorem by Ay–Jost–Lê–Schwachhöfer that characterizes the Fisher metric by its invariance under sufficient statistics.

Appendix: The Chentsov uniqueness theorem

In this appendix we recall a reformulation of the Chentsov theorem (Chentsov 1978, Theorem 11.1, p. 159) on the uniqueness of the Fisher metric in the language of information geometry by Amari and Nagaoka (Proposition 24), which is simpler than the original formulation by Chentsov in the category language. In Proposition 25 we formulate a result in Ay et al. (2015) that characterizes the Fisher metric on finite sample spaces via the monotonicity. Then we discuss in Remark 26 some problems in generalizing the Chentsov theorem to a larger class of measure spaces that contains also non-discrete measure spaces.

Let us denote by ${\mathcal P}_+ ({\Omega }_n)$ the subset of ${\mathcal P}({\Omega }_n)$ that consists of positive measures.

Proposition 24

(Amari and Nagaoka 2000, Theorem 2.6, p. 38) Assume that $\{(g_n)\}_{n=1}^\infty$ is a sequence of Riemannian metrics on ${\mathcal P}_+ ({\Omega }_n)$ for each n that are invariant with respect to sufficient statistics;  i.e., for all $n, m, S \subset {\mathcal P}_+({\Omega }_n),$ and $F : {\Omega }_n \rightarrow {\Omega }_m$ such that F is a sufficient statistic for S,  the induced metrics on S and $F_*(S)$ are assumed to be invariant. Then there exists a positive real number c such that,  for all n,  $g_n$ coincides with the Fisher metric on ${\mathcal P}_ +({\Omega }_n)$ scaled by a factor of c.

Amari and Nagaoka did not supply their proof of Proposition 24. We recommend the reader to Campbell (1986) for a slight generalization of the Chentsov theorem, whose proof is close to the original Chentsov’s proof. For the reader convenience we recall the following monotonicity characterization of the Fisher metric on finite sample spaces.

Proposition 25

(Ay et al. 2015, Corollary 4.11) Let F be a continuous local statistical quadratic 2-form defined on statistical models associated with finite sample spaces $\{{\Omega }_n\}$ such that F is monotone under statistics. Then F coincides with the Fisher metric up to a multiplicative constant.

Remark 26

• 1. Chentsov defined the Fisher metric only on the positive sector ${\mathcal P}_+({\Omega }_n)$ of the space of all probability measures, because the expression for the Fisher metric in (2) is well defined only on ${\mathcal P}_+({\Omega }_n)$. In this paper we follow the approach in Ay et al. (2015) by requiring that an information metric F is obtained by (1) from the associated 2-form $\tilde{F}$, which is not only defined on ${\mathcal P}_+({\Omega }_n)$ but also defined on ${\mathcal M}({\Omega }_n)$ (in general case, on ${\mathcal M}({\Omega })$) and hence on ${\mathcal P}({\Omega }_n)$ (resp. on ${\mathcal P}({\Omega })$). This small difference is important, since for a non-discrete space ${\Omega }$ we do not know how to define a notion of a positive measure without using a reference measure $\mu _0$. Since the Fisher metric $g^F$ satisfies the mentioned requirement, see Example 19, Proposition 25 is equivalent to the Chentsov uniqueness theorem. Clearly, Theorem 4 generalizes Proposition 25.

• 2. As we mentioned above, the original Chentsov theorem can be equivalently reformulated in terms of the associated form $\tilde{F}$. Note that the space ${\mathcal P}({\Omega }_n)$ (resp. ${\mathcal M}({\Omega }_n)$) is not a manifold, or a manifold with boundary, but a stratified space which admits different embeddings into Euclidean spaces. In Ay et al. (2016) and in the present paper we do not consider smooth tensor fields on ${\mathcal P}({\Omega }_n)$ (resp. on ${\mathcal M}({\Omega }_n)$) but (strongly or point-wise) continuous tensor fields on ${\mathcal M}({\Omega })$ which do not require the notion of a smooth structure on ${\mathcal M}({\Omega })$.

• 3. In Morozova et al. (1991, §5) Morozova–Chentsov also suggested a method to extend the Chentsov uniqueness theorem to the case of non-discrete measure spaces ${\Omega }$. Their idea is similar to the Amari–Nagaoka idea, namely they wanted to consider a Riemannian metric on infinite measure spaces as limit of Riemannian metrics on finite measure spaces. They did not discuss a condition under which such a limit exists. In fact, they did not give a definition of limit of such metrics. If the limit exists they called it finitely generated. They stated that the Fisher metric is the unique finitely generated metric that is invariant under sufficient statistics (resp. that is monotone). One may speculate that since such a Riemannian metric depends on base measures $\mu$ and tangent vectors at $\mu$ Morozova–Chentsov’s approach requires a definition of topology on the space ${\mathcal L}_2 ^2 ({\Omega })$.