Abstract
The differential-geometric structure of the set of positive densities on a given measure space has raised the interest of many mathematicians after the discovery by C.R. Rao of the geometric meaning of the Fisher information. Most of the research is focused on parametric statistical models. In series of papers by author and coworkers a particular version of the nonparametric case has been discussed. It consists of a minimalistic structure modeled according the theory of exponential families: given a reference density other densities are represented by the centered log likelihood which is an element of an Orlicz space. This mappings give a system of charts of a Banach manifold. It has been observed that, while the construction is natural, the practical applicability is limited by the technical difficulty to deal with such a class of Banach spaces. It has been suggested recently to replace the exponential function with other functions with similar behavior but polynomial growth at infinity in order to obtain more tractable Banach spaces, e.g. Hilbert spaces. We give first a review of our theory with special emphasis on the specific issues of the infinite dimensional setting. In a second part we discuss two specific topics, differential equations and the metric connection. The position of this line of research with respect to other approaches is briefly discussed.
Keywords
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Amari, S.I.: Differential geometry of curved exponential families—curvatures and information loss. Ann. Statist. 10(2), 357–385 (1982)
Amari, S.: Differential-geometrical methods in statistics. Lecture Notes in Statistics, vol. 28. Springer, New York (1985)
Amari, S.: Differential geometrical theory of statistics. In: Differential Geometry in Statistical Inference. Institute of Mathematical Statistics Lecture Notes—Monograph Series, vol. 10, pp. 19–94. Institute of Mathematical Statistics, Hayward (1987)
Amari, S., Nagaoka, H.: Methods of information geometry. American Mathematical Society, Providence (2000); translated from the 1993 Japanese original by Daishi Harada
Bourbaki, N.: Variétés differentielles et analytiques. Fascicule de résultats / Paragraphes 1 à 7. Number XXXIII in Éléments de mathématiques. Hermann, Paris (1971)
Lang, S.: Differential and Riemannian manifolds, 3rd edn. Graduate Texts in Mathematics, vol. 160. Springer, New York (1995)
Pistone, G., Sempi, C.: An infinite-dimensional geometric structure on the space of all the probability measures equivalent to a given one. Ann. Statist. 23(5), 1543–1561 (1995)
Pistone, G., Rogantin, M.: The exponential statistical manifold: mean parameters, orthogonality and space transformations. Bernoulli 5(4), 721–760 (1999)
Gibilisco, P., Pistone, G.: Connections on non-parametric statistical manifolds by Orlicz space geometry. IDAQP 1(2), 325–347 (1998)
Cena, A.: Geometric structures on the non-parametric statistical manifold. PhD thesis, Dottorato in Matematica, Università di Milano (2002)
Cena, A., Pistone, G.: Exponential statistical manifold. Ann. Inst. Statist. Math. 59(1), 27–56 (2007)
Malagò, L., Matteucci, M., Dal Seno, B.: An information geometry perspective on estimation of distribution algorithms: boundary analysis. In: GECCO 2008: Proceedings of the 2008 GECCO Conference Companion on Genetic and Evolutionary Computation, pp. 2081–2088. ACM, New York (2008)
Imparato, D.: Exponential models and Fisher information. Geometry and applications. PhD thesis, DIMAT Politecnico di Torino (2008)
Brigo, D., Pistone, G.: Projecting the Fokker-Planck equation onto a finite dimensional exponential family. arXiv:0901.1308 (2009)
Malagò, L., Pistone, G.: A note on the border of an exponential family. arXiv:1012.0637v1 (2010)
Pistone, G.: κ-exponential models from the geometrical viewpoint. The European Physical Journal B Condensed Matter Physics 71(1), 29–37 (2009)
Pistone, G.: Algebraic varieties vs. differentiable manifolds in statistical models. In: Gibilisco, P., Riccomagno, E., Rogantin, M., Wynn, H.P. (eds.) Algebraic and Geometric Methods in Statistics, pp. 339–363. Cambridge University Press (2009)
Imparato, D., Trivellato, B.: Geometry of extended exponential models. In: Algebraic and geometric methods in statistics, pp. 307–326. Cambridge Univ. Press, Cambridge (2010)
Malagò, L., Matteucci, M., Pistone, G.: Towards the geometry of estimation of distribution algorithms based on the exponential family. In: Proceedings of the 11th Workshop on Foundations of Genetic Algorithms, FOGA 2011, pp. 230–242. ACM, New York (2011)
Malagò, L., Matteucci, M., Pistone, G.: Stochastic natural gradient descent by estimation of empirical covariances. In: 2011 IEEE Congress on Evolutionary Computation (CEC), pp. 949–956 (2011)
Malagò, L.: On the geometry of optimization based on the exponential family relaxation. PhD thesis, Politecnico di Milano (2012)
Malagò, L., Matteucci, M., Pistone, G.: Natural gradient, fitness modelling and model selection: A unifying perspective. Paper #1747 IEEE Congress on Evolutionary Computation IEEE CEC 2013, Cancń México, June 20–23 (2013)
Gibilisco, P., Isola, T.: Connections on statistical manifolds of density operators by geometry of noncommutative \(L\sp p\)-spaces. Infin. Dimens. Anal. Quantum Probab. Relat. Top. 2(1), 169–178 (1999)
Jenčová, A.: A construction of a nonparametric quantum information manifold. J. Funct. Anal. 239(1), 1–20 (2006)
Gibilisco, P., Riccomagno, E., Rogantin, M.P., Wynn, H.P. (eds.): Algebraic and geometric methods in statistics. Cambridge University Press, Cambridge (2010)
Efron, B.: Defining the curvature of a statistical problem (with applications to second order efficiency). Ann. Statist. 3(6), 1189–1242 (1975); With a discussion by Rao, C. R., Pierce, D. A., Cox, D. R., Lindley, D. V., LeCam, L., Ghosh, J. K., Pfanzagl, J., Keiding, N., Dawid, A. P., Reeds, J., with a reply by the author
Barndorff-Nielsen, O.E.: Information and Exponential Families in Statistical Theory. John Wiley & Sons, New York (1978)
Brown, L.D.: Fundamentals of statistical exponential families with applications in statistical decision theory. IMS Lecture Notes. Monograph Series, vol. 9. Institute of Mathematical Statistics, Hayward (1986)
Letac, G.: Lectures on natural exponential families and their variance functions. Monografías de Matemática, Mathematical Monographs, vol. 50. Instituto de Matemática Pura e Aplicada (IMPA), Rio de Janeiro (1992)
Dawid, A.P.: Discussion of a paper by Bradley Efron. Ann. Statist. 3(6), 1231–1234 (1975)
Dawid, A.P.: Further comments on: “Some comments on a paper by Bradley Efron”. Ann. Statist. 3, 1189–1242 (1975); Ann. Statist. 5(6), 1249 (1977)
Gzyl, H., Recht, L.: A geometry on the space of probabilities. I. The finite dimensional case. Rev. Mat. Iberoam 22(2), 545–558 (2006)
Gzyl, H., Recht, L.: A geometry on the space of probabilities. II. Projective spaces and exponential families. Rev. Mat. Iberoam 22(3), 833–849 (2006)
Krasnosel’skii, M.A., Rutickii, Y.B.: Convex Functions and Orlicz Spaces. Noordhoff, Groningen (1961); Russian original: Fizmatgiz, Moskva (1958)
Musielak, J.: Orlicz spaces and modular spaces. Lecture Notes in Mathematics, vol. 1034. Springer, Berlin (1983)
Rao, M.M., Ren, Z.D.: Applications of Orlicz spaces. Monographs and Textbooks in Pure and Applied Mathematics, vol. 250. Marcel Dekker Inc., New York (2002)
Adams, R.A., Fournier, J.J.F.: Sobolev spaces, 2nd edn. Pure and Applied Mathematics, vol. 140. Elsevier/Academic Press, Amsterdam (2003)
Gallavotti, G.: Statistical mechanics: A short treatise. Texts and Monographs in Physics. Springer, Berlin (1999)
Boros, E., Hammer, P.L.: Pseudo-Boolean optimization. Discrete Appl. Math. 123(1-3), 155–225 (2002); Workshop on Discrete Optimization, DO 1999 (Piscataway, NJ)
Grasselli, M.R.: Dual connections in nonparametric classical information geometry. Technical Report math-ph/0104031 v1, arXiv (2001)
Appell, J., Zabrejko, P.P.: Nonlinear superposition operators. Cambridge Tracts in Mathematics, vol. 95. Cambridge University Press, Cambridge (1990)
Upmeier, H.: Symmetric Banach manifolds and Jordan \(C\sp \ast\)-algebras. North-Holland Mathematics Studies, vol. 104. North-Holland Publishing Co., Amsterdam (1985); Notas de Matemática (Mathematical Notes), 96
Ambrosetti, A., Prodi, G.: A primer of nonlinear analysis. Cambridge Studies in Advanced Mathematics, vol. 34. Cambridge University Press, Cambridge (1993)
Barndorff-Nielsen, O.E., Jupp, P.E.: Statistics, yokes and symplectic geometry. Ann. Fac. Sci. Toulouse Math. 6(3), 389–427 (1997)
Shima, H.: The geometry of Hessian structures. World Scientific Publishing Co. Pte. Ltd., Hackensack (2007)
Ambrosio, L., Gigli, N., Savaré, G.: Gradient flows in metric spaces and in the space of probability measures, 2nd edn. Lectures in Mathematics ETH Zürich. Birkhäuser, Basel (2008)
Csiszár, I., Matúš, F.: Information projections revisited. IEEE Trans. Inform. Theory 49(6), 1474–1490 (2003)
Santacroce, M., Siri, P., Trivellato, B.: A dynamic approach to exponential statistical manifolds (in progress, 2013)
Arnold, L., Auger, A., Hansen, N., Ollivier, Y.: Information-Geometric Optimization Algorithms: A Unifying Picture via Invariance Principles. arXiv:1106.3708 (2011)
Otto, F.: The geometry of dissipative evolution equations: the porous medium equation. Comm. Partial Differential Equations 26(1-2), 101–174 (2001)
Parry, M., Dawid, A.P., Lauritzen, S.: Proper local scoring rules. Ann. Statist. 40(1), 561–592 (2012)
Majewski, W.A., Labuschagne, L.E.: On applications of orlicz spaces to statistical physics. arXiv:1302.3460 (2013)
Villani, C.: A review of mathematical topics in collisional kinetic theory. In: Handbook of Mathematical Fluid Dynamics, vol. I, pp. 71–305. North-Holland, Amsterdam (2002)
Grasselli, M.R.: Dual connections in nonparametric classical information geometry. Ann. Inst. Statist. Math. 62(5), 873–896 (2010)
Malliavin, P.: Integration and probability. Graduate Texts in Mathematics, vol. 157. Springer, New York (1995); With the collaboration of H. Airault, L. Kay, G. Letac, Edited and translated from the French by Kay, With a foreword by Mark Pinsky
Brigo, D., Hanzon, B., Le Gland, F.: Approximate nonlinear filtering by projection on exponential manifolds of densities. Bernoulli 5(3), 495–534 (1999)
Naudts, J.: Generalised Thermostatistics. Springer (2011)
Tsallis, C.: Possible generalization of Boltzmann-Gibbs statistics. J. Statist. Phys. 52(1-2), 479–487 (1988)
Kaniadakis, G.: Statistical mechanics in the context of special relativity. Physical Review E 66, 056125, 1–17 (2002)
Kaniadakis, G.: Statistical mechanics in the context of special relativity. ii. Phys. Rev. E 72(3), 036108 (2005)
Newton, N.J.: An infinite-dimensional statistical manifold modelled on Hilbert space. J. Funct. Anal. 263(6), 1661–1681 (2012)
Vigelis, R.F., Cavalcante, C.C.: On the φ-family of probability distributions. Journal of Theoretical Probability (2011) (online first)
Burdet, G., Combe, P., Nencka, H.: On real Hilbertian info-manifolds. In: Disordered and Complex Systems (London, 2000). AIP Conf. Proc. Amer. Inst. Phys., Melville, NY, vol. 553, pp. 153–158 (2001)
Eguchi, S.: Tubular modelling approach to statistical method for observational studies. In: 2nd International Symposium on Information Geometry and its Applications, Tokyo, December 12-16 (2005)
Zhang, J., Hästö, P.: Statistical manifold as an affine space: a functional equation approach. Journal of Mathematical Psychology 50(1), 60–65 (2006)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Pistone, G. (2013). Nonparametric Information Geometry. In: Nielsen, F., Barbaresco, F. (eds) Geometric Science of Information. GSI 2013. Lecture Notes in Computer Science, vol 8085. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-40020-9_3
Download citation
DOI: https://doi.org/10.1007/978-3-642-40020-9_3
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-40019-3
Online ISBN: 978-3-642-40020-9
eBook Packages: Computer ScienceComputer Science (R0)