Advertisement

Statistics and Computing

, Volume 26, Issue 1–2, pp 325–332 | Cite as

Computing with Fisher geodesics and extended exponential families

  • F. Critchley
  • P. MarriottEmail author
Article

Abstract

Recent progress using geometry in the design of efficient Markov chain Monte Carlo (MCMC) algorithms have shown the effectiveness of the Fisher Riemannian structure. Furthermore, the theory of the underlying geometry of spaces of statistical models has made an important breakthrough by extending the classical theory on exponential families to their closures, the so-called extended exponential families. This paper looks at the underlying geometry of the Fisher information, in particular its limiting behaviour near boundaries, which illuminates the excellent behaviour of the corresponding geometric MCMC algorithms. Further, the paper shows how Fisher geodesics in extended exponential families smoothly attach the boundaries of extended exponential families to their relative interior. We conjecture that this behaviour could be exploited for trans-dimensional MCMC algorithms.

Keywords

Extended exponential family Fisher geodesic Fisher metric Information geometry Markov chain Monte Carlo 

Notes

Acknowledgments

Thanks to EPSRC for grants EP/E017878/1 and EP/L010429/1

References

  1. Altham, P.: Two generalizations of the binomial distribution. Appl. Stat. 27, 162–167 (1978)zbMATHMathSciNetCrossRefGoogle Scholar
  2. Amari, S.I.: Differential-Geometrical Methods in Statistics. Springer, Berlin (1990)zbMATHGoogle Scholar
  3. Anaya-Izquierdo, K., Critchley, F., Marriott, P.: When are first order asymptotics adequate? a diagnostic. STAT 3, 17–22 (2014)CrossRefGoogle Scholar
  4. Anaya-Izquierdo, K., Critchley, F., Marriott, P., Vos, P.: Computational information geometry: foundations. In: Proceedings of Geometric Science of Information 2013. Lecture Notes in Computer Science, pp. 311–318. Springer, Berlin (2013).Google Scholar
  5. Barndorff-Nielsen, O.: Information and Exponential Families in Statistical Theory. Wiley, New York (1978)zbMATHGoogle Scholar
  6. Brown, L.: Fundamentals of Statistical Exponential Families: with Applications in Statistical Decision Theory. Institute of Mathematical Statistics, Hayward (1986)zbMATHGoogle Scholar
  7. Critchley, F., Marriott, P.: Computational information geometry in statistics: theory and practice. Entropy 16(5), 2454–2471 (2014)MathSciNetCrossRefGoogle Scholar
  8. Csiszar, I., Matus, F.: Closures of exponential families. Ann. Probab. 33(2), 582–600 (2005)zbMATHMathSciNetCrossRefGoogle Scholar
  9. Fienberg, S., Rinaldo, A.: Maximum likelihood estimation in log-linear models: theory and algorithms. Ann. Stat. 40, 996–1023 (2012)zbMATHMathSciNetCrossRefGoogle Scholar
  10. Geyer, C.J.: Likelihood inference in exponential families and directions of recession. Electron. J. Stat. 3, 259–289 (2009)zbMATHMathSciNetCrossRefGoogle Scholar
  11. Girolami, M., Calderhead, B.: Riemann manifold Langevin and Hamiltonian Monte Carlo methods. J. R. Stat. Soc. B 73(2), 123–214 (2011)MathSciNetCrossRefGoogle Scholar
  12. Kass, R., Vos, P.: Geometrical Foundations of Asymptotic Inference. Wiley, New York (1997)zbMATHCrossRefGoogle Scholar
  13. Lauritzen, S.: Graphical Models. Oxford University Press, Oxford (1996)Google Scholar
  14. Livingstone, S., Girolami, M.: Information-geometric Markov chain Monte Carlo methods using diffusions. Entropy 16, 3074–3102 (2014)MathSciNetCrossRefGoogle Scholar
  15. R Development Core Team: R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria (2006). http://www.R-project.org. ISBN 3-900051-07-0. Accessed Feb 2014
  16. Rinaldo, A., Feinberg, S., Zhou, Y.: On the geometry of discrete exponential families with applications to exponential random graph models. Electron. J. Stat. 3, 446–484 (2009)zbMATHMathSciNetCrossRefGoogle Scholar
  17. Tuy, H.: Convex Analysis and Global Optimization. Klumer academic publishers, London (1998)zbMATHCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2014

Authors and Affiliations

  1. 1.Department of Mathematics and StatisticsThe Open UniversityMilton KeynesUK
  2. 2.Department of Statistics and Actuarial ScienceUniversity of WaterlooWaterlooCanada

Personalised recommendations