Skip to main content
SpringerLink
Log in
Book cover

International Conference on Geometric Science of Information

GSI 2023: Geometric Science of Information pp 268–277Cite as

Categorical Information Geometry

Part of the Lecture Notes in Computer Science book series (LNCS,volume 14071)

Abstract

Information geometry is the study of interactions between random variables by means of metric, divergences, and their geometry. Categorical probability has a similar aim, but uses algebraic structures, primarily monoidal categories, for that purpose. As recent work shows, we can unify the two approaches by means of enriched category theory into a single formalism, and recover important information-theoretic quantities and results, such as entropy and data processing inequalities.

Keywords

  • Category Theory
  • Markov Categories
  • Graphical Models
  • Divergences
  • Information Geometry

Supported by Sam Staton’s ERC grant “BLaSt – A Better Language for Statistics”.

This is a preview of subscription content, access via your institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   69.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   89.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    Despite the name, Markov categories are not only suited to model Markov processes, but arbitrary stochastic processes. Indeed, arbitrary joint distributions can be formed, and the Markov property states that the stochastic dependencies between the variables are faithfully represented by a particular graph. If the graph is (equivalent to) a single chain, we have a Markov process. In general, the graph is more complex. In this respect, Markov categories are similar to, but more general than, Markov random fields. See [6] for more details on this.

  2. 2.

    The geometry of the category of Markov kernels studied by Chentsov in [2, Sects. 4 and 6] is not metric geometry, it is a study of invariants in the sense of Klein’s Erlangen Program. More related to the present work are, rather, the invariant information characteristics of Sect. 8 of [2]. Much of classical information geometry, and hence indirectly this work, is built upon those notions.

References

  1. Chentsov, N.N.: The categories of mathematical statistics. Dokl. Akad. Nauk SSSR 164, 511–514 (1965)

    MathSciNet  Google Scholar 

  2. Chentsov, N.N.: Statistical decision rules and optimal inference. Nauka (1972)

    Google Scholar 

  3. Cho, K., Jacobs, B.: Disintegration and Bayesian inversion via string diagrams. Math. Struct. Comput. Sci. 29, 938–971 (2019). https://doi.org/10.1017/S0960129518000488

    CrossRef  MathSciNet  MATH  Google Scholar 

  4. Fong, B.: Causal theories: a categorical perspective on Bayesian networks. Master’s thesis, University of Oxford (2012). arXiv:1301.6201

  5. Fritz, T.: A synthetic approach to Markov kernels, conditional independence and theorems on sufficient statistics. Adv. Math. 370, 107239 (2020). arXiv:1908.07021

  6. Fritz, T., Klingler, A.: The d-separation criterion in categorical probability (2022). arXiv:2207.05740

  7. Fritz, T., Liang, W.: Free GS-monoidal category and free Markov categories (2022). arXiv:2204.02284

  8. Gadducci, F.: On the algebraic approach to concurrent term rewriting. Ph.D. thesis, University of Pisa (1996)

    Google Scholar 

  9. Giry, M.: A categorical approach to probability theory. In: Banaschewski, B. (ed.) Categorical Aspects of Topology and Analysis. LNM, vol. 915, pp. 68–85. Springer, Heidelberg (1982). https://doi.org/10.1007/BFb0092872

    CrossRef  Google Scholar 

  10. Golubtsov, P.V.: Axiomatic description of categories of information transformers. Problemy Peredachi Informatsii 35(3), 80–98 (1999)

    MathSciNet  MATH  Google Scholar 

  11. Gromov, M.: In search for a structure, Part 1: on entropy (2013). https://www.ihes.fr/~gromov/wp-content/uploads/2018/08/structre-serch-entropy-july5-2012.pdf

  12. Lawvere, F.W.: The category of probabilistic mappings (1962). Unpublished notes

    Google Scholar 

  13. Lawvere, W.: Metric spaces, generalized logic and closed categories. Rendiconti del seminario matematico e fisico di Milano 43 (1973). http://www.tac.mta.ca/tac/reprints/articles/1/tr1abs.html

  14. Leinster, T.: Entropy and Diversity. Cambridge University Press (2021)

    Google Scholar 

  15. Mac Lane, S.: Categories for the Working Mathematician. Graduate Texts in Mathematics, vol. 5, 2nd edn. Springer, New York (1998). https://doi.org/10.1007/978-1-4757-4721-8

    CrossRef  MATH  Google Scholar 

  16. Morozova, E., Chentsov, N.N.: Natural geometry on families of probability laws. Itogi Nauki i Tekhniki. Sovremennye Problemy Matematiki. Fundamental’nye Napravleniya 83, 133–265 (1991)

    Google Scholar 

  17. Perrone, P.: Lifting couplings in Wasserstein spaces (2021). arXiv:2110.06591

  18. Perrone, P.: Markov categories and entropy (2022). arXiv:2212.11719

Download references

Acknowledgements

The author would like to thank Tobias Fritz, Tomáš Gonda and Sam Staton for the helpful discussions and feedback, the anonymous reviewers for their constructive comments, and Swaraj Dash for the help with translating from Russian.

Author information

Authors and Affiliations

  1. University of Oxford, Oxford, UK

    Paolo Perrone

Authors
  1. Paolo Perrone
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Paolo Perrone .

Editor information

Editors and Affiliations

  1. Sony Computer Science Laboratories Inc., Tokyo, Japan

    Frank Nielsen

  2. THALES Land and Air Systems, Meudon, France

    Frédéric Barbaresco

Rights and permissions

Reprints and Permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Perrone, P. (2023). Categorical Information Geometry. In: Nielsen, F., Barbaresco, F. (eds) Geometric Science of Information. GSI 2023. Lecture Notes in Computer Science, vol 14071. Springer, Cham. https://doi.org/10.1007/978-3-031-38271-0_27

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-38271-0_27

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-38270-3

  • Online ISBN: 978-3-031-38271-0

  • eBook Packages: Computer ScienceComputer Science (R0)

search

Navigation