Information geometry is the study of interactions between random variables by means of metric, divergences, and their geometry. Categorical probability has a similar aim, but uses algebraic structures, primarily monoidal categories, for that purpose. As recent work shows, we can unify the two approaches by means of enriched category theory into a single formalism, and recover important information-theoretic quantities and results, such as entropy and data processing inequalities.
- Category Theory
- Markov Categories
- Graphical Models
- Information Geometry
Supported by Sam Staton’s ERC grant “BLaSt – A Better Language for Statistics”.
This is a preview of subscription content, access via your institution.
Tax calculation will be finalised at checkout
Purchases are for personal use onlyLearn about institutional subscriptions
Despite the name, Markov categories are not only suited to model Markov processes, but arbitrary stochastic processes. Indeed, arbitrary joint distributions can be formed, and the Markov property states that the stochastic dependencies between the variables are faithfully represented by a particular graph. If the graph is (equivalent to) a single chain, we have a Markov process. In general, the graph is more complex. In this respect, Markov categories are similar to, but more general than, Markov random fields. See  for more details on this.
The geometry of the category of Markov kernels studied by Chentsov in [2, Sects. 4 and 6] is not metric geometry, it is a study of invariants in the sense of Klein’s Erlangen Program. More related to the present work are, rather, the invariant information characteristics of Sect. 8 of . Much of classical information geometry, and hence indirectly this work, is built upon those notions.
Chentsov, N.N.: The categories of mathematical statistics. Dokl. Akad. Nauk SSSR 164, 511–514 (1965)
Chentsov, N.N.: Statistical decision rules and optimal inference. Nauka (1972)
Cho, K., Jacobs, B.: Disintegration and Bayesian inversion via string diagrams. Math. Struct. Comput. Sci. 29, 938–971 (2019). https://doi.org/10.1017/S0960129518000488
Fong, B.: Causal theories: a categorical perspective on Bayesian networks. Master’s thesis, University of Oxford (2012). arXiv:1301.6201
Fritz, T.: A synthetic approach to Markov kernels, conditional independence and theorems on sufficient statistics. Adv. Math. 370, 107239 (2020). arXiv:1908.07021
Fritz, T., Klingler, A.: The d-separation criterion in categorical probability (2022). arXiv:2207.05740
Fritz, T., Liang, W.: Free GS-monoidal category and free Markov categories (2022). arXiv:2204.02284
Gadducci, F.: On the algebraic approach to concurrent term rewriting. Ph.D. thesis, University of Pisa (1996)
Giry, M.: A categorical approach to probability theory. In: Banaschewski, B. (ed.) Categorical Aspects of Topology and Analysis. LNM, vol. 915, pp. 68–85. Springer, Heidelberg (1982). https://doi.org/10.1007/BFb0092872
Golubtsov, P.V.: Axiomatic description of categories of information transformers. Problemy Peredachi Informatsii 35(3), 80–98 (1999)
Gromov, M.: In search for a structure, Part 1: on entropy (2013). https://www.ihes.fr/~gromov/wp-content/uploads/2018/08/structre-serch-entropy-july5-2012.pdf
Lawvere, F.W.: The category of probabilistic mappings (1962). Unpublished notes
Lawvere, W.: Metric spaces, generalized logic and closed categories. Rendiconti del seminario matematico e fisico di Milano 43 (1973). http://www.tac.mta.ca/tac/reprints/articles/1/tr1abs.html
Leinster, T.: Entropy and Diversity. Cambridge University Press (2021)
Mac Lane, S.: Categories for the Working Mathematician. Graduate Texts in Mathematics, vol. 5, 2nd edn. Springer, New York (1998). https://doi.org/10.1007/978-1-4757-4721-8
Morozova, E., Chentsov, N.N.: Natural geometry on families of probability laws. Itogi Nauki i Tekhniki. Sovremennye Problemy Matematiki. Fundamental’nye Napravleniya 83, 133–265 (1991)
Perrone, P.: Lifting couplings in Wasserstein spaces (2021). arXiv:2110.06591
Perrone, P.: Markov categories and entropy (2022). arXiv:2212.11719
The author would like to thank Tobias Fritz, Tomáš Gonda and Sam Staton for the helpful discussions and feedback, the anonymous reviewers for their constructive comments, and Swaraj Dash for the help with translating from Russian.
Editors and Affiliations
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Perrone, P. (2023). Categorical Information Geometry. In: Nielsen, F., Barbaresco, F. (eds) Geometric Science of Information. GSI 2023. Lecture Notes in Computer Science, vol 14071. Springer, Cham. https://doi.org/10.1007/978-3-031-38271-0_27
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-38270-3
Online ISBN: 978-3-031-38271-0