Abstract
We introduce Bayesian strategies, a new interpretation of probabilistic programs in game semantics. This interpretation can be seen as a refinement of Bayesian networks.
Bayesian strategies are based on a new form of event structure, with two causal dependency relations respectively modelling control flow and data flow. This gives a graphical representation for probabilistic programs which resembles the concrete representations used in modern implementations of probabilistic programming.
From a theoretical viewpoint, Bayesian strategies provide a rich setting for denotational semantics. To demonstrate this we give a model for a general higher-order programming language with recursion, conditional statements, and primitives for sampling from continuous distributions and trace re-weighting. This is significant because Bayesian networks do not easily support higher-order functions or conditionals.
Chapter PDF
Similar content being viewed by others
References
Abbes, S., Benveniste, A.: True-concurrency probabilistic models: Branching cells and distributed probabilities for event structures. Information and Computation 204(2), 231–274 (2006)
Abbes, S., Benveniste, A.: True-concurrency probabilistic models: Markov nets and a law of large numbers. Theoretical computer science 390(2-3), 129–170 (2008)
Abramsky, S., Jagadeesan, R., Malacaria, P.: Full abstraction for PCF. Information and Computation 163(2), 409–470 (2000)
Abramsky, S., McCusker, G.: Call-by-value games. In: International Workshop on Computer Science Logic. pp. 1–17. Springer (1997)
Alcolei, A.: Jeux concurrents enrichis: témoins pour les preuves et les ressources. Ph.D. thesis, ENS Lyon (2019)
Bénabou, J.: Introduction to bicategories. In: Reports of the midwest category seminar. pp. 1–77. Springer (1967)
Benveniste, A., Fabre, E., Haar, S.: Markov nets: probabilistic models for distributed and concurrent systems. IEEE Transactions on Automatic Control 48(11), 1936–1950 (2003)
Billingsley, P.: Probability and measure. John Wiley & Sons (2008)
Bingham, E., Chen, J.P., Jankowiak, M., Obermeyer, F., Pradhan, N., Karaletsos, T., Singh, R., Szerlip, P., Horsfall, P., Goodman, N.D.: Pyro: Deep universal probabilistic programming. The Journal of Machine Learning Research 20(1), 973–978 (2019)
Blass, A.: A game semantics for linear logic. Annals of Pure and Applied logic 56(1-3), 183–220 (1992)
Borgström, J., Dal Lago, U., Gordon, A.D., Szymczak, M.: A lambda-calculus foundation for universal probabilistic programming. In: ACM SIGPLAN Notices. vol. 51, pp. 33–46. ACM (2016)
Bruni, R., Melgratti, H., Montanari, U.: Concurrency and probability: Removing confusion, compositionally. In: Proceedings of the 33rd Annual ACM/IEEE Symposium on Logic in Computer Science. pp. 195–204 (2018)
Bruni, R., Melgratti, H., Montanari, U.: Bayesian network semantics for petri nets. Theoretical Computer Science 807, 95–113 (2020)
Castellan, S.: The causality project, http://iso.mor.phis.me/software/causality/
Castellan, S., Clairambault, P., Paquet, H., Winskel, G.: The concurrent game semantics of probabilistic PCF. In: Logic in Computer Science (LICS), 2018 33rd Annual ACM/IEEE Symposium on, ACM/IEEE (2018)
Castellan, S., Clairambault, P., Winskel, G.: The parallel intensionally fully abstract games model of PCF. In: 2015 30th Annual ACM/IEEE Symposium on Logic in Computer Science. pp. 232–243. IEEE (2015)
Castellan, S., Clairambault, P., Winskel, G.: Thin games with symmetry and concurrent Hyland-Ong games. Logical Methods in Computer Science (2019)
Castellan, S., Paquet, H.: Probabilistic programming inference via intensional semantics. In: European Symposium on Programming. pp. 322–349. Springer (2019)
Castellan, S., Yoshida, N.: Two sides of the same coin: session types and game semantics: a synchronous side and an asynchronous side. Proceedings of the ACM on Programming Languages 3(POPL), 1–29 (2019)
Clairambault, P., De Visme, M., Winskel, G.: Game semantics for quantum programming. Proceedings of the ACM on Programming Languages 3(POPL), 1–29 (2019)
Clairambault, P., de Visme, M.: Full abstraction for the quantum lambda-calculus. Proceedings of the ACM on Programming Languages 4(POPL), 1–28 (2019)
Claret, G., Rajamani, S.K., Nori, A.V., Gordon, A.D., Borgström, J.: Bayesian inference using data flow analysis. In: Proceedings of the 2013 9th Joint Meeting on Foundations of Software Engineering. pp. 92–102 (2013)
Cusumano-Towner, M.F., Saad, F.A., Lew, A.K., Mansinghka, V.K.: Gen: a general-purpose probabilistic programming system with programmable inference. In: Proceedings of the 40th ACM SIGPLAN Conference on Programming Language Design and Implementation. pp. 221–236 (2019)
Dahlqvist, F., Kozen, D.: Semantics of higher-order probabilistic programs with conditioning. Proceedings of the ACM on Programming Languages 4(POPL), 1–29 (2019)
Faggian, C., Piccolo, M.: Partial orders, event structures and linear strategies. In: International Conference on Typed Lambda Calculi and Applications. pp. 95–111. Springer (2009)
Fiore, M., Honda, K.: Recursive types in games: Axiomatics and process representation. In: Proceedings. Thirteenth Annual IEEE Symposium on Logic in Computer Science (Cat. No. 98CB36226). pp. 345–356. IEEE (1998)
Fong, B.: Causal theories: A categorical perspective on Bayesian networks. arXiv preprint arXiv:1301.6201 (2013)
Gabler, P., Trapp, M., Ge, H., Pernkopf, F.: Graph tracking in dynamic probabilistic programs via source transformations. In: Symposium on Advances in Approximate Bayesian Inference (AABI) (2019)
Girard, J.Y.: Linear logic. Theoretical computer science 50(1), 1–101 (1987)
Goodman, N.D., Mansinghka, V.K., Roy, D., Bonawitz, K., Tenenbaum, J.B.: Church: a language for generative models. In: Proceedings of the Twenty-Fourth Conference on Uncertainty in Artificial Intelligence. pp. 220–229 (2008)
Honda, K., Yoshida, N.: Game theoretic analysis of call-by-value computation. In: International Colloquium on Automata, Languages, and Programming. pp. 225–236. Springer (1997)
Hur, C.K., Nori, A.V., Rajamani, S.K., Samuel, S.: A provably correct sampler for probabilistic programs. In: LIPIcs-Leibniz International Proceedings in Informatics. vol. 45. Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik (2015)
Hyland, J.M.E., Ong, C.H.: On full abstraction for PCF: I, II, and III. Information and computation 163(2) (2000)
Jacobs, B.: A channel-based exact inference algorithm for bayesian networks. arXiv preprint arXiv:1804.08032 (2018)
Jeffrey, A.: Premonoidal categories and a graphical view of programs. Preprint, Dec (1997)
Joyal, A., Street, R.: The geometry of tensor calculus, I. Advances in mathematics 88(1), 55–112 (1991)
Koller, D., Friedman, N.: Probabilistic graphical models: principles and techniques. MIT press (2009)
Lago, U., Hoshino, N.: The geometry of Bayesian programming. In: 2019 34th Annual ACM/IEEE Symposium on Logic in Computer Science (LICS). pp. 1–13 (2019)
Lew, A.K., Cusumano-Towner, M.F., Sherman, B., Carbin, M., Mansinghka, V.K.: Trace types and denotational semantics for sound programmable inference in probabilistic languages. Proceedings of the ACM on Programming Languages 4(POPL), 1–32 (2019)
Mak, C., Ong, C.H.L., Paquet, H., Wagner, D.: Densities of almost-surely terminating probabilistic programs are differentiable almost everywhere. In: European Symposium on Programming. Springer (2021)
Mansinghka, V., Selsam, D., Perov, Y.: Venture: a higher-order probabilistic programming platform with programmable inference. arXiv preprint arXiv:1404.0099 (2014)
van de Meent, J.W., Paige, B., Yang, H., Wood, F.: An introduction to probabilistic programming. arXiv preprint arXiv:1809.10756 (2018)
Mellies, P.A.: Asynchronous games 4: A fully complete model of propositional linear logic. In: 20th Annual IEEE Symposium on Logic in Computer Science (LICS’05). pp. 386–395. IEEE (2005)
Melliès, P.A., Mimram, S.: Asynchronous games: Innocence without alternation. In: International Conference on Concurrency Theory. pp. 395–411. Springer (2007)
Minka, T., Winn, J.: Gates. In: Advances in Neural Information Processing Systems. pp. 1073–1080 (2009)
Nielsen, M., Plotkin, G., Winskel, G.: Petri nets, event structures and domains, part i. Theoretical Computer Science 13(1), 85–108 (1981)
Ong, L., Vákár, M.: S-finite kernels and game semantics for probabilistic programming. In: POPL’18 Workshop on Probabilistic Programming Semantics (PPS) (2018)
Paquet, H.: Probabilistic concurrent game semantics. Ph.D. thesis, University of Cambridge (2020)
Paquet, H., Winskel, G.: Continuous probability distributions in concurrent games. Electronic Notes in Theoretical Computer Science 341, 321–344 (2018)
Power, J., Robinson, E.: Premonoidal categories and notions of computation. Mathematical structures in computer science 7(5), 453–468 (1997)
Rideau, S., Winskel, G.: Concurrent strategies. In: 2011 IEEE 26th Annual Symposium on Logic in Computer Science. pp. 409–418. IEEE (2011)
Schulman, J., Heess, N., Weber, T., Abbeel, P.: Gradient estimation using stochastic computation graphs. In: Advances in Neural Information Processing Systems. pp. 3528–3536 (2015)
Ścibior, A., Kammar, O., Vákár, M., Staton, S., Yang, H., Cai, Y., Ostermann, K., Moss, S.K., Heunen, C., Ghahramani, Z.: Denotational validation of higher-order bayesian inference. Proceedings of the ACM on Programming Languages 2(POPL),  60 (2017)
Staton, S.: Commutative semantics for probabilistic programming. In: European Symposium on Programming. pp. 855–879. Springer (2017)
Tran, D., Kucukelbir, A., Dieng, A.B., Rudolph, M., Liang, D., Blei, D.M.: Edward: A library for probabilistic modeling, inference, and criticism. arXiv preprint arXiv:1610.09787 (2016)
Tsukada, T., Ong, C.L.: Nondeterminism in game semantics via sheaves. In: 2015 30th Annual ACM/IEEE Symposium on Logic in Computer Science. pp. 220–231. IEEE (2015)
Vákár, M., Kammar, O., Staton, S.: A domain theory for statistical probabilistic programming. Proceedings of the ACM on Programming Languages 3(POPL), 1–29 (2019)
Varacca, D., Völzer, H., Winskel, G.: Probabilistic event structures and domains. In: International Conference on Concurrency Theory. pp. 481–496. Springer (2004)
Varacca, D., Yoshida, N.: Probabilistic \(\pi \)-calculus and event structures. Electronic Notes in Theoretical Computer Science 190(3), 147–166 (2007)
Winskel, G.: Event structures. In: Advances in Petri Nets. pp. 325–392 (1986)
Winskel, G.: Event structures with symmetry. Electronic Notes in Theoretical Computer Science 172, 611–652 (2007)
Winskel, G.: Distributed probabilistic and quantum strategies. Electr. Notes Theor. Comput. Sci. 298, 403–425 (2013)
Winskel, G., Rideau, S., Clairambault, P., Castellan, S.: Games and strategies as event structures. Logical Methods in Computer Science 13 (2017)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Open Access This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.
The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.
Copyright information
© 2021 The Author(s)
About this paper
Cite this paper
Paquet, H. (2021). Bayesian strategies: probabilistic programs as generalised graphical models. In: Yoshida, N. (eds) Programming Languages and Systems. ESOP 2021. Lecture Notes in Computer Science(), vol 12648. Springer, Cham. https://doi.org/10.1007/978-3-030-72019-3_19
Download citation
DOI: https://doi.org/10.1007/978-3-030-72019-3_19
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-72018-6
Online ISBN: 978-3-030-72019-3
eBook Packages: Computer ScienceComputer Science (R0)