Measure Transformer Semantics for Bayesian Machine Learning

  • Johannes Borgström
  • Andrew D. Gordon
  • Michael Greenberg
  • James Margetson
  • Jurgen Van Gael
Conference paper

DOI: 10.1007/978-3-642-19718-5_5

Part of the Lecture Notes in Computer Science book series (LNCS, volume 6602)
Cite this paper as:
Borgström J., Gordon A.D., Greenberg M., Margetson J., Van Gael J. (2011) Measure Transformer Semantics for Bayesian Machine Learning. In: Barthe G. (eds) Programming Languages and Systems. ESOP 2011. Lecture Notes in Computer Science, vol 6602. Springer, Berlin, Heidelberg

Abstract

The Bayesian approach to machine learning amounts to inferring posterior distributions of random variables from a probabilistic model of how the variables are related (that is, a prior distribution) and a set of observations of variables. There is a trend in machine learning towards expressing Bayesian models as probabilistic programs. As a foundation for this kind of programming, we propose a core functional calculus with primitives for sampling prior distributions and observing variables. We define combinators for measure transformers, based on theorems in measure theory, and use these to give a rigorous semantics to our core calculus. The original features of our semantics include its support for discrete, continuous, and hybrid measures, and, in particular, for observations of zero-probability events. We compile our core language to a small imperative language that has a straightforward semantics via factor graphs, data structures that enable many efficient inference algorithms. We use an existing inference engine for efficient approximate inference of posterior marginal distributions, treating thousands of observations per second for large instances of realistic models.

Download to read the full conference paper text

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Johannes Borgström
    • 1
  • Andrew D. Gordon
    • 1
  • Michael Greenberg
    • 2
  • James Margetson
    • 1
  • Jurgen Van Gael
    • 1
  1. 1.Microsoft ResearchUSA
  2. 2.University of PennsylvaniaUSA

Personalised recommendations