Deriving Probability Density Functions from Probabilistic Functional Programs

  • Sooraj Bhat
  • Johannes Borgström
  • Andrew D. Gordon
  • Claudio Russo
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7795)


The probability density function of a probability distribution is a fundamental concept in probability theory and a key ingredient in various widely used machine learning methods. However, the necessary framework for compiling probabilistic functional programs to density functions has only recently been developed. In this work, we present a density compiler for a probabilistic language with discrete and continuous distributions, and discrete observations, and provide a proof of its soundness. The compiler greatly reduces the development effort of domain experts, which we demonstrate by solving inference problems from various scientific applications, such as modelling the global carbon cycle, using a standard Markov chain Monte Carlo framework.


  1. Bhat, S., Agarwal, A., Vuduc, R.W., Gray, A.G.: A type theory for probability density functions. In: Field, J., Hicks, M. (eds.) POPL, pp. 545–556. ACM (2012)Google Scholar
  2. Borgström, J., Gordon, A.D., Greenberg, M., Margetson, J., Van Gael, J.: Measure Transformer Semantics for Bayesian Machine Learning. In: Barthe, G. (ed.) ESOP 2011. LNCS, vol. 6602, pp. 77–96. Springer, Heidelberg (2011), CrossRefGoogle Scholar
  3. Gilks, W.R., Thomas, A., Spiegelhalter, D.J.: A language and program for complex Bayesian modelling. The Statistician 43, 169–178 (1994)CrossRefGoogle Scholar
  4. Giry, M.: A categorical approach to probability theory. In: Banaschewski, B. (ed.) Categorical Aspects of Topology and Analysis. Lecture Notes in Mathematics, vol. 915, pp. 68–85. Springer, Heidelberg (1982)CrossRefGoogle Scholar
  5. Gordon, A.D., Aizatulin, M., Borgström, J., Claret, G., Graepel, T., Nori, A., Rajamani, S., Russo, C.: A model-learner pattern for Bayesian reasoning. In: POPL (2013)Google Scholar
  6. Griewank, A., Walther, A.: Evaluating Derivatives: Principles and Techniques of Algorithmic Differentiation, 2nd edn. SIAM (2008)Google Scholar
  7. Kiselyov, O., Shan, C.-C.: Embedded Probabilistic Programming. In: Taha, W.M. (ed.) DSL 2009. LNCS, vol. 5658, pp. 360–384. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  8. McInerny, G., Purves, D.: Fine-scale environmental variation in species distribution modelling: regression dilution, latent variables and neighbourly advice. Methods in Ecology and Evolution 2(3), 248–257 (2011)CrossRefGoogle Scholar
  9. Neal, R.M.: Probabilistic inference using Markov chain Monte Carlo methods. Technical Report CRG-TR-93-1, Dept. of Computer Science, University of Toronto (September 1993)Google Scholar
  10. Panangaden, P.: The category of Markov kernels. Electronic Notes in Theoretical Computer Science 22, 171–187 (1999)MathSciNetCrossRefGoogle Scholar
  11. Ramsey, N., Pfeffer, A.: Stochastic lambda calculus and monads of probability distributions. In: POPL, pp. 154–165 (2002)Google Scholar
  12. Schumann, J., Pressburger, T., Denney, E., Buntine, W., Fischer, B.: AutoBayes program synthesis system users manual. Technical Report NASA/TM–2008–215366, NASA Ames Research Center (2008)Google Scholar
  13. Smith, M.J., Vanderwel, M.C., Lyutsarev, V., Emmott, S., Purves, D.W.: The climate dependence of the terrestrial carbon cycle; including parameter and structural uncertainties. Biogeosciences Discussions 9, 13439–13496 (2012)CrossRefGoogle Scholar
  14. Syme, D., Granicz, A., Cisternino, A.: Expert F#. Apress (2007)Google Scholar
  15. Wingate, D., Stuhlmueller, A., Goodman, N.: Lightweight implementations of probabilistic programming languages via transformational compilation. In: Proceedings of the 14th Intl. Conf. on Artificial Intelligence and Statistics, p. 131 (2011)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Sooraj Bhat
    • 1
  • Johannes Borgström
    • 2
  • Andrew D. Gordon
    • 3
  • Claudio Russo
    • 3
  1. 1.Georgia Institute of TechnologyUSA
  2. 2.Uppsala UniversitySweden
  3. 3.Microsoft ResearchUK

Personalised recommendations