Running Probabilistic Programs Backwards

  • Neil TorontoEmail author
  • Jay McCarthy
  • David Van Horn
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9032)


Many probabilistic programming languages allow programs to be run under constraints in order to carry out Bayesian inference. Running programs under constraints could enable other uses such as rare event simulation and probabilistic verification—except that all such probabilistic languages are necessarily limited because they are defined or implemented in terms of an impoverished theory of probability. Measure-theoretic probability provides a more general foundation, but its generality makes finding computational content difficult.

We develop a measure-theoretic semantics for a first-order probabilistic language with recursion, which interprets programs as functions that compute preimages. Preimage functions are generally uncomputable, so we derive an abstract semantics. We implement the abstract semantics and use the implementation to carry out Bayesian inference, stochastic ray tracing (a rare event simulation), and probabilistic verification of floating-point error bounds.


Probability Semantics Domain-Specific Languages 


  1. 1.
    Haskell 98 language and libraries, the revised report (December 2002),
  2. 2.
    Aumann, R.J.: Borel structures for function spaces. Illinois Journal of Mathematics 5, 614–630 (1961)zbMATHMathSciNetGoogle Scholar
  3. 3.
    Bonawitz, K.A.: Composable Probabilistic Inference with Blaise. Ph.D. thesis, Massachusetts Institute of Technology (2008)Google Scholar
  4. 4.
    Borgström, J., Gordon, A.D., Greenberg, M., Margetson, J., Van Gael, J.: Measure transformer semantics for Bayesian machine learning. In: Barthe, G. (ed.) ESOP 2011. LNCS, vol. 6602, pp. 77–96. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  5. 5.
    Cousot, P., Monerau, M.: Probabilistic abstract interpretation. In: Seidl, H. (ed.) ESOP 2012. LNCS, vol. 7211, pp. 169–193. Springer, Heidelberg (2012)Google Scholar
  6. 6.
    Darais, D., Might, M., Van Horn, D.: Galois transformers and modular abstract interpreters. CoRR abs/1411.3962 (2014),
  7. 7.
    DeGroot, M., Schervish, M.: Probability and Statistics. Addison Wesley Publishing Company, Inc. (2012)Google Scholar
  8. 8.
    Goodman, N., Mansinghka, V., Roy, D., Bonawitz, K., Tenenbaum, J.: Church: A language for generative models. In: Uncertainty in Artificial Intelligence (2008)Google Scholar
  9. 9.
    Gulwani, S., Jojic, N.: Program verification as probabilistic inference. In: Principles of Programming Languages, pp. 277–289 (2007)Google Scholar
  10. 10.
    Hughes, J.: Generalizing monads to arrows. Science of Computer Programming 37, 67–111 (2000)CrossRefzbMATHMathSciNetGoogle Scholar
  11. 11.
    Hurd, J.: Formal Verification of Probabilistic Algorithms. Ph.D. thesis. University of Cambridge (2002)Google Scholar
  12. 12.
    Jones, C.: Probabilistic Non-Determinism. Ph.D. thesis, Univ. of Edinburgh (1990)Google Scholar
  13. 13.
    Kiselyov, O., Shan, C.: Monolingual probabilistic programming using generalized coroutines. In: Uncertainty in Artificial Intelligence (2008)Google Scholar
  14. 14.
    Koller, D., McAllester, D., Pfeffer, A.: Effective Bayesian inference for stochastic programs. In: 14th National Conference on Artificial Intelligence (August 1997)Google Scholar
  15. 15.
    Kozen, D.: Semantics of probabilistic programs. In: Foundations of Computer Science (1979)Google Scholar
  16. 16.
    Lindley, S., Wadler, P., Yallop, J.: Idioms are oblivious, arrows are meticulous, monads are promiscuous. In: Workshop on Mathematically Structured Functional Programming (2008)Google Scholar
  17. 17.
    Lunn, D.J., Thomas, A., Best, N., Spiegelhalter, D.: WinBUGS – a Bayesian modelling framework. Statistics and Computing 10(4) (2000)Google Scholar
  18. 18.
    McAllester, D., Milch, B., Goodman, N.D.: Random-world semantics and syntactic independence for expressive languages. Tech. rep. MIT (2008)Google Scholar
  19. 19.
    McBride, C., Paterson, R.: Applicative programming with effects. Journal of Functional Programming 18(1) (2008)Google Scholar
  20. 20.
    Milch, B., Marthi, B., Russell, S., Sontag, D., Ong, D., Kolobov, A.: BLOG: Probabilistic models with unknown objects. In: International Joint Conference on Artificial Intelligence (2005)Google Scholar
  21. 21.
    Minka, T., Winn, J., Guiver, J., Webster, S., Zaykov, Y., Yangel, B., Spengler, A., Bronskill, J.: Infer.NET 2.6 (2014),
  22. 22.
    Moggi, E.: Computational lambda-calculus and monads. In: IEEE Symposium on Logic in Computer Science, pp. 14–23 (1989)Google Scholar
  23. 23.
    Nori, A.V., Hur, C.K., Rajamani, S.K., Samuel, S.: R2: An efficient MCMC sampler for probabilistic programs. In: AAAI Conference on Artificial Intelligence (2014)Google Scholar
  24. 24.
    Pfeffer, A.: The design and implementation of IBAL: A general-purpose probabilistic language. In: Statistical Relational Learning. MIT Press (2007)Google Scholar
  25. 25.
    Ramsey, N., Pfeffer, A.: Stochastic lambda calculus and monads of probability distributions. In: Principles of Programming Languages (2002)Google Scholar
  26. 26.
    Sergey, I., Devriese, D., Might, M., Midtgaard, J., Darais, D., Clarke, D., Piessens, F.: Monadic abstract interpreters. In: Programming Language Design and Implementation, pp. 399–410 (2013)Google Scholar
  27. 27.
    Tobin-Hochstadt, S., Felleisen, M.: The design and implementation of typed Scheme. In: Principles of Programming Languages, pp. 395–406 (2008)Google Scholar
  28. 28.
    Toronto, N.: Trustworthy, Useful Languages for Probabilistic Modeling and Inference. Ph.D. thesis, Brigham Young University (2014),
  29. 29.
    Toronto, N., McCarthy, J.: Computing in Cantor’s paradise with λ ZFC. In: Functional and Logic Programming Symposium, pp. 290–306 (2012)Google Scholar
  30. 30.
    Veach, E., Guibas, L.J.: Metropolis light transport. In: ACM SIGGRAPH, pp. 65–76 (1997)Google Scholar
  31. 31.
    Wadler, P.: Monads for functional programming. In: Jeuring, J., Meijer, E. (eds.) Advanced Functional Programming (2001)Google Scholar
  32. 32.
    Wingate, D., Goodman, N.D., Stuhlmüller, A., Siskind, J.M.: Nonstandard interpretations of probabilistic programs for efficient inference. In: Neural Information Processing Systems, pp. 1152–1160 (2011)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2015

Authors and Affiliations

  1. 1.University of MarylandCollege ParkUSA
  2. 2.Vassar CollegePoughkeepsieUSA

Personalised recommendations