Probabilistic Programming Language and its Incremental Evaluation

  • Oleg Kiselyov
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10017)


This system description paper introduces the probabilistic programming language Hakaru10, for expressing, and performing inference on (general) graphical models. The language supports discrete and continuous distributions, mixture distributions and conditioning. Hakaru10 is a DSL embedded in Haskell and supports Monte-Carlo Markov Chain (MCMC) inference.

Hakaru10 is designed to address two main challenges of probabilistic programming: performance and correctness. It implements the incremental Metropolis-Hastings method, avoiding all redundant computations. In the presence of conditional branches, efficiently maintaining dependencies and correctly computing the acceptance ratio are non-trivial problems, solved in Hakaru10. The implementation is unique in being explicitly designed to satisfy the common equational laws of probabilistic programs. Hakaru10 is typed; specifically, its type system statically prevents meaningless conditioning, enforcing that the values to condition upon must indeed come from outside the model.


Markov Chain Monte Carlo Directed Acyclic Graph Markov Chain Monte Carlo Method Markov Chain Monte Carlo Algorithm Probabilistic Programming 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



I am indebted Rob Zinkov and Chung-chieh Shan for many helpful discussions. Comments and suggestions by anonymous reviewers are gratefully acknowledged. The work on Hakaru10 was supported by DARPA grant FA8750-14-2-0007.


  1. 1.
    AISTATS, number 33. MIT Press, Cambridge (2014)Google Scholar
  2. 2.
    De Raedt, L., Kimmig, A., Toivonen, H.: ProbLog: a probabilistic Prolog and its application in link discovery. In: Veloso, M.M. (ed.) Proceedings of the 20th International Joint Conference on Artificial Intelligence, pp. 6–12, January 2007Google Scholar
  3. 3.
    Erwig, M., Kollmansberger, S.: Probabilistic functional programming in Haskell. J. Funct. Program. 16(1), 21–34 (2006)CrossRefzbMATHGoogle Scholar
  4. 4.
    Getoor, L., Taskar, B.: Introduction to Statistical Relational Learning. MIT Press, Cambridge, November 2007Google Scholar
  5. 5.
    Goodman, N.D.: The principles and practice of probabilistic programming. In: POPL 2013: Conference Record of the Annual ACM Symposium on Principles of Programming Languages, pp. 399–402. ACM Press, New York, January 2013Google Scholar
  6. 6.
    Goodman, N.D., Mansinghka, V.K., Roy, D., Bonawitz, K., Tenenbaum, J.B.: Church: a language for generative models. In: McAllester, D.A., Myllymäki, P. (eds.) Proceedings of the 24th Conference on Uncertainty in Artificial Intelligence, pp. 220–229, Corvallis, Oregon, 9–12. AUAI Press, July 2008Google Scholar
  7. 7.
    Goodman, N.D., Stuhlmüller, A.: The design and implementation of probabilistic programming languages (2014).
  8. 8.
    Gordon, A.D., Henzinger, T.A., Nori, A.V., Rajamani, S.K.: Probabilistic programming. In: FOSE, pp. 167–181. ACM (2014)Google Scholar
  9. 9.
    Hoffman, M.D., Gelman, A.: The No-U-Turn Sampler: Adaptively setting path lengths in Hamiltonian Monte Carlo. e-Print 1111.4246, (2011)
  10. 10.
    Hur, C.K., Nori, A.V., Rajamani, S.K., Samuel, S.: A provably correct sampler for probabilistic programs. In: FSTTCS 2015 (2015)Google Scholar
  11. 11.
    Kiselyov, O.: Problems of the lightweight implementation of probabilistic programming. In: Proceedings of Workshop on Probabilistic Programming Semantics (2016)Google Scholar
  12. 12.
    Kiselyov, O., Shan, C.C.: Monolingual probabilistic programming using generalized coroutines. In: Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence, pp. 285–292, Corvallis, Oregon, 19–21. AUAI Press, June 2009Google Scholar
  13. 13.
    McBride, C., Paterson, R.: Applicative programming with effects. J. Funct. Program. 18(1), 1–13 (2008)CrossRefzbMATHGoogle Scholar
  14. 14.
    Milch, B., Marthi, B., Russell, S., Sontag, D., Ong, D.L., Kolobov, A.: BLOG: probabilistic models with unknown objects. In: Getoor and Taskar [4], chapter 13, pp. 373–398Google Scholar
  15. 15.
    Minka, T., Winn, J.M., Guiver, J.P., Kannan, A.: Infer.NET 2.2. Microsoft Research Cambridge (2009).
  16. 16.
    Murphy, K.: Software for graphical models: a review. Int. Soc. Bayesian Anal. Bull. 14(4), 13–15 (2007)Google Scholar
  17. 17.
    Pearl, J.: Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference, 2nd edn. Morgan Kaufmann, San Francisco (1988)zbMATHGoogle Scholar
  18. 18.
    Pfeffer, A., Figaro: an object-oriented probabilistic programming language. Technical report 137, Charles River Analytics (2009)Google Scholar
  19. 19.
    Sato, T.: A glimpse of symbolic-statistical modeling by PRISM. J. Intell. Inf. Syst. 31(2), 161–176 (2008)CrossRefGoogle Scholar
  20. 20.
    Ścibior, A., Ghahramani, Z., Gordon, A.D.: Practical probabilistic programming with monads. In: Proceedings of the 8th ACM SIGPLAN Symposium on Haskell, pp. 165–176. ACM Press, New York (2015)Google Scholar
  21. 21.
    Wingate, D., Stuhlmüller, A., Goodman, N.D.: Lightweight implementations of probabilistic programming languages via transformational compilation. In: AISTATS, no. 15, pp. 770–778, Revision 3, February 8, 2014. MIT Press, Cambridge (2011)Google Scholar
  22. 22.
    Wood, F., van de Meent, J.W., Mansinghka, V.: A new approach to probabilistic programming inference. In: AISTATS 2014 [1], pp. 1024–1032 (2014)Google Scholar
  23. 23.
    Yang, L., Hanrahan, P., Goodman, N.D.: Generating efficient MCMC kernels from probabilistic programs. In: AISTATS [1], pp. 1068–1076 (2014)Google Scholar
  24. 24.
    Zinkov, R., Shan, C-C.: Probabilistic programming language Hakaru. v1. DARPA PPAML Report (2014)Google Scholar

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  1. 1.Tohoku UniversitySendaiJapan

Personalised recommendations