Skip to main content
SpringerLink
Log in

Probabilistic Programming Language and its Incremental Evaluation

  • Conference paper
  • First Online:
Programming Languages and Systems (APLAS 2016)

Part of the book series: Lecture Notes in Computer Science ((LNPSE,volume 10017))

Included in the following conference series:

Abstract

This system description paper introduces the probabilistic programming language Hakaru10, for expressing, and performing inference on (general) graphical models. The language supports discrete and continuous distributions, mixture distributions and conditioning. Hakaru10 is a DSL embedded in Haskell and supports Monte-Carlo Markov Chain (MCMC) inference.

Hakaru10 is designed to address two main challenges of probabilistic programming: performance and correctness. It implements the incremental Metropolis-Hastings method, avoiding all redundant computations. In the presence of conditional branches, efficiently maintaining dependencies and correctly computing the acceptance ratio are non-trivial problems, solved in Hakaru10. The implementation is unique in being explicitly designed to satisfy the common equational laws of probabilistic programs. Hakaru10 is typed; specifically, its type system statically prevents meaningless conditioning, enforcing that the values to condition upon must indeed come from outside the model.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    These signatures are somewhat simplified. We will later see that dist is overloaded.

  2. 2.

    Dirac distribution, or Dirac delta, is taken as density of a discrete random variable with the single value.

  3. 3.

    http://ppaml.galois.com/wiki/wiki/CP4SmallProblemsCollection.

  4. 4.

    http://www.darpa.mil/program/probabilistic-programming-for-advancing-machine-Learning.

  5. 5.

    Imposing the upper bound on the number of balls may be undesirable, especially for the Poisson distribution. In principle, Hakaru10 could instantiate conditional branches of a model lazily; in which case balls could be an infinite list. We are investigating this possibility.

  6. 6.

    The actual implementation has one more level of indirection, but the description given here is a good approximation.

References

  1. AISTATS, number 33. MIT Press, Cambridge (2014)

    Google Scholar 

  2. De Raedt, L., Kimmig, A., Toivonen, H.: ProbLog: a probabilistic Prolog and its application in link discovery. In: Veloso, M.M. (ed.) Proceedings of the 20th International Joint Conference on Artificial Intelligence, pp. 6–12, January 2007

    Google Scholar 

  3. Erwig, M., Kollmansberger, S.: Probabilistic functional programming in Haskell. J. Funct. Program. 16(1), 21–34 (2006)

    Article  MATH  Google Scholar 

  4. Getoor, L., Taskar, B.: Introduction to Statistical Relational Learning. MIT Press, Cambridge, November 2007

    Google Scholar 

  5. Goodman, N.D.: The principles and practice of probabilistic programming. In: POPL 2013: Conference Record of the Annual ACM Symposium on Principles of Programming Languages, pp. 399–402. ACM Press, New York, January 2013

    Google Scholar 

  6. Goodman, N.D., Mansinghka, V.K., Roy, D., Bonawitz, K., Tenenbaum, J.B.: Church: a language for generative models. In: McAllester, D.A., Myllymäki, P. (eds.) Proceedings of the 24th Conference on Uncertainty in Artificial Intelligence, pp. 220–229, Corvallis, Oregon, 9–12. AUAI Press, July 2008

    Google Scholar 

  7. Goodman, N.D., Stuhlmüller, A.: The design and implementation of probabilistic programming languages (2014). http://dippl.org

  8. Gordon, A.D., Henzinger, T.A., Nori, A.V., Rajamani, S.K.: Probabilistic programming. In: FOSE, pp. 167–181. ACM (2014)

    Google Scholar 

  9. Hoffman, M.D., Gelman, A.: The No-U-Turn Sampler: Adaptively setting path lengths in Hamiltonian Monte Carlo. e-Print 1111.4246, arXiv.org (2011)

  10. Hur, C.K., Nori, A.V., Rajamani, S.K., Samuel, S.: A provably correct sampler for probabilistic programs. In: FSTTCS 2015 (2015)

    Google Scholar 

  11. Kiselyov, O.: Problems of the lightweight implementation of probabilistic programming. In: Proceedings of Workshop on Probabilistic Programming Semantics (2016)

    Google Scholar 

  12. Kiselyov, O., Shan, C.C.: Monolingual probabilistic programming using generalized coroutines. In: Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence, pp. 285–292, Corvallis, Oregon, 19–21. AUAI Press, June 2009

    Google Scholar 

  13. McBride, C., Paterson, R.: Applicative programming with effects. J. Funct. Program. 18(1), 1–13 (2008)

    Article  MATH  Google Scholar 

  14. Milch, B., Marthi, B., Russell, S., Sontag, D., Ong, D.L., Kolobov, A.: BLOG: probabilistic models with unknown objects. In: Getoor and Taskar [4], chapter 13, pp. 373–398

    Google Scholar 

  15. Minka, T., Winn, J.M., Guiver, J.P., Kannan, A.: Infer.NET 2.2. Microsoft Research Cambridge (2009). http://research.microsoft.com/infernet

  16. Murphy, K.: Software for graphical models: a review. Int. Soc. Bayesian Anal. Bull. 14(4), 13–15 (2007)

    Google Scholar 

  17. Pearl, J.: Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference, 2nd edn. Morgan Kaufmann, San Francisco (1988)

    MATH  Google Scholar 

  18. Pfeffer, A., Figaro: an object-oriented probabilistic programming language. Technical report 137, Charles River Analytics (2009)

    Google Scholar 

  19. Sato, T.: A glimpse of symbolic-statistical modeling by PRISM. J. Intell. Inf. Syst. 31(2), 161–176 (2008)

    Article  Google Scholar 

  20. Ścibior, A., Ghahramani, Z., Gordon, A.D.: Practical probabilistic programming with monads. In: Proceedings of the 8th ACM SIGPLAN Symposium on Haskell, pp. 165–176. ACM Press, New York (2015)

    Google Scholar 

  21. Wingate, D., Stuhlmüller, A., Goodman, N.D.: Lightweight implementations of probabilistic programming languages via transformational compilation. In: AISTATS, no. 15, pp. 770–778, Revision 3, February 8, 2014. MIT Press, Cambridge (2011)

    Google Scholar 

  22. Wood, F., van de Meent, J.W., Mansinghka, V.: A new approach to probabilistic programming inference. In: AISTATS 2014 [1], pp. 1024–1032 (2014)

    Google Scholar 

  23. Yang, L., Hanrahan, P., Goodman, N.D.: Generating efficient MCMC kernels from probabilistic programs. In: AISTATS [1], pp. 1068–1076 (2014)

    Google Scholar 

  24. Zinkov, R., Shan, C-C.: Probabilistic programming language Hakaru. v1. DARPA PPAML Report (2014)

    Google Scholar 

Download references

Acknowledgments

I am indebted Rob Zinkov and Chung-chieh Shan for many helpful discussions. Comments and suggestions by anonymous reviewers are gratefully acknowledged. The work on Hakaru10 was supported by DARPA grant FA8750-14-2-0007.

Author information

Authors and Affiliations

  1. Tohoku University, Sendai, Japan

    Oleg Kiselyov

Authors
  1. Oleg Kiselyov
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Oleg Kiselyov .

Editor information

Editors and Affiliations

  1. Kyoto University , Kyoto, Japan

    Atsushi Igarashi

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing AG

About this paper

Cite this paper

Kiselyov, O. (2016). Probabilistic Programming Language and its Incremental Evaluation. In: Igarashi, A. (eds) Programming Languages and Systems. APLAS 2016. Lecture Notes in Computer Science(), vol 10017. Springer, Cham. https://doi.org/10.1007/978-3-319-47958-3_19

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-47958-3_19

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-47957-6

  • Online ISBN: 978-3-319-47958-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation