Abstract
Probabilistic Programming (PP) extends the expressiveness and scalability of Bayesian networks via programmability. Influence Diagrams (IDs) extend Bayesian Networks with decision variables and utility functions, allowing them to model sequential decision problems. Limited-Memory IDs (LIMIDs) further allow some earlier events to be ignored or forgotten. We propose a generalisation of PP and LIMIDs called IDLP, implemented in Logic Programming and with a solver based on Reinforcement Learning and sampling. We show that IDLP can model and solve LIMIDs, and perform PP tasks including inference, finding most probable explanations, and maximum likelihood estimation.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
In standard Prolog notation a predicate P/A has name P and arity A.
- 2.
The underscore _ character is a Prolog anonymous variable that matches any term and indicates a “don’t care” value.
- 3.
A different policy is given in [9]: treat in month 3 if tests 1 and 2, or 3, are positive. We find that their policy has expected utility 725.884 while ours is optimal. They cite our expected utility so we believe this was simply a typographical error. To compute the expected value of a policy we use the variable elimination algorithm where, instead of maximising over the decision variables, we set their values according to the policy.
References
Birge, J.R., Louveaux, F.V.: Introduction to Stochastic Programming. Springer, New York (2011). doi:10.1007/978-1-4614-0237-4
van den Broeck, G., Thon, I., van Otterlo, M., de Raedt, L.: DTProbLog: a decision-theoretic probabilistic prolog. In: 24th AAAI Conference on Artificial Intelligence (2010)
Cano, A., Gómez, M., Moral, S.: A forward-backward Monte Carlo method for solving influence diagrams. Int. J. Approximate Reasoning 42, 119–135 (2006)
Charnes, J.M., Shenoy, P.P.: Multistage Monte Carlo Method for solving influence diagrams using local computation. Manage. Sci. 50(3), 405–418 (2004)
Gordon, A.D., Henzinger, T.A., Nori, A.V., Rajamani, S.K.: Probabilistic programming. in: International Conference on Software Engineering (2014)
Gutmann, B., Thon, I., De Raedt, L.: Learning the parameters of probabilistic logic programs from interpretations. In: Gunopulos, D., Hofmann, T., Malerba, D., Vazirgiannis, M. (eds.) ECML PKDD 2011. LNCS, vol. 6911, pp. 581–596. Springer, Heidelberg (2011). doi:10.1007/978-3-642-23780-5_47
Howard, R.A., Matheson, J.E.: Influence Diagrams. Readings in Decision Analysis, Strategic Decisions Group, Menlo Park, CA, Chap. 38, pp. 763–771 (1981)
Kimura, H.: Reinforcement learning in multi-dimensional state-action space using Random Rectangular Coarse Coding and Gibbs Sampling. In: International Conference on Intelligent Robots and Systems. IEEE (2007)
Lauritzen, S.L., Nilsson, D.: Representing and solving decision problems with limited information. Manage. Sci. 47, 1238–1251 (2001)
Pearl, J.: Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Series in Representation and Reasoning. Morgan Kaufman Publishers, San Mateo (1988)
Pfeffer, A.: Practical Probabilistic Programming. Manning Publications, Greenwich (2016)
De Raedt, L., Kimmig, A.: Probabilistic (logic) programming concepts. Mach. Learn. 100(1), 5–47 (2015). Springer New York LLC
De Raedt, L., Kimmig, A., Toivonen, H.: ProbLog: a probabilistic prolog and its application in link discovery. In: IJCAI 2007, Proceedings of the 20th International Joint Conference on Artificial Intelligence, pp. 2462–2467 (2007)
Raiffa, H.: Decision Analysis. Addison-Wesley, Reading (1968)
Sato, T.: A statistical learning method for logic programs with distribution semantics. In: Proceedings of the 12th International Conference on Logic Programming, pp. 715–729. MIT Press (1995)
Shachter, R.D.: Evaluating influence diagrams. Oper. Res. 34(6), 871–882 (1986)
Sutton, R.S., Barto, A.G.: Reinforcement Learning: an Introduction. MIT Press, Cambridge (1998)
Acknowledgements
This work was supported in part by Science Foundation Ireland (SFI) under Grant Number SFI/12/RC/2289.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Prestwich, S.D., Toffano, F., Wilson, N. (2017). A Probabilistic Programming Language for Influence Diagrams. In: Moral, S., Pivert, O., Sánchez, D., Marín, N. (eds) Scalable Uncertainty Management. SUM 2017. Lecture Notes in Computer Science(), vol 10564. Springer, Cham. https://doi.org/10.1007/978-3-319-67582-4_18
Download citation
DOI: https://doi.org/10.1007/978-3-319-67582-4_18
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-67581-7
Online ISBN: 978-3-319-67582-4
eBook Packages: Computer ScienceComputer Science (R0)