Online Bayesian Inference for the Parameters of PRISM Programs
This paper presents a method for approximating posterior distributions over the parameters of a given PRISM program. A sequential approach is taken where the distribution is updated one datapoint at a time. This makes it applicable to online learning situations where data arrives over time. The method is applicable whenever the prior is a mixture of products of Dirichlet distributions. In this case the true posterior will be a mixture of very many such products. An approximation is effected by merging products of Dirichlet distributions. An analysis of the quality of the approximation is presented. Due to the heavy computational burden of this approach, the method has been implemented in the Mercury logic programming language. Initial results using a hidden Markov model are presented.
KeywordsPosterior Distribution Logic Program Latent Dirichlet Allocation Dirichlet Distribution Ground Instance
Unable to display preview. Download preview PDF.
- 2.Cover, T.M., Thomas, J.A.: Elements of Information Theory. Wiley (1991)Google Scholar
- 3.Cowell, R.G., Dawid, A.P., Sebastiani, P.: A comparison of sequential learning methods for incomplete data. In: Bernado, J.M., Berger, J., Dawid, A.P., Smith, A.F.M. (eds.) Bayesian Statistics, vol. 5, pp. 533–541. Clarendon Press, Oxford (1995)Google Scholar
- 6.Penny, W.D.: KL-divergences of Normal, Gamma, Dirichlet and Wishart densities. Technical report, University College London (2001)Google Scholar
- 7.R Development Core Team: R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria (2011) ISBN 3-900051-07-0Google Scholar
- 10.West, M.: Modelling with mixtures. In: Bernado, J.M., Berger, J.O., Dawid, A.P., Smith, A.F.M. (eds.) Bayesian Statistics, vol. 4, pp. 503–524. Clarendon Press, Oxford (1992)Google Scholar