Abstract
The paper presents a new sampling methodology for Bayesian networks called cutset sampling that samples only a subset of the variables and applies exact inference for the others. We show that this approach can be implemented effciently when the sampled variables constitute a cycle-cutset for the Bayesian network and otherwise it is exponential in the induced-width of the network’s graph, whose sampled variables are removed. Cutset sampling is an instance of the well known Rao-Blakwellisation technique for variance reduction investigated in [5, 2, 16]. Moreover, the proposed scheme extends standard sampling methods to non-ergodic networks with ergodic subspaces. Our empirical results confirm those expectations and show that cycle cutset sampling is superior to Gibbs sampling for a variety of benchmarks, yielding a simple, yet powerful sampling scheme.
This work was supported in part by NSF grant IIS-0086529 and MURI ONR award N00014-00-1-0617.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
A. Becker, R. Bar-Yehuda, and D. Geiger. Random algorithms for the loop cutset problem. In Uncertainty in AI (UAI’99), 1999.
G. Casella and C.P. Robert. Rao-blackwellisation of sampling schemes. Biometrika, 83(1):81–94, 1996.
R. Dechter. Bucket elimination: A unifying framework for reasoning. Artificial Intelligence, 113: 41–85, 1999.
A. Doucet, N. de Freitas, K. Murphy, and S. Russell. Rao-blackwellised particle filtering for dynamic bayesian networks. In Uncertainty in AI, pages 176–183, 2000.
A.E. Gelfand and A.F.M. Smith. Sampling-based approaches to calculating marginal densities. Journal of the American Statistical Association, 85:398–409, 1990.
S. Geman and D. Geman. Stochastic relaxations, gibbs distributions and the bayesian restoration of images. IEEE Transaction on Pattern analysis and Machine Intelligence (PAMI-6), pages 721–42, 1984.
W. Gilks, S. Richardson, and D. Spiegelhalter. Markov chain Monte Carlo in practice. Chapman and Hall, 1996.
M.H. De Groot. Probability and Statistics, 2nd edition. Addison-Wesley, 1986.
K. Kask I. Rish and R. Dechter. Empirical evaluation of approximation algorithms for probabilistic decoding. In Uncertainty in AI (UAI’98), 1998.
C. Jensen, A. Kong, and U. Kjaerulff. Blocking gibbs sampling in very large probabilistic expert systems. International Journal of Human Computer Studies. Special Issue on Real-World Applications of Uncertain Reasoning., pages 647–666, 1995.
F. V. Jensen, S.L Lauritzen, and K.G. Olesen. Bayesian updating in causal probabilistic networks by local computation. Computational Statistics Quarterly, 4:269–282, 1990.
Y. Weiss K. P. Murphy and M. I. Jordan. Loopy belief propagation for approximate inference: An empirical study. In Uncertainty in AI (UAI’99), 1999.
Uffe Kjærulff. Hugs: Combining exact inference and gibbs sampling in junction trees. In Uncertainty in AI, pages 368–375. Morgan Kaufmann, 1995.
D. Koller, U. Lerner, and D. Angelov. A general algorithm for approximate inference and its application to hybrid bayes nets. In Uncertainty in AI, pages 324–333, 1998.
S. L. Lauritzen and D. J. Spiegelhalter. Local computation with probabilities on graphical structures and their application to expert systems. Journal of the Royal Statistical Society, Series B, 50(2):157–224, 1988.
W. H. Wong Liu, J. and A. Kong. Covariance structure of the gibbs sampler with applications to the comparison of estimators and augmentation schemes. Biometrika, pages 27–40, 1994.
D. J. C MacKay. Introduction to monte carlo methods. In Proceedings of NATO Advanced Study Institute on Learning in Graphical Models. Sept 27–Oct 7, pages 175–204, 1996.
J. Pearl. Probabilistic Reasoning in Intelligent Systems. Morgan Kaufmann, 1988.
M. A. Peot and R. D. Shachter. Fusion and proagation with multiple observations in belief networks. Artificial Intelligence, pages 299–318, 1992.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2003 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Bidyuk, B., Dechter, R. (2003). Cycle-Cutset Sampling for Bayesian Networks. In: Xiang, Y., Chaib-draa, B. (eds) Advances in Artificial Intelligence. Canadian AI 2003. Lecture Notes in Computer Science, vol 2671. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-44886-1_23
Download citation
DOI: https://doi.org/10.1007/3-540-44886-1_23
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-40300-5
Online ISBN: 978-3-540-44886-0
eBook Packages: Springer Book Archive