Statistics and Computing

, Volume 8, Issue 3, pp 243–251 | Cite as

A simple method for finding a legal configuration in complex Bayesian networks

  • Claus Skaanning Jensen


This paper deals with an important problem with large and complex Bayesian networks. Exact inference in these networks is simply not feasible owing to the huge storage requirements of exact methods. Markov chain Monte Carlo methods, however, are able to deal with these large networks but to do this they require an initial legal configuration to set off the sampler. So far nondeterministic methods such as forward sampling have often been used for this, even though the forward sampler may take an eternity to come up with a legal configuration. In this paper a novel algorithm will be presented that allows a legal configuration in a general Bayesian network to be found in polynomial time in almost all cases. The algorithm will not be proved deterministic but empirical results will demonstrate that this holds in most cases. Also, the algorithm will be justified by its simplicity and ease of implementation.

Complex Bayesian networks Markov chain Monte Carlo Gibbs sampling starting configuration 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Gelfand, A. E. and Smith, A. F. M. (1990) Sampling-based approaches to calculating marginal densities. Journal of American Statistical Association, 85(410), 398–409.Google Scholar
  2. Gelman, A. and Rubin, D. B. (1992) Inference from iterative simulation using single and multiple sequences (with discussion). Statistical Science, 7, 457–511.Google Scholar
  3. Geman, S. and Geman, D. (1984) Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 6(6), 721–41.Google Scholar
  4. Geyer, C. J. (1991) Markov chain Monte Carlo maximum likelihood, in Computing Science and Statistics, Proceedings of the 23rd Symposium on the Interface, pp. 156–63.Google Scholar
  5. Geyer, C. J. (1992) Practical Markov Chain Monte Carlo. Statistical Science, 7(4), 473–511.Google Scholar
  6. Geyer, C. J. and Thompson, E. A. (1995) Annealing Markov chain Monte Carlo with applications to ancestral inference. Journal of American Statistical Association, 90(431), 909–20.Google Scholar
  7. Gilberg, A., Gilberg, L., Gilberg, R. et al. (1978) Polar Eskimo Genealogy, Vol. 203No. 4 of Meddelelser om Grønland, Nyt Nordisk Forlag, Arnold Busck, København.Google Scholar
  8. Hastings, W. K. (1970) Monte Carlo sampling methods using Markov chains and their applications. Biometrika, 57(1), 97–109.Google Scholar
  9. Heath, S. C. (1998) Generating consistent genotypic configurations for multi-allelic loci and large complex pedigrees. Human Heredity, 48, 1–11.Google Scholar
  10. Jensen, C. S. (1997) Blocking Gibbs Sampling for Inference in Large and Complex Bayesian Networks with Applications in Genetics, PhD Thesis, Department of Computer Science, Aalborg University, Denmark.Google Scholar
  11. Jensen, C. S. and Kong, A. (1996) Linkage analysis in large pedigrees with many loops — blocking Gibbs, in European Mathematical Genetics Meeting, Clementi, M. and Forabosco, P. (eds), Cooperativa Libraria Editrice Università di Padova, pp. 84–98.Google Scholar
  12. Jensen, C. S. and Sheehan, N. (1997) Problems with the determination of the noncommunicating classes for MCMC applications in pedigree analysis. Technical Report R–97–5004, Department of Computer Science, Aalborg University, Denmark. Accepted by Biometrics for publication in 1998.Google Scholar
  13. Jensen, C. S., Kong, A. and Kjærulff, U. (1995) Blocking-Gibbs sampling in very large probabilistic expert systems. International Journal of Human-Computer Studies, 42, 647–66.Google Scholar
  14. Jensen, F. V. (1988) Junction trees and decomposable hypergraphs. Research report, Judex Datasystemer A/S, Aalborg, Denmark.Google Scholar
  15. Kong, A. (1991) Efficient methods for computing linkage likelihoods of recessive diseases in inbred pedigrees. Genetic Epidemiology, 8, 81–103.Google Scholar
  16. Lauritzen, S. L. (1992) Propagation of probabilities, means and variances in mixed graphical association models. Journal of American Statistical Association, 87(420), 1098–1108.Google Scholar
  17. Lauritzen, S. L. and Spiegelhalter, D. J. (1988) Local computations with probabilities on graphical structures and their application to expert systems. Journal of the Royal Statistical Society, Series B, 50(2), 157–224.Google Scholar
  18. Marinari, E. and Parisi, G. (1992) Simulated tempering: a new Monte Carlo scheme. Europhysics Letters, 19, 451–58.Google Scholar
  19. Metropolis, N., Rosenbluth, A., Rosenbluth, M. et al. (1953) Equations of state calculations by fast computing machines. Journal of Chemistry and Physics, 21, 1087–91Google Scholar
  20. Pearl, J. (1986) Fusion, propagation, and structuring in belief networks. Artificial Intelligence, 29, 241–88Google Scholar
  21. Pearl, J. (1988) Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference, Series in Representation and Reasoning, Morgan Kaufmann Publishers, San Mateo, California.Google Scholar
  22. Sheehan, N. and Thomas, A. (1993) On the irreducibility of a Markov chain defined on a space of genotype configurations by a sampling scheme. Biometrics, 49, 163–75.Google Scholar
  23. Shenoy, P. P. and Shafer, G. R. (1990) Axioms for probability and belief-function propagation, in Uncertainty in Artificial Intelligence 4, Shachter, R. D., Levitt, T. S., Kanal, L. N., and Lemmer, J. F. (eds), Elsevier Science Publishers B. V. (North-Holland), Amsterdam, pp. 169–98.Google Scholar
  24. Smith, A. F. M. and Roberts, G. O. (1993) Bayesian computation via the Gibbs sampler and related Markov chain Monte Carlo methods. Journal of the Royal Statistical Society, Series B, 55(1), 5–23.Google Scholar
  25. Thomas, A. (1988) Drawing pedigrees. IMA Journal of Mathematics Applied in Medicine & Biology, 5, 201–13.Google Scholar
  26. Thomas, A., Spiegelhalter, D. J. and Gilks, W. R. (1992) BUGS: A program to perform Bayesian inference using Gibbs sampling, in Bayesian Statistics 4, Bernardo, J. M., Berger, J. O., Dawid, A. P. and Smith, A. F. M. (eds), Clarendon Press, Oxford, UK, pp. 837–42.Google Scholar

Copyright information

© Kluwer Academic Publishers 1998

Authors and Affiliations

  • Claus Skaanning Jensen
    • 1
  1. 1.Department of Computer ScienceAalborg UniversityAalborg ØDenmark

Personalised recommendations