Statistics and Computing

, Volume 2, Issue 1, pp 7–17 | Cite as

Optimal decomposition of probabilistic networks by simulated annealing

  • Uffe Kjærulff


This paper investigates the applicability of a Monte Carlo technique known as ‘simulated annealing’ to achieve optimum or sub-optimum decompositions of probabilistic networks under bounded resources. High-quality decompositions are essential for performing efficient inference in probabilistic networks. Optimum decomposition of probabilistic networks is known to be NP-hard (Wen, 1990). The paper proves that cost-function changes can be computed locally, which is essential to the efficiency of the annealing algorithm. Pragmatic control schedules which reduce the running time of the annealing algorithm are presented and evaluated. Apart from the conventional temperature parameter, these schedules involve the radius of the search space as a new control parameter. The evaluation suggests that the inclusion of this new parameter is important for the success of the annealing algorithm for the present problem.


Simulated annealing global optimization Monte Carlo algorithms graph decomposition probabilistic networks NP-complete problems 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Andreassen, S., Jensen, F. V., Andersen, S. K. et al. (1989) MUNIN—an expert EMG assistant, inComputer-Aided Electromyography and Expert Systems, Desmedt, J. E. (ed), Elsevier Science Publishers, Amsterdam.Google Scholar
  2. Bohachevsky, I. O., Johnson, M. E. and Stein, M. L. (1986) Generalized simulated annealing for function optimization.Technometrics,28, 209–217.Google Scholar
  3. Cooper, G. F. (1987) Probabilistic inference using belief networks is NP-hard. Technical Report KSL-87-37, Knowledge Systems Laboratory, Medical Computer Science, Stanford University, Stanford, California.Google Scholar
  4. Fujisawa, T. and Orino, H. (1974) An efficient algorithm of finding a minimal triangulation of a graph, inIEEE International Symposium on Circuits and Systems, San Francisco, California, pp. 172–175.Google Scholar
  5. Geman, S. and Geman, D. (1984) Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images.IEEE Transactions on Pattern Analysis and Machine Intelligence,61, 721–741.Google Scholar
  6. Jensen, F. V., Andersen, S. K., Kjærulff, U. and Andreassen, S. (1987) MUNIN—on the case for probabilities in medical expert systems—a practical exercise, inProceedings of Conference on Artificial Intelligence in Medicine (AIME), Marseilles, France, pp. 149–160.Google Scholar
  7. Jensen, F. V., Lauritzen, S. L. and Olsen, K. G. (1990) Bayesian updating in causal probabilistic networks by local computations.Computational Statistics Quarterly,4, 269–282.Google Scholar
  8. Kirkpatrick, S., Gelatt, C. D. and Vecchi, M. P. (1983) Optimization by simulated annealing.Science,220, 671–680.Google Scholar
  9. Kjæulff, U. (1990) Graph triangulation—algorithms giving small total state space. Technical Report R 90-09, University of Aalborg, Denmark.Google Scholar
  10. Lauritzen, S. L. and Spiegelhalter, D. J. (1988) Local computations with probabilities on graphical structures and their application to expert systems.Journal of the Royal Statistical Society B,50, 157–224.Google Scholar
  11. Lundy, M. (1985) Applications of the annealing algorithm to combinatorial problems in statistics.Biometrika,72, 191–198.Google Scholar
  12. Lundy, M. and Mees, A. (1986) Convergence of an annealing algorithm.Mathematical Programming,34, 111–124.Google Scholar
  13. Olesen, K. G., Kjærulff, U., Jensen, al. (1989) A MUNIN network for the median nerve—a case study on loops.Applied Artificial Intelligence,3, Special issuc: Towards Causal AI Models in Practice.Google Scholar
  14. Pearl, J. (1988)Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference, Series in Representation and Reasoning, Morgan Kaufmann Publishers, San Mateo, CaliforniaGoogle Scholar
  15. Rose, D. J. (1970) Triangulated graphs and the elimination process.Journal of Mathematical Analysis and Applications,32, 597–609.Google Scholar
  16. Rose, D. J. (1973) A graph-theoretic study of the numerical solution of sparse positive definite systems of linear equations, inGraph Theory and Computing, Read, R. C. (ed), Academic Press, New York, pp. 183–217.Google Scholar
  17. Rose, D. J., Tarjan, R. E. and Lueker, G. S. (1976) Algorithmic aspects of vertex elimination on graphs.SIAM Journal on Computing,5, 266–283.Google Scholar
  18. Shachter, R. D. (1988) Probabilistic inference and influence diagrams.Operations Research,36, 589–604.Google Scholar
  19. Shafer, G. and Shenoy, P. P. (1988) Bayesian and belief-function propagation. Working Paper 121, School of Business, University of Kansas, Lawrence, Kansas.Google Scholar
  20. Spiegelhalter, D. J. (1986) Probabilistic reasoning in predictive expert systems, inUncertainty in Artificial Intelligence, Lemmer, J. F. and Kanal, L. N. (eds), Elsevier Science Publishers, Amsterdam.Google Scholar
  21. Suermondt, H. J. and Cooper, G. F. (1988) Updating probabilities in multiply-connected belief networks, inProceedings of the Fourth Workshop on Uncertainty in Artificial Intelligence, Minneapolis, pp. 335–343.Google Scholar
  22. Tarjan, R. E. and Yannakakis, M. (1984) Simple linear-time algorithms to test chordality of graphs, test acyclicity of hypergraphs, and selectively reduce acyclic hypergraphs.SIAM Journal on Computing,13, 566–579.Google Scholar
  23. Thomas, A. (1986) Optimal computation of probability functions for pedigree analysis.IMA Journal of Mathematics Applied in Medicine and Biology,3, 167–178.Google Scholar
  24. Wen, W. X. (1990) Optimal decomposition of belief networks, inProceedings of the Sixth Workshop on Uncertainty in Artificial Intelligence, Cambridge, Massachusetts, pp. 245–256.Google Scholar

Copyright information

© Chapman & Hall 1992

Authors and Affiliations

  • Uffe Kjærulff
    • 1
  1. 1.Department of Mathematics and Computer Science, Institute of Electronic SystemsAalborg UniversityAalborgDenmark

Personalised recommendations