Statistics and Computing

, Volume 7, Issue 1, pp 19–34 | Cite as

Decomposing Bayesian networks: triangulation of the moral graph with genetic algorithms

  • Pedro Larrañaga
  • Cindy M. H. Kuijpers
  • Mikel Poza
  • Roberto H. Murga
Article

Abstract

In this paper we consider the optimal decomposition of Bayesian networks. More concretely, we examine empirically the applicability of genetic algorithms to the problem of the triangulation of moral graphs. This problem constitutes the only difficult step in the evidence propagation algorithm of Lauritzen and Spiegelhalter (1988) and is known to be NP-hard (Wen, 1991). We carry out experiments with distinct crossover and mutation operators and with different population sizes, mutation rates and selection biasses. The results are analysed statistically. They turn out to improve the results obtained with most other known triangulation methods (Kjærulff, 1990) and are comparable to results obtained with simulated annealing (Kjærulff, 1990; Kjærulff, 1992).

Bayesian networks genetic algorithms optimal decomposition graph triangulation moral graph NP-hard problems statistical analysis 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Banzhaf, W. (1990) The ‘molecular’ traveling salesman. Biological Cybernetics, 64, 7–14.Google Scholar
  2. Chakraborty, U. K. and Dastidar, D. G. (1993) Using reliability analysis to estimate the number of generations to convergence in genetic algorithms. Information Processing Letters, 46, 199–209.Google Scholar
  3. Chavez, R. M. and Cooper, G. F. (1990) A randomized approximation algorithm for probabilistic inference on Bayesian belief networks. Networks, 20, 661–85.Google Scholar
  4. Cooper, G. F. (1990) The computational complexity of probabilistic inference using Bayesian belief networks. Artificial Intelligence, 42, 393–405.Google Scholar
  5. Dagum, P. and Luby, M. (1993) Approximating probabilistic inference in Bayesian belief networks is NP-hard. Artificial Intelligence, 60, 141–53.Google Scholar
  6. Dagum, P. and Horvitz, E. (1993) A Bayesian analysis of simulation algorithms for inference in belief networks. Networks, 23, 499–516.Google Scholar
  7. Davis, L. (1985) Applying adaptive algorithms to epistatic domains. In Proceedings International Joint Conference on Artificial Intelligence, Los Angeles, CA, pp. 162–4.Google Scholar
  8. Davis, L. (1991) Handbook of Genetic Algorithms. Van Nostrand Reinhold, New York.Google Scholar
  9. De Jong, K. A. (1975) An analysis of the behaviour of a class of genetic adaptive systems. Ph.D. Dissertation, University of Michigan.Google Scholar
  10. Eiben, A. E., Aarts, E. H. L. and Van Hee, K. M. (1990) Global convergence of genetics algorithms: an infinite Markov chain analysis. Computing Science Notes, Eindhoven University of Technology, The Netherlands.Google Scholar
  11. Fogel, D. B. (1990) A parallel processing approach to a multiple traveling salesman problem using evolutionary programming. In Proceedings of the Fourth Annual Parallel Processing Symposium, Fullerton, CA, pp. 318–26.Google Scholar
  12. Fox B. R. and McMahon, M. B. (1991) Genetic operators for sequencing problems. In Foundations of Genetic Algorithms: First Workshop on the Foundations of Genetic Algorithms and Classifier Systems (G. Rawlins, ed.), pp. 284–300. Morgan Kaufmann, Los Altos, CA.Google Scholar
  13. Fujisawa, T. and Orino, H. (1974) An efficient algorithm of finding a minimal triangulation of a graph. In IEEE International Symposium on Circuits and Systems, San Francisco, California, pp. 172–5.Google Scholar
  14. Fung, R. M. and Chang, K. C. (1990) Weighing and integrating evidence for stochastic simulation in Bayesian networks. In Uncertainty in Artificial Intelligence 5, (M. Henrion, R. D. Shachter, L. N. Kanal, and J. F. Lemmer, eds), pp. 209–20, Elsevier, Amsterdam.Google Scholar
  15. Goldberg, D. E. (1989) Genetic Algorithms in Search, Optimization and Machine Learning. Addison-Wesley, Reading, MA.Google Scholar
  16. Goldberg, D. E. and Lingle, Jr. R. (1985) Alleles, loci and the traveling salesman problem. In Proceedings of an International Conference on Genetic Algorithms and Their Applications, Pittsburgh, PA, pp. 154–9.Google Scholar
  17. Grefenstette, J., Gopal, R., Rosmaita, B. and Van Gucht, D. (1985) Genetic algorithms for the traveling salesman problem. In Proceedings of an International Conference on Genetic Algorithms and Their Applications, Pittsburgh, PA, pp. 160–5.Google Scholar
  18. Henrion, M. (1988) Propagating uncertainty in Bayesian networks by probabilistic logic sampling. In Uncertainty in Artificial Intelligence 2, (J. F. Lemmer and L. N. Kanal, eds), pp. 149–63, North-Holland, Amsterdam.Google Scholar
  19. Holland, J. (1975) Adaptation in Natural and Artificial Systems. University of Michigan Press, Ann Arbor.Google Scholar
  20. Homaifar, A. and Guan, S. (1991) A new approach on the traveling salesman problem by genetic algorithm. Technical Report, North Carolina A & T State University.Google Scholar
  21. Hryceij, T. (1990) Gibbs sampling in Bayesian networks. Artificial Intelligence, 46, 351–63.Google Scholar
  22. Jensen, F. (1994) Implementation aspects of various propagation algorithms in Hugin. Technical Report R 94–2014, University of Aalborg, Denmark.Google Scholar
  23. Jensen, F. V. (1996) An Introduction to Bayesian Networks. UCL Press, London.Google Scholar
  24. Jensen, C. S., Kong, A. and Kjærulff, U. (1993) Blocking Gibbs sampling in very large probabilistic expert systems. Technical Report R 93–2031, University of Aalborg, Denmark.Google Scholar
  25. Jog, P., Suh, J. Y. and Van Gucht, D. (1989) The effects of population size, heuristic crossover and local improvement on a genetic algorithm for the traveling salesman problem. In Proceedings on the Third International Conference on Genetic Algorithms, Arlington, VA, pp. 110–15.Google Scholar
  26. Kjærulff, U. (1990) Triangulation of graphs-Algorithms giving small total state space. Technical Report R 90–09, University of Aalborg, Denmark.Google Scholar
  27. Kjærulff, U. (1992) Optimal decomposition of probabilistic networks by simulated annealing. Statistics and Computing, 2, 7–17.Google Scholar
  28. Larrañaga, P., Graña, M., D'Anjou, A. and Torrealdea, F. J. (1993) Genetics algorithms elitist probabilistic of degree 1, a generalization of simulated annealing. In Proceedings of the Third Congress of the Italian Association for Artificial Intelligence, IA*AI '93, Torino, Italy, pp. 208–17.Google Scholar
  29. Larrañaga, P., Kuijpers, C. M. H., Poza, M. and Murga, R. H. (1994) Optimal decomposition of Bayesian networks by genetic algorithms. Internal Report EHU-KZAA-IKT–3–94, University of the Basque Country, Spain.Google Scholar
  30. Larrañaga, P., Kuijpers, C. M. H., Murga, R. H. and Yurramendi, Y. (1996a) Learning Bayesian network structures by searching for the best ordering with genetic algorithms. IEEE Transactions on Systems, Man and Cybernetics. Vol. 26, No. 4, pp. 487–93.Google Scholar
  31. Larrañaga, P., Poza, M., Yurramendi, Y., Murga, R. H., and Kuijpers, C. M. H. (1996b) Structure learning of Bayesian network by genetic algorithms: a performance analysis of control parameters. IEEE Transactions on Pattern Analysis and Machine Intelligence (to appear).Google Scholar
  32. Larrañaga, P., Murga, R. H., Poza, M. and Kuijpers, C. M. H. (1996c) Structure learning of Bayesian networks by hybrid genetic algorithms. In Learning from Data: Artificial Intelligence and Statistics V, (D. Fisher and H. Lenz, eds), Springer-Verlag, New York, pp. 165–74.Google Scholar
  33. Larrañaga, P., Kuijpers, C. M. H. and Murga, R. H. (1996d) >Evolutionary algorithms for the travelling salesman problem: A review of representations and operators. Artificial Intelligence Review (to appear).Google Scholar
  34. Lauritzen, S. L. and Spiegelhalter, D. J. (1988) Local computations with probabilities on graphical structures and their application to expert systems. Journal of the Royal Statistical Society, Series B, 50, 157–224.Google Scholar
  35. Lidd, M. L. (1991) The traveling salesman problem domain application of a fundamentally new approach to utilizing genetic algorithms. Technical Report, MITRE Corporation.Google Scholar
  36. Michalewicz, Z. (1992) Genetic Algorithms + Data Structures = Evolution Programs. Springer-Verlag, Berlin.Google Scholar
  37. Mühlenbein, H. (1989) Parallel genetic algorithms, population genetics and combinatorial optimization. In Proceedings on the Third International Conference on Genetic Algorithms, Arlington, VA, pp. 416–21.Google Scholar
  38. Neapolitan, R. E. (1990) Probabilistic Reasoning in Expert Systems, Theory and Algorithms. Wiley, New York.Google Scholar
  39. Oliver, I. M., Smith, D. J. and Holland, J. R. C. (1987) A study of permutation crossover operators on the TSP. In Genetic Algorithms and Their Applications: Proceedings of the Second International Conference, Cambridge, MA, pp. 224–30.Google Scholar
  40. Pearl, J. (1986) Fusion, propagation, and structuring in belief networks. Artificial Intelligence, 29, 241–88.Google Scholar
  41. Pearl, J. (1987) Evidential reasoning using stochastic simulation of causal models. Artificial Intelligence, 32, 245–57.Google Scholar
  42. Pearl, J. (1988) Probabilistic Reasoning in Intelligent Systems. Morgan Kaufmann, Palo Alto, CA.Google Scholar
  43. Rose, D. J., Tarjan, R. E. and Lueker, G. S. (1976) Algorithmic aspects of vertex elimination on graphs. SIAM Journal on Computing, 5, 266–83.Google Scholar
  44. Rudolph, G. (1994) Convergence analysis of canonical genetic algorithms. IEEETransactions on Neural Networks, 5, 96–101.Google Scholar
  45. Seniw, D. (1991) A genetic algorithm for the traveling salesman problem. M.Sc. Thesis, University of North Carolina at Charlotte.Google Scholar
  46. Shachter, R. D. (1988) Probabilistic inference and influence diagrams. Operations Research, 36, 589–604.Google Scholar
  47. Shachter, R. D. and Peot, M. A. (1990) Simulation approaches to general probabilistic inference on belief networks. In Uncertainty in Artificial Intelligence 5 (M. Henrion, R. D. Shachter, L. N. Kanal and J. F. Lemmer, eds), pp. 221–34. Elsevier, Amsterdam.Google Scholar
  48. Shwe, M. and Cooper, G. (1991) An empirical analysis of likelihood-weighting simulation on a large, multiply connected medical belief network. Computers and Biomedical Research, 24, 453–75.Google Scholar
  49. Suh, J. Y. and Van Gucht, D. (1987) Incorporating heuristic information into genetic search. In Genetic Algorithms and Their Applications: Proceedings of the Second International Conference, Cambridge, MA, pp. 100–7.Google Scholar
  50. Syswerda, G. (1991) Schedule optimization using genetic algorithms. In Handbook of Genetic Algorithms, (L. Davis, ed.), pp. 332–49. Van Nostrand Reinhold, New York.Google Scholar
  51. Tarjan, R. E. and Yannakakis, M. (1984) Simple linear-time algorithms to test chordality of graphs, test acyclicity of hypergraphs, and selectively reduce acyclic hypergraphs. SIAM Journal on Computing, 13, 566–79.Google Scholar
  52. Wen, W. X. (1990) Decomposing belief networks by simulated annealing. In (C. P. Tsang, ed.) Proceedings of the Australian 1990 Joint Conference on Artificial Intelligence, pp. 103–18. World Scientific Publishers, Perth, WA.Google Scholar
  53. Wen, W. X. (1991) Optimal decomposition of belief networks. In Uncertainty in Artificial Intelligence 6 (P. P. Bonissone, M. Henrion, L. N. Kanal, and J. F. Lemmer, eds), pp. 209–24. North-Holland, Amsterdam.Google Scholar
  54. Whitley, D. (1989) The GENITOR algorithm and selection pressure: Why rank-based allocation of reproductive trials is best. In Proceedings on the Third International Conference on Genetic Algorithms, Arlington, VA, pp. 116–21.Google Scholar
  55. Whitley D. and Hanson, T. (1987) Optimizing neural networks using faster, more accurate genetic search. In Genetic Algorithms and Their Applications: Proceedings of the Second International Conference, Cambridge, MA, pp. 391–6.Google Scholar
  56. Whitley, D., Starkweather, T. and Fuquay, D. (1989) Scheduling problems and travelling salesman: The genetic edge recombination operator. In Proceedings on the Third International Conference on Genetic Algorithms, Arlington, VA, pp. 133–40.Google Scholar
  57. Whitley, D., Starkweather, T. and Shaner, D. (1991) The traveling salesman and sequence scheduling: Quality solutions using genetic edge recombination. In Handbook of Genetic Algorithms (L. Davis, ed), pp. 350–72. Van Nostrand Reinhold, New York.Google Scholar

Copyright information

© Chapman and Hall 1997

Authors and Affiliations

  • Pedro Larrañaga
    • 1
  • Cindy M. H. Kuijpers
    • 2
  • Mikel Poza
    • 1
  • Roberto H. Murga
    • 1
  1. 1.Department of Computer Science and Artificial IntelligenceUniversity of the Basque CountrySan Sebastia´nSpain
  2. 2.Department of Applied MathematicsUniversity of TwenteEnschedeThe Netherlands

Personalised recommendations