# Decomposing Bayesian networks: triangulation of the moral graph with genetic algorithms

- 301 Downloads
- 30 Citations

## Abstract

In this paper we consider the optimal decomposition of Bayesian networks. More concretely, we examine empirically the applicability of genetic algorithms to the problem of the triangulation of moral graphs. This problem constitutes the only difficult step in the evidence propagation algorithm of Lauritzen and Spiegelhalter (1988) and is known to be NP-hard (Wen, 1991). We carry out experiments with distinct crossover and mutation operators and with different population sizes, mutation rates and selection biasses. The results are analysed statistically. They turn out to improve the results obtained with most other known triangulation methods (Kjærulff, 1990) and are comparable to results obtained with simulated annealing (Kjærulff, 1990; Kjærulff, 1992).

## Preview

Unable to display preview. Download preview PDF.

### References

- Banzhaf, W. (1990) The ‘molecular’ traveling salesman.
*Biological Cybernetics*,**64**, 7–14.Google Scholar - Chakraborty, U. K. and Dastidar, D. G. (1993) Using reliability analysis to estimate the number of generations to convergence in genetic algorithms.
*Information Processing Letters*,**46**, 199–209.Google Scholar - Chavez, R. M. and Cooper, G. F. (1990) A randomized approximation algorithm for probabilistic inference on Bayesian belief networks.
*Networks*,**20**, 661–85.Google Scholar - Cooper, G. F. (1990) The computational complexity of probabilistic inference using Bayesian belief networks.
*Artificial Intelligence*,**42**, 393–405.Google Scholar - Dagum, P. and Luby, M. (1993) Approximating probabilistic inference in Bayesian belief networks is NP-hard.
*Artificial Intelligence*,**60**, 141–53.Google Scholar - Dagum, P. and Horvitz, E. (1993) A Bayesian analysis of simulation algorithms for inference in belief networks.
*Networks*,**23**, 499–516.Google Scholar - Davis, L. (1985) Applying adaptive algorithms to epistatic domains. In
*Proceedings International Joint Conference on Artificial Intelligence*, Los Angeles, CA, pp. 162–4.Google Scholar - Davis, L. (1991)
*Handbook of Genetic Algorithms*. Van Nostrand Reinhold, New York.Google Scholar - De Jong, K. A. (1975) An analysis of the behaviour of a class of genetic adaptive systems. Ph.D. Dissertation, University of Michigan.Google Scholar
- Eiben, A. E., Aarts, E. H. L. and Van Hee, K. M. (1990) Global convergence of genetics algorithms: an infinite Markov chain analysis. Computing Science Notes, Eindhoven University of Technology, The Netherlands.Google Scholar
- Fogel, D. B. (1990) A parallel processing approach to a multiple traveling salesman problem using evolutionary programming. In
*Proceedings of the Fourth Annual Parallel Processing Symposium*, Fullerton, CA, pp. 318–26.Google Scholar - Fox B. R. and McMahon, M. B. (1991) Genetic operators for sequencing problems. In
*Foundations of Genetic Algorithms: First Workshop on the Foundations of Genetic Algorithms and Classifier Systems*(G. Rawlins, ed.), pp. 284–300. Morgan Kaufmann, Los Altos, CA.Google Scholar - Fujisawa, T. and Orino, H. (1974) An efficient algorithm of finding a minimal triangulation of a graph. In
*IEEE International Symposium on Circuits and Systems*, San Francisco, California, pp. 172–5.Google Scholar - Fung, R. M. and Chang, K. C. (1990) Weighing and integrating evidence for stochastic simulation in Bayesian networks. In
*Uncertainty in Artificial Intelligence 5*, (M. Henrion, R. D. Shachter, L. N. Kanal, and J. F. Lemmer, eds), pp. 209–20, Elsevier, Amsterdam.Google Scholar - Goldberg, D. E. (1989)
*Genetic Algorithms in Search, Optimization and Machine Learning*. Addison-Wesley, Reading, MA.Google Scholar - Goldberg, D. E. and Lingle, Jr. R. (1985) Alleles, loci and the traveling salesman problem. In
*Proceedings of an International Conference on Genetic Algorithms and Their Applications*, Pittsburgh, PA, pp. 154–9.Google Scholar - Grefenstette, J., Gopal, R., Rosmaita, B. and Van Gucht, D. (1985) Genetic algorithms for the traveling salesman problem. In
*Proceedings of an International Conference on Genetic Algorithms and Their Applications*, Pittsburgh, PA, pp. 160–5.Google Scholar - Henrion, M. (1988) Propagating uncertainty in Bayesian networks by probabilistic logic sampling. In
*Uncertainty in Artificial Intelligence 2*, (J. F. Lemmer and L. N. Kanal, eds), pp. 149–63, North-Holland, Amsterdam.Google Scholar - Holland, J. (1975)
*Adaptation in Natural and Artificial Systems*. University of Michigan Press, Ann Arbor.Google Scholar - Homaifar, A. and Guan, S. (1991) A new approach on the traveling salesman problem by genetic algorithm. Technical Report, North Carolina A & T State University.Google Scholar
- Hryceij, T. (1990) Gibbs sampling in Bayesian networks.
*Artificial Intelligence*,**46**, 351–63.Google Scholar - Jensen, F. (1994) Implementation aspects of various propagation algorithms in Hugin. Technical Report R 94–2014, University of Aalborg, Denmark.Google Scholar
- Jensen, F. V. (1996)
*An Introduction to Bayesian Networks*. UCL Press, London.Google Scholar - Jensen, C. S., Kong, A. and Kjærulff, U. (1993) Blocking Gibbs sampling in very large probabilistic expert systems. Technical Report R 93–2031, University of Aalborg, Denmark.Google Scholar
- Jog, P., Suh, J. Y. and Van Gucht, D. (1989) The effects of population size, heuristic crossover and local improvement on a genetic algorithm for the traveling salesman problem. In
*Proceedings on the Third International Conference on Genetic Algorithms*, Arlington, VA, pp. 110–15.Google Scholar - Kjærulff, U. (1990) Triangulation of graphs-Algorithms giving small total state space. Technical Report R 90–09, University of Aalborg, Denmark.Google Scholar
- Kjærulff, U. (1992) Optimal decomposition of probabilistic networks by simulated annealing.
*Statistics and Computing*,**2**, 7–17.Google Scholar - Larrañaga, P., Graña, M., D'Anjou, A. and Torrealdea, F. J. (1993) Genetics algorithms elitist probabilistic of degree 1, a generalization of simulated annealing. In
*Proceedings of the Third Congress of the Italian Association for Artificial Intelligence, IA*AI '93*, Torino, Italy, pp. 208–17.Google Scholar - Larrañaga, P., Kuijpers, C. M. H., Poza, M. and Murga, R. H. (1994) Optimal decomposition of Bayesian networks by genetic algorithms. Internal Report EHU-KZAA-IKT–3–94, University of the Basque Country, Spain.Google Scholar
- Larrañaga, P., Kuijpers, C. M. H., Murga, R. H. and Yurramendi, Y. (1996a) Learning Bayesian network structures by searching for the best ordering with genetic algorithms.
*IEEE Transactions on Systems, Man and Cybernetics*. Vol. 26, No. 4, pp. 487–93.Google Scholar - Larrañaga, P., Poza, M., Yurramendi, Y., Murga, R. H., and Kuijpers, C. M. H. (1996b) Structure learning of Bayesian network by genetic algorithms: a performance analysis of control parameters.
*IEEE Transactions on Pattern Analysis and Machine Intelligence*(to appear).Google Scholar - Larrañaga, P., Murga, R. H., Poza, M. and Kuijpers, C. M. H. (1996c) Structure learning of Bayesian networks by hybrid genetic algorithms. In
*Learning from Data: Artificial Intelligence and Statistics V*, (D. Fisher and H. Lenz, eds), Springer-Verlag, New York, pp. 165–74.Google Scholar - Larrañaga, P., Kuijpers, C. M. H. and Murga, R. H. (1996d) >Evolutionary algorithms for the travelling salesman problem: A review of representations and operators.
*Artificial Intelligence Review*(to appear).Google Scholar - Lauritzen, S. L. and Spiegelhalter, D. J. (1988) Local computations with probabilities on graphical structures and their application to expert systems.
*Journal of the Royal Statistical Society*, Series B,**50**, 157–224.Google Scholar - Lidd, M. L. (1991) The traveling salesman problem domain application of a fundamentally new approach to utilizing genetic algorithms. Technical Report, MITRE Corporation.Google Scholar
- Michalewicz, Z. (1992)
*Genetic Algorithms + Data Structures = Evolution Programs*. Springer-Verlag, Berlin.Google Scholar - Mühlenbein, H. (1989) Parallel genetic algorithms, population genetics and combinatorial optimization. In
*Proceedings on the Third International Conference on Genetic Algorithms*, Arlington, VA, pp. 416–21.Google Scholar - Neapolitan, R. E. (1990)
*Probabilistic Reasoning in Expert Systems, Theory and Algorithms*. Wiley, New York.Google Scholar - Oliver, I. M., Smith, D. J. and Holland, J. R. C. (1987) A study of permutation crossover operators on the TSP. In
*Genetic Algorithms and Their Applications: Proceedings of the Second International Conference*, Cambridge, MA, pp. 224–30.Google Scholar - Pearl, J. (1986) Fusion, propagation, and structuring in belief networks.
*Artificial Intelligence*,**29**, 241–88.Google Scholar - Pearl, J. (1987) Evidential reasoning using stochastic simulation of causal models.
*Artificial Intelligence*,**32**, 245–57.Google Scholar - Pearl, J. (1988)
*Probabilistic Reasoning in Intelligent Systems*. Morgan Kaufmann, Palo Alto, CA.Google Scholar - Rose, D. J., Tarjan, R. E. and Lueker, G. S. (1976) Algorithmic aspects of vertex elimination on graphs.
*SIAM Journal on Computing*,**5**, 266–83.Google Scholar - Rudolph, G. (1994) Convergence analysis of canonical genetic algorithms.
*IEEETransactions on Neural Networks*,**5**, 96–101.Google Scholar - Seniw, D. (1991) A genetic algorithm for the traveling salesman problem. M.Sc. Thesis, University of North Carolina at Charlotte.Google Scholar
- Shachter, R. D. (1988) Probabilistic inference and influence diagrams.
*Operations Research*,**36**, 589–604.Google Scholar - Shachter, R. D. and Peot, M. A. (1990) Simulation approaches to general probabilistic inference on belief networks. In
*Uncertainty in Artificial Intelligence 5*(M. Henrion, R. D. Shachter, L. N. Kanal and J. F. Lemmer, eds), pp. 221–34. Elsevier, Amsterdam.Google Scholar - Shwe, M. and Cooper, G. (1991) An empirical analysis of likelihood-weighting simulation on a large, multiply connected medical belief network.
*Computers and Biomedical Research*,**24**, 453–75.Google Scholar - Suh, J. Y. and Van Gucht, D. (1987) Incorporating heuristic information into genetic search. In
*Genetic Algorithms and Their Applications: Proceedings of the Second International Conference*, Cambridge, MA, pp. 100–7.Google Scholar - Syswerda, G. (1991) Schedule optimization using genetic algorithms. In
*Handbook of Genetic Algorithms*, (L. Davis, ed.), pp. 332–49. Van Nostrand Reinhold, New York.Google Scholar - Tarjan, R. E. and Yannakakis, M. (1984) Simple linear-time algorithms to test chordality of graphs, test acyclicity of hypergraphs, and selectively reduce acyclic hypergraphs.
*SIAM Journal on Computing*,**13**, 566–79.Google Scholar - Wen, W. X. (1990) Decomposing belief networks by simulated annealing. In (C. P. Tsang, ed.)
*Proceedings of the Australian 1990 Joint Conference on Artificial Intelligence*, pp. 103–18. World Scientific Publishers, Perth, WA.Google Scholar - Wen, W. X. (1991) Optimal decomposition of belief networks. In
*Uncertainty in Artificial Intelligence 6*(P. P. Bonissone, M. Henrion, L. N. Kanal, and J. F. Lemmer, eds), pp. 209–24. North-Holland, Amsterdam.Google Scholar - Whitley, D. (1989) The GENITOR algorithm and selection pressure: Why rank-based allocation of reproductive trials is best. In
*Proceedings on the Third International Conference on Genetic Algorithms*, Arlington, VA, pp. 116–21.Google Scholar - Whitley D. and Hanson, T. (1987) Optimizing neural networks using faster, more accurate genetic search. In
*Genetic Algorithms and Their Applications: Proceedings of the Second International Conference*, Cambridge, MA, pp. 391–6.Google Scholar - Whitley, D., Starkweather, T. and Fuquay, D. (1989) Scheduling problems and travelling salesman: The genetic edge recombination operator. In
*Proceedings on the Third International Conference on Genetic Algorithms*, Arlington, VA, pp. 133–40.Google Scholar - Whitley, D., Starkweather, T. and Shaner, D. (1991) The traveling salesman and sequence scheduling: Quality solutions using genetic edge recombination. In
*Handbook of Genetic Algorithms*(L. Davis, ed), pp. 350–72. Van Nostrand Reinhold, New York.Google Scholar