On the Use of a Non-redundant Encoding for Learning Bayesian Networks from Data with a GA

  • Steven van Dijk
  • Dirk Thierens
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3242)


We study the impact of the choice of search space for a GA that learns Bayesian networks from data. The most convenient search space is redundant and therefore allows for multiple representations of the same solution and possibly disruption during crossover. An alternative search space eliminates this redundancy, and potentially allows a more efficient search to be conducted. On the other hand, a non-redundant encoding requires a more complicated implementation. We experimentally compare several plausible approaches (GAs) to study the impact of this and other design decisions.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Chickering, D.M., Meek, C., Heckerman, D.: Large-sample learning of Bayesian networks is NP-hard. In: [24], pp. 124–133Google Scholar
  2. 2.
    van Dijk, S., Thierens, D., van der Gaag, L.C.: Building a GA from design principles for learning Bayesian networks. In: Cantú-Paz, E., Foster, J.A., Deb, K., Davis, L., Roy, R., O’Reilly, U.-M., Beyer, H.-G., Kendall, G., Wilson, S.W., Harman, M., Wegener, J., Dasgupta, D., Potter, M.A., Schultz, A., Dowsland, K.A., Jonoska, N., Miller, J., Standish, R.K. (eds.) GECCO 2003. LNCS, vol. 2723, pp. 886–897. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  3. 3.
    Castelo, R., Kočka, T.: On inclusion-driven learning of Bayesian networks. Journal of Machine Learning Research 4, 527–574 (2003)CrossRefGoogle Scholar
  4. 4.
    Chickering, D.M.: Optimal structure identification with greedy search. Journal of Machine Learning Research 3, 507–554 (2002)CrossRefMathSciNetGoogle Scholar
  5. 5.
    Heckerman, D., Geiger, D., Chickering, D.M.: Learning Bayesian networks: The combination of knowledge and statistical data. Machine Learning 20, 197–243 (1995)MATHGoogle Scholar
  6. 6.
    Lam, W., Bacchus, F.: Learning Bayesian belief networks. an approach based on the MDL principle. Computational Intelligence 10, 269–293 (1994)CrossRefGoogle Scholar
  7. 7.
    Larrañaga, P., Poza, M., Yurramendi, Y., Murga, R., Kuijpers, C.: Structure learning of Bayesian networks by genetic algorithms: A performance analysis of control parameters. IEEE Trans. on Pattern Analysis and Machine Intelligence 18, 912–926 (1996)CrossRefGoogle Scholar
  8. 8.
    Spirtes, P., Meek, C.: Learning Bayesian networks with discrete variables from data. In: Fayyad, U.M., Uthurusamy, R. (eds.) Proc. of the First Int. Conf. on Knowledge Discovery and Data Mining, pp. 294–299. AAAI Press, Menlo Park (1995)Google Scholar
  9. 9.
    Chickering, D.M.: Learning equivalence classes of Bayesian-network structures. Journal of Machine Learning Research 2, 445–498 (2002)MATHCrossRefMathSciNetGoogle Scholar
  10. 10.
    Nielsen, J.D., Kocka, T., Pena, J.M.: On local optima in learning Bayesian networks. In: [24]Google Scholar
  11. 11.
    Acid, S., de Campos, L.M.: Searching for Bayesian network structures in the space of restricted acyclic partially directed graphs. Journal of Artificial Intelligence Research 18, 445–490 (2003)MATHMathSciNetGoogle Scholar
  12. 12.
    Jones, T., Forrest, S.: Fitness distance correlation as a measure of problem difficulty for genetic algorithms. In: Eshelman, L.J. (ed.) Proc. of the 6th Int. Conf. on Genetic Algorithms, pp. 184–192. Morgan-Kaufmann, San Francisco (1995)Google Scholar
  13. 13.
    Altenberg, L.: Fitness distance correlation analysis: An instructive counterexample. In: Bäck, T. (ed.) Proc. of the 7th Int. Conf. on Genetic Algorithms, pp. 57–64. Morgan-Kaufmann, San Francisco (1997)Google Scholar
  14. 14.
    Smith, T., Husbands, P., Layzell, P., O’Shea, M.: Fitness landscapes and evolvability. Evolutionary Computation 10, 1–34 (2002)CrossRefGoogle Scholar
  15. 15.
    Manderick, B., de Weger, M., Spiessens, P.: The genetic algorithm and the structure of the fitness landscape. In: Belew, R., Booker, L. (eds.) Proc. of the Fourth Int. Conf. on Genetic Algorithms and their Applications, Morgan-Kaufmann, San Francisco (1991)Google Scholar
  16. 16.
    Thierens, D.: Non-redundant genetic coding of neural networks. In: Proc. of the IEEE Int. Conf. on Evolutionary Computation, pp. 571–575. IEEE Press, Los Alamitos (1996)CrossRefGoogle Scholar
  17. 17.
    Naudts, B., Kallel, L.: A comparison of predictive measures of problem difficulty in evolutionary algorithms. IEEE Trans. on Evolutionary Computation 4, 1 (2000)CrossRefGoogle Scholar
  18. 18.
    Rothlauf, F., Goldberg, D.E.: Redundant representations in evolutionary computation. Evolutionary Computation 11, 381–415 (2003)CrossRefGoogle Scholar
  19. 19.
    Pearl, J.: Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann, San Francisco (1988)Google Scholar
  20. 20.
    van Dijk, S., van der Gaag, L.C., Thierens, D.: A skeleton-based approach to learning Bayesian networks from data. In: Lavrač, N., Gamberger, D., Todorovski, L., Blockeel, H. (eds.) PKDD 2003. LNCS (LNAI), vol. 2838, pp. 132–143. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  21. 21.
    van Dijk, S., Thierens, D., de Berg, M.: On the design and analysis of competent selectorecombinative GAs. Evolutionary Computation 12 (2004)Google Scholar
  22. 22.
    van der Gaag, L.C., Renooij, S., Witteman, C., Aleman, B., Taal, B.: Probabilities for a probabilistic network: A case-study in oesophageal cancer. Artificial Intelligence in Medicine 25, 123–148 (2002)CrossRefGoogle Scholar
  23. 23.
    Peek, N., Ottenkamp, J.: Developing a decision-theoretic network for a congenital heart disease. In: Keravnou, E., et al. (eds.) Proc. of the 6th European Conf. on Artificial Intelligence in Medicine, pp. 157–168. Springer, Heidelberg (1997)CrossRefGoogle Scholar
  24. 24.
    Meek, C., Kjærulff, U. (eds.): Proc. of the 19th Conf. on Uncertainty in Artificial Intelligence. Morgan-Kaufmann, San Francisco (2003)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2004

Authors and Affiliations

  • Steven van Dijk
    • 1
  • Dirk Thierens
    • 1
  1. 1.Institute of Information and Computing SciencesUtrecht UniversityUtrechtThe Netherlands

Personalised recommendations