Advertisement

The Impact of Exact Probabilistic Learning Algorithms in EDAs Based on Bayesian Networks

  • Carlos Echegoyen
  • Roberto Santana
  • Jose A. Lozano
  • Pedro Larrañaga
Part of the Studies in Computational Intelligence book series (SCI, volume 157)

Summary

This paper discusses exact learning of Bayesian networks in estimation of distribution algorithms. The estimation of Bayesian network algorithm (EBNA) is used to analyze the impact of learning the optimal (exact) structure in the search. By applying recently introduced methods that allow learning optimal Bayesian networks, we investigate two important issues in EDAs. First, we analyze the question of whether learning more accurate (exact) models of the dependencies implies a better performance of EDAs. Secondly, we are able to study the way in which the problem structure is translated into the probabilistic model when exact learning is accomplished. The results obtained reveal that the quality of the problem information captured by the probability model can improve when the accuracy of the learning algorithm employed is increased. However, improvements in model accuracy do not always imply a more efficient search.

Keywords

Bayesian Network Bayesian Information Criterion Evolutionary Computation Frequency Matrice Distribution Algorithm 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Baluja, S., Davies, S.: Using optimal dependency-trees for combinatorial optimization: Learning the structure of the search space. In: Proceedings of the 14th International Conference on Machine Learning, pp. 30–38. Morgan Kaufmann, San Francisco (1997)Google Scholar
  2. 2.
    Bengoetxea, E.: Inexact Graph Matching Using Estimation of Distribution Algorithms. PhD thesis, Ecole Nationale Supérieure des Télécommunications (2003)Google Scholar
  3. 3.
    Blanco, R., Lozano, J.A.: Estimation of Distribution Algorithms. A New Tool for Evolutionary Computation. In: Larrañaga, P., Lozano, J.A. (eds.) An empirical comparison of discrete Estimation of Distribution Algorithms, pp. 163–176. Kluwer Academic Publishers, Dordrecht (2002)Google Scholar
  4. 4.
    Buntine, W.: Theory refinement on Bayesian networks. In: Proceedings of the Seventh Conference on Uncertainty in Artificial Intelligence, pp. 52–60 (1991)Google Scholar
  5. 5.
    Castillo, E., Gutierrez, J.M., Hadi, A.S.: Expert Systems and Probabilistic Network Models. Springer, Heidelberg (1997)Google Scholar
  6. 6.
    Chickering, D.M., Geiger, D., Heckerman, D.: Learning Bayesian networks is NP-hard. Technical Report MSR-TR-94-17, Microsoft Research, Redmond, WA (1994)Google Scholar
  7. 7.
    Chuang, C.Y., Chen, Y.P.: On the effectiveness of distribution estimated by probabilistic model building. Technical Report: NCL-TR-2008001, Natural Computing Laboratory NCLab. Department of Computer Science. National Chiao Tung University (February 2008)Google Scholar
  8. 8.
    Coffin, D.J., Smith, R.E.: The limitations of distribution sampling for linkage learning. In: Proceedings of the 2007 Congress on Evolutionary Computation CEC 2007, pp. 364–369. IEEE Press, Los Alamitos (2007)Google Scholar
  9. 9.
    Cotta, C.: Protein structure prediction using evolutionary algorithms hybridized with backtracking. In: Mira, J., Álvarez, J.R. (eds.) IWANN 2003. LNCS, vol. 2687, pp. 321–328. Springer, Heidelberg (2003)Google Scholar
  10. 10.
    Cutello, V., Nicosia, G., Pavone, M., Timmis, J.: An immune algorithm for protein structure prediction on lattice models. IEEE Transactions on Evolutionary Computation 11(1), 101–117 (2007)CrossRefGoogle Scholar
  11. 11.
    Dill, K.A.: Theory for the folding and stability of globular proteins. Biochemistry 24(6), 1501–1509 (1985)CrossRefGoogle Scholar
  12. 12.
    Eaton, D., Murphy, K.: Exact Bayesian structure learning from uncertain interventions. In: Proceedings of the Eleventh International Conference on Artificial Intelligence and Statistics (2007)Google Scholar
  13. 13.
    Echegoyen, C., Lozano, J.A., Santana, R., Larrañaga, P.: Exact Bayesian network learning in estimation of distribution algorithms. In: Proceedings of the 2007 Congress on Evolutionary Computation CEC 2007, pp. 1051–1058. IEEE Press, Los Alamitos (2007)Google Scholar
  14. 14.
    Etxeberria, R., Larrañaga, P.: Global optimization using Bayesian networks. In: Ochoa, A., Soto, M.R., Santana, R. (eds.) Proceedings of the Second Symposium on Artificial Intelligence (CIMAF 1999), Havana, Cuba, pp. 151–173 (1999)Google Scholar
  15. 15.
    Goldberg, D.E.: Genetic Algorithms in Search, Optimization, and Machine Learning. Addison-Wesley, Reading (1989)zbMATHGoogle Scholar
  16. 16.
    Hauschild, M., Pelikan, M., Lima, C., Sastry, K.: Analyzing probabilistic models in hierarchical BOA on traps and spin glasses. In: Thierens, D., et al. (eds.) Proceedings of the Genetic and Evolutionary Computation Conference GECCO 2007, vol. I, pp. 523–530. ACM Press, London (2007)CrossRefGoogle Scholar
  17. 17.
    Hauschild, M., Pelikan, M., Sastry, K., Goldberg, D.E.: Using previous models to bias structural learning in the hierarchical BOA. MEDAL Report No. 2008003, Missouri Estimation of Distribution Algorithms Laboratory (MEDAL) (2008)Google Scholar
  18. 18.
    Heckerman, D., Geiger, D., Chickering, D.M.: Learning Bayesian networks: The combination of knowledge and statistical data. Machine Learning 20, 197–243 (1995)zbMATHGoogle Scholar
  19. 19.
    Inza, I., Larrañaga, P., Sierra, B.: Estimation of Distribution Algorithms. A New Tool for Evolutionary Computation. In: Larrañaga, P., Lozano, J.A. (eds.) Feature weighting for nearest neighbor by estimation of distribution algorithms, pp. 291–308. Kluwer Academic Publishers, Dordrecht (2002)Google Scholar
  20. 20.
    Koivisto, M., Sood, K.: Exact Bayesian structure discovery in Bayesian networks. Journal of Machine Learning Research 5, 549–573 (2004)MathSciNetGoogle Scholar
  21. 21.
    Larrañaga, P., Lozano, J.A. (eds.): Estimation of Distribution Algorithms. A New Tool for Evolutionary Computation. Kluwer Academic Publishers, Dordrecht (2002)zbMATHGoogle Scholar
  22. 22.
    Lima, C.F., Pelikan, M., Goldberg, D.E., Lobo, F.G., Sastry, K., Hauschild, M.: Influence of selection and replacement strategies on linkage learning in BOA. In: Proceedings of the 2007 Congress on Evolutionary Computation CEC 2007, pp. 1083–1090. IEEE Press, Los Alamitos (2007)Google Scholar
  23. 23.
    Lozano, J.A., Larrañaga, P., Inza, I., Bengoetxea, E. (eds.): Towards a New Evolutionary Computation: Advances on Estimation of Distribution Algorithms. Springer, Heidelberg (2006)zbMATHGoogle Scholar
  24. 24.
    Lozano, J.A., Sagarna, R., Larrañaga, P.: Parallel estimation of Bayesian networks algorithms. In: Evolutionary Computation and Probabilistic Graphical Models. Proceedings of the Third Symposium on Adaptive Systems (ISAS 2001), Havana, Cuba, pp. 137–144 (March 2001)Google Scholar
  25. 25.
    Mendiburu, A., Lozano, J., Miguel-Alonso, J.: Parallel implementation of EDAs based on probabilistic graphical models. IEEE Transactions on Evolutionary Computation 9(4), 406–423 (2005)CrossRefGoogle Scholar
  26. 26.
    Mendiburu, A., Santana, R., Lozano, J.A.: Introducing belief propagation in estimation of distribution algorithms: A parallel framework. Technical Report EHU-KAT-IK-11/07, Department of Computer Science and Artificial Intelligence, University of the Basque Country (October 2007)Google Scholar
  27. 27.
    Mühlenbein, H., Höns, R.: The estimation of distributions and the minimum relative entropy principle. Evolutionary Computation 13(1), 1–27 (2005)CrossRefGoogle Scholar
  28. 28.
    Mühlenbein, H., Mahnig, T.: Evolutionary synthesis of Bayesian networks for optimization. In: Patel, M., Honavar, V., Balakrishnan, K. (eds.) Advances in Evolutionary Synthesis of Intelligent Agents, pp. 429–455. MIT Press, Cambridge (2001)Google Scholar
  29. 29.
    Mühlenbein, H., Mahnig, T., Ochoa, A.: Schemata, distributions and graphical models in evolutionary optimization. Journal of Heuristics 5(2), 213–247 (1999)CrossRefGoogle Scholar
  30. 30.
    Mühlenbein, H., Paaß, G.: From recombination of genes to the estimation of distributions I. Binary parameters. In: Ebeling, W., Rechenberg, I., Voigt, H.-M., Schwefel, H.-P. (eds.) PPSN 1996. LNCS, vol. 1141, pp. 178–187. Springer, Heidelberg (1996)CrossRefGoogle Scholar
  31. 31.
    Ochoa, A., Soto, M.R.: Linking entropy to estimation of distribution algorithms. In: Lozano, J.A., Larrañaga, P., Inza, I., Bengoetxea, E. (eds.) Towards a New Evolutionary Computation: Advances on Estimation of Distribution Algorithms, pp. 1–38. Springer, Heidelberg (2006)Google Scholar
  32. 32.
    Pelikan, M.: Hierarchical Bayesian Optimization Algorithm. Toward a New Generation of Evolutionary Algorithms. Studies in Fuzziness and Soft Computing. Springer, Heidelberg (2005)zbMATHGoogle Scholar
  33. 33.
    Pelikan, M., Goldberg, D.E., Cantú-Paz, E.: BOA: The Bayesian optimization algorithm. In: Banzhaf, W., Daida, J., Eiben, A.E., Garzon, M.H., Honavar, V., Jakiela, M., Smith, R.E. (eds.) Proceedings of the Genetic and Evolutionary Computation Conference GECCO 1999, Orlando, FL, vol. I, pp. 525–532. Morgan Kaufmann Publishers, San Francisco (1999)Google Scholar
  34. 34.
    Santana, R.: Estimation of distribution algorithms with Kikuchi approximations. Evolutionary Computation 13(1), 67–97 (2005)CrossRefGoogle Scholar
  35. 35.
    Santana, R., Larrañaga, P., Lozano, J.A.: Protein folding in 2-dimensional lattices with estimation of distribution algorithms. In: Barreiro, J.M., Martín-Sánchez, F., Maojo, V., Sanz, F. (eds.) ISBMDA 2004. LNCS, vol. 3337, pp. 388–398. Springer, Heidelberg (2004)Google Scholar
  36. 36.
    Santana, R., Larrañaga, P., Lozano, J.A.: Interactions and dependencies in estimation of distribution algorithms. In: Proceedings of the 2005 Congress on Evolutionary Computation CEC 2005, Edinburgh, U.K, pp. 1418–1425. IEEE Press, Los Alamitos (2005)Google Scholar
  37. 37.
    Santana, R., Larrañaga, P., Lozano, J.A.: The role of a priori information in the minimization of contact potentials by means of estimation of distribution algorithms. In: Marchiori, E., Moore, J.H., Rajapakse, J.C. (eds.) EvoBIO 2007. LNCS, vol. 4447, pp. 247–257. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  38. 38.
    Santana, R., Larrañaga, P., Lozano, J.A.: Component weighting functions for adaptive search with EDAs. In: Proceedings of the 2008 Congress on Evolutionary Computation CEC 2008, Hong Kong. IEEE Press, Los Alamitos (accepted for publication, 2008)Google Scholar
  39. 39.
    Santana, R., Larrañaga, P., Lozano, J.A.: Estimation of distribution algorithms with affinity propagation methods. Technical Report EHU-KZAA-IK-1/08, Department of Computer Science and Artificial Intelligence, University of the Basque Country (January 2008), http://www.sc.ehu.es/ccwbayes/technical.htm
  40. 40.
    Santana, R., Larrañaga, P., Lozano, J.A.: Protein folding in simplified models with estimation of distribution algorithms. IEEE Transactions on Evolutionary Computation (to appear, 2008)Google Scholar
  41. 41.
    Schwarz, G.: Estimating the dimension of a model. Annals of Statistics 7(2), 461–464 (1978)CrossRefGoogle Scholar
  42. 42.
    Silander, T., Myllymaki, P.: A simple approach for finding the globally optimal Bayesian network structure. In: Proceedings of the 22th Annual Conference on Uncertainty in Artificial Intelligence (UAI 2006). Morgan Kaufmann Publishers, San Francisco (2006)Google Scholar
  43. 43.
    Singh, A., Moore, A.: Finding optimal Bayesian networks by dynamic programming. Technical report, Carnegie Mellon University (June 2005)Google Scholar
  44. 44.
    Soto, M.R., Ochoa, A.: A factorized distribution algorithm based on polytrees. In: Proceedings of the 2000 Congress on Evolutionary Computation CEC 2000, La Jolla Marriott Hotel La Jolla, California, USA, July 6-9, pp. 232–237. IEEE Press, Los Alamitos (2000)CrossRefGoogle Scholar
  45. 45.
    Soto, M.R., Ochoa, A., Acid, S., Campos, L.M.: Bayesian evolutionary algorithms based on simplified models. In: Ochoa, A., Soto, M.R., Santana, R. (eds.) Proceedings of the Second Symposium on Artificial Intelligence (CIMAF 1999), Havana, Cuba, pp. 360–367 (March 1999)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Carlos Echegoyen
    • 1
  • Roberto Santana
    • 1
  • Jose A. Lozano
    • 1
  • Pedro Larrañaga
    • 2
  1. 1.Intelligent Systems Group Department of Computer Science and Artificial IntelligenceUniversity of the Basque CountryDonostia - San SebastianSpain
  2. 2.Department of Artificial IntelligenceTechnical University of MadridMadridSpain

Personalised recommendations