Constrained Score+(Local)Search Methods for Learning Bayesian Networks

  • José A. Gámez
  • J. Miguel Puerta
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3571)


The dominant approach for learning Bayesian networks from data is based on the use of a scoring metric, that evaluates the fitness of any given candidate network to the data, and a search procedure, that explores the space of possible solutions. The most used method inside this family is (iterated) hill climbing, because its good trade-off between CPU requirements, accuracy of the obtained model, and ease of implemetation. In this paper we focus on the searh space of dags and in the use of hill climbing as search engine. Our proposal consists in the reduction of the candidate dags or neighbors to be explored at each iteration, making the method more efficient on CPU time, but without decreasing the quality of the model discovered. Thus, initially the parent set for each variable is not restricted and so all the neighbors are explored, but during this exploration we take advantage of locally consistent metrics properties and remove some nodes from the set of candidate parents, constraining in this way the process for subsequent iterations. We show the benefits of our proposal by carrying out several experiments in three different domains.


Local Search Bayesian Network Hill Climbing Local Search Method Directed Cycle 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Acid, S., de Campos, L.M.: A hybrid methodology for learning belief networks: Benedict. International. Journal of Approximate Reasoning 27(3), 235–262 (2001)zbMATHCrossRefGoogle Scholar
  2. 2.
    Acid, S., de Campos, L.M.: Searching for Bayesian Network Structures in the Space of Restricted Acyclic Partially Directed Graphs. Journal of Artificial Intelligence Research 18, 445–490 (2003)zbMATHMathSciNetGoogle Scholar
  3. 3.
    Beinlich, I.A., Suermondt, H.J., Chavez, R.M., Cooper, G.F.: The ALARM monitoring system: A case study with two probabilistic inference techniques for belief networks. In: Proc. of the 2nd European Conf. on Artificial Intelligence in Medicine, pp. 247–256 (1989)Google Scholar
  4. 4.
    Binder, J., Koller, D., Russell, S., Kanazawa, K.: Adaptive probabilistic networks with hidden variables. Machine Learning 29(2), 213–244 (1997)zbMATHCrossRefGoogle Scholar
  5. 5.
    Buntine, W.: Theory refinement of Bayesian networks. In: Proc. of the 7th Conf. on Uncertainty in Artificial Intelligence, pp. 52–60 (1991)Google Scholar
  6. 6.
    de Campos, L.M., Fernández-Luna, J.M., Gámez, J.A., Puerta, J.M.: Ant colony optimization for learning Bayesian networks. International Journal of Approximate Reasoning 31, 291–311 (2002)zbMATHCrossRefMathSciNetGoogle Scholar
  7. 7.
    de Campos, L.M., Puerta, J.M.: Stochastic local and distributed search algorithms for learning belief networks. In: Proc. of 3rd Int. Symp. on Adaptive Systems: Evolutionary Computation and Probabilistic Graphical Model, pp. 109–115 (2001)Google Scholar
  8. 8.
    de Campos, L.M., Puerta, J.M.: Stochastic local search algorithms for learning belief networks: Searching in the space of orderings. In: Benferhat, S., Besnard, P. (eds.) ECSQARU 2001. LNCS (LNAI), vol. 2143, pp. 228–239. Springer, Heidelberg (2001)CrossRefGoogle Scholar
  9. 9.
    Chickering, D.M.: Optimal structure identification with greedy search. Journal of Machine Learning Research 3, 507–554 (2002)CrossRefMathSciNetGoogle Scholar
  10. 10.
    Chickering, D.M., Geiger, D., Heckerman, D.: Learning Bayesian networks is NP-Complete. In: Fisher, D., Lenz, H. (eds.) Learning from Data: Artificial Intelligence and Statistics V, pp. 121–130. Springer, Heidelberg (1996)Google Scholar
  11. 11.
    Friedman, N., Nachman, I., Peér, D.: Learning Bayesian networks from massive datasets: The ”sparse candidate” algorithm. In: Proc. of the 15th Conf. on Uncertainty in Artificial Intelligence, pp. 201–210 (1999)Google Scholar
  12. 12.
    Heckerman, D., Geiger, D., Chickering, D.M.: Learning Bayesian networks: The combination of knowledge and statistical data. Machine Learning 20, 197–244 (1995)zbMATHGoogle Scholar
  13. 13.
    Jensen, F.V.: Bayesian Networks and Decision Graphs. Springer, Heidelberg (2001)zbMATHGoogle Scholar
  14. 14.
    Neapolitan, R.: Learning Bayesian Networks. Prentice-Hall, Englewood Cliffs (2003)Google Scholar
  15. 15.
    Pearl, J.: Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann, San Mateo (1988)Google Scholar
  16. 16.
    Spirtes, P., Glymour, C., Scheines, R.: Causation, Prediction, and Search. Lecture Notes in Statistics, vol. 81. Springer, Heidelberg (1993)zbMATHGoogle Scholar
  17. 17.
    Verma, T., Pearl, J.: Equivalence and synthesis of causal models. In: Proc. of the 6th Conf. on Uncertainty in Artificial Intelligence, pp. 220–227 (1991)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2005

Authors and Affiliations

  • José A. Gámez
    • 1
  • J. Miguel Puerta
    • 1
  1. 1.Dpto. de Informática and SIMD-i3AUniversidad de Castilla-La ManchaAlbaceteSpain

Personalised recommendations