Skip to main content
SpringerLink
Log in

Learning Bayesian Networks by Floating Search Methods

  • Chapter
Book cover Advances in Bayesian Networks

Part of the book series: Studies in Fuzziness and Soft Computing ((STUDFUZZ,volume 146))

  • 954 Accesses

Abstract

In this work, two novel sequential algorithms for learning Bayesian networks are proposed. The presented sequential search methods are an adaptation of a pair of algorithms proposed to feature subset selection: Sequential Forward Floating Selection and Sequential Backward Floating Selection. As far as we know, these algorithms have never been used for learning Bayesian networks. An empirical comparison among the results of the proposed algorithms and the results of two sequential algorithm (the classical B-algorithm and its extension, the B3 algorithm) is carried out over four databases from literature. The results show promising results for the floating approach to the learning Bayesian network problem.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. B. Abramson, J. M. Brown, W. Edwards, A. Murphy, and R. L. Winkler, ‘Hailfinder: A Bayesian system for forecasting severe weather’, International Journal of Forecasting, 12, 57–71, (1996) .

    Article  Google Scholar 

  2. S. Acid, L. M. de Campos, and J. F. Huete, ‘The search of causal orderings: a short cut for learning belief networks’, in Proceedings of the Sixth European Conference on Symbolic and Quantitative Approache to Reasoning with Uncertainty, pp. 228–239, (2001) .

    Google Scholar 

  3. I.A. Beinlinch, H.J. Suermondt, R.M. Chavez, and G.F. Coooper, ‘The ALARM monitoring system: a case study with two probabilistic inference techniques for belief networks’, in Proceedings of the Second European Conference on Artificial Intelligence in Medicine, pp. 247–257, (1989) .

    Google Scholar 

  4. J. Binder, D. Koller, S. Russell, and K. Kanazawa, ‘Adaptive probabilistic networks with hidden variables’, Machine Learning, 29(2–3), 213–244, (1997).

    Article  MATH  Google Scholar 

  5. R. Blanco, I. Inza, and P. Larrañaga, ‘Learning Bayesian networks in the space of structures by estimation of distribution algorithms’, International Journal of Intelligent Systems, 18, 205–220, (2003).

    Article  MATH  Google Scholar 

  6. C. Borgelt and R. Kruse, ‘An empirical investigation of the K2 metric’, in Proceedings of the Sixth European Conference on Symbolic and Quantitative Approache to Reasoning with Uncertainty, pp. 240–251, (2001).

    Chapter  Google Scholar 

  7. W. Buntine, ‘Theory refinement in Bayesian networks’, in Proceedings of the Seventh Conference on Uncertainty in Artificial Intelligence, pp. 52–60, (1991) .

    Google Scholar 

  8. W. Buntine, ‘A guide to the literature on learning probabilistic networks from data’, IEEE Transactions on Knowledge and Data Engineering, 8(2), 195–210, (1996).

    Article  Google Scholar 

  9. E. Castillo, J.M. Gutiérrez, and A.S. Hadi, Expert Systems and Probabilistic Network Models, Springer-Verlag, New York, 1997.

    Book  Google Scholar 

  10. J. Cheng, R. Greiner, J. Kelly, D. Bell, and W. Liu, ‘Learning Bayesian networks from data: an information-theory based approach’, Artificial Intelligence, 137(1–2), 43–90, (2002).

    Article  MathSciNet  MATH  Google Scholar 

  11. D.M. Chickering, ‘Learning equivalence classes of Bayesian network structures’, Journal of Machine Learning Research, 2, 445–498, (2002) .

    MathSciNet  MATH  Google Scholar 

  12. D.M. Chickering, D. Geiger, and D. Heckerman, ‘Learning Bayesian networks is NP-hard’, Technical Report MSR-TR-94–17, Microsoft Research, Advanced Technology Division, Microsoft Corporation, Redmond, WA, (1994) .

    Google Scholar 

  13. D.M. Chickering, D. Geiger, and D. Heckerman, ‘Learning Bayesian networks: Search methods and experimental results’, in Preliminary Papers of the Fifth International Workshop on Artificial Intelligence and Statistics, pp. 112–128, (1995) .

    Google Scholar 

  14. G.F. Cooper and E.A. Herskovits, ‘A Bayesian method for the induction of probabilistic networks from data’, Machine Learning, 9, 309–347, (1992) .

    Google Scholar 

  15. C. Cotta and J. Muruzábal, ‘Towards more efficient evolutionary induction of Bayesian networks’, in Parallel Problem Solving From Nature VIII. Lecture Notes in Computer Science 2439, pp. 730–739, (2002) .

    Google Scholar 

  16. R.G. Cowell, ‘Conditions under which conditional independence and scoring methods lead to identical selection of Bayesian network models’, in Proceedings of the Seventeenth Conference on Uncertainty in Artificial Intelligence, pp. 91–97, (2001) .

    Google Scholar 

  17. A.P. Dawid, ‘Conditional independence in statistical theory’, Journal of the Royal Statistics Society, Series B, 41, 1–31, (1979) .

    MathSciNet  MATH  Google Scholar 

  18. L.M. de Campos, J.M. Fernández-Luna, J.J. Gámez, and J.M. Puerta, ‘Ant colony optimization for learning Bayesian networks’, International Journal on Artificial Reasoning, 31(3), 109–136, (2002) .

    Google Scholar 

  19. L.M. de Campos and J.F. Huete, ‘A new approach for learning belief networks using independence criteria’, International Journal of Approximate Reasoning, 24(1), 11–37, (2000).

    Article  MathSciNet  MATH  Google Scholar 

  20. L.M. de Campos and J.M. Puerta, ‘Stochastic local algorithms for learning belief networks: searching in the space of the orderings, symbolic and quantitative approaches to reasoning with uncertainty’, Lecture Notes in Artificial Intelligence 2143, 228–239, (2001).

    Google Scholar 

  21. N. Friedman, M. Goldszmidt, and A. Wyner, ‘On the application of the bootstrap for computing confidence measures on features of induced Bayesian networks’, in Proceedings of the Seventh International Workshop on Artificial Intelligence and Statistics, (1999) .

    Google Scholar 

  22. N. Friedman and D. Koller, ‘Being Bayesian about network structure: A Bayesian approach to structure discovery in Bayesian networks’, Machine Learning, 50(1–2), 95–125, (2003).

    Article  MATH  Google Scholar 

  23. S. Gillispie and M. Perlman, ‘Enumerating Markov equivalence classes of acyclic digraph models’, in Proceedings of the Seventeenth Conference on Uncertainty in Artificial Intelligence, pp. 171–177, (2001).

    Google Scholar 

  24. J. Habrant, ‘Structure learning of Bayesian networks from databases by genetic algorithms’, in Proceedings of the International Conference on Enterprise Information Systems, (1999) .

    Google Scholar 

  25. D. Heckerman, ‘A tutorial on learning with Bayesian networks’, Technical Report MSR-TR-95–06, Microsoft Advanced Technology Division, Microsoft Corporation, Seattle, Washington, (1995) .

    Google Scholar 

  26. D. Heckerman and D. Geiger, ‘Likelihoods and parameters priors for Bayesian networks’, Technical Report MST-TR-95–54, Microsoft Advanced Technology Division, Microsoft Corporation, Seattle, Washington, (1995) .

    Google Scholar 

  27. D. Heckerman, D. Geiger, and D.M. Chickering, ‘Learning Bayesian networks: The combination of knowledge and statistical data’, Machine Learning, 20, 197–243, (1995) .

    MATH  Google Scholar 

  28. E.A. Herskovits and G.F. Cooper, ‘Kutató: An entropy-driven system for construction of probabilistic expert systems from database’, in Proceedings of the Sixth Conference on Uncertainty in Artificial Intelligence, pp. 54–62, (1990).

    Google Scholar 

  29. A.L. Jensen and F.V. Jensen, ‘MIDAS — An influence diagram for management of mildew in winther wheat’, in Proceedings of the Twelfeth Conference on Uncertainty in Artificial Intelligence, pp. 349–356, San Francisco, (1996) . Morgan Kaufmann.

    Google Scholar 

  30. F.V. Jensen, Bayesian Networks and Decision Graphs, Springer Verlag, 2001.

    MATH  Google Scholar 

  31. J. Kittler, ‘Feature set search algorithms’, in Pattern Recognition and Signal Processing, ed., C.H. Chen, pp. 41–60. Sithoff and Noordhoff, (1978) .

    Chapter  Google Scholar 

  32. T. Kočka and R. Castelo, ‘Improved learning of Bayesian networks’, in Proceedings of the Seventeenth Conference on Uncertainty in Artificial Intelligence, pp. 269–276, (2001).

    Google Scholar 

  33. P. Larrañaga, C.M.H. Kuijpers, R.H. Murga, and Y. Yurramendi, ‘Learning Bayesian network structures by searching for the best ordering with genetic algorithms’, IEEE Transactions on Systems, Man, and Cybernetics — Part A: Systems and Humans, 26(4), 487–493, (1996) .

    Article  Google Scholar 

  34. P. Larrañaga, M. Poza, Y. Yurramendi, R.H. Murga, and C.M.H. Kuijpers, ‘Structure learning of Bayesian networks by genetic algorithms: A performance analysis of control parameters’, IEEE Transactions on Pattern Analysis and Machine Intelligence, 18(9), 912–926, (1996).

    Article  Google Scholar 

  35. H.B. Mann and D.R. Whitney, ‘On a test of whether one of two random variables is stochastically larger than the other’, Annals of Mathematical Statistics, 18, 50–60, (1947) .

    Article  MathSciNet  MATH  Google Scholar 

  36. J.W. Myers, K.B. Laskey, and T. Levitt, ‘Learning Bayesian networks from incomplete data with stochastic search algorithms’, in Proceedings of the Fifteenth Conference on Uncertainty in Artificial Intelligence, pp. 476–485, (1999).

    Google Scholar 

  37. J. Pearl, Probabilistic Reasoning in Intelligent Systems, Morgan Kaufmann, 1988.

    Google Scholar 

  38. J.M. Peña, J.A. Lozano, and P. Larrañaga, ‘Unsupervised learning of Bayesian networks via estimation of distribution algorithms’, in Proceedings of the First European Workshop on Probabilistics Graphical Models, pp. 144–151, (2002) .

    Google Scholar 

  39. P. Pudil, J. Novovicova, and J. Kittler, ‘Floating search methods in feature selection’, Pattern Recognition Letters, 15(1), 1119–1125, (1994).

    Article  Google Scholar 

  40. R.W. Robinson, ‘Counting unlabelled acyclic digraphs’, in Lecture Notes in Mathematics: Combinatorial Mathematics V, pp. 28–43. Springer-Verlag, (1977) .

    Google Scholar 

  41. G. Schwarz, ‘Estimating the dimension of a model’, Annals of Statistics, 7(2), 461–464, (1978) .

    Article  Google Scholar 

  42. C.E. Shannon, ‘A mathematical theory of communication’, The Bell System Technical Journal, 27, 379–423, (1948) .

    MathSciNet  MATH  Google Scholar 

  43. P. Spirtes, C. Glymour, and R. Scheines, Causation, Prediction, and Search, Lecture Notes in Statistics 81, Springer-Verlag, 1993.

    Book  MATH  Google Scholar 

  44. S.D. Stearns, ‘On selecting features for pattern classifiers’, in Third International Conference on Pattern Recognition, pp. 71–75, (1976).

    Google Scholar 

  45. H. Steck, ‘On the use of skeletons when learning in Bayesian networks’, in Proceedings of the Sixteenth Conference on Uncertainty in Artificial Intelligence, pp. 558–565, (2000).

    Google Scholar 

  46. J. Tian, ‘A branch and bound algorithm for MDL learning Bayesian networks’, in Proceedings of the Sixteenth Conference on Uncertainty in Artificial Intelligence, pp. 580–588, (2000).

    Google Scholar 

  47. G. Wahba, ‘Comments on ’monotone regression splines in action’ , Statistical Science, 3, 456–458, (1988) .

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Intelligent System Group, Deparment of Computer Science and Artificial Intelligence, University of Basque Country, P.O. Box 649, 20080, San Sebastián - Donostia, Spain

    Rosa Blanco, Iñaki Inza & Pedro Larrañaga

Authors
  1. Rosa Blanco
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Iñaki Inza
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Pedro Larrañaga
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Escuela Politecnica Superior, Depto. Informatica, Universidad de Castilla-La Mancha, Campus Universitario s/n, 02071, Albacete, Spain

    José A. Gámez

  2. ETS Ingenieria Informatica, Depto. Ciencias Computacion, Universidad de Granada, Campus Universitario Fuentenueva, 18071, Granada, Spain

    Serafín Moral

  3. ETS Ingenieria Informatica, Depto. Estadistica y Matemática Aplicada, Universidad de Almeria, La Cañada de San Urbano s/n, 04120, Almeria, Spain

    Antonio Salmerón

Rights and permissions

Reprints and permissions

Copyright information

© 2004 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Blanco, R., Inza, I., Larrañaga, P. (2004). Learning Bayesian Networks by Floating Search Methods. In: Gámez, J.A., Moral, S., Salmerón, A. (eds) Advances in Bayesian Networks. Studies in Fuzziness and Soft Computing, vol 146. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-39879-0_10

Download citation

  • DOI: https://doi.org/10.1007/978-3-540-39879-0_10

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-05885-1

  • Online ISBN: 978-3-540-39879-0

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation