Skip to main content

Advertisement

SpringerLink
Log in

Evolving dynamic Bayesian networks with Multi-objective genetic algorithms

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

A dynamic Bayesian network (DBN) is a probabilistic network that models interdependent entities that change over time. Given example sequences of multivariate data, we use a genetic algorithm to synthesize a network structure that models the causal relationships that explain the sequence. We use a multi-objective evaluation strategy with a genetic algorithm. The multi-objective criteria are a network's probabilistic score and structural complexity score. Our use of Pareto ranking is ideal for this application, because it naturally balances the effect of the likelihood and structural simplicity terms used in the BIC network evaluation heuristic. We use a basic structural scoring formula, which tries to keep the number of links in the network approximately equivalent to the number of variables. We also use a simple representation that favors sparsely connected networks similar in structure to those modeling biological phenomenon. Our experiments show promising results when evolving networks ranging from 10 to 30 variables, using a maximal connectivity of between 3 and 4 parents per node. The results from the multi-objective GA were superior to those obtained with a single objective GA.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Charniak E (1991) Bayesian networks without tears. AI Magazine, Winter, pp 50–63

    Google Scholar 

  2. Pearl J (2000) Causality. Cambridge

  3. Russell S, Norvig P (2003) Artificial intelligence: a modern approach, 2nd edn. Prentice Hall

  4. Castillo E, Gutierrez JM, Hadi AS (1997) Expert systems and probabilistic network models. Springer Verlag

  5. Markowetz F (2005) A bibliography on learning causal networks of gene interactions. http://www.molgen.mpg.de/~markowet/docs/network-bib.pdf. Last accessed April 1, 2005

  6. Nefian AV, Liang L, Pi X, Liu X, Murphy K (2002) Dynamic Bayesian networks for audio-visual speech recognition. EURASIP J Appl Signal Process pp 1–15

  7. Davies S, Moore A (1999) Bayesian networks for lossless dataset compression. In: Proceedings of the 5th ACM SIGKDD international conference on knowledge discovery and data mining, pp 387–391

  8. Pelikan M, Goldberg DE, Cantu-Paz E (1999) BOA: the Bayesian optimization algorithm. In: Banzhaf W et al. (eds) Proceedings of the GECCO-99, pp 525–532

  9. Friedman N, Murphy K, Russell S (1998) Learning the structure of dynamic probabilistic networks. In: Proceedings of the uncertainty in AI (UAI’98), Morgan Kaufman

  10. Murphy KP (2005) Dynamic Bayesian networks. http://www.cs.ubc.ca/~murphyk/Papers/dbnchapter.pdf. To appear in Probabilistic Graphical Models, M. Jordan. Last accessed April 13 2005

  11. Husmeier D (2003) Sensitivity and specificity of inferring genetic regulatory interactions from microarray experiments with dynamic Bayesian networks. Bioinformatics 19(17):2271–2282

    Article  Google Scholar 

  12. Kim SY, Imoto S, Miyano S (2003) Inferring gene networks from time series microarray data using dynamic Bayesian networks. Briefings in Bioinformatics 4(3):228–235

    Article  Google Scholar 

  13. Perrin B-E, Ralaivola L, Mazurie A, Bottani S, Mallet J, d’Alche Buc F (2003) Gene networks inference using dynamic Bayesian networks. Bioinformatics 19:ii138–ii148, Supplement 2

    Google Scholar 

  14. Wu CC, Huang HC, Juan HF, Chen ST (2004) GeneNetwork: an interactive tool for reconstruction of genetic networks using microarray data. Bioinformatics 20(18):3691–3693

    Article  Google Scholar 

  15. Yu J, Smith VA, Wang PP, Hartemink AJ, Jarvis ED (2004) Advances to Bayesian network inference for generating causal networks from observational biological data. Bioinformatics 20(18):3594–3603

    Article  Google Scholar 

  16. Buntine WL (1994) Operations for learning with graphical models. J Artif Intell Res 2:159–225

    Google Scholar 

  17. Heckerman D (1995) A tutorial on learning with Bayesian networks. Technical Report MSR-TR-95-06, Microsoft Research, March 1995

  18. Chrisman L (2005) A roadmap to research on Bayesian networks and other decomposable probabilistic models. citeseer.nj.nec.com/ chrisman98roadmap.html. Last accessed April 1 2005

  19. Stephenson TA (2000) An introduction to Bayesian network theory and usage. Technical Report 00–03, IDIAP, February 2000

  20. Guo H, Hsu W (2002) A survey of algorithms for real-time Bayesian network inference. In: Proceedings of the joint AAAI-02/KDD-02/UAI-02 workshop on realtime decision support and diagnosis systems

  21. Cooper G, Herskovits E (1992) A Bayesian method for the induction of probabilistic networks from data. Mach Learn 9:309–347

    Google Scholar 

  22. Chickering DM (1996) Learning Bayesian networks is NP-complete. In: Fisher D, Lenz H-J (eds) Learning from data: AI and statistics. Springer Verlag

  23. Bouckaert RR (1994) Probabilistic network construction using the minimum description length principle. Technical Report UU-CS-1994-27, Utrecht University, Dept. of Computer Science, July 1994

  24. Lam W, Wong ML, Leung KS, Ngan PS (1998) Discovering probabilistic knowledge from databases using evolutionary computation and minimum description length principle. In: Koza JR et al (eds) Proceeding of the genetic programming 1998. Morgan Kaufmann, pp 786–794

  25. Sierra B, Larranaga P (1998) Predicting the survival in malignant skin melanoma using Bayesian networks automatically induced by genetic algorithms. An empirical comparison between different approaches. Artif Intell Med 14(1–2):215–230

    Article  Google Scholar 

  26. de Campos LM, Huete JF (1999) Approximating causal orderings for Bayesian networks using genetic algorithms and simulated Annealing. Technical Report #DECSAI-99021, U. of Grenada, Uncertainty Treatment in Artificial Intelligence Group, May 1999

  27. Myers JW, Laskey KB, DeJong KA (1999) Learning Bayesian networks from incomplete data using evolutionary algorithms. In Banzhaf W et al (eds) Proceedings of the GECCO-99, pp 458– 465

  28. Cotta C, Murzabal J (2002) Towards a more efficient evolutionary induction of Bayesian networks. In: Proceedings of the PPSN 2002, pp 730–739

  29. Hsu WH, Guo H, Perry BB, Stilson JA (2002) A permutation genetic algorithm for variable ordering in learning Bayesian networks from data. In: Langdon WB et al (eds) Proceedings of the GECCO 2002, pp. 383–390, Morgan Kaufmann

  30. Harwood S, Scheines R (2002) Genetic algorithm search over causal models. Technical Report CMU-PHIL-131, Carnegie Mellon University, Dept. of Philosophy

  31. Wong ML, Lee SY, Leung KS (2002) A hybrid data mining approach to discover bayesian networks using evolutionary programming. In: Langdon WB et al (eds) Proceedings of the GECCO 2002, pp 214–221

  32. van Dijk S, Thierens D, van der Gaag LC (2003) Building a GA from design principles for learning Bayesian networks. In: Cantu-Paz E et al (eds) Proceedings of the GECCO 2003, Springer-Verlag, pp 886–897

  33. Larranaga P, Poza M, Yurramendi Y, Murga RH, Kuijpers CMH (1996) Structure learning of Bayesian networks by genetic algorithms: A performance analysis of control parameters. IEEE Trans Pattern Anal Mach Intell 18(9):912–926

    Article  Google Scholar 

  34. Tucker A, Liu X, Garway-Heath D (2003) Spatial operators for evolving dynamic Bayesian networks from spatio-temporal data. In: Cantu-Paz E et al (eds) Proceedings of the GECCO 2003, Springer-Verlag, pp 2360–2371

  35. Tucker A, Liu X (2003) Learning dynamic Bayesian networks from multivariate time series with changing dependencies. In: Proceedings of the 5th intelligent data analysis conference (IDA 2003), Springer-Verlag, pp 100–110, 2003. LNCS 2810

  36. Coello Coello CA, van Veldhuizen DA, Lamont GB (2002) Evolutionary algorithms for solving multi-objective problems, Kluwer Academic Publishers

  37. Fonseca CM, Fleming PJ (1995) An overview of evolutionary algorithms in multiobjective optimization. Evolutionary Comput 3(1):1–16

    Google Scholar 

  38. van Veldhuizen DA, Lamont GB (2000) Multiobjective evolutionary algorithms: Analyzing the state-of-the-art. Evolutionary Computation 8(2):125–147

    Article  Google Scholar 

  39. Holland JH (1992) Adaptation in natural and artificial systems. MIT Press

  40. Goldberg DE (1989) Genetic algorithms in search, optimization, and machine learning. Addison Wesley

  41. Schwarz G (1978) Estimating the dimension of a model. Ann Stat 6(2):461–464

    Google Scholar 

  42. Rissanen J (1986) Stochastic complexity and modeling. Ann Stat 14(3):1080–1100

    MathSciNet  Google Scholar 

  43. Barabasi A-L, Albert R (1999) Emergence of scaling in random networks. Science 286(15):509–512

    MathSciNet  Google Scholar 

  44. Jeong H, Tombor B, Albert R, Oltvai ZN, Barabasi A-L (2000) The large-scale organization of metabolic networks. Nature 407:651–654

    Article  Google Scholar 

  45. Murphy KP (2005) Bayes net toolbox for matlab. http://www.cs.ubc.ca/~murphyk/Software/BNT/bnt.html. Last accessed March 16 2005.

  46. Gruau F (1994) Genetic micro programming of neural networks. In: Kinnear KE (ed) Advances in genetic programming, MIT Press, pp 495–518

  47. Gruau F (1994) Genetic micro programming of neural networks.In: Kinnear KE (ed)Advances in genetic programming, MIT Press, pp 495–518

Download references

Author information

Author notes

  1. Eduardo Zuviria

    Present address: Department of Information Processing and Operations Research, University of Montreal, Montreal, Quebec, Canada

Authors and Affiliations

  1. Department of Computer Science, Brock University, St. Catharines, Ontario, L2S 3A1, Canada

    Brian J. Ross & Eduardo Zuviria

Authors
  1. Brian J. Ross
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Eduardo Zuviria
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Brian J. Ross.

Additional information

Brian J. Ross is a professor of computer science at Brock University, where he has worked since 1992. He obtained his BCSc at the University of Manitoba, Canada, in 1984, his M.Sc. at the University of British Columbia, Canada, in 1988, and his Ph.D. at the University of Edinburgh, Scotland, in 1992. His research interests include evolutionary computation, language induction, concurrency, and logic programming. He is also interested in computer applications in the fine arts.

Eduardo Zuviria received a BS degree in Computer Science from Brock University in 2004 and a MS degree in Computer Science from Queen's University in 2006 where he held jobs as teacher and research assistant. Currently, he is attending a Ph.D. program at the University of Montreal. He holds a diploma in electronics from a technical college and has worked for eight years in the computer industry as a software developer and systems administrator. He has received several scholarships including the Ontario Graduate Scholarship, Queen's Graduate Scholarship and a NSERC- USRA scholarship.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Ross, B.J., Zuviria, E. Evolving dynamic Bayesian networks with Multi-objective genetic algorithms. Appl Intell 26, 13–23 (2007). https://doi.org/10.1007/s10489-006-0002-6

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-006-0002-6

Keywords

Search

Navigation