Skip to main content
SpringerLink
Log in

Ensemble bayesian networks evolved with speciation for high-performance prediction in data mining

  • Methodologies and Application
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

Bayesian networks (BNs) can be easily refined (or learn) using data given prior knowledge about a changing environment. Furthermore, by exploring multiple diverse BNs in parallel, it is expected that an intelligent system may adapt quickly to changes in the environment, resulting in robust prediction. Recently, there have been attempts to design BN structures using evolutionary algorithms; however, most of these have used only the fittest solution from the final generation. Because it is difficult to combine all of the important factors into a single evaluation function, the solution is often biased and of limited adaptability. Here we describe a method of generating diverse BN structures via speciation and selective combination for adaptive prediction. Experiments using the seven benchmark networks show that the proposed method can result in improved accuracy in handling uncertainty by exploiting ensembles of BNs evolved by speciation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  • Barber D (2012) Bayesian reasoning and machine learning. Cambridge University Press, Cambridge

    MATH  Google Scholar 

  • Beinlich IA, Suermondt HJ, Chavez RM, Cooper GF (1989) The ALARM monitoring system: a case study with two probabilistic inference techniques for belief networks. In: Proceedings of the Second European Conference on Artificial Intelligence in Medicine, pp 247–256

  • Binder J, Koller D, Russell S, Kanazawa K (1997) Adaptive probabilistic networks with hidden variables. Mach Learn 29(2–3):213–244

    Article  MATH  Google Scholar 

  • Chickering DM, Geiger D, Heckerman D (1994) Learning Bayesian networks is NP-hard, Technical Report MSR-TR-94-17, Microsoft Research

  • Colace F, De Santo M, Greco L (2014) Learning Bayesian network structure using a multiexpert approach. Int J Softw Eng Knowl Eng 24(2):269–284

    Article  Google Scholar 

  • Cooper GF, Herskovits E (1992) A Bayesian method for the induction of probabilistic networks from data. Mach Learn 9:309–347

    MATH  Google Scholar 

  • Daly R, Shen Q, Aitken S (2011) Learning Bayesian networks: approaches and issues. Knowl Eng Rev 26(2):99–157

    Article  Google Scholar 

  • Feng G, Zhang J-D, Liao SS (2014) A novel method for combining Bayesian networks, theoretical analysis, and its applications. Pattern Recognit 47(5):2057–2069

    Article  MATH  Google Scholar 

  • Gamez JA, Mateo JL, Puerta JM (2011) Learning Bayesian networks by hill climbing: efficient methods based on progressive restriction of the neighborhood. Data Min Knowl Discov 22(1–2):106–148

    Article  MathSciNet  MATH  Google Scholar 

  • Garg A, Pavlovic V, Rehg JM (2003) Boo sted learning in dynamic Bayesian networks for multimodal speaker detection. Proc IEEE 91(9):1355–1369

    Article  Google Scholar 

  • GENIE & SMILE. http://genie.sis.pitt.edu

  • Goldberg DE (2008) Genetic algorithms in search, optimization, and machine learning, 1st edn. Addison-Wesley Professional

  • Gouvea MM Jr., Araujo AFR (2010) Diversity-based adaptive evolutionary algorithms, Chapter 1. New Achievements in Evolutionary Computation

  • Heckerman D (2008) A tutorial on learning with Bayesian networks. Innov Bayesian Netw 156:33–82

    Article  MATH  Google Scholar 

  • Hu L, Wang L (2013) Using consensus Bayesian network to model the reactive oxygen species regulatory pathway. PLOS One 8(2):e56832. doi:10.1371/journal.pone.0056832

  • Hwang K-S, Cho S-B (2009) Landmark detection from mobile life log using a modular Bayesian network model. Expert Syst Appl 36:12065–12076

    Article  Google Scholar 

  • Jensen FV, Kjærulff U, Olesen KG, Pedersen J (1989) An expert system for control of waste water treatment—a pilot project. Technical report. Judex Datasystemer A/S, Aalborg (in Danish)

    Google Scholar 

  • Kim K-J, Cho S-B (2005) Systematically incorporating domain-specific knowledge into evolutionary speciated checkers players. IEEE Trans Evol Comput 9(6):615–627

    Article  Google Scholar 

  • Kim K-J, Cho S-B (2008) Evolutionary ensemble of diverse artificial neural networks using speciation. Neurocomputing 71(7–9):1604–1618

    Article  Google Scholar 

  • Kim K-J, Cho S-B (2012) Automated synthesis of multiple analog circuits using evolutionary computation for redundancy-based fault-tolerance. Appl Soft Comput 12(4):1309–1321

    Article  Google Scholar 

  • Kim K, Mckay R (2012) Stochastic diversity loss and scalability in estimation of distribution genetic programming. IEEE Trans Evol Comput 17(3):301–320

    Google Scholar 

  • Kim K-J, Park J-G, Cho S-B (2011) Correlation analysis and performance evaluation of distance measures for evolutionary neural networks. J Intell Fuzzy Syst 22:83–92

    MathSciNet  MATH  Google Scholar 

  • Kim KJ, Yoo JO, Cho SB (2005) Robust inference of Bayesian networks using speciated evolution and ensemble. In: International Symposium on Methodologies for Intelligent Systems, pp 92–101

  • Koller D, Friedman N (2009) Probabilistic graphical models: principles and techniques. MIT Press, Cambridge

    MATH  Google Scholar 

  • Korb KB, Nicholson AE (2010) Bayesian artificial intelligence, 2nd edn. CRC Press, Boca Raton

    Google Scholar 

  • Larranaga P, Karshenas H, Bielza C, Santana R (2013) A review on evolutionary algorithms in Bayesian network learning and inference tasks. Inf Sci 233(1):109–125

    Article  MathSciNet  MATH  Google Scholar 

  • Larranaga P, Kuijpers CMH, Murga RH, Yurramendi Y (1996) Learning Bayesian network structures by searching for the best ordering with genetic algorithm. IEEE Trans Syst Man Cybern Part A 26(4):487–493

  • Larranaga P, Poza M, Yurramendi Y, Murga RH, Kuijpers CMH (1996) Structure learning of Bayesian networks by genetic algorithms: a performance analysis of control parameters. IEEE Trans Pattern Anal Mach Intell 18(9):912–926

  • Lauritzen S-L, Spiegelhalter DJ (1988) Local computations with probabilities on graphical structures and their applications on expert systems. J R Stat Soc B 50(2):157–224

    MathSciNet  MATH  Google Scholar 

  • Li XL, He XD, Yuan SM (2005) Learning Bayesian networks structures from incomplete data based on extending evolutionary programming. In: Proceedings of the Fourth International Conference on Machine Learning and Cybernetics, pp 2039–2043

  • Li W, Liu W, Yue K (2008) Recovering the global structure from multiple local Bayesian networks. Int J Artif Intell Tools 17(6):1067–1088

    Article  Google Scholar 

  • Luo X, Ouyang Y, Xiong Z (2011) Improving matrix factorization-based recommender via ensemble methods. Int J Inf Technol Decision Making 10(3):539–561

    Article  Google Scholar 

  • Mahfoud SW (1995) Niching methods for genetic algorithms. Ph.D. Dissertation, University of Illinois at Urbana-Champaign

  • Muruzabal J, Cotta C (2007) A Study on the evolution of Bayesian network graph structures. Adv Probab Graph Models 193–214

  • Myers JW, Laskey KB, Dejong KA (1999) Learning Bayesian networks from incomplete data using evolutionary algorithm. In: Proceedings of the Genetic and Evolutionary Computation Conference, pp 458–465

  • Na Y, Yang J (2010) Distributed Bayesian network structure learning. In: IEEE International Symposium on Industrial Electronics, pp 1607–161

  • Pena JM, Kocka T, Nielsen JD (2004) Featuring multiple local optima to assist the user in the interpretation of induced Bayesian network models. In: Proceedings of the Tenth International Conference on Information Processing and Management of Uncertainty in Knowledge-based Systems, pp 1683–1690

  • Peng Y, Kou G, Wang G, Wu W, Shi Y (2011) Ensemble of software defect predictors: an AHP-based evaluation method. Int J Inf Technol Decision Making 10(1):187–206

    Article  Google Scholar 

  • Robles V, Larranaga P, Pena JM, Menasalvas E, Perez MS, Herves V, Wasilewska A (2004) Bayesian network multi-classifiers for protein secondary structure prediction. Artif Intell Med 31(2):117–136

    Article  Google Scholar 

  • Rogers A, Prugel-Bennett A (1999) Genetic drift in genetic algorithm selection schemes. IEEE Trans Evol Comput 3(4):298–303

    Article  Google Scholar 

  • Schadt EE, Linderman MD, Sorenson J, Lee L, Nolan GP (2010) Computational solutions to large-scale data management and analysis. Nat Rev Genet 11(9):647–657

    Article  Google Scholar 

  • Scutari M, Denis JB (2014) Bayesian networks: with examples in R. Chapman & Hall, London

    MATH  Google Scholar 

  • Shen C-W (2009) A Bayesian networks approach to modeling financial risks of e-logistics investments. Int J Inf Technol Decision Making 8(4):711–726

    Article  MATH  Google Scholar 

  • Su X, Khoshgoftaar TM (2008) Collaborative filtering for multi-class data using Bayesian networks. Int J Artif Intell Tools 17(1):71–85

    Article  Google Scholar 

  • Vafaee F (2014) Learning the structure of large-scale Bayesian networks using genetic algorithm. In: Proceedings of the Conference on Genetic and Evolutionary Computation, pp 855–862

  • Wong ML, Lam W, Leung KS (1999) Using evolutionary programming and minimum description length principle for data mining of Bayesian networks. IEEE Trans Pattern Anal Mach Intell 21(2):174–178

  • Wong ML, Lee SY, Leung KS (2004) Data mining of Bayesian networks using cooperative coevolution. Decision Support Syst 38:451–472

    Article  Google Scholar 

  • Zhou ZH (2012) Ensemble methods: foundations and algorithms. Chapman & Hall/CRC, London

    Google Scholar 

Download references

Acknowledgments

This work was supported by the National Research Foundation of Korea (NRF) Grant funded by the Korean Government (MSIP) (2013 R1A2A2A01016589) and the Industrial Strategic Technology Development Program, 10044828, Development of augmenting multisensory technology for enhancing significant effect on service industry, funded by the Ministry of Trade, Industry & Energy (MI, Korea).

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, Sejong University, Seoul, South Korea

    Kyung-Joong Kim

  2. Department of Computer Science, Yonsei University, Seoul, South Korea

    Sung-Bae Cho

Authors
  1. Kyung-Joong Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sung-Bae Cho
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kyung-Joong Kim.

Additional information

Communicated by V. Loia.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kim, KJ., Cho, SB. Ensemble bayesian networks evolved with speciation for high-performance prediction in data mining. Soft Comput 21, 1065–1080 (2017). https://doi.org/10.1007/s00500-015-1841-z

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-015-1841-z

Keywords

Search

Navigation