Soft Computing

, Volume 21, Issue 22, pp 6713–6738 | Cite as

A comparative study on swarm intelligence for structure learning of Bayesian networks

  • Junzhong Ji
  • Cuicui Yang
  • Jiming Liu
  • Jinduo Liu
  • Baocai Yin
Methodologies and Application
  • 236 Downloads

Abstract

A Bayesian network (BN) is an important probabilistic model in the field of artificial intelligence and a powerful formalism used to describe uncertainty in the real world. As science and technology develop, considerable data on complex systems have been acquired by various means, which presents a significant challenge regarding how to accurately and robustly learn a network structure for a complex system. To address this challenge, many BN structure learning methods based on swarm intelligence have been developed. In this study, we perform a systematic comparison of three typical methods based on ant colony optimization, artificial bee colony algorithm, and bacterial foraging optimization. First, we analyze and summarize their main characteristics from the perspective of stochastic searching. Second, we conduct thorough experimental comparisons to examine the roles of different mechanisms in each method by means of multiaspect metrics, i.e., the K2 score, structural differences, and execution time. Next, we perform further experiments to validate the robustness of different algorithms on some benchmark data sets with noise. Finally, we present the prospects and references for researchers who are engaged in learning BN networks.

Keywords

Bayesian network structure learning Swarm intelligence Ant colony optimization Artificial bee colony algorithm Bacterial foraging optimization 

References

  1. Alcobó JR (2004) Incremental hill-climbing search applied to Bayesian network structure learning. In: Proceedings of the 15th European conference on machine learning, Pisa, ItalyGoogle Scholar
  2. Aouay S, Jamoussi S, Ayed YB (2013) Particle swarm optimization based method for Bayesian network structure learning. In: Proceedings of the 5th international conference on modeling, simulation and applied optimization (ICMSAO’13), pp 1–6Google Scholar
  3. Bang-Jensen J, Gutin GZ (2008) Digraphs: theory, algorithms and applications. Springer, BerlinMATHGoogle Scholar
  4. Beinlich IA, Suermondt HJ, Chavez RM, Cooper GF (1989) The alarm monitoring system: a case study with two probabilistic inference techniques for belief networks. AIME, Lecture notes in medical informatics, vol 38, pp 247–256Google Scholar
  5. Bielza C, Larraňaga P (2014) Bayesian networks in neuroscience: a survey. Front Comput Neurosci 8:131CrossRefMATHGoogle Scholar
  6. Buntine W (1996) A guide to the literature on learning probabilistic networks from data. IEEE Trans Knowl Data Eng 8(2):195–210CrossRefGoogle Scholar
  7. Castro PAD, Von Zuben FJ (2005) An immune-inspired approach to Bayesian networks. In: Proceedings of the 5th international conference on hybrid intelligent systems (HIS’05), IEEE, pp 6–9Google Scholar
  8. Cheng J, Bell DA, Liu W (1997) Learning belief networks from data: an information theory based approach. In: Proceedings of the 6th international conference on information and knowledge management (CIKM’97), pp 325–331Google Scholar
  9. Chickering DM, Geiger D, Heckerman D (1994) Learning Bayesian networks is NP-Hard. Technical Report MSR-TR-94-17, Microsoft ResearchGoogle Scholar
  10. Cooper GF, Herskovits E (1992) A Bayesian method for the introduction of probabilistic networks from data. Mach Learn 9(4):309–347MATHGoogle Scholar
  11. Daly R, Shen Q, Aitken S (2011) Learning Bayesian networks: approaches and issues. Knowl Eng Rev 26(02):99–157CrossRefGoogle Scholar
  12. Daly R, Shen Q (2009) Learning Bayesian network equivalence classes with ant colony optimization. J Artif Intell Res 35(1):391–447MathSciNetMATHGoogle Scholar
  13. De Campos LM, Puerta JM (2001) Stochastic local and distributed search algorithms for learning belief networks. In: Proceedings of third international symposium on adaptive systems: evolutionary computation and probabilistic graphical model, pp 109–115Google Scholar
  14. De Campos LM, Fernández-Luna JM, Gámez JA, Puerta JM (2002) Ant colony optimization for learning Bayesian networks. Int J Approx Reason 31(3):291–311MathSciNetCrossRefMATHGoogle Scholar
  15. De Campos LM, Gámez JA, Puerta JM (2008) Learning Bayesian networks by ant colony optimisation: searching in two different spaces. Mathw Soft Comput 9(3):251–268MathSciNetMATHGoogle Scholar
  16. De Campos LM, Huete JF (2000) A new approach for learning belief networks using independence criteria. Int J Approx Reason 24(1):11–37MathSciNetCrossRefMATHGoogle Scholar
  17. De CT Gomes L, De Sousa JS, Bezerra GB, De Castro LN, Von Zuben FJ (2003) Copt-aiNet and the gene ordering problem. In: Second Brazilian workshop on bioinformaticsGoogle Scholar
  18. Dorigo M, Maniezzo VA, Colorni A (1996) The ant system: optimization by a colony of cooperating agents. IEEE Trans Syst Man Cybern B 26(1):29–41CrossRefGoogle Scholar
  19. Eberhart RC, Shi Y, Kennedy J (2001) Swarm intelligence. Morgan Kaufmann, Los AltosGoogle Scholar
  20. Heckerman DE, Geiger D, Chickering DM (1995) Learning Bayesian networks: the combination of knowledge and statistical data. Mach Learn 20(3):197–243MATHGoogle Scholar
  21. Heckerman D (1998) A tutorial on learning Bayesian networks: learning in graphical models. Kluwer, DordrechtCrossRefMATHGoogle Scholar
  22. Heng XC, Qin Z, Tian L, Shao LP (2007) Learning Bayesian network structures with discrete particle swarm optimization algorithm. In: IEEE symposium on foundations of computational intelligence (FOCI’07). IEEE, pp 47–52Google Scholar
  23. Henrion M (1986) Propagating uncertainty by logic sampling in Bayes networks. In: Proceedings of the AAAI workshop on uncertainty in aritificial intelligence, PhiladelphiaGoogle Scholar
  24. Ji JZ, Wei HK, Liu CN, Yin BC (2013) Artificial bee colony algorithm merged with pheromone communication mechanism for the 0–1 multidimensional knapsack problem. Math Probl Eng 2013, Article ID 676275Google Scholar
  25. Ji JZ, Zhang HX, Hu RB, Liu CN (2008) A tabu-search based Bayesian network structure learning algorithm. Beijing Gongye Daxue Xuebao 37(8):1274–1280Google Scholar
  26. Ji JZ, Zhang HX, Hu RB, Liu CN (2009) A Bayesian network learning algorithm based on independence test and ant colony optimization. Acta Autom Sin 35(3):281–288CrossRefMATHGoogle Scholar
  27. Ji JZ, Hu RB, Zhang HX, Liu CN (2011) A hybrid method for learning the Bayesian networks based on ant colony optimization. Appl Soft Comput 11(4):3373–3384CrossRefGoogle Scholar
  28. Ji JZ, Wei HK, Liu CN (2013) An artificial bee colony algorithm for learning Bayesian networks. Soft Comput 6:983–994CrossRefGoogle Scholar
  29. Karaboga D (2005) An idea based on honey bee swarm for numerical optimization. Technical Report TR06, Computer Engineering Department, Erciyes University, TurkeyGoogle Scholar
  30. Kennedy J, Eberhart R (1995) Particle swarm optimization. In: Proceedings of the 1995 IEEE international conference on neural networks, pp 1942–1948Google Scholar
  31. Koller D, Friedman N (2009) Probabilistic graphical models: principles and techniques. MIT Press, CambridgeMATHGoogle Scholar
  32. Larrañaga P, Karshenas H, Bielza C, Santana R (2013) A review on evolutionary algorithms in Bayesian network learning and inference tasks. Inf Sci 233(1):109–125MathSciNetCrossRefMATHGoogle Scholar
  33. Li XL (2010) A particle swarm optimization and immune theory-based algorithm for structure learning of Bayesian networks. Int J Database Theory Appl 3(2):61–69Google Scholar
  34. Martens D, Baesens B, Fawcett T (2011) Editorial survey: swarm intelligence for data mining. Mach Learn 82(1):1–42MathSciNetCrossRefGoogle Scholar
  35. Mori M, Tsukiyama M, Fukuda T (1993) Immune algorithm with searching diversity and its application to resource allocation problem. Trans Inst Electron Eng Jpn C 113:872–878Google Scholar
  36. Mumford JA, Ramsey JD (2014) Bayesian networks for fMRI: a primer. NeuroImage 86:573–582CrossRefGoogle Scholar
  37. Needham CJ, Bradford JR, Bulpitt AJ, Westhead DR (2007) A primer on learning in Bayesian networks for computational biology. PLoS Comput Biol 3(8):e129CrossRefGoogle Scholar
  38. Passino KM (2002) Biomimicry of bacterial foraging for distributed optimization and control. Control Syst 22(3):52–67CrossRefGoogle Scholar
  39. Pearl J (1988) Reasoning in intelligent systems: networks of plausible inference. Morgan Kaufamann, San MateoMATHGoogle Scholar
  40. Pinto PC, Nägele A, Dejori M, Runkler TA, Sousa JMC (2009) Using a local discovery ant algorithm for Bayesian network structure learning. IEEE Trans Evol Comput 13(4):767–779CrossRefGoogle Scholar
  41. Rissanen J (1978) Modeling by shortest data description. Automatica 14(5):465–471CrossRefMATHGoogle Scholar
  42. Robinson RW (1977) Counting unlabeled acyclic digraphs. In: Little CHC (ed) Combinatorial mathematics V. Springer, BerlinGoogle Scholar
  43. Romero C, Ventura S (2010) Educational data mining: a review of the state of the art. IEEE Trans Syst Man Cybern C Appl Rev 40(6):601–618CrossRefGoogle Scholar
  44. Rubio-Largo Á, Vega-Rodríguez MA, Gómez-Pulido JA, Sánchez-Pérez JM (2012) A comparative study on multiobjective swarm intelligence for the routing and wavelength assignment problem. IEEE Trans Syst Man Cybern C Appl Rev 42(6):1644–1655CrossRefGoogle Scholar
  45. Schwarz G (1978) Estimating the dimension of a model. Ann Stat 6(2):461–464MathSciNetCrossRefMATHGoogle Scholar
  46. Shan C, Gong S, McOwan PW (2009) Facial expression recognition based on local binary patterns: a comprehensive study. Image Visi Comput 27(6):803–816CrossRefGoogle Scholar
  47. Spiegelhalter DJ, Dawid AP, Lauritzen SL, Cowell RG (1993) Bayesian analysis in expert systems. Stat Sci 8(3):219–247MathSciNetCrossRefMATHGoogle Scholar
  48. Suzuki J (1999) Learning Bayesian belief networks based on the minimum description length principle: basic properties. IEICE Trans Fundam Electron Commun Comput Sci 82(10):2237–2245Google Scholar
  49. Tsamardinos I, Brown LE, Aliferis CF (2006) The max–min hill-climbing Bayesian network structure learning algorithm. Mach Learn 65(1):31–78CrossRefGoogle Scholar
  50. Tsamardinos I, Aliferis CF, Statnikov A (2003) Scaling-up Bayesian network learning to thousands of variables using local learning technique. Technical Report DSL TR-03-02, Department of Biomedical Informatics, Vanderbilt UniversityGoogle Scholar
  51. Wang T, Yang J (2010) A heuristic method for learning Bayesian networks using discrete particle swarm optimization. Knowl Inf Syst 24(2):269–281CrossRefGoogle Scholar
  52. Weber P, Medina-Oliva G, Simon C, Iung B (2012) Overview on Bayesian networks applications for dependability, risk analysis and maintenance areas. Eng Appl Artif Intell 25(4):671–682CrossRefGoogle Scholar
  53. Wong ML, Lam W, Leung KS (1999) Using evolutionary programming and minimum description length principle for data mining of Bayesian networks. IEEE Trans Pattern Anal Mach Intell 21(2):174–178CrossRefGoogle Scholar
  54. Wu Y, McCall J, Corne D (2010) Two novel ant colony optimization approaches for Bayesian network structure learning. In: Proceedings of the 2010 IEEE congress on evolutionary computation (CEC’10), pp 1–7Google Scholar
  55. Yang CC, Ji JZ, Liu JM, Liu JD, Yin BC (2016) Structural learning of Bayesian networks by bacterial foraging optimization algorithm. Int J Approx Reason 69:147–167Google Scholar
  56. Zhao J, Sun J, Xu W, Zhou D (2009) Structure learning of Bayesian networks based on discrete binary quantum-behaved particle swarm optimization algorithm. In: Proceedings of the 5th international conference on natural computation (ICNC’09), 86-90Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2016

Authors and Affiliations

  • Junzhong Ji
    • 1
  • Cuicui Yang
    • 1
  • Jiming Liu
    • 2
  • Jinduo Liu
    • 1
  • Baocai Yin
    • 1
  1. 1.Beijing Municipal Key Laboratory of Multimedia and Intelligent Software Technology, College of Computer Science and TechnologyBeijing University of TechnologyBeijingChina
  2. 2.Department of Computer Science and TechnologyHong Kong Baptist UniversityKowloon TongHong Kong

Personalised recommendations