# A comparative study on swarm intelligence for structure learning of Bayesian networks

- 245 Downloads

## Abstract

A Bayesian network (BN) is an important probabilistic model in the field of artificial intelligence and a powerful formalism used to describe uncertainty in the real world. As science and technology develop, considerable data on complex systems have been acquired by various means, which presents a significant challenge regarding how to accurately and robustly learn a network structure for a complex system. To address this challenge, many BN structure learning methods based on swarm intelligence have been developed. In this study, we perform a systematic comparison of three typical methods based on ant colony optimization, artificial bee colony algorithm, and bacterial foraging optimization. First, we analyze and summarize their main characteristics from the perspective of stochastic searching. Second, we conduct thorough experimental comparisons to examine the roles of different mechanisms in each method by means of multiaspect metrics, i.e., the K2 score, structural differences, and execution time. Next, we perform further experiments to validate the robustness of different algorithms on some benchmark data sets with noise. Finally, we present the prospects and references for researchers who are engaged in learning BN networks.

### Keywords

Bayesian network structure learning Swarm intelligence Ant colony optimization Artificial bee colony algorithm Bacterial foraging optimization## Notes

### Acknowledgments

We would like to thank the anonymous referees for their many valuable suggestions and comments. We thank the authors of the corresponding algorithms for sharing the binary executable systems. This work is partly supported by the NSFC Research Program (61375059, 61332016), the National “973” Key Basic Research Program of China (2014CB744601), the Specialized Research Fund for the Doctoral Program of Higher Education (20121103110031), and the Beijing Municipal Education Research Plan key Project (Beijing Municipal Fund Class B) (KZ201410005004).

### Compliance with ethical standards

### Conflict of interest

The authors declare that they have no competing interests.

### References

- Alcobó JR (2004) Incremental hill-climbing search applied to Bayesian network structure learning. In: Proceedings of the 15th European conference on machine learning, Pisa, ItalyGoogle Scholar
- Aouay S, Jamoussi S, Ayed YB (2013) Particle swarm optimization based method for Bayesian network structure learning. In: Proceedings of the 5th international conference on modeling, simulation and applied optimization (ICMSAO’13), pp 1–6Google Scholar
- Bang-Jensen J, Gutin GZ (2008) Digraphs: theory, algorithms and applications. Springer, BerlinMATHGoogle Scholar
- Beinlich IA, Suermondt HJ, Chavez RM, Cooper GF (1989) The alarm monitoring system: a case study with two probabilistic inference techniques for belief networks. AIME, Lecture notes in medical informatics, vol 38, pp 247–256Google Scholar
- Bielza C, Larraňaga P (2014) Bayesian networks in neuroscience: a survey. Front Comput Neurosci 8:131CrossRefMATHGoogle Scholar
- Buntine W (1996) A guide to the literature on learning probabilistic networks from data. IEEE Trans Knowl Data Eng 8(2):195–210CrossRefGoogle Scholar
- Castro PAD, Von Zuben FJ (2005) An immune-inspired approach to Bayesian networks. In: Proceedings of the 5th international conference on hybrid intelligent systems (HIS’05), IEEE, pp 6–9Google Scholar
- Cheng J, Bell DA, Liu W (1997) Learning belief networks from data: an information theory based approach. In: Proceedings of the 6th international conference on information and knowledge management (CIKM’97), pp 325–331Google Scholar
- Chickering DM, Geiger D, Heckerman D (1994) Learning Bayesian networks is NP-Hard. Technical Report MSR-TR-94-17, Microsoft ResearchGoogle Scholar
- Cooper GF, Herskovits E (1992) A Bayesian method for the introduction of probabilistic networks from data. Mach Learn 9(4):309–347MATHGoogle Scholar
- Daly R, Shen Q, Aitken S (2011) Learning Bayesian networks: approaches and issues. Knowl Eng Rev 26(02):99–157CrossRefGoogle Scholar
- Daly R, Shen Q (2009) Learning Bayesian network equivalence classes with ant colony optimization. J Artif Intell Res 35(1):391–447MathSciNetMATHGoogle Scholar
- De Campos LM, Puerta JM (2001) Stochastic local and distributed search algorithms for learning belief networks. In: Proceedings of third international symposium on adaptive systems: evolutionary computation and probabilistic graphical model, pp 109–115Google Scholar
- De Campos LM, Fernández-Luna JM, Gámez JA, Puerta JM (2002) Ant colony optimization for learning Bayesian networks. Int J Approx Reason 31(3):291–311MathSciNetCrossRefMATHGoogle Scholar
- De Campos LM, Gámez JA, Puerta JM (2008) Learning Bayesian networks by ant colony optimisation: searching in two different spaces. Mathw Soft Comput 9(3):251–268MathSciNetMATHGoogle Scholar
- De Campos LM, Huete JF (2000) A new approach for learning belief networks using independence criteria. Int J Approx Reason 24(1):11–37MathSciNetCrossRefMATHGoogle Scholar
- De CT Gomes L, De Sousa JS, Bezerra GB, De Castro LN, Von Zuben FJ (2003) Copt-aiNet and the gene ordering problem. In: Second Brazilian workshop on bioinformaticsGoogle Scholar
- Dorigo M, Maniezzo VA, Colorni A (1996) The ant system: optimization by a colony of cooperating agents. IEEE Trans Syst Man Cybern B 26(1):29–41CrossRefGoogle Scholar
- Eberhart RC, Shi Y, Kennedy J (2001) Swarm intelligence. Morgan Kaufmann, Los AltosGoogle Scholar
- Heckerman DE, Geiger D, Chickering DM (1995) Learning Bayesian networks: the combination of knowledge and statistical data. Mach Learn 20(3):197–243MATHGoogle Scholar
- Heckerman D (1998) A tutorial on learning Bayesian networks: learning in graphical models. Kluwer, DordrechtCrossRefMATHGoogle Scholar
- Heng XC, Qin Z, Tian L, Shao LP (2007) Learning Bayesian network structures with discrete particle swarm optimization algorithm. In: IEEE symposium on foundations of computational intelligence (FOCI’07). IEEE, pp 47–52Google Scholar
- Henrion M (1986) Propagating uncertainty by logic sampling in Bayes networks. In: Proceedings of the AAAI workshop on uncertainty in aritificial intelligence, PhiladelphiaGoogle Scholar
- Ji JZ, Wei HK, Liu CN, Yin BC (2013) Artificial bee colony algorithm merged with pheromone communication mechanism for the 0–1 multidimensional knapsack problem. Math Probl Eng 2013, Article ID 676275Google Scholar
- Ji JZ, Zhang HX, Hu RB, Liu CN (2008) A tabu-search based Bayesian network structure learning algorithm. Beijing Gongye Daxue Xuebao 37(8):1274–1280Google Scholar
- Ji JZ, Zhang HX, Hu RB, Liu CN (2009) A Bayesian network learning algorithm based on independence test and ant colony optimization. Acta Autom Sin 35(3):281–288CrossRefMATHGoogle Scholar
- Ji JZ, Hu RB, Zhang HX, Liu CN (2011) A hybrid method for learning the Bayesian networks based on ant colony optimization. Appl Soft Comput 11(4):3373–3384CrossRefGoogle Scholar
- Ji JZ, Wei HK, Liu CN (2013) An artificial bee colony algorithm for learning Bayesian networks. Soft Comput 6:983–994CrossRefGoogle Scholar
- Karaboga D (2005) An idea based on honey bee swarm for numerical optimization. Technical Report TR06, Computer Engineering Department, Erciyes University, TurkeyGoogle Scholar
- Kennedy J, Eberhart R (1995) Particle swarm optimization. In: Proceedings of the 1995 IEEE international conference on neural networks, pp 1942–1948Google Scholar
- Koller D, Friedman N (2009) Probabilistic graphical models: principles and techniques. MIT Press, CambridgeMATHGoogle Scholar
- Larrañaga P, Karshenas H, Bielza C, Santana R (2013) A review on evolutionary algorithms in Bayesian network learning and inference tasks. Inf Sci 233(1):109–125MathSciNetCrossRefMATHGoogle Scholar
- Li XL (2010) A particle swarm optimization and immune theory-based algorithm for structure learning of Bayesian networks. Int J Database Theory Appl 3(2):61–69Google Scholar
- Martens D, Baesens B, Fawcett T (2011) Editorial survey: swarm intelligence for data mining. Mach Learn 82(1):1–42MathSciNetCrossRefGoogle Scholar
- Mori M, Tsukiyama M, Fukuda T (1993) Immune algorithm with searching diversity and its application to resource allocation problem. Trans Inst Electron Eng Jpn C 113:872–878Google Scholar
- Mumford JA, Ramsey JD (2014) Bayesian networks for fMRI: a primer. NeuroImage 86:573–582CrossRefGoogle Scholar
- Needham CJ, Bradford JR, Bulpitt AJ, Westhead DR (2007) A primer on learning in Bayesian networks for computational biology. PLoS Comput Biol 3(8):e129CrossRefGoogle Scholar
- Passino KM (2002) Biomimicry of bacterial foraging for distributed optimization and control. Control Syst 22(3):52–67CrossRefGoogle Scholar
- Pearl J (1988) Reasoning in intelligent systems: networks of plausible inference. Morgan Kaufamann, San MateoMATHGoogle Scholar
- Pinto PC, Nägele A, Dejori M, Runkler TA, Sousa JMC (2009) Using a local discovery ant algorithm for Bayesian network structure learning. IEEE Trans Evol Comput 13(4):767–779CrossRefGoogle Scholar
- Rissanen J (1978) Modeling by shortest data description. Automatica 14(5):465–471CrossRefMATHGoogle Scholar
- Robinson RW (1977) Counting unlabeled acyclic digraphs. In: Little CHC (ed) Combinatorial mathematics V. Springer, BerlinGoogle Scholar
- Romero C, Ventura S (2010) Educational data mining: a review of the state of the art. IEEE Trans Syst Man Cybern C Appl Rev 40(6):601–618CrossRefGoogle Scholar
- Rubio-Largo Á, Vega-Rodríguez MA, Gómez-Pulido JA, Sánchez-Pérez JM (2012) A comparative study on multiobjective swarm intelligence for the routing and wavelength assignment problem. IEEE Trans Syst Man Cybern C Appl Rev 42(6):1644–1655CrossRefGoogle Scholar
- Schwarz G (1978) Estimating the dimension of a model. Ann Stat 6(2):461–464MathSciNetCrossRefMATHGoogle Scholar
- Shan C, Gong S, McOwan PW (2009) Facial expression recognition based on local binary patterns: a comprehensive study. Image Visi Comput 27(6):803–816CrossRefGoogle Scholar
- Spiegelhalter DJ, Dawid AP, Lauritzen SL, Cowell RG (1993) Bayesian analysis in expert systems. Stat Sci 8(3):219–247MathSciNetCrossRefMATHGoogle Scholar
- Suzuki J (1999) Learning Bayesian belief networks based on the minimum description length principle: basic properties. IEICE Trans Fundam Electron Commun Comput Sci 82(10):2237–2245Google Scholar
- Tsamardinos I, Brown LE, Aliferis CF (2006) The max–min hill-climbing Bayesian network structure learning algorithm. Mach Learn 65(1):31–78CrossRefGoogle Scholar
- Tsamardinos I, Aliferis CF, Statnikov A (2003) Scaling-up Bayesian network learning to thousands of variables using local learning technique. Technical Report DSL TR-03-02, Department of Biomedical Informatics, Vanderbilt UniversityGoogle Scholar
- Wang T, Yang J (2010) A heuristic method for learning Bayesian networks using discrete particle swarm optimization. Knowl Inf Syst 24(2):269–281CrossRefGoogle Scholar
- Weber P, Medina-Oliva G, Simon C, Iung B (2012) Overview on Bayesian networks applications for dependability, risk analysis and maintenance areas. Eng Appl Artif Intell 25(4):671–682CrossRefGoogle Scholar
- Wong ML, Lam W, Leung KS (1999) Using evolutionary programming and minimum description length principle for data mining of Bayesian networks. IEEE Trans Pattern Anal Mach Intell 21(2):174–178CrossRefGoogle Scholar
- Wu Y, McCall J, Corne D (2010) Two novel ant colony optimization approaches for Bayesian network structure learning. In: Proceedings of the 2010 IEEE congress on evolutionary computation (CEC’10), pp 1–7Google Scholar
- Yang CC, Ji JZ, Liu JM, Liu JD, Yin BC (2016) Structural learning of Bayesian networks by bacterial foraging optimization algorithm. Int J Approx Reason 69:147–167Google Scholar
- Zhao J, Sun J, Xu W, Zhou D (2009) Structure learning of Bayesian networks based on discrete binary quantum-behaved particle swarm optimization algorithm. In: Proceedings of the 5th international conference on natural computation (ICNC’09), 86-90Google Scholar