AMBN 2015: Advanced Methodologies for Bayesian Networks pp 32-45 | Cite as
Learning Bayesian Network Parameters from Small Data Set: A Spatially Maximum a Posteriori Method
Abstract
To learn accurate BN parameters from small data set, combined with data, domain knowledge is often incorporated into the learning process as parameter constraints. Currently, most of the existing parameter learning methods take parameter learning problem as an exact optimization problem and regard the optimal solutions as the final parameters. However, due to the scarcity of data, objective functions constructed from the data, like likelihood function and entropy function, are not accurate. Therefore, parameters derived from the objective functions do not approach the true parameters well while some suboptimal parameters fit the true parameters better. Thus, searching more reasonable suboptimal parameters is a possible approach to learn better BN parameters. In this paper, we propose to visualize suboptimal parameters with parallel coordinate system and propose a Spatially Maximum a Posteriori (SMAP) method. Experimental results reveal that the proposed method outperforms most of the existing parameter learning methods.
Keywords
Bayesian Networks Parameter learning Small data set Convex optimization Linear programmingNotes
Acknowledgements
This work is supported in part by National Nature Science Foundation of China (grant 61573285) and the Doctoral Fund of Ministry of Education of China (grant 20116102110026).
References
- 1.Pearl, J.: Probabilistic Reasoning in Intelligent Systems. Morgan Kaufmann, Burlington (1988)MATHGoogle Scholar
- 2.Tamda, Y., Imoto, S., Araki, H.: Estimating genome-wide gene networks using nonparametric Bayesian network models on massively parallel computers. IEEE Trans. Comput. Biol. Bioinform. 3, 683–697 (2011)CrossRefGoogle Scholar
- 3.Ibrahim, W., Beiu, V.: Using Bayesian networks to accurately calculate the reliability of complementary metal oxide semiconductor gates. IEEE Trans. Reliab. 60, 538–549 (2011)CrossRefGoogle Scholar
- 4.Infantes, G., Ghallab, M., Ingrand, F.: Learning the behavior model of a robot. Auton. Robot. 30, 157–177 (2011)CrossRefGoogle Scholar
- 5.Steven, M., Ann, N., Kevin, K.: Anomaly detection in vessel tracks using Bayesian networks. Int. J. Approximate Reasoning 55, 84–98 (2014)CrossRefGoogle Scholar
- 6.Neil, W., Mahmood, R., Kevin, K.: Detection and classification of non-stationary transient signals using sparse approximations and Bayesian networks. IEEE/ACM Trans. Audio Speech Lang. Process. 22, 1750–1764 (2014)CrossRefGoogle Scholar
- 7.Dries, L., Steven, B., Rob, D., Guy, E., Joris, A.: A review of Bayesian belief networks in ecosystem service modeling. Environ. Model. Softw. 46, 1–11 (2013)CrossRefGoogle Scholar
- 8.Redner, R., Walker, H.: Mixture densities, maximum likelihood and the EM algorithm. SIAM Rev. 26, 195–239 (1984)MathSciNetCrossRefGoogle Scholar
- 9.Helsper, E., Gaag, L., Groenendal, F.: Designing a procedure for the acquisition of probability constraints for Bayesian networks. In: Proceedings of the Fourteenth Conference on Engineering Knowledge in the Age of the Semantic Web, pp. 280–292 (2004)CrossRefGoogle Scholar
- 10.Wittig, F., Jameson, A.: Exploiting qualitative knowledge in the learning of conditional probabilities of Bayesian networks. In: Proceedings of the Sixteenth International Conference on Uncertainty in Artificial Intelligence, pp. 644–652 (2000)Google Scholar
- 11.Altendorf, E., Restificar, A., Dietterich, T.: Learning from sparse data by exploiting monotonicity constraints. In: Proceedings of the Twenty First International Conference on Uncertainty in Artificial Intelligence, pp. 18–26 (2005)Google Scholar
- 12.Feelders, A., Gaag, L.: Learning Bayesian networks parameters under order constraints. Int. J. Approximate Reasoning 42, 37–53 (2006)MathSciNetCrossRefGoogle Scholar
- 13.Isozaki, T., Kato, N., Ueno, M.: Data temperature in minimum free energies for parameter learning of Bayesian networks. Int. J. Artif. Intell. Tools 18, 653–671 (2009)CrossRefGoogle Scholar
- 14.Niculescu, R., Mitchell, T., Rao, B.: Bayesian network learning with parameter constraints. J. Mach. Learn. Res. 7, 1357–1383 (2006)MathSciNetMATHGoogle Scholar
- 15.Campos, C., Yan, T., Qiang, J.: Constrained maximum likelihood learning of Bayesian networks for facial action recognition. In: Proceedings of the Tenth European Conference on Computer Vision, pp. 168–181 (2008)Google Scholar
- 16.Campos, C., Qiang, J.: Improving Bayesian network parameter learning using constraints. In: Proceedings of the Nineteenth International Conference on Pattern Recognition, pp. 1–4 (2008)Google Scholar
- 17.Rui, C., Shoemaker, R., Wei, W.: A novel knowledge-driven systems biology approach for phenotype prediction upon genetic intervention. IEEE Trans. Comput. Biol. Bioinform. 1, 1170–1181 (2011)Google Scholar
- 18.Zhi-gao, G., Xiao-guang, G., Ruo-hai, D.: Learning Bayesian network parameters under dual constraints from small data set. Acta Automatica Sinica 40, 1509–1516 (2014)Google Scholar
- 19.Yun, Z., Fenton, N., Neil, M.: Bayesian network approach to multinomial parameter learning using data and expert judgments. Int. J. Approximate Reasoning 55, 1252–1268 (2014)MathSciNetCrossRefGoogle Scholar
- 20.Dasgupta, S.: The sample complexity of learning fixed-structure Bayesian networks. Mach. Learn. 29, 165–180 (1997)CrossRefGoogle Scholar
- 21.Boyd, S.: CVX: Matlab software for disciplined convex programming (2015). http://cvxr.com/cvx
- 22.Kullback, S., Leibler, R.: On information and sufficiency. Ann. Math. Stat. 22, 79–86 (1951)MathSciNetCrossRefGoogle Scholar
Copyright information
Open Access This chapter is distributed under the terms of the Creative Commons Attribution Noncommercial License, which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.