Data Mining and Knowledge Discovery

, Volume 30, Issue 3, pp 576–604 | Cite as

Parameter learning in hybrid Bayesian networks using prior knowledge

  • Inmaculada Pérez-Bernabé
  • Antonio Fernández
  • Rafael Rumí
  • Antonio Salmerón
Article

Abstract

Mixtures of truncated basis functions have been recently proposed as a generalisation of mixtures of truncated exponentials and mixtures of polynomials for modelling univariate and conditional distributions in hybrid Bayesian networks. In this paper we analyse the problem of learning the parameters of marginal and conditional MoTBF densities when both prior knowledge and data are available. Incorporating prior knowledge provide a valuable tool for obtaining useful models, especially in domains of applications where data are costly or scarce, and prior knowledge is available from practitioners. We explore scenarios where the prior knowledge can be expressed as an MoTBF density that is afterwards combined with another MoTBF density estimated from the available data. The resulting model remains within the MoTBF class which is a convenient property from the point of view of inference in hybrid Bayesian networks. The performance of the proposed method is tested in a series of experiments carried out over synthetic and real data.

Keywords

Hybrid Bayesian networks Mixtures of truncated basis functions Parameter learning Prior information 

References

  1. Aguilera PA, Fernández A, Reche F, Rumí R (2010) Hybrid Bayesian network classifiers: application to species distribution models. Environ Model Softw 25:1630–1639CrossRefGoogle Scholar
  2. Alcalá-Fdez J, Fernandez A, Luengo J, Derrac J, García S, Sánchez L, Herrera F (2011) Keel data-mining software tool: data set repository, integration of algorithms and experimental analysis framework. J Mult Valued Logic Soft Comput 17:255–287Google Scholar
  3. Bache K, Lichman M (2013) UCI machine learning repository. http://archive.ics.uci.edu/ml
  4. Bernardo JM, Smith AF (2009) Bayesian theory, vol 405. Wiley, New YorkGoogle Scholar
  5. Clemen R, Winkler R (1999) Combining probability distributions from experts in risk analysis. Risk Anal 19(2):187–203Google Scholar
  6. Fernández A, Gámez JA, Rumí R, Salmerón A (2014) Data clustering using hidden variables in hybrid Bayesian networks. Prog Artif Intell 2:141–152CrossRefGoogle Scholar
  7. Fernández A, Nielsen JD, Salmerón A (2010) Learning Bayesian networks for regression from incomplete databases. Int J Uncertain Fuzziness Knowl Based Syst 18:69–86MathSciNetCrossRefGoogle Scholar
  8. Fernández A, Pérez-Bernabé I, Rumí R, Salmerón A (2013) Incorporating prior knowledge when learning mixtures of truncated basis functions from data. In: Jaeger M, Nielsen TD, Viappiani P (eds) Proceedings of the 12th Scandinavian AI conference (SCAI’2013) pp 95–104Google Scholar
  9. Fernández A, Pérez-Bernabé I, Salmerón A (2013) On using the PC algorithm for learning continuous Bayesian networks: an Experimental Analysis. In: Proceedings of the 15th conference of the Spanish Association for Artificial Intelligence (CAEPIA’2013). Lecture Notes in Computer Science, vol 8109. Springer, Berlin, pp 342–351Google Scholar
  10. Fernández A, Rumí R, del Sagrado J, Salmerón A (2014) Supervised classification using hybrid probabilistic decision graphs. In: Proceedings of the 7th European workshop on probabilistic graphical models (PGM’2014). Lecture Notes in Artificial Intelligence, vol 8754. Springer, Berlin, pp 206–221Google Scholar
  11. Flores J, Gámez JA, Martínez AM, Salmerón A (2011) Mixtures of truncated exponentials in supervised classification: case study for the naive Bayes and averaged one-dependence estimators. In: Ventura S, Abraham A, Cios KJ, Romero C, Marcelloni F, Benítez JM, Gibaja EL (eds) Proceedings of the 11th international conference on intelligent systems design and applications (ISDA’2011), pp 593–598Google Scholar
  12. Heckerman D (1997) Bayesian networks for data mining. Data Min Knowl Discov 1:79–119CrossRefGoogle Scholar
  13. Kanamori T, Takenouchi T (2013) Improving Logitboost with prior knowledge. Inf Fusion 14:208–219CrossRefGoogle Scholar
  14. Langseth H, Nielsen T, Pérez-Bernabé I, Salmerón A (2014) Learning mixtures of truncated basis functions from data. Int J Approx Reason 55:940–956MathSciNetCrossRefMATHGoogle Scholar
  15. Langseth H, Nielsen T, Rumí R, Salmerón A (2012) Mixtures of truncated basis functions. Int J Approx Reason 53:212–227MathSciNetCrossRefMATHGoogle Scholar
  16. Langseth H, Nielsen T, Salmerón A (2012) Learning mixtures of truncated basis functions from data. In: Cano A, Gómez-Olmedo M, Nielsen TD (eds) Proceedings of the 6th European workshop on probabilistic graphical models (PGM’2012), pp 163–170Google Scholar
  17. Lauritzen S (1992) Propagation of probabilities, means and variances in mixed graphical association models. J Am Stat Assoc 87:1098–1108MathSciNetCrossRefMATHGoogle Scholar
  18. López-Cruz PL, Bielza C, Larrañaga P (2012) Learning mixtures of polynomials from data using B-spline interpolation. In: Cano A, Gómez-Olmedo M, Nielsen TD (eds) Proceedings of the 6th European workshop on probabilistic graphical models (PGM’12), pp 211–218Google Scholar
  19. López-Cruz PL, Bielza C, Larrañaga P (2014) Learning mixtures of polynomials of multidimensional probability densities from data using B-spline interpolation. Int J Approx Reason 55:989–1010MathSciNetCrossRefMATHGoogle Scholar
  20. Luengo JC, Rumí R (2015) Naive Bayes classifier with mixtures of polynomials. In: De Marsico M, Figueiredo M, Fred A (eds) Proceedings of the 4th international conference on pattern recognition applications and methods (ICPRAM’2015), vol 1, pp 14–24Google Scholar
  21. Moral S, Rumí R, Salmerón A (2001) Mixtures of truncated exponentials in hybrid Bayesian networks. In: Proceedings of the 6th European conference on symbolic and quantitative approaches to reasoning with uncertainty (ECSQARU’2001). Lecture Notes in Artificial Intelligence, vol 2143, pp 135–143Google Scholar
  22. Moral S, Rumí R, Salmerón A (2003) Approximating conditional MTE distributions by means of mixed trees. In: Proceedings of the 7th European conference on symbolic and quantitative approaches to reasoning with uncertainty (ECSQARU’2003). Lecture Notes in Artificial Intelligence, vol 2711, pp 173–183Google Scholar
  23. Morales M, Rodríguez C, Salmerón A (2007) Selective naive Bayes for regression using mixtures of truncated exponentials. Int J Uncertain Fuzziness Knowl Based Syst 15:697–716CrossRefMATHGoogle Scholar
  24. Pearl J (1988) Probabilistic reasoning in intelligent systems. Morgan-Kaufmann, San MateoMATHGoogle Scholar
  25. R Development Core Team (2011) R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria. http://www.R-project.org/. ISBN 3-900051-07-0
  26. Rumí R, Salmerón A, Moral S (2006) Estimating mixtures of truncated exponentials in hybrid Bayesian networks. Test 15:397–421MathSciNetCrossRefMATHGoogle Scholar
  27. Schwarz G (1978) Estimating the dimension of a model. Ann Stat 6:461–464MathSciNetCrossRefMATHGoogle Scholar
  28. Shenoy P, Shafer G (1990) Axioms for probability and belief function propagation. In: Shachter R, Levitt T, Lemmer J, Kanal L (eds) Uncertainty in artificial intelligence 4. North Holland, Amsterdam, pp 169–198Google Scholar
  29. Shenoy P, West J (2011) Inference in hybrid Bayesian networks using mixtures of polynomials. Int J Approx Reason 52:641–657MathSciNetCrossRefMATHGoogle Scholar
  30. Wong T (2009) Alternative prior assumptions for improving the performance of naïve Bayesian classifiers. Data Min Knowl Discov 18:183–213MathSciNetCrossRefGoogle Scholar
  31. Zhang N, Poole D (1996) Exploiting causal independence in Bayesian network inference. J Artif Intell Res 5:301–328MathSciNetMATHGoogle Scholar

Copyright information

© The Author(s) 2015

Authors and Affiliations

  • Inmaculada Pérez-Bernabé
    • 1
  • Antonio Fernández
    • 1
  • Rafael Rumí
    • 1
  • Antonio Salmerón
    • 1
  1. 1.Department of MathematicsUniversity of AlmeríaAlmeríaSpain

Personalised recommendations