Parameter learning in hybrid Bayesian networks using prior knowledge

Data Mining and Knowledge Discovery volume 30, pages576–604(2016)Cite this article

Abstract

Mixtures of truncated basis functions have been recently proposed as a generalisation of mixtures of truncated exponentials and mixtures of polynomials for modelling univariate and conditional distributions in hybrid Bayesian networks. In this paper we analyse the problem of learning the parameters of marginal and conditional MoTBF densities when both prior knowledge and data are available. Incorporating prior knowledge provide a valuable tool for obtaining useful models, especially in domains of applications where data are costly or scarce, and prior knowledge is available from practitioners. We explore scenarios where the prior knowledge can be expressed as an MoTBF density that is afterwards combined with another MoTBF density estimated from the available data. The resulting model remains within the MoTBF class which is a convenient property from the point of view of inference in hybrid Bayesian networks. The performance of the proposed method is tested in a series of experiments carried out over synthetic and real data.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19

References

  1. Aguilera PA, Fernández A, Reche F, Rumí R (2010) Hybrid Bayesian network classifiers: application to species distribution models. Environ Model Softw 25:1630–1639

    Article  Google Scholar 

  2. Alcalá-Fdez J, Fernandez A, Luengo J, Derrac J, García S, Sánchez L, Herrera F (2011) Keel data-mining software tool: data set repository, integration of algorithms and experimental analysis framework. J Mult Valued Logic Soft Comput 17:255–287

    Google Scholar 

  3. Bache K, Lichman M (2013) UCI machine learning repository. http://archive.ics.uci.edu/ml

  4. Bernardo JM, Smith AF (2009) Bayesian theory, vol 405. Wiley, New York

    Google Scholar 

  5. Clemen R, Winkler R (1999) Combining probability distributions from experts in risk analysis. Risk Anal 19(2):187–203

    Google Scholar 

  6. Fernández A, Gámez JA, Rumí R, Salmerón A (2014) Data clustering using hidden variables in hybrid Bayesian networks. Prog Artif Intell 2:141–152

    Article  Google Scholar 

  7. Fernández A, Nielsen JD, Salmerón A (2010) Learning Bayesian networks for regression from incomplete databases. Int J Uncertain Fuzziness Knowl Based Syst 18:69–86

    MathSciNet  Article  Google Scholar 

  8. Fernández A, Pérez-Bernabé I, Rumí R, Salmerón A (2013) Incorporating prior knowledge when learning mixtures of truncated basis functions from data. In: Jaeger M, Nielsen TD, Viappiani P (eds) Proceedings of the 12th Scandinavian AI conference (SCAI’2013) pp 95–104

  9. Fernández A, Pérez-Bernabé I, Salmerón A (2013) On using the PC algorithm for learning continuous Bayesian networks: an Experimental Analysis. In: Proceedings of the 15th conference of the Spanish Association for Artificial Intelligence (CAEPIA’2013). Lecture Notes in Computer Science, vol 8109. Springer, Berlin, pp 342–351

  10. Fernández A, Rumí R, del Sagrado J, Salmerón A (2014) Supervised classification using hybrid probabilistic decision graphs. In: Proceedings of the 7th European workshop on probabilistic graphical models (PGM’2014). Lecture Notes in Artificial Intelligence, vol 8754. Springer, Berlin, pp 206–221

  11. Flores J, Gámez JA, Martínez AM, Salmerón A (2011) Mixtures of truncated exponentials in supervised classification: case study for the naive Bayes and averaged one-dependence estimators. In: Ventura S, Abraham A, Cios KJ, Romero C, Marcelloni F, Benítez JM, Gibaja EL (eds) Proceedings of the 11th international conference on intelligent systems design and applications (ISDA’2011), pp 593–598

  12. Heckerman D (1997) Bayesian networks for data mining. Data Min Knowl Discov 1:79–119

    Article  Google Scholar 

  13. Kanamori T, Takenouchi T (2013) Improving Logitboost with prior knowledge. Inf Fusion 14:208–219

    Article  Google Scholar 

  14. Langseth H, Nielsen T, Pérez-Bernabé I, Salmerón A (2014) Learning mixtures of truncated basis functions from data. Int J Approx Reason 55:940–956

    MathSciNet  Article  MATH  Google Scholar 

  15. Langseth H, Nielsen T, Rumí R, Salmerón A (2012) Mixtures of truncated basis functions. Int J Approx Reason 53:212–227

    MathSciNet  Article  MATH  Google Scholar 

  16. Langseth H, Nielsen T, Salmerón A (2012) Learning mixtures of truncated basis functions from data. In: Cano A, Gómez-Olmedo M, Nielsen TD (eds) Proceedings of the 6th European workshop on probabilistic graphical models (PGM’2012), pp 163–170

  17. Lauritzen S (1992) Propagation of probabilities, means and variances in mixed graphical association models. J Am Stat Assoc 87:1098–1108

    MathSciNet  Article  MATH  Google Scholar 

  18. López-Cruz PL, Bielza C, Larrañaga P (2012) Learning mixtures of polynomials from data using B-spline interpolation. In: Cano A, Gómez-Olmedo M, Nielsen TD (eds) Proceedings of the 6th European workshop on probabilistic graphical models (PGM’12), pp 211–218

  19. López-Cruz PL, Bielza C, Larrañaga P (2014) Learning mixtures of polynomials of multidimensional probability densities from data using B-spline interpolation. Int J Approx Reason 55:989–1010

    MathSciNet  Article  MATH  Google Scholar 

  20. Luengo JC, Rumí R (2015) Naive Bayes classifier with mixtures of polynomials. In: De Marsico M, Figueiredo M, Fred A (eds) Proceedings of the 4th international conference on pattern recognition applications and methods (ICPRAM’2015), vol 1, pp 14–24

  21. Moral S, Rumí R, Salmerón A (2001) Mixtures of truncated exponentials in hybrid Bayesian networks. In: Proceedings of the 6th European conference on symbolic and quantitative approaches to reasoning with uncertainty (ECSQARU’2001). Lecture Notes in Artificial Intelligence, vol 2143, pp 135–143

  22. Moral S, Rumí R, Salmerón A (2003) Approximating conditional MTE distributions by means of mixed trees. In: Proceedings of the 7th European conference on symbolic and quantitative approaches to reasoning with uncertainty (ECSQARU’2003). Lecture Notes in Artificial Intelligence, vol 2711, pp 173–183

  23. Morales M, Rodríguez C, Salmerón A (2007) Selective naive Bayes for regression using mixtures of truncated exponentials. Int J Uncertain Fuzziness Knowl Based Syst 15:697–716

    Article  MATH  Google Scholar 

  24. Pearl J (1988) Probabilistic reasoning in intelligent systems. Morgan-Kaufmann, San Mateo

    Google Scholar 

  25. R Development Core Team (2011) R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria. http://www.R-project.org/. ISBN 3-900051-07-0

  26. Rumí R, Salmerón A, Moral S (2006) Estimating mixtures of truncated exponentials in hybrid Bayesian networks. Test 15:397–421

    MathSciNet  Article  MATH  Google Scholar 

  27. Schwarz G (1978) Estimating the dimension of a model. Ann Stat 6:461–464

    MathSciNet  Article  MATH  Google Scholar 

  28. Shenoy P, Shafer G (1990) Axioms for probability and belief function propagation. In: Shachter R, Levitt T, Lemmer J, Kanal L (eds) Uncertainty in artificial intelligence 4. North Holland, Amsterdam, pp 169–198

    Google Scholar 

  29. Shenoy P, West J (2011) Inference in hybrid Bayesian networks using mixtures of polynomials. Int J Approx Reason 52:641–657

    MathSciNet  Article  MATH  Google Scholar 

  30. Wong T (2009) Alternative prior assumptions for improving the performance of naïve Bayesian classifiers. Data Min Knowl Discov 18:183–213

    MathSciNet  Article  Google Scholar 

  31. Zhang N, Poole D (1996) Exploiting causal independence in Bayesian network inference. J Artif Intell Res 5:301–328

    MathSciNet  MATH  Google Scholar 

Download references

Acknowledgments

This research has been partly funded by the Spanish Ministry of Economy and Competitiveness, through projects TIN2010-20900-C04-02 and TIN2013-46638-C3-1-P and by Junta de Andalucía through Project P11-TIC-7821 and by ERDF funds.

Author information

Affiliations

  1. Department of Mathematics, University of Almería, 04120, Almería, Spain

    Inmaculada Pérez-Bernabé, Antonio Fernández, Rafael Rumí & Antonio Salmerón

Authors
  1. Inmaculada Pérez-Bernabé
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Antonio Fernández
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Rafael Rumí
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Antonio Salmerón
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Inmaculada Pérez-Bernabé.

Additional information

A preliminary version of this paper was presented at the SCAI 2013 conference Fernández et al. (2013a).

Responsible editor: Pierre Baldi.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Pérez-Bernabé, I., Fernández, A., Rumí, R. et al. Parameter learning in hybrid Bayesian networks using prior knowledge. Data Min Knowl Disc 30, 576–604 (2016). https://doi.org/10.1007/s10618-015-0429-7

Download citation

Keywords

  • Hybrid Bayesian networks
  • Mixtures of truncated basis functions
  • Parameter learning
  • Prior information