Skip to main content

Credal Model Averaging: An Extension of Bayesian Model Averaging to Imprecise Probabilities

  • Conference paper

Part of the Lecture Notes in Computer Science book series (LNAI,volume 5211)

Abstract

We deal with the arbitrariness in the choice of the prior over the models in Bayesian model averaging (BMA), by modelling prior knowledge by a set of priors (i.e., a prior credal set). We consider Dash and Cooper’s BMA applied to naive Bayesian networks, replacing the single prior over the naive models by a credal set; this models a condition close to prior ignorance about the models, which leads to credal model averaging (CMA). CMA returns an indeterminate classification, i.e., multiple classes, on the instances for which the learning set is not informative enough to smooth the effect of the choice of the prior. We give an algorithm to compute exact credal model averaging for naive networks. Extensive experiments show that indeterminate classifications preserve the reliability of CMA on the instances which are classified in a prior-dependent way by BMA.

Keywords

  • Credal model averaging
  • Bayesian model averaging
  • imprecise probabilities
  • naive Bayes
  • classification
  • naive Bayesian networks

References

  1. Hoeting, J., Madigan, D., Raftery, A., Volinsky, C.: Bayesian Model Averaging: a Tutorial. Statistical Science 14(4), 382–417 (1999)

    MATH  CrossRef  MathSciNet  Google Scholar 

  2. Clyde, M., George, E.I.: Model Uncertainty. Statistical Science 19, 81–94 (2004)

    MATH  CrossRef  MathSciNet  Google Scholar 

  3. Boullé, M.: Compression-Based Averaging of Selective Naive Bayes Classifiers. Journal of Machine Learning Research 8, 1659–1685 (2007)

    Google Scholar 

  4. Dash, D., Cooper, G.: Exact Model Averaging with Naive Bayesian Classifiers. In: Proceedings of the Nineteenth International Conference on Machine Learning, pp. 91–98 (2002)

    Google Scholar 

  5. Walley, P.: Statistical Reasoning with Imprecise Probabilities. Chapman and Hall, New York (1991)

    MATH  Google Scholar 

  6. Zaffalon, M.: The Naive Credal Classifier. Journal of Statistical Planning and Inference 105(1), 5–21 (2002)

    MATH  CrossRef  MathSciNet  Google Scholar 

  7. Corani, G., Zaffalon, M.: Learning Reliable Classifiers from Small or Incomplete Data Sets: the Naive Credal Classifier 2. Journal of Machine Learning Research 9, 581–621 (2008)

    Google Scholar 

  8. Abellán, J., Moral, S.: A New Score for Independence Based on the Imprecise Dirichlet model. In: Cozman, F.G., Nau, R., Seidenfeld, T. (eds.) ISIPTA 2005: Proceedings of the Fourth International Symposium on Imprecise Probabilities and Their Applications, Manno, Switzerland, pp. 1–10. SIPTA (2005)

    Google Scholar 

  9. Fayyad, U.M., Irani, K.B.: Multi-interval Discretization of Continuous-valued Attributes for Classification Learning. In: Proceedings of the 13th International Joint Conference on Artificial Intelligence, pp. 1022–1027. Morgan Kaufmann, San Francisco (1993)

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2008 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Corani, G., Zaffalon, M. (2008). Credal Model Averaging: An Extension of Bayesian Model Averaging to Imprecise Probabilities. In: Daelemans, W., Goethals, B., Morik, K. (eds) Machine Learning and Knowledge Discovery in Databases. ECML PKDD 2008. Lecture Notes in Computer Science(), vol 5211. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-87479-9_35

Download citation

  • DOI: https://doi.org/10.1007/978-3-540-87479-9_35

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-87478-2

  • Online ISBN: 978-3-540-87479-9

  • eBook Packages: Computer ScienceComputer Science (R0)