Abstract
Most hypothesis testing in machine learning is done using the frequentist null-hypothesis significance test, which has severe drawbacks. We review recent Bayesian tests which overcome the drawbacks of the frequentist ones.
Chapter PDF
Similar content being viewed by others
References
Demšar, J.: Statistical comparisons of classifiers over multiple data sets. Journal of Machine Learning Research 7, 1–30 (2006)
Garcıa, S., Herrera, F.: An extension on statistical comparisons of classifiers over multiple data sets for all pairwise comparisons. Journal of Machine Learning Research 9, 2677–2694 (2008)
Benavoli, A., Corani, G., Mangili, F.: Should we really use post-hoc tests based on mean-ranks? Journal of Machine Learning Research (2015) (in press)
Kruschke, J.: Doing Bayesian data analysis: A tutorial introduction with R. Academic Press (2010)
Müller, P., Parmigiani, G., Robert, C., Rousseau, J.: Optimal sample size for multiple testing: the case of gene expression microarrays. Journal of the American Statistical Association 99(468), 990–1001 (2004)
Benavoli, A., Mangili, F., Corani, G., Zaffalon, M., Ruggeri, F.: A Bayesian Wilcoxon signed-rank test based on the Dirichlet process. In: Proceedings of the 31st International Conference on Machine Learning (ICML 2014), pp. 1026–1034 (2014)
Elkan, C.: The foundations of cost-sensitive learning. In: International Joint Conference on Artificial Intelligence, vol. 17, pp. 973–978 (2001)
Corani, G., Benavoli, A.: A Bayesian approach for comparing cross-validated algorithms on multiple data sets. Machine Learning (2015) (in press)
Nadeau, C., Bengio, Y.: Inference for the generalization error. Machine Learning 52(3), 239–281 (2003)
Lacoste, A., Laviolette, F., Marchand, M.: Bayesian comparison of machine learning algorithms on single and multiple datasets. In: Proc. of the Fifteenth Int. Conf. on Artificial Intelligence and Statistics (AISTATS 2012), pp. 665–675 (2012)
Benavoli, A., Mangili, F., Corani, G., Zaffalon, M.: A Bayesian nonparametric procedure for comparing algorithms. In: Proceedings of the 32nd International Conference on Machine Learning (ICML 2015) (2015) (in press)
Ferguson, T.S.: A Bayesian analysis of some nonparametric problems. The Annals of Statistics 1(2), 209–230 (1973)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Corani, G., Benavoli, A., Mangili, F., Zaffalon, M. (2015). Bayesian Hypothesis Testing in Machine Learning. In: Bifet, A., et al. Machine Learning and Knowledge Discovery in Databases. ECML PKDD 2015. Lecture Notes in Computer Science(), vol 9286. Springer, Cham. https://doi.org/10.1007/978-3-319-23461-8_13
Download citation
DOI: https://doi.org/10.1007/978-3-319-23461-8_13
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-23460-1
Online ISBN: 978-3-319-23461-8
eBook Packages: Computer ScienceComputer Science (R0)