Abstract
In their seminal 1990 paper, Wasserman and Kadane establish an upper bound for the Bayes’ posterior probability of a measurable set A, when the prior lies in a class of probability measures \(\mathcal {P}\) and the likelihood is precise. They also give a sufficient condition for such upper bound to hold with equality. In this paper, we introduce a generalization of their result by additionally addressing uncertainty related to the likelihood. We give an upper bound for the posterior probability when both the prior and the likelihood belong to a set of probabilities. Furthermore, we give a sufficient condition for this upper bound to become an equality. This result is interesting on its own, and has the potential of being applied to various fields of engineering (e.g. model predictive control), machine learning, and artificial intelligence.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Recall that in the weak\(^\star \) topology, a net \((P_\alpha )_{\alpha \in I}\) converges to P if and only if \(P_\alpha (A) \rightarrow P(A)\), for all \(A \in \mathcal {F}\). See also results presented in [29, Appendix D3].
References
Augustin, T., Coolen, F.P.A., De Cooman, G., Troffaes, M.C.M. (eds.): Introduction to Imprecise Probabilities. Wiley Series in Probability and Statistics, Wiley, Hoboken (2014)
Berger, J.O.: The robust Bayesian viewpoint. In: Kadane, J.B. (ed.) Robustness of Bayesian Analyses. North-Holland, Amsterdam (1984)
Caprio, M., et al.: Imprecise Bayesian neural networks (2023). arXiv:2302.09656
Caprio, M., Gong, R.: Dynamic imprecise probability kinematics. In: Proceedings of Machine Learning Research, vol. 215, pp. 72–83 (2023)
Casella, G.: An introduction to empirical Bayes data analysis. Am. Stat. 39(2), 83–87 (1985)
Cerreia-Vioglio, S., Maccheroni, F., Marinacci, M.: Ergodic theorems for lower probabilities. Proc. Am. Math. Soc. 144, 3381–3396 (2015)
Cozman, F.G.: Credal networks. Artif. Intell. 120, 199–233 (2000)
Depeweg, S., Hernandez-Lobato, J.M., Doshi-Velez, F., Udluft, S.: Decomposition of uncertainty in Bayesian deep learning for efficient and risk-sensitive learning. In: Dy, J., Krause, A. (eds.) Proceedings of the 35th International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 80, pp. 1184–1193. PMLR (2018)
Ellsberg, D.: Risk, ambiguity, and the Savage axioms. Q. J. Econ. 75(4), 643–669 (1961)
Epstein, L.G., Wang, T.: Intertemporal asset pricing under Knightian uncertainty. Econometrica 62(2), 283–322 (1994)
de Finetti, B.: Theory of Probability, vol. 1. Wiley, New York (1974)
de Finetti, B.: Theory of Probability, vol. 2. Wiley, New York (1975)
Giacomini, R., Kitagawa, T.: Robust Bayesian inference for set-identified models. Econometrica 89(4), 1519–1556 (2021)
Gilboa, I., Marinacci, M.: Ambiguity and the Bayesian paradigm. In: Acemoglu, D., Arellano, M., Dekel, E. (eds.) Advances in Economics and Econometrics, Tenth World Congress, vol. 1. Cambridge University Press, Cambridge (2013)
Gong, R., Meng, X.L.: Judicious judgment meets unsettling updating: dilation, sure loss, and Simpson’s paradox. Stat. Sci. 36(2), 169–190 (2021)
Grünwald, P., van Ommen, T.: Inconsistency of Bayesian inference for misspecified linear models, and a proposal for repairing it. Bayesian Anal. 12(4), 1069–1103 (2017)
Huber, P.J., Ronchetti, E.M.: Robust Statistics. Wiley Series in Probability and Statistics, 2nd edn. Wiley, Hoboken (2009)
Hüllermeier, E., Waegeman, W.: Aleatoric and epistemic uncertainty in machine learning: an introduction to concepts and methods. Mach. Learn. 3(110), 457–506 (2021)
Kapoor, S., Maddox, W.J., Izmailov, P., Wilson, A.G.: On uncertainty, tempering, and data augmentation in Bayesian classification. In: Koyejo, S., Mohamed, S., Agarwal, A., Belgrave, D., Cho, K., Oh, A. (eds.) Advances in Neural Information Processing Systems, vol. 35, pp. 18211–18225. Curran Associates, Inc. (2022). https://proceedings.neurips.cc/paper_files/paper/2022/file/73e018a0123b35a3e64269526f9096c9-Paper-Conference.pdf
Kendall, A., Gal, Y.: What uncertainties do we need in Bayesian deep learning for computer vision? In: Advances in Neural Information Processing Systems, vol. 30 (2017)
Klibanoff, P., Hanany, E.: Updating preferences with multiple priors. Theor. Econ. 2(3), 261–298 (2007)
Marinacci, M., Montrucchio, L.: Introduction to the mathematics of ambiguity. In: Gilboa, I. (ed.) Uncertainty in Economic Theory: A Collection of Essays in Honor of David Schmeidler’s 65th Birthday. Routledge, London (2004)
Rakovi, S.V., Levine, W.S.: Handbook of Model Predictive Control, 1st edn. Birkhäuser, Basel (2018)
Rudin, W.: Principles of Mathematical Analysis, 3rd edn. McGraw-Hill, New York (1976)
Sale, Y., Caprio, M., Hüllermeier, E.: Is the volume of a credal set a good measure for epistemic uncertainty? In: Evans, R.J., Shpitser, I. (eds.) Proceedings of the Thirty-Ninth Conference on Uncertainty in Artificial Intelligence. Proceedings of Machine Learning Research, vol. 216, pp. 1795–1804. PMLR (2023)
Smith, L., Gal, Y.: Understanding measures of uncertainty for adversarial example detection. arXiv preprint arXiv:1803.08533 (2018)
Troffaes, M.C., de Cooman, G.: Lower Previsions. Wiley, Chichester (2014)
Walley, P.: Coherent lower (and upper) probabilities. Technical report, University of Warwick, Coventry (1981)
Walley, P.: Statistical Reasoning with Imprecise Probabilities, Monographs on Statistics and Applied Probability, vol. 42. Chapman and Hall, London (1991)
Wasserman, L.A., Kadane, J.B.: Bayes’ theorem for Choquet capacities. Ann. Stat. 18(3), 1328–1339 (1990)
Wimmer, L., Sale, Y., Hofman, P., Bischl, B., Hüllermeier, E.: Quantifying aleatoric and epistemic uncertainty in machine learning: are conditional entropy and mutual information appropriate measures? In: Evans, R.J., Shpitser, I. (eds.) Proceedings of the Thirty-Ninth Conference on Uncertainty in Artificial Intelligence. Proceedings of Machine Learning Research, vol. 216, pp. 2282–2292. PMLR (2023)
Acknowledgments
Michele Caprio would like to acknowledge partial funding by the Army Research Office (ARO MURI W911NF2010080). Yusuf Sale is supported by the DAAD programme Konrad Zuse Schools of Excellence in Artificial Intelligence, sponsored by the Federal Ministry of Education and Research.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Ethics declarations
Disclosure of Interests
The authors have no competing interests to declare that are relevant to the content of this article.
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Caprio, M., Sale, Y., Hüllermeier, E., Lee, I. (2024). A Novel Bayes’ Theorem for Upper Probabilities. In: Cuzzolin, F., Sultana, M. (eds) Epistemic Uncertainty in Artificial Intelligence . Epi UAI 2023. Lecture Notes in Computer Science(), vol 14523. Springer, Cham. https://doi.org/10.1007/978-3-031-57963-9_1
Download citation
DOI: https://doi.org/10.1007/978-3-031-57963-9_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-57962-2
Online ISBN: 978-3-031-57963-9
eBook Packages: Computer ScienceComputer Science (R0)