Abstract
It is sometimes claimed that the Bayesian framework automatically implements Ockham’s razor—that conditionalizing on data consistent with both a simple theory and a complex theory more or less inevitably favours the simpler theory. It is shown here that the automatic razor doesn’t in fact cut it for certain mundane curve-fitting problems.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
Here and throughout, these theories are to be understood as being incompatible: a polynomial of degree k is required to have a non-zero coefficient for \(x^k.\)
The basic phenomenon that drives the argument of Sect. 3 below arises whether we work with real or rational variables and polynomial coefficients: whatever probability a prior assigns to the linear polynomials, it assigns almost all of this probability to some bounded subset of the space of linear polynomials, and hence all but rules out linear polynomials of relatively large slope or with relatively large intercepts (thanks to an anonymous referee for this way of putting the point). The restriction to rational variables and coefficients allows the consequences of this phenomenon to be brought out in an especially stark fashion.
For a related point made in a somewhat different context, see Seidenfeld (1979, pp. 414 f.).
For a claim that something along these lines does in fact hold in contexts like ours, see Rosenkrantz (1983, p. 82).
Bayesians might rather rely on a Pr-relative notion of typicality of data sets at this point. But such a notion is not easy to come by in our context, since Pr doesn’t assign probabilities to the proposition that the first data points are given by D or that the first values of x sampled are given by \(\Delta .\)
References
Henderson, L., Goodman, N., Tenenbaum, J., & Woodward, J. (2010). The structure and dynamics of scientific theories: A hierarchical Bayesian perspective. Philosophy of Science, 77, 172–200.
Jefferys, W., & Berger, J. (1992). Ockham’s razor and Bayesian analysis. American Scientist, 80, 64–72.
Kelly, K., & Glymour, C. (2004). Why Bayesian confirmation does not capture the logic of scientific justification. In C. Hitchcock (Ed.), Contemporary debates in philosophy of science (pp. 94–114). Oxford: Blackwell.
McKay, D. (2003). Information theory, inference, and learning algorithms. Cambridge: Cambridge University Press.
Rosenkrantz, R. (1983). Why Glymour is a Bayesian. In J. Earman (Ed.), Testing scientific theories (pp. 69–97). Minneapolis: University of Minnesota Press.
Seidenfeld, T. (1979). Why I am not an objective Bayesian; some reflections prompted by Rosenkrantz. Theory and Decision, 11, 413–440.
White, R. (2005). Why favour simplicity? Analysis, 65, 205–210.
Acknowledgements
This material was presented at the Ninth Workshop in Decisions, Games, and Logic at the University of Michigan and at the Workshop on Probability and Learning at Columbia University. For helpful comments and discussion, thanks to Kenny Easwaran, Jim Joyce, Laura Ruetsche, and three very helpful anonymous referees.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Belot, G. An Automatic Ockham’s Razor for Bayesians?. Erkenn 84, 1361–1367 (2019). https://doi.org/10.1007/s10670-018-0011-y
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10670-018-0011-y