Evaluating Test Cases for Probabilistic Measures of Coherence
- 207 Downloads
How can we determine the adequacy of a probabilistic coherence measure? A widely accepted approach to this question besides formulating adequacy constraints is to employ paradigmatic test cases consisting of a scenario providing a joint probability distribution over some specified set of propositions coupled with a normative coherence assessment for this set. However, despite the popularity of the test case approach, a systematic evaluation of the proposed test cases is still missing. This paper’s aim is to change this. Using a custom written computer program for the necessary probabilistic calculations a large number of coherence measures in an extensive collection of test cases is examined. The result is a detailed overview of the test case performance of any probabilistic coherence measures proposed so far. It turns out that none of the popular coherence measures such as Shogenji’s, Glass’ and Olsson’s, Fitelson’s or Douven and Meijs’ but two rather unnoticed measures perform best. This, however, does not mean that the other measures can be rejected straightforwardly. Instead, the results presented here are to be understood as a contribution among others to the project of finding adequate probabilistic coherence measures.
KeywordsSupport Measure Joint Probability Distribution Coherence Measure Relative Score Alternative Generalization
I would like to thank Michael Schippers and Mark Siebel for helpful comments or discussion. This work was supported by grant SI 1731/1-1 to Mark Siebel from the Deutsche Forschungsgemeinschaft (DFG) as part of the priority program “New Frameworks of Rationality” (SPP 1516).
- BonJour, L. (1985). The structure of empirical knowledge. Cambridge: Harvard University Press.Google Scholar
- Bovens, L., & Hartmann, S. (2003). Bayesian epistemology. Oxford: Oxford University Press.Google Scholar
- Brewka, G. (1991). Nonmonotonic reasoning: Logical foundations of commonsense. Cambridge: Cambridge University Press.Google Scholar
- Carnap, R. (1950). Logical foundations of probability. Chicago: University of Chicago Press.Google Scholar
- Fitelson, B. (2004). Two technical corrections to my coherence measure. http://fitelson.org/coherence2.
- Gaifman, H. (1979). Subjective probability, natural predicates and Hempel’s ravens. Erkenntnis, 21, 105–147.Google Scholar
- Glass, D. H. (2002). Coherence, explanation, and Bayesian networks. In M. O’Neill, R. F. E. Sutcliffe, C. Ryan, M. Eaton & N. J. L. Griffith (Eds.), Artificial intelligence and cognitive science (pp. 177–182). 13th Irish conference, AICS 2002, Limerick, Ireland, September 2002. Berlin: Springer.Google Scholar
- Harris, A., & Hahn, U. (2009). Bayesian rationality in evaluating multiple testimonies: Incorporating the role of coherence. Journal of Experimental Psychology: Learning, Memory, and Cognition, 35(5), 1366–1373.Google Scholar
- Jekel, M., & Koscholke, J. (2013). An empirical study of coherence assessment (unpublished manuscript).Google Scholar
- Joyce, J. (2008). Bayes’ theorem. http://plato.stanford.edu/archives/fall2008/entries/bayes-theorem/.
- Keynes, J. (1921). A treatise on probability. London: Macmillan.Google Scholar
- Levi, I. (1962). Corroboration and rules of acceptance. British Journal for the Philosophy of Science, 13, 307–313.Google Scholar
- Meijs, W. (2005). Probabilistic measures of coherence. PhD thesis, Erasmus University, Rotterdam.Google Scholar
- Mortimer, H. (1988). The logic of induction. Paramus: Prentice Hall.Google Scholar
- Nozick, R. (1981). Philosophical explanations. Oxford: Clarendon.Google Scholar
- Olsson, E. J. (2013). Coherentist theories of epistemic justification. http://plato.stanford.edu/entries/justep-coherence/.
- Rescher, N. (1973). The coherence theory of truth. Oxford: Oxford University Press.Google Scholar