, Volume 81, Issue 1, pp 155–181 | Cite as

Evaluating Test Cases for Probabilistic Measures of Coherence

  • Jakob Koscholke
Original Article


How can we determine the adequacy of a probabilistic coherence measure? A widely accepted approach to this question besides formulating adequacy constraints is to employ paradigmatic test cases consisting of a scenario providing a joint probability distribution over some specified set of propositions coupled with a normative coherence assessment for this set. However, despite the popularity of the test case approach, a systematic evaluation of the proposed test cases is still missing. This paper’s aim is to change this. Using a custom written computer program for the necessary probabilistic calculations a large number of coherence measures in an extensive collection of test cases is examined. The result is a detailed overview of the test case performance of any probabilistic coherence measures proposed so far. It turns out that none of the popular coherence measures such as Shogenji’s, Glass’ and Olsson’s, Fitelson’s or Douven and Meijs’ but two rather unnoticed measures perform best. This, however, does not mean that the other measures can be rejected straightforwardly. Instead, the results presented here are to be understood as a contribution among others to the project of finding adequate probabilistic coherence measures.


Support Measure Joint Probability Distribution Coherence Measure Relative Score Alternative Generalization 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



I would like to thank Michael Schippers and Mark Siebel for helpful comments or discussion. This work was supported by grant SI 1731/1-1 to Mark Siebel from the Deutsche Forschungsgemeinschaft (DFG) as part of the priority program “New Frameworks of Rationality” (SPP 1516).


  1. Akiba, K. (2000). Shogenji’s probabilistic measure of coherence is incoherent. Analysis, 60, 356–359.CrossRefGoogle Scholar
  2. BonJour, L. (1985). The structure of empirical knowledge. Cambridge: Harvard University Press.Google Scholar
  3. Bovens, L., & Hartmann, S. (2003). Bayesian epistemology. Oxford: Oxford University Press.Google Scholar
  4. Bovens, L., & Hartmann, S. (2005). Why there cannot be a single probabilistic measure of coherence. Erkenntnis, 63, 361–374.CrossRefGoogle Scholar
  5. Brewka, G. (1991). Nonmonotonic reasoning: Logical foundations of commonsense. Cambridge: Cambridge University Press.Google Scholar
  6. Carnap, R. (1950). Logical foundations of probability. Chicago: University of Chicago Press.Google Scholar
  7. Cheng, P. W. (1997). From covariation to causation: A causal power theory. Psychological Review, 104, 367–405.CrossRefGoogle Scholar
  8. Christensen, D. (1999). Measuring confirmation. Journal of Philosophy, 96, 437–461.CrossRefGoogle Scholar
  9. Crupi, V., Tentori, K., & Gonzales, M. (2007). On Bayesian measures of evidential support: Theoretical and empirical issues. Philosophy of Science, 74, 229–252.CrossRefGoogle Scholar
  10. Douven, I., & Meijs, W. (2007). Measuring coherence. Synthese, 156, 405–425.CrossRefGoogle Scholar
  11. Eells, E., & Fitelson, B. (2002). Symmetries and asymmetries in evidential support. Philosophical Studies, 107, 129–142.CrossRefGoogle Scholar
  12. Finch, H. A. (1960). Confirming power of observations metricized for decisions among hypotheses. Philosophy of Science, 27, 293–307.CrossRefGoogle Scholar
  13. Fitelson, B. (2003). A probabilistic theory of coherence. Analysis, 63, 194–199.CrossRefGoogle Scholar
  14. Fitelson, B. (2004). Two technical corrections to my coherence measure.
  15. Gaifman, H. (1979). Subjective probability, natural predicates and Hempel’s ravens. Erkenntnis, 21, 105–147.Google Scholar
  16. Glass, D. H. (2002). Coherence, explanation, and Bayesian networks. In M. O’Neill, R. F. E. Sutcliffe, C. Ryan, M. Eaton & N. J. L. Griffith (Eds.), Artificial intelligence and cognitive science (pp. 177–182). 13th Irish conference, AICS 2002, Limerick, Ireland, September 2002. Berlin: Springer.Google Scholar
  17. Glass, D. H. (2005). Problems with priors in probabilistic measures of coherence. Erkenntnis, 63, 375–385.CrossRefGoogle Scholar
  18. Good, I. J. (1984). The best explicatum for weight of evidence. Journal of Statistical Computation and Simulation, 19, 294–299.CrossRefGoogle Scholar
  19. Harris, A., & Hahn, U. (2009). Bayesian rationality in evaluating multiple testimonies: Incorporating the role of coherence. Journal of Experimental Psychology: Learning, Memory, and Cognition, 35(5), 1366–1373.Google Scholar
  20. Jekel, M., & Koscholke, J. (2013). An empirical study of coherence assessment (unpublished manuscript).Google Scholar
  21. Kemeny, J., & Oppenheim, P. (1952). Degrees of factual support. Philosophy of Science, 1952, 307–324.CrossRefGoogle Scholar
  22. Keynes, J. (1921). A treatise on probability. London: Macmillan.Google Scholar
  23. Kuipers, T. A. F. (2000). From instrumentalism to constructive realism. Dordrecht: Reidel.CrossRefGoogle Scholar
  24. Levi, I. (1962). Corroboration and rules of acceptance. British Journal for the Philosophy of Science, 13, 307–313.Google Scholar
  25. Meijs, W. (2005). Probabilistic measures of coherence. PhD thesis, Erasmus University, Rotterdam.Google Scholar
  26. Meijs, W. (2006). Coherence as generalized logical equivalence. Erkenntnis, 64, 231–252.CrossRefGoogle Scholar
  27. Meijs, W., & Douven, I. (2005). Bovens and Hartmann on coherence. Mind, 114, 355–363.CrossRefGoogle Scholar
  28. Moretti, L., & Akiba, K. (2007). Probabilistic measures of coherence and the problem of belief individuation. Synthese, 154, 73–95.CrossRefGoogle Scholar
  29. Mortimer, H. (1988). The logic of induction. Paramus: Prentice Hall.Google Scholar
  30. Nozick, R. (1981). Philosophical explanations. Oxford: Clarendon.Google Scholar
  31. Olsson, E. J. (2002). What is the problem of coherence and truth? The Journal of Philosophy, 94, 246–272.CrossRefGoogle Scholar
  32. Olsson, E. J. (2005). Against coherence: Truth, probability and justification. Oxford: Oxford University Press.CrossRefGoogle Scholar
  33. Olsson, E. J. (2013). Coherentist theories of epistemic justification.
  34. Popper, K. R. (1954). Degree of confirmation. British Journal for the Philosophy of Science, 5, 143–149.CrossRefGoogle Scholar
  35. Rescher, N. (1958). Theory of evidence. Philosophy of Science, 25, 83–94.CrossRefGoogle Scholar
  36. Rescher, N. (1973). The coherence theory of truth. Oxford: Oxford University Press.Google Scholar
  37. Rips, L. J. (2001). Two kinds of reasoning. Psychological Science, 12, 129–134.CrossRefGoogle Scholar
  38. Roche, W. (2013). Coherence and probability: A probabilistic account of coherence. In M. Araszkiewicz & J. Savelka (Eds.), Coherence: Insights from philosophy, jurisprudence and artificial intelligence (pp. 59–91). Dordrecht: Springer.CrossRefGoogle Scholar
  39. Schippers, M. (2014). Probabilistic measures of coherence: From adequacy constraints towards pluralism. Synthese, 191(16), 3821–3845.CrossRefGoogle Scholar
  40. Schupbach, J. N. (2011). New hope for Shogenji’s coherence measure. British Journal for the Philosophy of Science, 62(1), 125–142.CrossRefGoogle Scholar
  41. Shogenji, T. (1999). Is coherence truth conducive? Analysis, 59, 338–345.CrossRefGoogle Scholar
  42. Shogenji, T. (2012). The degree of epistemic justification and the conjunction fallacy. Synthese, 184, 29–48.CrossRefGoogle Scholar
  43. Siebel, M. (2004). On Fitelson’s measure of coherence. Analysis, 64, 189–190.CrossRefGoogle Scholar
  44. Siebel, M. (2005). Against probabilistic measures of coherence. Erkenntnis, 63, 335–360.CrossRefGoogle Scholar
  45. Siebel, M., & Wolff, W. (2008). Equivalent testimonies as a touchstone of coherence measures. Synthese, 161, 167–182.CrossRefGoogle Scholar
  46. Tentori, K., Crupi, V., Bonini, N., & Osherson, D. (2007). Comparison of confirmation measures. Cognition, 103, 107–119.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2015

Authors and Affiliations

  1. 1.Philosophy DepartmentUniversity of OldenburgOldenburgGermany

Personalised recommendations