Abstract
Over the years several nonequivalent probabilistic measures of coherence have been discussed in the philosophical literature. In this paper we examine these measures with respect to their empirical adequacy. Using test cases from the coherence literature as vignettes for psychological experiments we investigate whether the measures can predict the subjective coherence assessments of the participants. It turns out that the participants’ coherence assessments are best described by Roche’s (Insights from philosophy, jurisprudence and artificial intelligence, 2013) coherence measure based on Douven and Meijs’ (Synthese 156:405–425, 2007) average mutual support approach and the conditional probability.
Similar content being viewed by others
Notes
We excluded the Siebel and Schippers’ inconsistent testimony case from all analyses because most measures have undefined function values in this test case (see Appendix 2).
Six participants did not answer the questionnaire prior to the lab study and were therefore excluded from this analysis.
References
Akiba, K. (2000). Shogenji’s probabilistic measure of coherence is incoherent. Analysis, 60, 356–359.
BonJour, L. (1985). The structure of empirical knowledge. Cambridge: Harvard University Press.
Bovens, L., & Hartmann, S. (2003). Bayesian epistemology. Oxford: Oxford University Press.
Carnap, R. (1950). Logical foundations of probability. Chicago: University of Chicago Press.
Cheng, P. W. (1997). From covariation to causation: A causal power theory. Psychological Review, 104, 367–405.
Cialdini, R. B., Trost, M. R., & Newsom, J. T. (1995). Preference for consistency: The development of a valid measure and the discovery of surprising behavioral implications. Journal of Personality and Social Psychology, 69, 318–328.
Crupi, V., Tentori, K., & Gonzales, M. (2007). On Bayesian measures of evidential support: Theoretical and empirical issues. Philosophy of Science, 74, 229–252.
Douven, I., & Meijs, W. (2007). Measuring coherence. Synthese, 156, 405–425.
Festa, R. (2012). For unto every one that hath shall be given. Matthew properties for incremental confirmation. Synthese, 184, 89–100.
Finch, H. A. (1960). Confirming power of observations metricized for decisions among hypotheses. Philosophy of Science, 27, 293–307.
Fitelson, B. (2004). Two technical corrections to my coherence measure. http://fitelson.org/coherence2.
Fitelson, B. (2003). A probabilistic theory of coherence. Analysis, 63, 194–199.
Frederick, S. (2005). Cognitive reflection and decision making. Journal of Economic Perspectives, 19, 25–42.
Gaifman, H. (1979). Subjective probability, natural predicates and Hempel’s ravens. Erkenntnis, 21, 105–147.
Glass, D. H. (2002). Coherence, explanation, and Bayesian networks. In O’Neill, M., Sutcliffe, R. F. E., Ryan, C., Eaton, M., & Griffith, N. J. L. (Eds.), Artificial intelligence and cognitive science. 13th Irish conference, AICS 2002, Limerick, Ireland, September 2002 (pp. 177–182). Berlin: Springer.
Glass, D. H. (2005). Problems with priors in probabilistic measures of coherence. Erkenntnis, 63, 375–385.
Good, I. J. (1984). The best explicatum for weight of evidence. Journal of Statistical Computation and Simulation, 19, 294–299.
Greiner, B. (2004). An online recruitment system for economic experiments. In K. Kremer & V. Macho (Eds.), Forschung und wissenschaftliches Rechnen 2003, GWDG Bericht 63 (pp. 79–93). Goettingen: Ges. fuer Wiss. Datenverarbeitung.
Harris, A., & Hahn, U. (2009). Bayesian rationality in evaluating multiple testimonies: Incorporating the role of coherence. Journal of Experimental Psychology: Learning, Memory, and Cognition, 35(5), 1366–1373.
Jeffreys, H. (1961). Theory of probability. Oxford: Oxford University Press.
Joyce, J. (2008). Bayes’ theorem. http://plato.stanford.edu/archives/fall2008/entries/bayestheorem/.
Kemeny, J., & Oppenheim, P. (1952). Degrees of factual support. Philosophy of Science, 1952, 307–324.
Keynes, J. (1921). A treatise on probability. London: Macmillan.
Kolmogorov, A. (1956). Foundations of the theory of probability. New York: AMS Chelsea Publishing.
Koscholke, J. (2015). Evaluating test cases for probabilistic measures of coherence. Erkenntnis. doi:10.1007/s1067001597341.
Kuipers, T. A. F. (2000). From instrumentalism to constructive realism. Dordrecht: Reidel.
Levi, I. (1962). Corroboration and rules of acceptance. British Journal for the Philosophy of Science, 13, 307–313.
Meijs, W. (2005). Probabilistic measures of coherence. PhD thesis, Erasmus University, Rotterdam.
Meijs, W. (2006). Coherence as generalized logical equivalence. Erkenntnis, 64, 231–252.
Meijs, W., & Douven, I. (2007). On the alleged impossibility of coherence. Synthese, 157(3), 347–360.
Mortimer, H. (1988). The logic of induction. Paramus: Prentice Hall.
Nozick, R. (1981). Philosophical explanations. Oxford: Clarendon.
Olsson, E. J. (2002). What is the problem of coherence and truth? The Journal of Philosophy, 94, 246–272.
Olsson, E. J. (2005). Against coherence: Truth, probability and justification. Oxford: Oxford University Press.
Pfeiffer, P. (1990). Probability for applications. New York: Springer.
Pinheiro, J., Bates, D., DebRoy, S., Sarkar, D., & R Core Team (2013). nlme: Linear and nonlinear mixed effects models. http://CRAN.Rproject.org/package=nlme.
Popper, K. R. (1954). Degree of confirmation. British Journal for the Philosophy of Science, 5, 143–149.
R Core Team (2015). R: A language and environment for statistical computing. Vienna: R Foundation for Statistical Computing.
Rescher, N. (1958). Theory of evidence. Philosophy of Science, 25, 83–94.
Rescher, N. (1973). The coherence theory of truth. Oxford: Oxford University Press.
Rips, L. J. (2001). Two kinds of reasoning. Psychological Science, 12, 129–134.
Roche, W. (2013). Coherence and probability: A probabilistic account of coherence. In M. Araszkiewicz & J. Savelka (Eds.), Coherence: Insights from philosophy, jurisprudence and artificial intelligence (pp. 59–91). Dordrecht: Springer.
Schippers, M. (2014). Probabilistic measures of coherence: From adequacy constraints towards pluralism. Synthese, 191(16), 3821–3845.
Schippers, M., & Siebel, M. (2015). Inconsistency as a touchstone for coherence measures. Theoria, 30, 11–41.
Schupbach, J. N. (2011). New hope for Shogenji’s coherence measure. British Journal for the Philosophy of Science, 62(1), 125–142.
Schwarz, G. (1978). Estimating the dimension of a model. The Annals of Statistics, 6(2), 461–464.
Shogenji, T. (1999). Is coherence truth conducive? Analysis, 59, 338–345.
Shogenji, T. (2012). The degree of epistemic justification and the conjunction fallacy. Synthese, 184, 29–48.
Siebel, M. (2004). On Fitelson’s measure of coherence. Analysis, 64, 189–190.
Siebel, M., & Wolff, W. (2008). Equivalent testimonies as a touchstone of coherence measures. Synthese, 161, 167–182.
Wagenmakers, E. J. (2007). A practical solution to the pervasive problems of \(p\) values. Psychonomic Bulletin & Review, 14, 779–804.
Weller, J. A., Dieckmann, N. F., Tusler, M., Mertz, C. K., Burns, W. J., & Peters, E. (2013). Development and testing of an abbreviated numeracy scale: A Rasch analysis approach. Journal of Behavioral Decision Making, 26, 198–212.
Acknowledgments
We would like to thank (in alphabetical order) Arndt Bröder, Andreas Glöckner, Björn Meder, Michael Schippers and Mark Siebel for their contributions. We would also like to thank the participants of the Operationalization Workshop 2013 in Freiburg for helpful comments. This work was supported by grant SI 1731/11 to Mark Siebel and grant GL 632/31 and BR 2130/81 to Andreas Glöckner and Arndt Bröder from the Deutsche Forschungsgemeinschaft (DFG) as part of the priority program “New Frameworks of Rationality” (SPP 1516).
Author information
Authors and Affiliations
Corresponding author
Appendices
Appendix 1: Test cases
1.1 Akiba’s (2000) Die case
Imagine rolling a fair die and consider the following three statements:

\(S_1:\) The die comes up 2.

\(S_2:\) The die comes up 2 or 4.

\(S_3:\) The die comes up 2 or 4 or 6.
Which pair of statements fits together better. Statement 1 and 2 or statement 1 and 3?
1.2 Bovens and Hartmann’s (2003) tweety case

Situation 1:
Consider a population of 100 animals. 50 out of 100 animals are birds and 50 out of 100 animals cannot fly. Among these 100 animals there is exactly one animal that is a bird and cannot fly. Randomly pick one animal and consider the following two statements:

\(S_1:\) The picked animal is a bird.

\(S_2:\) The picked animal cannot fly.


Situation 2:
Consider a population of 100 animals. 50 out of 100 animals are birds and 50 out of 100 animals cannot fly. Among these 100 animals there is exactly one penguin and therefore a bird that cannot fly. Randomly pick one animal and consider the following three statements:

\(S_1:\) The picked animal is a bird.

\(S_2:\) The picked animal cannot fly.

\(S_3:\) The picked animal is a penguin.

In which of the two situations do the respective sets of statements fit together better?
1.3 Bovens and Hartmann’s (2003) Tokyo murder case
Imagine that a murder has occurred in Toyko and the corpse is still to be found. In order to search more efficiently the map of Tokyo is separated into 100 equallysized where the probability of finding the corpse is the same for each square. Now, 5 pairs of equally reliable and independent witnesses give the following statements as witness reports:

Pair 1:

\(S_1:\) The corpse is in squares 50 to 60.

\(S_2:\) The corpse is in squares 51 to 61.


Pair 2:

\(S_1:\) The corpse is in squares 22 to 55.

\(S_2:\) The corpse is in squares 55 to 90.


Pair 3:

\(S_1:\) The corpse is in squares 20 to 61.

\(S_2:\) The corpse is in squares 50 to 91.


Pair 4:

\(S_1:\) The corpse is in squares 41 to 60.

\(S_2:\) The corpse is in squares 51 to 70.


Pair 5:

\(S_1:\) The corpse is in squares 39 to 61.

\(S_2:\) The corpse is in squares 50 to 72.

Which pair of statements fits together best, which worst? Can you give and ordering where the first pair is the best and the last pair the worst?
1.4 Glass’ (2005) Dodecahedron case

Situation 1: You are rolling a fair die.

Situation 2: You are rolling a fair dodecahedron.
Now consider the following two statements:

\(S_1:\) The result will be 2.

\(S_2:\) The result will be 2 or 4.
In which of the two situation do these two statements fit together better?
1.5 Meijs’ (2005) Samurai case

Situation 1:
There are 10,000,000 suspects in a murder case. 1059 of them are Japanese and also 1059 own a samurai sword such that in total there are 9 suspects who are Japanese and own a samurai sword at the same time.

Situation 2:
There are 100 suspects in a murder case. 10 of them are Japanese and also 10 own a samurai sword such that in total there are 9 suspects who are Japanese and own a samurai sword at the same time.
Now consider the following two statements:

\(S_1:\) The murderer is Japanese.

\(S_2:\) The murderer owns a samurai sword.
In which of the two situations do the two statements fit together better?
1.6 Meijs’ (2006) Albino rabbit case

Situation 1:
There are 102 rabbits on the first island. 101 out of these 102 rabbits are grey. Also, 101 out of 102 rabbits have two ears. In total there are 100 out of 102 rabbits which are grey and have two ears at the same time. Consequently, there is exactly one rabbit which is grey but does not have two ears and exactly one rabbits which is not grey but has two ears.

Situation 2:
There are 102 rabbits on the second island, too. 100 out of these 102 rabbits are grey. Also, 100 out of 102 rabbits have two ears. In total there are 100 out of 102 rabbits which are grey and have two ears at the same time. Consequently, every grey rabbit has two ears and every rabbit that has two ears is also grey.
Now, randomly pick one rabbit and consider the following two statements:

\(S_1:\) The rabbit is grey.

\(S_2:\) The rabbits has two ears.
In which of the two situations do these two statements fit together better?
1.7 Meijs and Douven’s (2007) plane lottery case
Imagine the following lottery. The chances are 4 / 100 for flying to the North pole, 49 / 100 for flying to the South pole and 47 / 100 for flying to New Zealand. The probability for seeing a penguin at the North pole is 0, at the South pole it is 10 / 49 and in New Zealand it is 1 / 47. Now consider the following two situations in which when having landed one is confronted with two statements.

Situation 1:

\(S_1:\) You are landing at the North pole.

\(S_2:\) The animal you see is a penguin.


Situation 2:

\(S_1:\) You are landing at the South pole.

\(S_2:\) The animal you see is a penguin.

In which of the two situations do the respective statements fit together better?
1.8 Schippers and Siebel’s (2015) inconsistent testimony case
Imagine there are 8 suspects for a robbery. It is certain that exactly one of them is the robber. Consider the following two situations in which two independent and equally reliable witnesses give statements about the robber:

Situation 1:

\(S_1:\) The robbery was committed by suspect 1 or 2.

\(S_2:\) The robbery was committed by suspect 2 or 3.

\(S_3:\) The robbery was committed by suspect 1 or 3.


Situation 2:

\(S_1:\) The robbery was committed by suspect 1 or 2.

\(S_2:\) The robbery was committed by suspect 3 or 4.

\(S_3:\) The robbery was committed by suspect 5 or 6.

In which of these two situations do the respective sets of statements fit together better?
1.9 Schupbach’s (2011) Robber case
Imagine there are 10 suspects for a robbery. It is certain that exactly one of them is the robber. Consider the following two situations in which two independent and equally reliable witnesses make give statements about the robber:

Situation 1:

\(S_1:\) The robbery was committed by suspect 1 or 2 or 3.

\(S_2:\) The robbery was committed by suspect 1 or 2 or 4.

\(S_3:\) The robbery was committed by suspect 1 or 3 or 4.


Situation 2:

\(S_1:\) The robbery was committed by suspect 1 or 2 or 3.

\(S_2:\) The robbery was committed by suspect 1 or 4 or 5.

\(S_3:\) The robbery was committed by suspect 1 or 6 or 7.

In which of these two situations do the respective sets of statements fit together better?
1.10 Siebel’s (2004) pickpocketing robber case
Imagine the following situation. There are 10 equally likely suspects for a murder. 8 out of 10 have committed a pickpocketing before, 8 out of 10 have committed a robbery and in total 6 out of 10 have committed a pickpocketing and a robbery. Now consider the following two statements:

\(S_1:\) The murderer has committed a robbery.

\(S_2:\) The murderer has committed a pickpocketing.
Do these two statements fit together or not?
Appendix 2: Test case results
See Table 2.
Rights and permissions
About this article
Cite this article
Koscholke, J., Jekel, M. Probabilistic coherence measures: a psychological study of coherence assessment. Synthese 194, 1303–1322 (2017). https://doi.org/10.1007/s1122901509966
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s1122901509966