Skip to main content
Log in

Probabilistic coherence measures: a psychological study of coherence assessment

  • Published:
Synthese Aims and scope Submit manuscript

Abstract

Over the years several non-equivalent probabilistic measures of coherence have been discussed in the philosophical literature. In this paper we examine these measures with respect to their empirical adequacy. Using test cases from the coherence literature as vignettes for psychological experiments we investigate whether the measures can predict the subjective coherence assessments of the participants. It turns out that the participants’ coherence assessments are best described by Roche’s (Insights from philosophy, jurisprudence and artificial intelligence, 2013) coherence measure based on Douven and Meijs’ (Synthese 156:405–425, 2007) average mutual support approach and the conditional probability.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Notes

  1. We excluded the Siebel and Schippers’ inconsistent testimony case from all analyses because most measures have undefined function values in this test case (see Appendix 2).

  2. Six participants did not answer the questionnaire prior to the lab study and were therefore excluded from this analysis.

References

  • Akiba, K. (2000). Shogenji’s probabilistic measure of coherence is incoherent. Analysis, 60, 356–359.

    Article  Google Scholar 

  • BonJour, L. (1985). The structure of empirical knowledge. Cambridge: Harvard University Press.

    Google Scholar 

  • Bovens, L., & Hartmann, S. (2003). Bayesian epistemology. Oxford: Oxford University Press.

    Google Scholar 

  • Carnap, R. (1950). Logical foundations of probability. Chicago: University of Chicago Press.

    Google Scholar 

  • Cheng, P. W. (1997). From covariation to causation: A causal power theory. Psychological Review, 104, 367–405.

    Article  Google Scholar 

  • Cialdini, R. B., Trost, M. R., & Newsom, J. T. (1995). Preference for consistency: The development of a valid measure and the discovery of surprising behavioral implications. Journal of Personality and Social Psychology, 69, 318–328.

    Article  Google Scholar 

  • Crupi, V., Tentori, K., & Gonzales, M. (2007). On Bayesian measures of evidential support: Theoretical and empirical issues. Philosophy of Science, 74, 229–252.

    Article  Google Scholar 

  • Douven, I., & Meijs, W. (2007). Measuring coherence. Synthese, 156, 405–425.

    Article  Google Scholar 

  • Festa, R. (2012). For unto every one that hath shall be given. Matthew properties for incremental confirmation. Synthese, 184, 89–100.

    Article  Google Scholar 

  • Finch, H. A. (1960). Confirming power of observations metricized for decisions among hypotheses. Philosophy of Science, 27, 293–307.

    Article  Google Scholar 

  • Fitelson, B. (2004). Two technical corrections to my coherence measure. http://fitelson.org/coherence2.

  • Fitelson, B. (2003). A probabilistic theory of coherence. Analysis, 63, 194–199.

    Article  Google Scholar 

  • Frederick, S. (2005). Cognitive reflection and decision making. Journal of Economic Perspectives, 19, 25–42.

    Article  Google Scholar 

  • Gaifman, H. (1979). Subjective probability, natural predicates and Hempel’s ravens. Erkenntnis, 21, 105–147.

    Google Scholar 

  • Glass, D. H. (2002). Coherence, explanation, and Bayesian networks. In O’Neill, M., Sutcliffe, R. F. E., Ryan, C., Eaton, M., & Griffith, N. J. L. (Eds.), Artificial intelligence and cognitive science. 13th Irish conference, AICS 2002, Limerick, Ireland, September 2002 (pp. 177–182). Berlin: Springer.

  • Glass, D. H. (2005). Problems with priors in probabilistic measures of coherence. Erkenntnis, 63, 375–385.

    Article  Google Scholar 

  • Good, I. J. (1984). The best explicatum for weight of evidence. Journal of Statistical Computation and Simulation, 19, 294–299.

    Article  Google Scholar 

  • Greiner, B. (2004). An online recruitment system for economic experiments. In K. Kremer & V. Macho (Eds.), Forschung und wissenschaftliches Rechnen 2003, GWDG Bericht 63 (pp. 79–93). Goettingen: Ges. fuer Wiss. Datenverarbeitung.

    Google Scholar 

  • Harris, A., & Hahn, U. (2009). Bayesian rationality in evaluating multiple testimonies: Incorporating the role of coherence. Journal of Experimental Psychology: Learning, Memory, and Cognition, 35(5), 1366–1373.

    Google Scholar 

  • Jeffreys, H. (1961). Theory of probability. Oxford: Oxford University Press.

    Google Scholar 

  • Joyce, J. (2008). Bayes’ theorem. http://plato.stanford.edu/archives/fall2008/entries/bayes-theorem/.

  • Kemeny, J., & Oppenheim, P. (1952). Degrees of factual support. Philosophy of Science, 1952, 307–324.

    Article  Google Scholar 

  • Keynes, J. (1921). A treatise on probability. London: Macmillan.

    Google Scholar 

  • Kolmogorov, A. (1956). Foundations of the theory of probability. New York: AMS Chelsea Publishing.

    Google Scholar 

  • Koscholke, J. (2015). Evaluating test cases for probabilistic measures of coherence. Erkenntnis. doi:10.1007/s10670-015-9734-1.

  • Kuipers, T. A. F. (2000). From instrumentalism to constructive realism. Dordrecht: Reidel.

    Book  Google Scholar 

  • Levi, I. (1962). Corroboration and rules of acceptance. British Journal for the Philosophy of Science, 13, 307–313.

    Google Scholar 

  • Meijs, W. (2005). Probabilistic measures of coherence. PhD thesis, Erasmus University, Rotterdam.

  • Meijs, W. (2006). Coherence as generalized logical equivalence. Erkenntnis, 64, 231–252.

    Article  Google Scholar 

  • Meijs, W., & Douven, I. (2007). On the alleged impossibility of coherence. Synthese, 157(3), 347–360.

    Article  Google Scholar 

  • Mortimer, H. (1988). The logic of induction. Paramus: Prentice Hall.

    Google Scholar 

  • Nozick, R. (1981). Philosophical explanations. Oxford: Clarendon.

    Google Scholar 

  • Olsson, E. J. (2002). What is the problem of coherence and truth? The Journal of Philosophy, 94, 246–272.

    Article  Google Scholar 

  • Olsson, E. J. (2005). Against coherence: Truth, probability and justification. Oxford: Oxford University Press.

    Book  Google Scholar 

  • Pfeiffer, P. (1990). Probability for applications. New York: Springer.

    Book  Google Scholar 

  • Pinheiro, J., Bates, D., DebRoy, S., Sarkar, D., & R Core Team (2013). nlme: Linear and nonlinear mixed effects models. http://CRAN.R-project.org/package=nlme.

  • Popper, K. R. (1954). Degree of confirmation. British Journal for the Philosophy of Science, 5, 143–149.

    Article  Google Scholar 

  • R Core Team (2015). R: A language and environment for statistical computing. Vienna: R Foundation for Statistical Computing.

  • Rescher, N. (1958). Theory of evidence. Philosophy of Science, 25, 83–94.

    Article  Google Scholar 

  • Rescher, N. (1973). The coherence theory of truth. Oxford: Oxford University Press.

    Google Scholar 

  • Rips, L. J. (2001). Two kinds of reasoning. Psychological Science, 12, 129–134.

    Article  Google Scholar 

  • Roche, W. (2013). Coherence and probability: A probabilistic account of coherence. In M. Araszkiewicz & J. Savelka (Eds.), Coherence: Insights from philosophy, jurisprudence and artificial intelligence (pp. 59–91). Dordrecht: Springer.

    Chapter  Google Scholar 

  • Schippers, M. (2014). Probabilistic measures of coherence: From adequacy constraints towards pluralism. Synthese, 191(16), 3821–3845.

    Article  Google Scholar 

  • Schippers, M., & Siebel, M. (2015). Inconsistency as a touchstone for coherence measures. Theoria, 30, 11–41.

    Article  Google Scholar 

  • Schupbach, J. N. (2011). New hope for Shogenji’s coherence measure. British Journal for the Philosophy of Science, 62(1), 125–142.

    Article  Google Scholar 

  • Schwarz, G. (1978). Estimating the dimension of a model. The Annals of Statistics, 6(2), 461–464.

    Article  Google Scholar 

  • Shogenji, T. (1999). Is coherence truth conducive? Analysis, 59, 338–345.

    Article  Google Scholar 

  • Shogenji, T. (2012). The degree of epistemic justification and the conjunction fallacy. Synthese, 184, 29–48.

    Article  Google Scholar 

  • Siebel, M. (2004). On Fitelson’s measure of coherence. Analysis, 64, 189–190.

    Article  Google Scholar 

  • Siebel, M., & Wolff, W. (2008). Equivalent testimonies as a touchstone of coherence measures. Synthese, 161, 167–182.

    Article  Google Scholar 

  • Wagenmakers, E. J. (2007). A practical solution to the pervasive problems of \(p\) values. Psychonomic Bulletin & Review, 14, 779–804.

    Article  Google Scholar 

  • Weller, J. A., Dieckmann, N. F., Tusler, M., Mertz, C. K., Burns, W. J., & Peters, E. (2013). Development and testing of an abbreviated numeracy scale: A Rasch analysis approach. Journal of Behavioral Decision Making, 26, 198–212.

    Article  Google Scholar 

Download references

Acknowledgments

We would like to thank (in alphabetical order) Arndt Bröder, Andreas Glöckner, Björn Meder, Michael Schippers and Mark Siebel for their contributions. We would also like to thank the participants of the Operationalization Workshop 2013 in Freiburg for helpful comments. This work was supported by grant SI 1731/1-1 to Mark Siebel and grant GL 632/3-1 and BR 2130/8-1 to Andreas Glöckner and Arndt Bröder from the Deutsche Forschungsgemeinschaft (DFG) as part of the priority program “New Frameworks of Rationality” (SPP 1516).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Jakob Koscholke.

Appendices

Appendix 1: Test cases

1.1 Akiba’s (2000) Die case

Imagine rolling a fair die and consider the following three statements:

  • \(S_1:\) The die comes up 2.

  • \(S_2:\) The die comes up 2 or 4.

  • \(S_3:\) The die comes up 2 or 4 or 6.

Which pair of statements fits together better. Statement 1 and 2 or statement 1 and 3?

1.2 Bovens and Hartmann’s (2003) tweety case

  1. Situation 1:

    Consider a population of 100 animals. 50 out of 100 animals are birds and 50 out of 100 animals cannot fly. Among these 100 animals there is exactly one animal that is a bird and cannot fly. Randomly pick one animal and consider the following two statements:

    • \(S_1:\) The picked animal is a bird.

    • \(S_2:\) The picked animal cannot fly.

  2. Situation 2:

    Consider a population of 100 animals. 50 out of 100 animals are birds and 50 out of 100 animals cannot fly. Among these 100 animals there is exactly one penguin and therefore a bird that cannot fly. Randomly pick one animal and consider the following three statements:

    • \(S_1:\) The picked animal is a bird.

    • \(S_2:\) The picked animal cannot fly.

    • \(S_3:\) The picked animal is a penguin.

In which of the two situations do the respective sets of statements fit together better?

1.3 Bovens and Hartmann’s (2003) Tokyo murder case

Imagine that a murder has occurred in Toyko and the corpse is still to be found. In order to search more efficiently the map of Tokyo is separated into 100 equally-sized where the probability of finding the corpse is the same for each square. Now, 5 pairs of equally reliable and independent witnesses give the following statements as witness reports:

  • Pair 1: 

    • \(S_1:\) The corpse is in squares 50 to 60.

    • \(S_2:\) The corpse is in squares 51 to 61.

  • Pair 2: 

    • \(S_1:\) The corpse is in squares 22 to 55.

    • \(S_2:\) The corpse is in squares 55 to 90.

  • Pair 3: 

    • \(S_1:\) The corpse is in squares 20 to 61.

    • \(S_2:\) The corpse is in squares 50 to 91.

  • Pair 4: 

    • \(S_1:\) The corpse is in squares 41 to 60.

    • \(S_2:\) The corpse is in squares 51 to 70.

  • Pair 5: 

    • \(S_1:\) The corpse is in squares 39 to 61.

    • \(S_2:\) The corpse is in squares 50 to 72.

Which pair of statements fits together best, which worst? Can you give and ordering where the first pair is the best and the last pair the worst?

1.4 Glass’ (2005) Dodecahedron case

  • Situation 1: You are rolling a fair die.

  • Situation 2: You are rolling a fair dodecahedron.

Now consider the following two statements:

  • \(S_1:\) The result will be 2.

  • \(S_2:\) The result will be 2 or 4.

In which of the two situation do these two statements fit together better?

1.5 Meijs’ (2005) Samurai case

  1. Situation 1:

    There are 10,000,000 suspects in a murder case. 1059 of them are Japanese and also 1059 own a samurai sword such that in total there are 9 suspects who are Japanese and own a samurai sword at the same time.

  2. Situation 2:

    There are 100 suspects in a murder case. 10 of them are Japanese and also 10 own a samurai sword such that in total there are 9 suspects who are Japanese and own a samurai sword at the same time.

Now consider the following two statements:

  • \(S_1:\) The murderer is Japanese.

  • \(S_2:\) The murderer owns a samurai sword.

In which of the two situations do the two statements fit together better?

1.6 Meijs’ (2006) Albino rabbit case

  1. Situation 1:

    There are 102 rabbits on the first island. 101 out of these 102 rabbits are grey. Also, 101 out of 102 rabbits have two ears. In total there are 100 out of 102 rabbits which are grey and have two ears at the same time. Consequently, there is exactly one rabbit which is grey but does not have two ears and exactly one rabbits which is not grey but has two ears.

  2. Situation 2:

    There are 102 rabbits on the second island, too. 100 out of these 102 rabbits are grey. Also, 100 out of 102 rabbits have two ears. In total there are 100 out of 102 rabbits which are grey and have two ears at the same time. Consequently, every grey rabbit has two ears and every rabbit that has two ears is also grey.

Now, randomly pick one rabbit and consider the following two statements:

  • \(S_1:\) The rabbit is grey.

  • \(S_2:\) The rabbits has two ears.

In which of the two situations do these two statements fit together better?

1.7 Meijs and Douven’s (2007) plane lottery case

Imagine the following lottery. The chances are 4 / 100 for flying to the North pole, 49 / 100 for flying to the South pole and 47 / 100 for flying to New Zealand. The probability for seeing a penguin at the North pole is 0, at the South pole it is 10 / 49 and in New Zealand it is 1 / 47. Now consider the following two situations in which when having landed one is confronted with two statements.

  • Situation 1: 

    • \(S_1:\) You are landing at the North pole.

    • \(S_2:\) The animal you see is a penguin.

  • Situation 2: 

    • \(S_1:\) You are landing at the South pole.

    • \(S_2:\) The animal you see is a penguin.

In which of the two situations do the respective statements fit together better?

1.8 Schippers and Siebel’s (2015) inconsistent testimony case

Imagine there are 8 suspects for a robbery. It is certain that exactly one of them is the robber. Consider the following two situations in which two independent and equally reliable witnesses give statements about the robber:

  • Situation 1: 

    • \(S_1:\) The robbery was committed by suspect 1 or 2.

    • \(S_2:\) The robbery was committed by suspect 2 or 3.

    • \(S_3:\) The robbery was committed by suspect 1 or 3.

  • Situation 2: 

    • \(S_1:\) The robbery was committed by suspect 1 or 2.

    • \(S_2:\) The robbery was committed by suspect 3 or 4.

    • \(S_3:\) The robbery was committed by suspect 5 or 6.

In which of these two situations do the respective sets of statements fit together better?

1.9 Schupbach’s (2011) Robber case

Imagine there are 10 suspects for a robbery. It is certain that exactly one of them is the robber. Consider the following two situations in which two independent and equally reliable witnesses make give statements about the robber:

  • Situation 1: 

    • \(S_1:\) The robbery was committed by suspect 1 or 2 or 3.

    • \(S_2:\) The robbery was committed by suspect 1 or 2 or 4.

    • \(S_3:\) The robbery was committed by suspect 1 or 3 or 4.

  • Situation 2: 

    • \(S_1:\) The robbery was committed by suspect 1 or 2 or 3.

    • \(S_2:\) The robbery was committed by suspect 1 or 4 or 5.

    • \(S_3:\) The robbery was committed by suspect 1 or 6 or 7.

In which of these two situations do the respective sets of statements fit together better?

1.10 Siebel’s (2004) pickpocketing robber case

Imagine the following situation. There are 10 equally likely suspects for a murder. 8 out of 10 have committed a pickpocketing before, 8 out of 10 have committed a robbery and in total 6 out of 10 have committed a pickpocketing and a robbery. Now consider the following two statements:

  • \(S_1:\) The murderer has committed a robbery.

  • \(S_2:\) The murderer has committed a pickpocketing.

Do these two statements fit together or not?

Appendix 2: Test case results

See Table 2.

Table 2 Summary of the results

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Koscholke, J., Jekel, M. Probabilistic coherence measures: a psychological study of coherence assessment. Synthese 194, 1303–1322 (2017). https://doi.org/10.1007/s11229-015-0996-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11229-015-0996-6

Keywords

Navigation