Abstract
Sometimes, the availability of more evidence for a conclusion provides a reason to believe in its falsity. This counter-intuitive phenomenon is related to the idea of higher-order evidence, which has attracted broad interest in recent epistemological literature. Occasionally, providing more evidence for something weakens the case in its favor, by casting doubt on the probative value of other evidence of the same sort or on the fact-finder’s cognitive performance. We analyze this phenomenon, discuss its rationality, and outline possible application to evidence law and to the law in general. It is suggested, inter alia, that such higher-order evidence may explain how judicial experience-based expertise in fact-finding is possible despite the absence of a feedback mechanism; and that fact-finders’ self-doubt regarding their own competence in making ‘beyond-reasonable doubt’ judgments might be reasonable and should not be ignored.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
The alternative theories of Shakespearean authorship are discussed, e.g., in Michael Dobson, “Authorship Controversy”, in Michael Dobson and Stanley Wells (eds.), The Oxford Companion to Shakespeare (New York: Oxford University Press, 2001), pp. 30–31; Gabriel Egan, “A History of Shakespearean Authorship Attribution”, in G. Taylor and G. Egan (eds.), The New Oxford Shakespeare Authorship Companion (Oxford: Oxford University Press, 2017), pp. 27–47. Our discussion neither assumes nor relies on the accuracy of any of these theories, which we use here for the sake of illustration only.
For example, it demonstrates correspondences between the philosophical ideas of Bacon and the works of Shakespeare; points at some autobiographical allusions and cryptographic ciphers and codes in Shakespeare plays and poems, which reveal Bacon's authorship; etc.
See, e.g., Robert S. Wyer and William H. Unverzagt, “Effects of Instructions to Disregard Information on Its Subsequent Recall and Use in Making Judgments”, Journal of Personality and Social Psychology 48(3) (1985): pp. 533–549; and more generally: Timothy D. Wilson and Nancy Brekke, “Mental Contamination and Mental Correction: Unwanted Influences on Judgments and Evaluations”, Psychological Bulletin 116(1) (1994): pp. 117–142.
We use the terms “second-order evidence” and “higher-order” evidence interchangeably.
The distinction between defeating higher-order evidence about the probative value of other (first-order) evidence and higher-order evidence that indicates epistemic failure in the process of inferring from the evidence or of assessing its evidential weight is not exactly clear. Yet we do not intend here to disambiguate this distinction, which seems to be related to general epistemological issues such as the internalism–externalism debate. Exploration of these questions spans beyond the scope of this paper.
For a similar distinction between rebutting (or “opposing”) defeaters that are counterevidence against the hypothesis supported by one’s original evidence and undercutting (or “undermining”) defeaters that “attack the connection between the evidence and the conclusion”, see Lewis J. Pollock, Contemporary Theories of Knowledge (Totowa, NJ: Rowman and Littlefield, 1986), p. 39.
In the epistemological literature, the term “higher-order evidence” is usually understood as “evidence about what your evidence supports” (See, e.g., Paulina Sliwa and Sophie Horowitz, “Respecting All the Evidence”, Philosophical Studies 172(11) (2015): pp. 2835–2858), or as “evidence about evidential relations”: when I get evidence of my own epistemic malfunction, this serves as evidence that the evidential relations may not be as I’ve taken them to be (See, e.g., Richard Feldman, “Respecting the Evidence”, Philosophical Perspectives 19 (2005): pp. 95–119; Thomas Kelly, “The Epistemic Significance of Disagreement”, in J. Hawthorne and T. Gendler (eds.), Oxford Studies in Epistemology 1 (Oxford: Oxford University Press, 2005), pp. 167–196). David Christensen similarly distinguishes between “two ways that evidence rationalizes change of belief correspond to two ways in which I’m a fallible thinker. One dimension of my fallibility is that my beliefs are based on limited evidence. So the conclusions I draw—no matter how competently I react to my evidence—may turn out to be inaccurate. … The second dimension of my fallibility is that I may make mistakes in thinking. I sometimes fail to draw the conclusions that are best supported by my evidence, often because I make mistakes in judging what my evidence supports.” See David Christensen, “Higher-Order Evidence”, Philosophy and Phenomenological Research 81(1) (2010): pp. 185–215, 185–186. This literature thus focuses on the second conjunct in our formulation; i.e., evidence regarding the cognitive process of inferring from the evidence.
For details, see, e.g., Patrick Harding and Patricia H. Field, “Breathalyzer Accuracy in Actual Law Enforcement Practice: A Comparison of Blood and Breath-Alcohol Results in Wisconsin Drivers”, Journal of Forensic Sciences 32(5) (1987): pp. 1235–1240; Paul Schop, “Is DWI DOA?: Admissibility of Breath Testing Evidence in the Wake of Recent Challenges to Breath Testing Devices”, Southwestern University Law Review 20 (1991): pp. 247–288. For the LiDAR see, e.g., John D. Chillemi, “The Need for Speed and Judicial Notice: New York's Admissibility of Lidar Technology in Law Enforcement”, St. John’s Law Review 89 (2015): pp. 367–396; Ryan V. Cox and Carl Fors, “Admitting Light Detection and Ranging (LIDAR) Evidence in Texas: A Call for Statewide Judicial Notice”, St. Mary's Law Journal 42 (2010–2011): pp. 837–872.
For details see, e.g., Christian A. Meissner and John C. Brigham, “Thirty Years of Investigating the Own-Race Bias in Memory for Faces: A Meta-Analytic Review”, Psychology, Public Policy, and Law 7(1) (2001): pp. 3–35.
See Sheri L. Johnson, “Cross-Racial Identification Errors in Criminal Cases”, Cornell Law Review 69 (1984): pp. 934–987; Stephanie J. Platz and Harmon M. Hosch, “Cross-Racial/Ethnic Eyewitness Identification: A Field Study”, Journal of Applied Social Psychology 18(11) (1988): pp. 972–984; People v. McDonald, 690 P.2d 709, 717–718 (Cal. 1984); but cf., State v. Cromedy, 158 N.J. 112 (1999).
In a jury trial, the fact-finders are the jurors, whereas in a non-jury trial, the judge sits both as the fact-finder and as the trier of law. When we refer to judges as “fact-finders,” we assume a non-jury system and refer to a judge’s role in determining the facts from the evidence (as opposed to a judge’s role as the trier of law).
But see Section 5.1 for a brief discussion of institutional reasons against the admissibility of such second-order evidence about the cognitive failures of legal fact-finders.
See, e.g., Deena S. Weisberg et al., “The Seductive Allure of Neuroscience Explanations”, Journal of Cognitive Neuroscience 20(3) (2008): pp. 470–477 (“dazzled by neurobabble”); Spencer E. Compton, “Not Guilty by Reason of Neuroimaging: The Need for Cautionary Jury Instructions for Neuroscience Evidence in Criminal Trials”, Vanderbilt Journal of Entertainment and Technology Law 12(2) (2009): pp. 333–354. Of course, we are not committed to the truth of this view. For a less skeptical view regarding evidence generated by fMRI, see, e.g. Martha J. Farah and Cayce J. Hook, “The Seductive Allure of ‘Seductive Allure’”, Perspectives on Psychological Science 8(1) (2013): pp. 88–90.
For example, the evidence for Edward de Vere’s authorship serves as first-order evidence against the Shakespearean Hypothesis H, while at the same time undermining one body of evidence for the conclusion not-H. This phenomenon—the possibility of such dual functioning of a single piece of evidence—is discussed and demonstrated in recent epistemological literature: suppose that smoke tends to make you have auditory hallucinations, and imagine that you have an auditory experience of an approaching fire engine. Thus, a smoky smell provides first-order evidence for your conclusion that there is a fire engine nearby, while at the same time undermining your auditory evidence for that conclusion. See James Pryor, “Problems for Credulism”, in Chris Tucker (ed.), Seemings and Justification: New Essays on Dogmatism and Phenomenal Conservatism 89 (New York: Oxford University Press, 2013).
This analysis resembles the proper attitude towards conspiracy theories: even though a conspiracy theory might be persuasive at first glance, the fact that you can always come up with an alluring conspiracy theory in itself comprises a reason to doubt any conspiracy theory.
In a probability-raising model of evidential support, according to which E supports H if and only if \(P(H|E) > P(H)\), the assumption in the text can be formalized as: \(P(H|E_{i} ) > P(H)\) for all i.
In a Bayesian framework that assumes the probability-raising model of evidential support and the rule of conditionalization \(P_{new} (H) = P_{old} (H|E)\) (i.e., one’s new probability function, \(P_{new}\), is obtained from one’s old probability function, \(P_{old}\), by conditionalizing on the new evidence E), the phenomenon described in the text is analyzed as follows: as a result of accumulating merely pieces of evidence Ei, which individually support it, H may become less probable on the basis of one’s evidence after a series of updates via conditionalization. That is, although \(P(H|E_{i} ) > P(H)\) for all i, there is a threshold k, such that:
\(P(H|E_{1} \wedge E_{2} \wedge E_{3} \wedge \cdots \wedge E_{k + 1} ) < P(H|E_{1} \wedge E_{2} \wedge E_{3} \wedge \cdots \wedge E_{k} ).\)
We assume, for the sake of our illustrative use of the Shakespearean case, that if Bacon was the author then Marlowe could not be the author; if Marlowe was the author, Bacon could not be the author; and so on. The truth about the Shakespearean authorship puzzle is much more nuanced and includes more complex possibilities, but it is not our concern here.
A similar suspicion is that the phenomenon of “more is less” is limited to cases in which the individual pieces of evidence comprise evidence of disproof (of the occurrence of certain events) rather than of proof. Our following discussion and examples reply to this concern as well.
For a similar point, see Lachlan J. Gunn, et al., “Too Good to Be True: When Overwhelming Evidence Fails to Convince”, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences 472.2187 (2016)20150748.
In this paper, we do not take a side on recent epistemological debates regarding whether the Bayesian framework presented above (note 16–17 supra) is always suitable for analyzing higher-order defeaters. Arguably, there are cases of self-misleading evidence in which, in light of a new evidence E, H becomes less well supported on the basis of our evidence, namely \(P_{new} (H) < P_{old} (H),\) even though \(P_{old} (H|E) > P_{old} (H).\) This mismatch between the doxastic attitudes after having acquired the evidence and the agent’s conditional doxastic attitudes before acquiring the evidence might be explained by the alleged normative (epistemic) significance of whether the evidence is actually obtained or merely hypothetically obtained. Thus, for instance in “broken heart,” when you find yourself seeing your former love everywhere, you may learn something new about yourself—that you are misled by this evidence E—which requires you not merely to respond to the higher-order evidence by conditionalizing on it, but also to revise your doxastic attitude towards the conclusion that she is present, namely towards H conditional on E. For further discussion, see Mattias Skipper, “Higher-Order Defeat and the Impossibility of Self-Misleading Evidence”, in Mattias Skipper and Asbjørn Steglich-Petersen (eds.), Higher-Order Evidence: New Essays (Oxford: Oxford University Press, 2019), pp. 189–208.
See, e.g., Brian L. Cutler, Steven D. Penrod and Todd K. Martens, “The Reliability of Eyewitness Identification”, Law and Human Behavior 11(3) (1987): pp. 233–258; Steven Penrod and Brian H. Bornstein, “Generalizing Eyewitness Reliability Research”, in R. C. L. Lindsay et al. (eds.), The Handbook of Eyewitness Psychology, vol 2 (Mahwah, NJ: Lawrence Erlbaum Associates Publishers, 2007), p. 529.
In this method of corrupting a police lineup, the police arrange for the witness to 'accidentally' confront the preferred suspect in a context that seems incriminating.
For a discussion of this example, see note 20 supra, where the authors stress the idea of systematic bias. The idea that sometimes more evidence may corrupt epistemological processes was suggested, on different grounds, also in Kenneth M. Ehrenberg, “Less Evidence, Better Knowledge”, McGill Law Journal 60(2) (2015): pp. 173–214.
This is a general feature of explanatory reasoning in justifying hypotheses (“abduction”), namely, the violation of monotonicity: it may be possible to infer certain conclusions from a subset of a set of premises which cannot be inferred from the set as a whole. See Igor Douven, “Abduction”, in Edward N. Zalta (ed.), The Stanford Encyclopedia of Philosophy (Summer 2017 Edition),
https://plato.stanford.edu/archives/sum2017/entries/abduction.
For a discussion of the substantial role that explanatory considerations and the processes of inference to the best explanation play in judicial proof see Michael S. Pardo and Ronald J. Allen, “Juridical Proof and the Best Explanation”, Law and Philosophy 27(3) (2008): pp. 223–268.
See note 1 supra.
Note again that we do not commit ourselves to the real historical story of Shakespeare. The claim in the text is thus a stipulation.
We focus on inferences, which are necessarily involved in cases in which circumstantial evidence is considered. However, second-order defeating evidence might also cast doubt on the fact-finder’s non-inferential grounds for her belief, which is typically present in the case of direct evidence; for example, the fact-finder’s intuitive (non-inferential) impression of an eyewitness. We will not explore this line in full here.
For example, sometimes the fact-finder cannot know in favor of whom (the plaintiff or defendant) she might be mistaken. This seems to be the case in calculation errors (e.g., in damages, etc.): if all you know is only that you are not good enough at making the calculation, this knowledge implies nothing about who is going to benefit from your error.
See, e.g., the examples in notes 8–10 supra. In criminal trials, experts employed by the defense play an important role in scrutinizing the prosecution’s evidence, carrying out checks and further testing, and briefing defense lawyers about the strengths and weaknesses of the prosecution’s scientific evidence. According to USCS Fed Rules Evid., r. 702 (2011), if scientific, technical, or otherwise specialized knowledge will help the fact-finder to assess the reliability of the evidence, an “expert” witness may testify thereto in the form of an opinion or otherwise.
An expert witness may be confronted with a learned treatise as substantive evidence under the hearsay exception set out in USCS Fed Rules Evid., r. 903 (2011). See McCormick on Evidence (Kenneth S. Broun et al. eds., 7th ed. (St. Paul, MN: West, 2013), para 13.
See, e.g., Douglas Walton, Appeal to Expert Opinion: Arguments from Authority (Pennsylvania: Pennsylvania State Press, 1997), pp. 20–21. For a recent legal discussion of the question of how character evidence should be characterized (in the context American Federal rules of evidence) see, e.g., Barrett J. Anderson, “Recognizing Character: A New Perspective on Character Evidence”, Yale Law Journal 121 (2011): pp. 1912–1969. Yet, other explanations for the general inadmissibility of character evidence, which are not related to the accuracy of fact-finding, have been proposed in the literature. For such explanation, in terms of the efficient incentives for potential criminal offenders, See Chris W. Sanchirico, “Character Evidence and the Object of Trial”, Columbia Law Review 101 (2001): pp. 1227–1311.
See 28 U.S.C. § 144 (1949) and 28 U.S.C. § 455(b) (1988); and more generally Charles G. Geyh, Judicial Disqualification: An Analysis of Federal Law, 2nd ed. (Washington, DC: Federal Judicial Center, 2010). A similar approach is reflected in other legal systems. Cf. Wayne Gorman, “Reasonable Apprehension of Judicial Bias in Trial Proceedings”, Criminal Law Quarterly 55 (2009): pp. 46–93.
This seems possible because of the gap between epistemic (legal) justification and (legal) truth. For instance, a conviction of an innocent person can be justified (if there is clear and convincing evidence of his guilt) but, nevertheless, wrong.
See, e.g., Richard P. Larrick, “Debiasing”, in D.J. Koehler and N. Harvey (eds.), Blackwell Handbook of Judgment and Decision Making (Oxford: Blackwell Publishing, 2004), pp. 316–338; Christine Jolls and Cass R. Sunstein, “Debiasing Through Law”, Journal of Legal Studies 35(1) (2006): pp. 199–242. For the idea of self-debiasing see also Mario J. Rizzo and Douglas G. Whitman, “The Knowledge Problem of New Paternalism”, Brigham Young University Law Review 2009(4) (2009): pp. 905–968, 943–951.
From a strategic perspective, a practice of such discounting yields a dilemma for a client between hiring a big-shot lawyer and assuming the risk of the judge discounting her argument or taking a less successful lawyer. This, in turn, might lead to a pooling separation between parties that have right claims and wrong claims, since the latter would benefit more from taking big-shot lawyer and be willing to pay the price of discounting. This line of reasoning might lead to the conclusion that taking a big-shot lawyer should be taken as a signal for the weakness of one’s case.
See, e.g., Bella M. DePaulo and Roger L. Pfeifer, “On-the-Job Experience and Skill at Detecting Deception”, Journal of Applied Social Psychology 16(3) (1986): pp. 249–267; Ray Bull, “Can Training Enhance the Detection of Deception?”, in J.C Yuille (ed.), Credibility Assessment (New York: Springer, Dordrecht, 1989), pp. 83–99.
See, e.g., Saul M. Kassin, Christian A. Meissner, and Rebecca J. Norwick, “‘I’d Know a False Confession if I Saw One’: A Comparative Study of College Students and Police Investigators”, Law and Human Behavior 29(2) (2005): pp. 211–227. But see Stefan M. Herzog and Ralph Hertwig, “The Wisdom of Many in One Mind: Improving Individual Judgments with Dialectical Bootstrapping”, Psychological Science 20(2) (2009): pp. 231–237.
For that reason, the judge should be exposed to data about all the line-ups in a certain police department, since obviously there is a selection in the line-ups that end up in court; with no identification, the judge will not learn about the line-up.
See e.g., Kim L. Scheppele, “Just the Facts, Ma'am: Sexualized Violence, Evidentiary Habits and the Revision of Truth”, New York Law School Law Review 27 (1992): pp. 123–172; Laura E. Boeschen, Bruce D. Sales and Mary P. Koss, “Rape Trauma Experts in the Courtroom”, Psychology, Public Policy, and Law 4(1–2) (1998): pp. 414–432; Patricia Frazier and Eugene Borgida, “Rape Trauma Syndrome: A Review of Case Law and Psychological Research”, Law and Human Behavior 16(3) (1992): pp. 293–311.
Of course, there might be independent practical reasons why repeat juries would be unworkable, yet our point here is theoretical rather than practical.
Under English law, juries are no longer instructed to consider whether the defendant’s guilt is proved “beyond reasonable doubt.” Instead, the juries merely should be instructed that they must be “sure” before they can convict (R. v. Majid [2009] EWCA Crim 2563). Yet this merely changes the language of jury instructions, rather than the standard of proof, as “sure” does not mean 100% certainty; indeed, the court held that “sure” and “beyond reasonable doubt” mean the same (see JL [2018] 2 Crim LR 184).
Apparently, if the fact-finder reasonably believes that she is an “anti-expert” (namely, that her judgment that the plaintiff’s evidence is stronger than the defendant’s gives her a reason to believe the opposite), she might rationally be required to switch her belief and rule for the defendant whenever she thinks the plaintiff proved his case. But such phenomenon is peculiar, of course. See also, Andy Egan and Adam Elga, “I Can't Believe I'm Stupid”, Philosophical Perspectives 19 (2005): pp. 77–93.
Similar analysis can be applied to other standards of proof beyond the balance of probabilities, such as the ‘clear and convincing’ standard.
As evidence from an innocence project, or such. See https://www.innocenceproject.org.
The probabilistic approach to legal fact finding, which associates standards of proof with (exact) probabilities, is controversial. See, e.g., Neil B. Cohen, “Confidence in Probability: Burdens of Persuasion in a World of Imperfect Knowledge”, New York University Law Review 60 (1985): pp. 385–422; Ronald J. Allen and Michael S. Pardo, “The Problematic Value of Mathematical Models of Evidence”, The Journal of Legal Studies 36(1) (2007): pp. 107–140; Fredrick E. Vars, “Toward a General Theory of Standards of Proof”, Catholic University Law Review 60(1) (2010): pp. 1–46, 18–24. In addition, one might argue that probabilistic belief is not sufficient for criminal conviction because a proof beyond reasonable doubt requires that the fact-finder have some specific probabilistic knowledge. See Sarah Moss, Probabilistic Knowledge (Oxford: Oxford University Press, 2018), p. 202. However, nothing in our discussion depends on accepting the probabilistic approach, and we use it here only for the sake of simplicity.
Surely, beyond these considerations, which dictate what the judge ought to believe, there might be other institutional or pragmatic considerations that would affect what the judge should do in such cases.
These questions relate, e.g., to the debate about “epistemic akrasia.” While many accept the non-acratic constraint according to which an ideal rational agent cannot believe something that she believes to be unsupported by the evidence, recently some have argued that if you have good evidence for H, and good evidence that your evidence doesn’t support H, the rational response might be to have high confidence in both of these propositions. See, e.g., Sophie Horowitz, “Epistemic Akrasia”, Noûs 48(4) (2014): pp. 718–744; Daniel Greco, “A Puzzle About Epistemic Akrasia”, Philosophical Studies 167(2) (2014): pp. 201–219; Allen Coates, “Rational Epistemic Akrasia”, American Philosophical Quarterly 49(2) (2012): pp. 113–124; Eugene Chislenko, “Moore's Paradox and Akratic Belief”, Philosophy and Phenomenological Research 92(3) (2016): pp. 669–690; Alex Worsnip, “The Conflict of Evidence and Coherence”, Philosophy and Phenomenological Research 96(1) (2018): pp. 3–44; Maria Lasonen-Aarnio, “Enkrasia or Evidentialism? Learning to Love Mismatch”, Philosophical Studies (2019): pp. 1–36.
Acknowledgements
This research was supported by the Israel Science Foundation, Grant No. 1322/18, and the Barak Center for Interdisciplinary Legal Research at the Hebrew University of Jerusalem. For valuable comments and discussions, we thank Ilan Ben Shalom, Binyamin Blum, David Enoch, Alon Harel, Ehud Guttel, Adi Leibovitch, Orit Malka, Ittay Nissan-Rozen, Ittai Paldor, Doron Teichman, Ruth Weintraub, Re’em Segev and Eyal Zamir.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Malcai, O., Rivlin, R. Reasonable Self-doubt. Criminal Law, Philosophy 15, 25–45 (2021). https://doi.org/10.1007/s11572-020-09528-7
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11572-020-09528-7