Abstract
A central task in legal factfinding is evaluating the warrant for a finding or the soundness of an inference from the evidentiary propositions to a conclusion. This task is especially difficult when there is much at stake, but the evidence is incomplete and the soundness of the inference is uncertain. Analyses of how to improve such inferences have been made at various levels of generality, and for different types of evidence. For example, one general problem is distinguishing “scientific knowledge” from “junk science,” as required for admissibility in judicial proceedings under Federal Rule of Evidence 702, following the Daubert v. Merrell Dow Pharmaceuticals, Inc.1 decision.2 Another general problem is evaluating inferences about unique historical events, the kind of factfinding necessary in criminal cases.3 As opposed to such general problems, some theorists address only particular areas where inferences are difficult in law, such as the “lost chance” cases,4 cases involving “indeterminate plaintiffs,”5 inferences from “naked statistical evidence,”6 or inferences based on DNA identification.7 Such problems of correct inference cannot be solved purely by formal logic, nor can theorists merely duplicate the role of the factfinder by evaluating the specific evidence in a particular case. To be useful as theories of inference, accounts can be neither too general nor too specific. They must provide useful models for handling recurring types of inference in situations where findings must be warranted by incomplete evidence.8
Preparation of this Article was supported in part by a research grant from Hofstra University.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Reference
U.S. 579 (1993).
See, e.g., Gen. Elec. Co. v. Joiner, 522 U.S. 136 (1997); Kennedy v. Collagen Corp., 161 F.3d 1226 (9th Cir. 1998); cf. Kumho Tire Co. v. Carmichael, 526 U.S. 137, 147 (1999) (holding that Federal Rule of Evidence 702 mandates a judicial gatekeeping obligation to determine evidentiary reliability for all expert “knowledge,” not merely “scientific knowledge”).
See, e.g.,Joseph b. Kadane & David a. Schum, a Probabilistic Analysis of the Sacco and Vanzetti Evidence (1996).
See, e.g.,Joseph H. King, Jr., Causation, Valuation,and Chance in Personal Injury Torts Involving Preexisting Conditions and Future Consequences, 90 Ysle L.J. 1353 (1981); Vern R. Walker, Direct Inference in the Lost Chance Cases: Factfinding Constraints Under Minimal Fairness to Parties, 23 Hofstra L. Rev. 247 (1994).
See, e.g.,Bert Black & David E. Lilienfeld, Epidemiologic Proof in Toxic Tort Litigation, 52 Fordham L. REV. 732, 767–69, 782–84 (1984); Michael Dore, A Commentary on the Use of Epidemiological Evidence in Demonstrating Cause-in-Fact, 7 Harv. Envtl. L. REV. 429 (1983); Khristine L. Hall & Ellen K. Silbergeld, Reappraising Epidemiology: A Response to Mr. Dore, 7 Harv. Envtl. L. Rev. 441 (1983); Steve Gold, Note, Causa-tion in Toxic Torts: Burdens of Proof Standards of Persuasion, and Statistical Evidence, 96 Yale L.J. 376 (1986).
See, e.g., Craig R. Callen, Adjudication and the Appearance of Statistical Evidence, 65 Tut. L. Rev. 457 (1991); David Kaye, The Limits of the Preponderance of the Evidence Standard: Justifiably Naked Statistical Evidence and Multiple Causation, 1982 AM. B. Found. Res. J. 487; David Kaye, The Paradox of the Gatecrasher and Other Stories, 1979 Ariz. ST. L.J. 101 (1979); Richard Lempert, The New Evidence Scholarship: Analyzing the Process of Proof, 66 B.U. L. Rev. 439 (1986); David Kaye, Naked Statistical Evidence, 89 Yale L.J. 601 (1980) (book review).
See, e.g., 2 Paul C. Giannelli & Edward J. Imwinkelried, Scientific Evidence 1–39 (2d ed. 1993); D.H. Kay, Science in Evidence 153–258 (1997); National Research Council, The Evaluation Of Forensic Dna Evidence (1996); Judith A. McKenna et al., Reference Guide on Forensic DNA Evidence, in fed. Judicial ctr., reference manual on Scientific Evidence 273–329 (1994); Kenneth E. Melson, Determining Individuality by DNA, in Andre A. Moenssens, scientific Evidence In Civil and Criminal Cases 870–963 (4th ed. 1995).
A number of legal theorists have discussed the general problem of how to take completeness of the evidence into account in factfinding. See, e.g., Craig R. Callen, Kicking Rocks with Dr. Johnson: A Comment on Professor Allen’s Theory, 13 Cardozo L. Rev. 423, 431–39 (1991); Neil B. Cohen, Conceptualizing Proof and Calculating Probabilities: A Response to Professor Kaye, 73 Cornell L. Rev. 78, 85–86 (1987); D.H. Kaye, Do We Need a Calculus of Weight to Understand Proof Beyond a Reasonable Doubt?, 66 B.U. L. Rev. 657, 658 (1986); Richard O. Lempert, Modeling Relevance, 75 Mich. L. Rev. 1021, 1047–48 (1977); Lempert, supra note 6, at 473–74; Laurence H. Tribe, Trial by Mathematics: Precision and Ritual in the Legal Process, 84 Harv. L. Rev. 1329, 134950 (1971); Walker, supra note 4, at 286–97.
See, e.g.,Irving M. Copi & Carl Cohen, Introduction To Logic 25 (10th ed. 1998).
See, e.g.,Gerhard Brewka Et Al., Nonmonotonic Reasoning: An Overview 1, 24 (1997); Gilbert Harman, Reasoning, Meaning and Mind 30–32 (1999).
A deductive inference is “sound” if it is deductively valid and all of its premises are in fact true. See Copt & Cohen, supra note 9, at 32.
For discussions of nonmonotonic reasoning, see, for example, Brewka Et Al., supra note 10, at 1–3, 23–24; Harman, supra note 10, at 30–32. Such reasoning includes, but is not limited to, the use of so-called statistical syllogisms. A traditionally formulated statistical syllogism has the following form: “Most A’s are B’s. This is an A. Therefore, this is (probably) a B.” john L. Pollock, Nomic Probability And The Foundations Of Induction 76 (1990). For an early recognition of the difficulty of such inferences in legal theory, see George F. James, Relevancy, Probability,and the Law, 29 Cal. L. Rev. 689 (1941), arguing that the reasoning “[n]ine-tenths of all As are X, B is an A, therefore the chances are nine to one that B is X” is not logically valid “except upon the assumption that As may be treated as a uniform class with respect to the probability of their being X.” Id. at 697.
U.S. 136 (1997).
See, e.g.,David A. Schum, The Evidential Foundations Of Probabilistic Reasoning (1994); Kadane & Schum, supra note 3.
See Vern R. Walker, Preponderance, Probability and Warranted Factfinding, 62 Brook. L. Rev. 1075, 1079 (1996) (legal factfinding has the goal of producing “knowledge” in the sense of “warranted true belief”). For philosophical discussions of warrant as being the difference between mere true belief and human knowledge, see, for example, Alvin Plantinga, Warrant: The Current Debate (1993), and John L. Pollock & Joseph Cruz, Contemporary Theories Of Knowledge 7–10 (2d ed. 1999).
Stated in terms of propositional logic, we are inclined to believe three propositions: P, Q and Not-both-P-and-Q. Let P be the proposition that a person is in place P at a certain time, while Q is the proposition that the same person is in place Q at precisely the same time, with locations P and Q being very different. Not-both-P-and-Q states our belief that the person could not be in both places at the same time. The set of these three propositions leads deductively to contradictions. For example, from P and Not-both-P-and-Q we deduce Not-Q which yields the contradiction Q-and-Not-Q. Discarding any one of the original three propositions eliminates the contradiction. However, in this example the proposition we are least likely to discard is Not-both-P-and-Q because it is too well warranted by our physical theories and general experience.
It is a question for psychology and psycholinguistics whether we should say that human beings hold inconsistent beliefs or that they withhold belief when they become aware of an inconsistency. There is no need or basis in this Article for deciding which is the better description of human behavior. It would seem to make no difference to the concept of a theory of uncertainty. The important point here is that inconsistency among propositions leads to the search for warrant. Whether human beings can simultaneously believe such inconsistent propositions is irrelevant.
For an attempt to chart all of the evidence in a single legal case, see Kadane & Schum, supra note 3. Some aspects of this theory are discussed infra Part 1.3.
See, e.g., Copi & Cohen, supra note 9, at 5–7.
See id. at 5. Linguistic expressions comprise a very broad category, including not only natural languages, but also technical or specialized languages, and the formal or artificial languages found in mathematics and symbolic logic.
For an example of such a difference between scientific and legal concepts, see infra note 55.
See, e.g.,Copi & Cohen, supra note 9, at chs. 10–11; Albert E. Blumberg, Logic, Modern, in 5 The Encyclopedia of Philosophy 12 (Paul Edwards ed., 1967).
See Copt & Cohen, supra note 9, at ch. 12.
See, e.g.,A.N. Prior, Logic, Modal,in 5 The Encyclopedia of Philosophy, supra note 22,at5.
This is also known as the principle of contradiction, or the law of noncontradiction. See Copi & Cohen, supra note 9, at 389.
Because arguments are expressed in language, there is always a question of whether a logical structure accurately models what is being argued by the linguistic expressions used. If the logical model is itself constructed in a formalist language, the translation of the natural language argument into formal logic can be considered part of the linguistic dimension of a theory of uncertainty. On this view, whether an argument form written in symbolic logic adequately models the meaning of a natural language argument is part of the linguistic task of a theory of uncertainty. Even on this view, however, the theory of why only certain argument forms in symbolic notation are deductively valid is part of logical theory, not linguistics.
See, e.g., Brewka Et. Al, supra note 10 (examining “default logic”); Isaac Levi, The Fixation of Belief And Its Undoing (1991) (examining the justification for revising a system of beliefs by “contraction”); POLLOCK, supra note 12 (examining “prima facie reasoning” and “defeaters”).
On the role of stories in litigation, see, for example, Ronald J. Allen, Factual Ambiguity and a Theory of Evidence, 87 Nw. U. L. Rev. 604 (1994), and Ronald J. Allen, The Nature of Juridical Proof, 13 Cardozo L. Rev. 373 (1991).
See, e.g., 5 U.S.C. §§ 553–57 (2000).
See, e.g.,Vern R. Walker, Keeping the WTO from Becoming the “World Trans-science Organization ”: Scientific Uncertainty, Science Policy, and Factfinding in the Growth Hormones Dispute, 31 Cornell Int’l L.J. 251, 258–67 (1998) (discussing the nature of “science policies” adopted by administrative agencies, and giving examples of science policies adopted by the U.S. EPA for use in assessing the carcinogenic risk to humans posed by chemical agents).
See, e.g.,Envtl. Def. Fund, Inc. v. EPA, 548 F.2d 998 (D.C. Cir. 1976), supplemental opinion (1977), cert. denied sub nom., Velsicol Chem. Corp. v. EPA, 431 U.S. 925 (1977) (deciding burden of proof in administrative proceedings to suspend pesticide registrations).
The basic theory has been presented in detail in Vern R. Walker, The Siren Songs of Science: Toward a Taxonomy of Scientific Uncertainty for Decisionmakers, 23 Conn. L. Rev. 567 (1991) [hereinafter Walker, Siren Songs]. The theory has been applied to explain determinations of baseline risk in tort law. See Vern R. Walker, The Concept of Baseline Risk in Tort Litigation, 80 Ky. L.J. 631, 647–72 (1991–92) [hereinafter Walker, Baseline Risk].
A proposition about generic causation can be either affirmative or negative — that is, it can either assert or deny generic causation.
For discussions of types of measurement variables, see Edwin E. Ghiselli et al., Measurement Theory for the Behavioral Sciences (1981); Herman J. Loether & Donald G. McTavish, Descriptive and Inferential Statistics: An Introduction 16–24 (4th ed. 1993).
See infra text accompanying note 48.
This may be true about many studies of generic causation conducted within wealthy nations, but each decision to undertake a study to resolve uncertainty occurs within a particular socioeconomic context. The same studies might not be undertaken in a developing country, where scarce economic or occupational health resources might be put to better use. Fortunately, information about generic toxicity is often applicable to many circumstances around the globe, and such studies do not need to be replicated within each culture. Behavioral studies, on the other hand, might be far more culture specific.
Another example where a study might not be economically feasible, even in a wealthy nation, is in the context of a private lawsuit. If plaintiffs would have to finance a study that has variables tailored to the particular lawsuit, the private parties might not have the resources to do so, even assuming that such a study would be methodologically feasible.Hence, it is frequently a question in toxic tort or products liability litigation whether the studies available in the public domain are sufficient to warrant factfinding in a particular case. See, e.g., Gen. Elec. Co. v. Joiner, 522 U.S. 136 (1997) (upholding the district court’s exclusion from evidence of expert testimony based on animal and epidemiologic studies, where issue of fact was causation in a human being).
Not all descriptive inaccuracies are measurement errors, however. Propositions that inaccurately describe a generic causal relation may have sources of error other than measurement error. Sufficient measurement accuracy is a necessary, but not sufficient, condition for warranting descriptive accuracy.
See Edward G. Carmines & Richard A. Zeller, Reliability and Validity Assessment 13 (1979); Ghiselli., supra note 34, at 184, 191.
See, e.g., Theodore Peters & James O. Westgard, Evaluation of Methods, in Textbook of Clinical Chemistry 410, 412 (Norbert W. Tietz ed., 1986).
See, e.g.,David Freedman., Statistics 90–97 (2d ed. 1991).
See Carmines & Zeller, supra note 38, at 12; Ghiselli., supra note 34, at 266.
See,e.g., Peters & Westgard, supra note 39, at 412.
See, e.g.,William L. Hays, Statistics 224–25 (5th ed. 1994).
See, e.g.,David H. Kaye & David A. Freedman, Reference Guide on Statistics, in Reference Manual on Scientific Evidence 373–93 (1994).
See Walker, Siren Songs, supra note 32, at 598–608 (discussing linear regression models); Walker, Baseline Risk, supra note 32, at 651–62 (discussing relative risk model); see also Linda A. Bailey et al., Reference Guide on Epidemiology, in Reference Manual on Scientific Evidence, supra note 44, at 147–56 (discussing relative risk and odds ratio in epidemiologic studies); Daniel L. Rubinfeld, Reference Guide on Multiple Regression, in Reference Manual on Scientific Evidence, supra note 44, at 415–69.
The formula used in the mathematical model is correct, but what is incorrect is the value assigned to a constant in that formula.
In addition, even when no regular concurrence has been observed, there may be underlying causation involved, as when some third type of event counteracts or masks the otherwise causal action. See James A. Davis, The Logic of Causal Order 24–27 (1985); Bailey, supra note 45, at 157–70. For example, the complexities of human metabolism often make it very difficult to determine what is causing what within the human body. The design of a controlled experiment is intended to create a situation in which a statistically significant difference between the test group and the control group would warrant an inference of causation. Controlled laboratory experiments are not always feasible, however. The world of events is so complicated that without controlled experiments, conclusions about causal action are often difficult to warrant.
See supra text accompanying note 35.
See, e.g., Glenn Shafer, The Art of Causal Conjecture 91–111, 299–357 (1996).
See supra Part 1.2.1 (Concept Uncertainty).
See, e.g.,M. Granger Morgan & Max Henrion, Uncertainty 39 (1990) (“Sensitivity analysis is the computation of the effect of changes in input values or assumptions (including boundaries and model functional form) on the outputs.”).
See discussion of Schum’s theory infra Part L3.
Daubert v. Merrell Dow Pharm., Inc., 509 U.S. 579 (1993).
Gen. Elec. Co. v. Joiner, 522 U.S. 136 (1997).
In Daubert, the Supreme Court used the phrase “evidentiary reliability” to indicate a legal concept. Daubert, 509 U.S. at 589–92. That concept probably should include the pragmatic assessment that the evidence under scrutiny is “reliable enough” or “trustworthy enough” for judicial purposes, when the legal problem being addressed is admissibility. This pragmatic assessment goes much further than the scientific concept of “measurement reliability,” or even “measurement validity,” discussed supra Part 1.2.1 (Measurement Uncertainty). See id. at 590 n.9. These scientific concepts are useful to scientists because they allow assessments of measurement techniques divorced from the pragmatic question of whether those techniques are reliable enough or valid enough for a particular purpose. By contrast, the legal concept of evidentiary reliability probably includes a judgment about whether the residual inconsistencies (if any) between inferences drawn by experts based on the proffered evidence are tolerable for purposes of the admissibility decision. On this reading, what experts are expected to provide federal judges in a Daubert hearing are theories of uncertainty tailored to the proffered evidence and the findings at issue in the particular legal case. The point of contention between proponents and opponents of the evidence is what the residual uncertainties are and whether they should be tolerated within the body of evidence admitted into the case.
See, e.g., Fleming James, Jr. et al., Civil Procedure 357–409 (4th ed. 1992).
See Schum, supra note 14, at 100–09; Kadane & Schum, supra note 3, at 53–60.
See Schum, supra note 14, at 2, 77, 109, 138; Kadane & Schum, supra note 3, at 71, 76.
Schum, supra note 14, at 81–82, 472 (stating “generalizations are required”); Kadane & Schum, supra note 3, at 88–89 (“[B]ehind every arc or link in a chain of reasoning, there must reside an appropriate generalization that licenses the probabilistic inferential step at this arc.”).
Schum, supra note 14, at 82. At one point, Schum suggests that the term “generalization” is synonymous in this context with “warrant.” Id. at 81. This is not the meaning given to “warrant” in this Article. In my use, “warrant” is the account or argument, including the evidence, that justifies finding that the conclusion is true or probably true. “Warrant” as “generalization” would merely refer to a major premise in a particular kind of warranting argument. When the word “warrant” is used throughout this Article, however, it has my broader sense, not Schum’s narrower meaning.
Id. at 102. For additional examples, see id. at 86–92, 101–02, 110–11.
Id. at 81–82, 101–02, 110–11. It is not clear whether the hedge term is about frequency (how often the asserted sequence of events occurs), warrant (how good the evidential support is), or subjective confidence (how convinced the speaker is). On the one hand, Schum states that “[w]hich hedge I choose depends upon the strength of my own belief based upon the experiences I have had in evaluating this kind of evidence.” Id. at 81–82. On the other hand, Schum thinks that the nature of the evidence is important. He states: In situations in which our inferences involve replicable processes, we may have statistical or frequentist backings for these generalizations. In nonreplicable situa-tions involving singular or unique events, we may either support or weaken a generalization on the basis of ancillary evidence resulting from a variety of dif-ferent tests of these generalizations.
Id. at 210.
See id. at 91, 472.
Id at 472.
Id. at 261–69.
Id. at 261–63. Concerning the linguistic and logical dimensions of inferences, Schum’s analysis assumes that linguistic expressions can be found to describe the evidence accurately and to formulate the conclusions adequately. While there is some discussion of incorporating Zadeh’s fuzzy sets to capture the vagueness in set membership, Schum’s theory provides little insight into the possible sources of error below the propositional level. In order to use Schum’s method in an actual case, it would have to be supplemented by an account of linguistic meaning and logical structure that clarifies the evidence and conclusions in that case.
See id. at 262.
See id. at 263–64 (providing a possible membership function for “usually”).
See id. at 262.
Id. at 264.
Id. at 82, 263. Supporting generalizations with ancillary evidence is especially important in factfinding about unique events. As Kadane and Schum state: The basis for any defensible probability assessments for singular or unique events… is given by the strength of the generalization we can assert in defense of a link together with ancillary evidence that either supports or weakens the generalization’s being applicable in the inference at hand. Absent any ancillary evidence, our inference at a link would be based only upon an unsupported generalization. A chain of reasoning based only on unsupported generalizations cannot be very strong…
Kadane & Schum, supra note 3, at 87.
See Schum, supra note 14, at 71, 82, 112–14, 187–92; Kadane & Schum, supra note 3, at 85–88.
See Schum, supra note 14, at 82.
See id. at 249–50.
Kadane & Schum, supra note 3, at 51.
See Fed. R. Evid. 401; Kadane & Schum, supra note 3, at 50–52.
Schum, supra note 14, at 117; see also Kadane & Schum, supra note 3, at 52–53.
See Schum, supra note 14, at 112, 157–60, 187–91, 207; Kadane & Schum, supra note 3, at 52–53, 85–88.
See Schum, supra note 14, at 210, 249–51; see also Kadane & Schum, supra note 3, at 85–88. As Schum states: “A generalization is supported to the extent that this generalization survives our best attempts to show that it is invalid in the particular instance of concern.” Schum, supra note 14, at 251.
See Schum, supra note 14, at 200–69; Kadane & Schum, supra note 3, at 150–55.
See Schum, supra note 14, at 213–22, 290–449; Kadane & Schum, supra note 3, at 12157, 119–21.
See Vern R.Walker, Language, Meaning, and Warrant: An Essay on the Use of Bayesian Probability Systems in Legal Factfinding, 39 Jurimetrics 391, 397–404 (1999).
Formally, the likelihood ratio quantifies the change in odds on a proposition (such as the ultimate conclusion E) as a new proposition is taken into account (here, the evidence E*). See Schum, supra note 14, at 215–22; Kadane & Schum, supra note 3, at 124–26.
See Kadane & Schum, supra note 3, at 92, 290 chart 4.
See Schum, supra note 14, at 215–22; Kadane & Schum, supra note 3, at 116–50.
See Kadane & Schum, supra note 3, at 125.
See Schum, supra note 14, at 263; cf. Kadane & Schum, supra note 3, at 82 (stating that even tangible evidence, such as bullets or weapons, are charted in his method as propositions, since “[n]o item of tangible evidence `speaks for itself”).
See supra note 64 and accompanying text.
The requirement of a “hedge” within a generalization is not a sufficient answer. A hedge term can indicate the intended quantification for the proposition — for example, that the generalization might be true in “all,” “many,” “forty-two percent,” or “at least one” of the cases. But there can still be uncertainty about whether the generalization (with its intended quantification) is true or false. Uncertainty is about the potential for error, whatever the generalization’s quantification.
See supra Part 1.2.2
See supra note 52 and accompanying text.
Perhaps Schum merely assumes what logicians call a “direct inference” from the statistical generalization to the probability about a specific case. This is, however, a complicated inference when it comes to providing warrant. See Walker, supra note 4, at 279307.
See Schum,supra note 14, at 188–90.
See id. at 150–55.
See id. at 187–92, 245–51, 272–74, 301–06.
Schum considers probabilities, including likelihoods (conditional probabilities), to be subjective in nature. See id. at 40, 52–54, 209–10, 263–65; Kadane & Schum, supra note 3, at 24, 117–18, 267–68.
On the use of reasonably well-specified and reasonably stable causal systems to warrant assigning probabilities to sequences of events, see Walker, supra note 4, at 279–81, 29297.
This is a different problem than the regress that is possible because any inference can be decomposed further, see Kadane & Schum, supra note 3, at 52, 246, and because more generalizations are needed to support the inferential use of ancillary evidence, see Schum, supra note 14, at 187–92. The problem of adjusting probative force to reflect multiple generalizations must be faced in order to apply the method to any inference at all. In this respect, it resembles the problem of combining multiple items of evidence, to which Schum devotes considerable attention. See id. At 366–449.
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2002 Physica-Verlag Heidelberg
About this chapter
Cite this chapter
Walker, V.R. (2002). Theories of Uncertainty: Explaining the Possible Sources of Error in Inferences. In: MacCrimmon, M., Tillers, P. (eds) The Dynamics of Judicial Proof. Studies in Fuzziness and Soft Computing, vol 94. Physica, Heidelberg. https://doi.org/10.1007/978-3-7908-1792-8_10
Download citation
DOI: https://doi.org/10.1007/978-3-7908-1792-8_10
Publisher Name: Physica, Heidelberg
Print ISBN: 978-3-662-00323-7
Online ISBN: 978-3-7908-1792-8
eBook Packages: Springer Book Archive