Artificial Intelligence and Law

, Volume 22, Issue 1, pp 1–28 | Cite as

Calculating and understanding the value of any type of match evidence when there are potential testing errors

  • Norman Fenton
  • Martin Neil
  • Anne Hsu


It is well known that Bayes’ theorem (with likelihood ratios) can be used to calculate the impact of evidence, such as a ‘match’ of some feature of a person. Typically the feature of interest is the DNA profile, but the method applies in principle to any feature of a person or object, including not just DNA, fingerprints, or footprints, but also more basic features such as skin colour, height, hair colour or even name. Notwithstanding concerns about the extensiveness of databases of such features, a serious challenge to the use of Bayes in such legal contexts is that its standard formulaic representations are not readily understandable to non-statisticians. Attempts to get round this problem usually involve representations based around some variation of an event tree. While this approach works well in explaining the most trivial instance of Bayes’ theorem (involving a single hypothesis and a single piece of evidence) it does not scale up to realistic situations. In particular, even with a single piece of match evidence, if we wish to incorporate the possibility that there are potential errors (both false positives and false negatives) introduced at any stage in the investigative process, matters become very complex. As a result we have observed expert witnesses (in different areas of speciality) routinely ignore the possibility of errors when presenting their evidence. To counter this, we produce what we believe is the first full probabilistic solution of the simple case of generic match evidence incorporating both classes of testing errors. Unfortunately, the resultant event tree solution is too complex for intuitive comprehension. And, crucially, the event tree also fails to represent the causal information that underpins the argument. In contrast, we also present a simple-to-construct graphical Bayesian Network (BN) solution that automatically performs the calculations and may also be intuitively simpler to understand. Although there have been multiple previous applications of BNs for analysing forensic evidence—including very detailed models for the DNA matching problem, these models have not widely penetrated the expert witness community. Nor have they addressed the basic generic match problem incorporating the two types of testing error. Hence we believe our basic BN solution provides an important mechanism for convincing experts—and eventually the legal community—that it is possible to rigorously analyse and communicate the full impact of match evidence on a case, in the presence of possible errors.


Bayes Likelihood ratio Forensic match Evidence 



We are indebted to the following for providing comments, corrections, relevant information, and contacts: David Balding, Daniel Berger, Sheila Bird, Tiernan Coyle, David Kaye, Joseph Kadane, Jay Koehler, Margarita Kotti, David Lagnado, Amber Marks, William Marsh, Geoff Morrison, Richard Nobles, David Ormerod, Mike Redmayne, David Schiff, Bill Thompson, Patricia Wiltshire.


  1. ABS Consulting (2002) Marine safety: tools for risk-based decision making, Government InstitutesGoogle Scholar
  2. AgenaRisk software (2013).
  3. Aitken CGG, Taroni F (2004) Statistics and the evaluation of evidence for forensic scientists, 2nd edn. Wiley, New JerseyCrossRefzbMATHGoogle Scholar
  4. Aitken C et al (2011) Expressing evaluative opinions: a position statement. Sci Justice 51(1):1–2CrossRefGoogle Scholar
  5. Balding D (2004) Comment on: why the effect of prior odds should accompany the likelihood ratio when reporting DNA evidence. Law Probab Risk 3(1):63–64CrossRefGoogle Scholar
  6. Balding DJ (2005) Weight-of-evidence for forensic DNA profiles. Wiley, New JerseyCrossRefzbMATHGoogle Scholar
  7. Bedford T, Cooke R (2001) Probabilistic risk analysis, foundations and method. Cambridge University Press, CambridgeCrossRefGoogle Scholar
  8. Berger CEH, Buckleton J, Champod C, Evett I, Jackson G (2011) Evidence evaluation: a response to the court of appeal judgement in R v T. Sci Justice 51:43–49CrossRefGoogle Scholar
  9. Bex FJ, van Koppen PJ, Prakken H, Verheij B (2010) A hybrid formal theory of arguments, stories and criminal evidence. Artif Intell Law 18(2):123–152CrossRefGoogle Scholar
  10. Broeders T (2009) Decision-making in the forensic arena. In: Kaptein H, Prakkenx H and Verheij H, Ashgate (eds) Legal evidence and proof: statistics, stories and logic, p 71–92Google Scholar
  11. Buckleton J, Triggs CM, Walsh SJ (2005). Forensic DNA Evidence Interpretation, CRC PressGoogle Scholar
  12. Cowell RG, Dawid AP, Lauritzen SL, Spiegelhalter DJ (1999) Probabilistic networks and expert systems. Springer, New YorkzbMATHGoogle Scholar
  13. Cowell RG, Lauritzen SL, Mortera J (2008) Probabilistic modelling for DNA mixture analysis. Forensic Sci Int Genet Suppl Series 1(1):640–642CrossRefGoogle Scholar
  14. Dawid AP (2004) Which likelihood ratio (comment on ‘why the effect of prior odds should accompany the likelihood ratio when reporting DNA evidence). Law Probab Risk 3(1):65–71CrossRefGoogle Scholar
  15. Dawid AP, Evett IW (1997) Using a graphical model to assist the evaluation of complicated patterns of evidence. J Forensic Sci 42:226–231Google Scholar
  16. Dawid AP, Mortera J, Pascali VL, Van Boxel D (2002) Probabilistic expert systems for forensic inference from genetic markers. Scand J Stat 29(4):577–595CrossRefzbMATHGoogle Scholar
  17. Dawid AP, Mortera J, Vicard P (2007) Object-oriented Bayesian networks for complex forensic DNA profiling problems. Forensic Sci Int 169:195–205CrossRefGoogle Scholar
  18. Evett IW, Weir BS (1998) Interpreting DNA evidence : statistical genetics for forensic scientists. Sinauer Associates, SunderlandGoogle Scholar
  19. Evett IW, Foreman LA, Jackson G, Lambert JA (2000) DNA profiling: a discussion of issues relating to the reporting of very small match probabilities. Crim Law Rev (May) 341–355Google Scholar
  20. Fenton NE (2011) Science and law: improve statistics in court. Nature 479:36–37CrossRefGoogle Scholar
  21. Fenton N, Neil M (2010) Comparing risks of alternative medical diagnosis using Bayesian arguments. J Biomed Inform 43:485–495CrossRefGoogle Scholar
  22. Fenton NE, Neil M (2011) Avoiding legal fallacies in practice using Bayesian networks. Aust J Legal Philos 36:114–150Google Scholar
  23. Fenton N, Neil M (2012) Risk assessment and decision analysis with bayesian networks. CRC Press, Boca RatonGoogle Scholar
  24. Fenton NE, Neil M, Lagnado D (2012) A general structure for legal arguments about evidence using Bayesian networks. Cognit Sci 37(1):61–102CrossRefGoogle Scholar
  25. Fenton NE, Berger D, Lagnado D, Neil M, Hsu A (2013) When ‘neutral’ evidence still has probative value (with implications from the Barry George Case), science and justice
  26. Foreman LA, Evett IW (2001) Statistical analysis to support forensic interpretation for a new ten-locus STR profiling system. Int J Legal Med 114(3):147–155CrossRefGoogle Scholar
  27. Gigerenzer G (2002) Reckoning with risk: learning to live with uncertainty. Penguin Books, LondonGoogle Scholar
  28. Gill R (2013) Forensic statistics: ready for consumption?
  29. Gittelson S, Biedermann A, Bozza S, Taroni F (2013) Modeling the forensic two-trace problem with Bayesian networks. Artif Intell Law 21:221–252CrossRefGoogle Scholar
  30. Hepler AB, Dawid AP, Leucari V (2007) Object-oriented graphical representations of complex patterns of evidence. Law Probab Risk 6(1–4):275–293CrossRefGoogle Scholar
  31. Kadane JB, Schum DA (1996) A probabilistic analysis of the Sacco and Vanzetti evidence. Wiley, New JerseyGoogle Scholar
  32. Kaye DH (2009) Identification, individualization, uniqueness. Law Probab Risk 8(2):85–94CrossRefMathSciNetGoogle Scholar
  33. Kaye DH, Bernstein DE, Mnookin JL (2010) The new wigmore: a treatise on evidence—expert evidence. Aspen Publishers, Second EditionGoogle Scholar
  34. Koehler JJ (1993) Error and exaggeration in the presentation of DNA evidence at trial. Jurimetrics 34:21–39Google Scholar
  35. Koehler JJ (1996) On conveying the probative value of DNA evidence: frequencies, likelihood ratios and error rates. Univers Colo Law Rev 67:859–886Google Scholar
  36. Koehler JJ (2012) Proficiency tests to estimate error rates in the forensic sciences. Law Probab Risk 12(1):89–98. doi: 10.1093/lpr/mgs013 CrossRefGoogle Scholar
  37. Meester R, Sjerps M (2004) Why the effect of prior odds should accompany the likelihood ratio when reporting DNA evidence. Law Probab Risk 3(1):51–62CrossRefGoogle Scholar
  38. Morrison GM (2012) The likelihood ratio framework and forensic evidence in court: a response to RvT. Int J Evidence Proof 16(1)Google Scholar
  39. Mortera J, Dawid AP, Lauritzen SL (2003) Probabilistic expert systems for DNA mixture profiling. Theor Pop Biol 63:191–205CrossRefzbMATHGoogle Scholar
  40. Nordgaard A, Hedell R, Ansell R (2012) Assessment of forensic findings when alternative explanations have different likelihoods-”Blame-the-brother”-syndrome. Sci Justice 52:226–236CrossRefGoogle Scholar
  41. Puch-Solis R, Roberts P, Pope S, Aitken C (2012). Practitioner guide no 2: assessing the probative value of DNA evidence, guidance for judges, lawyers, forensic scientists and expert witnesses, royal statistical Society.
  42. Queen Mary University of London, ERC Advanced Grant (2013) Effective Bayesian modelling with knowledge before data (BAYES-KNOWLEDGE)
  43. R v Adams (1996) 2 Cr App R 467, [1996] Crim LR 898, CA and R v Adams [1998] 1 Cr App R 377Google Scholar
  44. Redmayne M (2001) Expert evidence and criminal justice. Oxford University Press, OxfordCrossRefGoogle Scholar
  45. Redmayne M, Roberts P, Aitken C, Jackson G (2011) Forensic science evidence in question. Crim Law Rev 5:347–356Google Scholar
  46. Robertson B, Vignaux T (1995) Interpreting evidence: evaluating forensic science in the courtroom. Wiley, New jerseyGoogle Scholar
  47. Robertson B, Vignaux GA, Berger CEH (2011) Extending the confusion about Bayes. Mod Law Rev 74(3):444–455CrossRefGoogle Scholar
  48. Saks MJ, Koehler JJ (2007) The individualization fallacy in forensic science evidence
  49. Schum DA, Starace S (2001) The evidential foundations of probabilistic reasoning. Northwestern University Press, EvanstonGoogle Scholar
  50. Shaw A (2013) Do people trust Bayesian calculations better if they are shown a simple version first? MSc Thesis, University of LondonGoogle Scholar
  51. Sjerps M, Berger C (2012) How clear is transparent? Reporting expert reasoning in legal cases. Law Probab Risk 11(4):317–329CrossRefGoogle Scholar
  52. Sjerps M, Meesters R (2009) Selection effects and database screening in forensic science. Forensic Sci Int 192(1–3):56–61CrossRefGoogle Scholar
  53. Taroni F, Aitken C, Garbolino P, Biedermann A (2006) Bayesian networks and probabilistic inference in forensic science. Wiley, New JerseyCrossRefzbMATHGoogle Scholar
  54. Thompson WC (2008) The potential for error in forensic DNA testing (and how that complicates the use of DNA databases for criminal identification). In council for responsible genetics (CRG) National Conference: forensic DNA databases and race: Issues, Abuses and Actions, June 19-20, 2008, New York University
  55. Thompson WC, Taroni F, Aitken CGG (2003) How the probability of a false positive affects the value of DNA evidence. J Forensic Sci 48(1):47–54Google Scholar
  56. Triggs CM, Buckleton JS (2004) Comment on: why the effect of prior odds should accompany the likelihood ratio when reporting DNA evidence. Law Probab Risk 3(1):73–82CrossRefGoogle Scholar
  57. Verheij B (2007) Argumentation support software: boxes-and-arrows and beyond. Law Probab Risk 6:187–208CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2013

Authors and Affiliations

  1. 1.Risk and Information Management Research GroupQueen Mary University of LondonLondonUK
  2. 2.Computer Science and StatisticsQueen Mary University of LondonLondonUK
  3. 3.Queen Mary University of LondonLondonUK
  4. 4.Agena LtdCambridgeUK

Personalised recommendations