Bayes Factor for Evaluative Purposes

Abstract

This chapter presents and discusses the use of the Bayes factor for the evaluation of scientific evidence in the form of discrete, continuous and continuous multivariate data. The latter may present a complex dependence structure that will be handled by means of multilevel models. The notion of “evaluative purpose” is understood here as referring to situations in which material of known source (control material) and evidential material of unknown source (recovered or questioned material) is collected and analyzed. The purpose is to evaluate the effect of the output of the examinations, in the form of scores or measurements of features, on the odds in favor of a proposition put forward by the prosecution, compared to an alternative proposition advanced by the defence. A discussion is included of the sensitivity of the described Bayes factor procedures to changes in the features of recovered and control materials, the available background information, as well as to choices made during probabilistic modelling and prior elicitation.

3.1 Introduction

Consider a case where material of known source (control material) and evidential material of unknown source (recovered or questioned material) are collected and analyzed. Interpretation of scientific evidence then amounts to assessing the probative value of the observations made during comparative examinations. The evidence is evaluated in terms of its effect on the odds in favor of a proposition H ₁ put forward by the prosecution, compared to an alternative proposition H ₂ advanced by the defense.

During comparative examinations, observations and measurements are made, leading to either discrete or continuous data. Forensic laboratories may also have equipment and methodologies that can lead to output in the form of multivariate data. Thus, scientific evidence is often described by more than one variable. For example, glass fragments from a crime scene can be compared with fragments collected on the clothing of a person of interest on the basis of several chemical components, as well as physical characteristics. It should be noted, however, that the assessment of a Bayes factor for multivariate data may be challenging. For example, data may not present enough regularity so that standard parametric distributions cannot be used. Data may also present a complex dependence structure with several levels of variation. In addition, a feature-based approach might not be always feasible, and it may be necessary to derive a Bayes factor on the basis of scores.

This chapter is structured as follows. Sections 3.2 and 3.3 address the problem of evaluation of evidence for various types of discrete and continuous data, respectively. Section 3.4 presents an extension to continuous multivariate data.

3.2 Evidence Evaluation for Discrete Data

This section deals with measurement results in the form of counts, using the binomial model (Sect. 3.2.1), the multinomial model (Sect. 3.2.2), and the Poisson model (Sect. 3.2.3).

3.2.1 Binomial Model

In many practical applications, data derive from realizations of experiments that may take one of two mutually exclusive outcomes. Examples include general features (so-called class characteristics) observed on questioned and known items or materials (e.g., fired bullets, fibers) when the question of interest is whether the compared materials come from the same source.

Consider a hypothetical case involving a questioned document for which results of analyses of black toner are available. On the questioned document, black bi-component toner is present. It is of the same type as that used by a given printing machine (known source). A question that may be of interest in such a case is how this analytical information should affect one’s belief in the proposition according to which the questioned document has been printed using the device of interest (Biedermann et al., 2009, 2011a). The competing propositions can thus be defined as follows:

H ₁ : :

The questioned document has been printed with the device of interest.

H ₂ : :

The questioned document has been printed with an unknown device.

Let T denote the observed toner type, either single component (T _S) or bi-component (T _B). Suppose that a database of the toner type (magnetism) of samples of black toner from N machines is available, n of which use a bi-component toner. Denote by θ the proportion of the population of printing devices equipped with bi-component toner. Available counts can be treated as realizations of Bernoulli trials (Sect. 2.2.1) with constant probability of success θ, $\Pr (T_B\mid \theta )=\theta$. Suppose a conjugate beta prior distribution Be(α, β) is used to model uncertainty about θ, where α and β can be elicited using the available background knowledge as in (1.42) and (1.43).

Denote by E _y the observations made on recovered material and by E _x the observations made on control material (i.e., documents printed with the device of interest). If the questioned document originates from the device of interest, the probability of the evidence becomes

$$\displaystyle \begin{aligned} \begin{array}{rcl} \Pr(E_y=T_B,E_x=T_B\mid H_1)& =&\displaystyle \int_{\varTheta}\Pr(T_B\mid\theta)\cdot \theta^{\alpha-1}(1-\theta)^{\beta-1}d\theta/\mbox{B}(\alpha,\beta)\\ & =&\displaystyle \int_{\varTheta}\theta\cdot \theta^{\alpha-1}(1-\theta)^{\beta-1}d\theta/\mbox{B}(\alpha,\beta). \end{array} \end{aligned} $$

If the questioned document originates from an unknown device (i.e., two distinct devices have been used), the probability of the evidence becomes

$$\displaystyle \begin{aligned} \begin{array}{rcl} \Pr(E_y=T_B,E_x=T_B\mid H_2)=\int_{\varTheta}\theta^2\cdot \theta^{\alpha-1}(1-\theta)^{\beta-1}d\theta/\mbox{B}(\alpha,\beta). \end{array} \end{aligned} $$

The Bayes factor can be computed as

$$\displaystyle \begin{aligned} \begin{array}{rcl}{} \mbox{BF}& =&\displaystyle \frac{\int_{\varTheta}\theta\cdot \theta^{\alpha-1}(1-\theta)^{\beta-1}d\theta}{\int_{\varTheta}\theta^2 \cdot \theta^{\alpha-1}(1-\theta)^{\beta-1}d\theta}\\ & =&\displaystyle \frac{\mbox{B}(\alpha+1,\beta)}{\mbox{B}(\alpha+2,\beta)}\int_{\varTheta}\frac{\theta^{\alpha}(1-\theta)^{\beta-1}}{\theta^{\alpha+1}(1-\theta)^{\beta-1}}\frac{\mbox{ B}(\alpha+2,\beta)}{\mbox{B}(\alpha+1,\beta)}\\ & =&\displaystyle \frac{\alpha+\beta+1}{\alpha+1}. \end{array} \end{aligned} $$

(3.1)

Example 3.1 (Questioned Documents)

Consider the case of a printed document of unknown origin. Analyses reveal that the toner present on the printed document is of type “bi-component.” The printing device that is thought to have been used to print the questioned document is equipped with a bi-component toner. In an available database with a total of N = 100 samples of black toner, n = 23 are bi-component (see Table 3.1). Using this information, the parameters of the beta prior distribution about θ can be elicited as follows:

Table 3.1 Results obtained following the analysis of, respectively, the component type (magnetism) and the resin type of 100 samples of black toner (Biedermann et al., 2011a)

This leads to a Be(23, 76).

The Bayes factor in (3.1) can be computed straightforwardly as follows:

The Bayes factor provides weak support for the proposition H ₁ according to which the questioned document has been printed with the printing device of interest rather than with an unknown printing device (H ₂).

It is worth noting that there is an alternative development described in the forensic statistics literature that considers background information derived from a population database as part of the evidence, (e.g., Ommen et al., 2016; Dawid, 2017). According to this line of reasoning, if proposition H ₁ is true (numerator), there are (n + 1) counts of bi-component toners. That is, the questioned item and the known item are assumed to come from the same source, hence adding one count to the database. Conversely, if proposition H ₂ is true (denominator), there are (n + 2) counts of bi-component toner. Here, it is assumed that the questioned item and the known item come from different sources, hence adding two counts to the database. The Bayes factor can then be obtained as

$$\displaystyle \begin{aligned} \begin{array}{rcl}{} \mbox{BF}& =&\displaystyle \frac{\int_{\varTheta}\theta^{n+1}(1-\theta)^{N-n}\theta^{\alpha-1}(1-\theta)^{\beta-1}d\theta}{\int_{\varTheta}\theta^{n+2}(1-\theta)^{N-n}\theta^{\alpha-1}(1-\theta)^{\beta-1}d\theta}\\ & =&\displaystyle \frac{\alpha+\beta+N+1}{\alpha+n+1}. \end{array} \end{aligned} $$

(3.2)

One can immediately verify that this corresponds to the BF in (3.1) with parameter α replaced by α + n, and parameter β replaced by β + N − n. However, it may be questioned whether the available database should be considered as evidence, rather than as conditioning information, because the database contains only general data unrelated to the case under investigation (Aitken et al., 2021).

3.2.2 Multinomial Model

The analyses described in Sect. 3.2.1 can be extended to situations where experiments can lead to more than two mutually exclusive outcomes.

Consider again the case involving printed documents, introduced in Sect. 3.2.1. Laboratories often analyze resins of toner on printed documents by means of Fourier Infrared Spectroscopy (FTIR). The results can be classified into one of several (k) categories (Table 3.1). Suppose that the resin type (R) recovered on the questioned document belongs to category j, which is also found in the toner used by a given printing machine. The question of interest is similar to the one considered in Sect. 3.2.1, that is, how the available analytical information should affect one’s belief in the proposition according to which a questioned document has been printed using a given device, called the potential source, rather than by some unknown printing device.

Denote by θ _j the proportion of the population that is of type (category) R _j, j = 1, …, k, $\Pr (R_j\mid \theta _j)=\theta _j$. Assume that observations of distinct categories can be treated as independent: available counts n ₁, …, n _k can be treated as realizations from a multinomial distribution Mult(n, θ ₁, …, θ _k)

$$\displaystyle \begin{aligned} \begin{array}{rcl} f(n_1,\dots,n_k\mid\theta_1,\dots,\theta_k)=\frac{N!}{n_1!\cdot\dots\cdot n_k!}\theta_1^{n_1}\cdot\dots\cdot\theta_k^{n_k}. \end{array} \end{aligned} $$

A conjugate Dirichlet prior probability distribution Dir(α ₁, …, α _k) is considered for modeling uncertainty about the population proportions θ ₁, …, θ _k:

$$\displaystyle \begin{aligned} \begin{array}{rcl} f(\theta_1,\dots,\theta_k\mid\alpha_1,\dots,\alpha_k)=\theta_1^{\alpha_1-1}\cdot\dots\cdot\theta_k^{\alpha_k-1}/\mbox{B}(\alpha), \end{array} \end{aligned} $$

with $\mbox{B}(\alpha )=\frac {\prod _{i=1}^{k}\varGamma (\alpha _i)}{\varGamma (\alpha )}$ and $\alpha =\sum _{i=1}^{k}\alpha _i$.

Denote by E _y the observations made on the recovered material and by E _x the observations made on the control material (i.e., documents printed with the device of interest). If the questioned document originates from the device of interest, the probability of the findings E = (E _y, E _x) becomes

$$\displaystyle \begin{aligned} \begin{array}{rcl} \Pr(E_y=R_j,E_x=R_j\mid H_1)& =&\displaystyle \int_{\varTheta}\Pr(R_j\mid\theta_j)\cdot \theta_1^{\alpha_1-1}\cdot\dots\cdot \theta_j^{\alpha_j-1}\cdot\dots\cdot \theta_k^{\alpha_k-1}d\boldsymbol\theta/\mbox{B}(\alpha)\\ & =&\displaystyle \int_{\varTheta}\theta_j\cdot \theta_1^{\alpha_1-1}\cdot\dots\cdot \theta_j^{\alpha_j-1}\cdot\dots\cdot \theta_k^{\alpha_k-1}d\boldsymbol\theta/\mbox{B}(\alpha). \end{array} \end{aligned} $$

If the questioned documents originate from an unknown device (i.e., two distinct devices have been used), the probability of the findings E becomes

$$\displaystyle \begin{aligned} \begin{array}{rcl} \Pr(E_y=R_j,E_x=R_j\mid H_2)=\int_{\varTheta}\theta_j^2\cdot \theta_1^{\alpha_1-1}\cdot\dots\cdot \theta_j^{\alpha_j-1}\cdot\dots\cdot \theta_k^{\alpha_k-1}d\boldsymbol\theta/\mbox{B}(\alpha). \end{array} \end{aligned} $$

The Bayes factor can be computed as

$$\displaystyle \begin{aligned} \begin{array}{rcl}{} \mbox{BF} & =&\displaystyle \frac{\int\theta_j\cdot\theta_1^{\alpha_1-1}\cdot\dots\cdot\theta_j^{\alpha_j-1}\cdot\dots\cdot \theta_k^{\alpha_k-1}d\boldsymbol\theta}{\int \theta_j^2\cdot \theta_1^{\alpha_1-1}\cdot\dots\cdot\theta_j^{\alpha_j-1}\cdot\dots\cdot \theta_k^{\alpha_k-1}d\boldsymbol\theta}\\ & =&\displaystyle \frac{\alpha+1}{{\alpha_j+1}}. \end{array} \end{aligned} $$

(3.3)

Example 3.2 (Questioned Documents—Continued)

Recall Example 3.1, involving questioned documents on which black toner is present. Suppose now that laboratory analyses focus on the toner’s resin component. Suppose that the parameters of the Dirichlet prior probability distribution are elicited as

Suppose that the rather common resin group Epoxy-A (category j = 2 in Table 3.1) is observed on both the questioned and known documents. The Bayes factor in (3.3) can be computed straightforwardly as

The Bayes factor provides, again, weak support for the proposition H ₁ according to which the questioned document has been printed with the printing device of interest, rather than with an unknown printing device (H ₂).

Suppose that a database of the resin type of samples of black toner from N machines is available, n ₁ (n ₂, …) of which belong to category 1 (2, …), as in Table 3.1. These data can be used to elicit the Dirichlet prior probability distribution. Following the methodology proposed by Zapata-Vazquez et al. (2014), the hyperparameters α ₁, …, α _k can be assessed by starting from expert judgments (e.g., a vector of quantiles) about proportions of items belonging to each category. Tools for eliciting prior probability distributions from experts’ opinions are also available in the R package SHELF. An example will be presented in Sect. 4.2.2.

3.2.3 Poisson Model

Some forensic science applications focus on the number of occurrences of particular events or observations that take place at given intervals of time or space. Practical examples are the number of gunshot residue particles (GSR) collected on the surface of the hands of individuals suspected to be involved in the discharge of a firearm (Cardinetti et al., 2006), or the number of corresponding matching striations in the comparative examination of marks left by firearms on fired bullets (Bunch, 2000).

Consider the following hypothetical case. A fired bullet is found at a crime scene, and a person of interest is apprehended, carrying a gun. The following propositions are of interest:

H ₁ : :

The bullet found at the crime scene was fired with the seized gun.

H ₂ : :

The bullet found at the crime scene was fired with an unknown gun.

The recovered bullet and bullets fired with the seized gun are compared. Consecutive matching striations (CMS) is a simple concept to quantify the extent of agreement between marks. The number of observed consecutively matching striations can be interpreted as a score . Let Δ(x, y) be the maximum CMS count for a given comparison. For the evaluation of a CMS count, data on comparisons made between pairs of bullets test-fired with the seized gun and between pairs of bullets test-fired with different guns are needed. The (score-based) Bayes factor therefore is

$$\displaystyle \begin{aligned} \begin{array}{rcl} \mbox{sBF}=\frac{{g}(\varDelta(x,y)\mid H_1)}{{g}(\varDelta(x,y)\mid H_2)}. \end{array} \end{aligned} $$

A statistical model commonly used in the forensic science literature for the type of data encountered in the example here assumes that counts follow a Poisson distribution Pn(λ)

$$\displaystyle \begin{aligned} \begin{array}{rcl} g(\varDelta(x,y)\mid\lambda_i)=\frac{e^{-\lambda_i}\lambda_i^{\varDelta(x,y)}}{\varDelta(x,y)!},\qquad \qquad \varDelta(x,y)=0,1,\dots\; ;\; \lambda_i\ge 0, \end{array} \end{aligned} $$

where parameter λ _i, i = 1, 2, represents the weighted average maximum CMS count.

Suppose that two datasets are compiled. The first relates to pairs of bullets fired with the seized gun, and the second to pairs of bullets fired with different guns. Such data can be used to inform the probability distribution g(⋅) at the score value Δ(x, y) as discussed in Sect. 1.5.2 and to compute the Bayes factor as

$$\displaystyle \begin{aligned} \begin{array}{rcl} \mbox{sBF} = \frac{\hat g(\varDelta(x,y)\mid x,H_1)}{\hat g(\varDelta(x,y)\mid H_2)}. \end{array} \end{aligned} $$

Bunch (2000) describes a likelihood ratio procedure for inference about competing propositions. This account is based on a frequentist perspective because it uses the maximum likelihood estimates $\hat \lambda _1$ and $\hat \lambda _2$ for parameters λ ₁ and λ ₂, calculated under the assumption that either proposition H ₁ or proposition H ₂ is true. Using these two estimates in the component Poisson likelihoods leads to the following likelihood ratio :

$$\displaystyle \begin{aligned} \begin{array}{rcl} \mbox{LR}=\frac{e^{-\hat\lambda_1}\hat\lambda_1^{\varDelta(x,y)}}{e^{-\hat\lambda_2}\hat\lambda_2^{\varDelta(x,y)}}. \end{array} \end{aligned} $$

In Bayesian statistics, the most common prior distribution for λ _i is the gamma distribution Ga(α _i, β _i) with shape parameter α and rate parameter β (e.g. Bernardo and Smith, 2000):

$$\displaystyle \begin{aligned} \begin{array}{rcl} f(\lambda_i\mid\alpha_i,\beta_i)=\frac{\beta_i^{\alpha_i}}{\varGamma(\alpha_i)}\lambda_i^{\alpha_i-1}e^{-\beta_i\lambda_i}, \qquad \qquad \lambda_i>0\; ;\; \alpha_i,\beta_i>0. \end{array} \end{aligned} $$

Since the Poisson and gamma distributions are conjugate (Sect. 1.10), the posterior distribution of λ is still in the family of gamma distributions, with parameters α and β updated according to well-known updating rules (see, e.g., Lee, 2012). When we have a realization of a random sample from a Poisson distribution, Pn(λ), say (z ₁, …, z _n), we end up with a Ga(α′, β′), where $\alpha '=\alpha +\sum _{i=1}^{n}z_i$ and β′ = β + n. Note that in this case there is only one observation, Δ(x, y); therefore, α′ = α + Δ(x, y) and β′ = β + 1. See also Biedermann et al. (2011b) for further illustrations of the Poisson–gamma model in forensic science applications.

The marginal distribution in the numerator and denominator of the Bayes factor is known in closed form here. It is a Poisson–gamma distribution :

$$\displaystyle \begin{aligned} \begin{array}{rcl} g(\varDelta(x,y)| \alpha_i,\beta_i)& =&\displaystyle \int_{\lambda_i}g(\varDelta(x,y)|\lambda_i)f(\lambda_i|\alpha_i,\beta_i)d\lambda_i\\ & =&\displaystyle \frac{1}{\varDelta(x,y)!}\frac{\beta_i^{\alpha_i}}{\varGamma(\alpha_i)}\frac{\varGamma(\alpha_i+\varDelta(x,y))}{(\beta_i+1)^{\alpha_i+\varDelta(x,y)}}.{} \end{array} \end{aligned} $$

(3.4)

The score-based Bayes factor then becomes

$$\displaystyle \begin{aligned} \begin{array}{rcl} {} \mbox{sBF}=\frac{\beta_1^{\alpha_1}\varGamma(\alpha_2)\varGamma(\alpha_1+\varDelta(x,y))(\beta_2+1)^{\alpha_2+\varDelta(x,y)}}{\beta_2^{\alpha_2}\varGamma(\alpha_1)\varGamma(\alpha_2+\varDelta(x,y))(\beta_1+1)^{\alpha_1+\varDelta(x,y)}}. \end{array} \end{aligned} $$

(3.5)

Another example of the use of the Poisson distribution for data in the form of independent counts can be found in Aitken and Gold (2013). These authors considered the number of occurrences of selected characteristics of speech recorded in a succession of time periods. In this application, a feature-based Bayes factor is used to assess findings with respect to the proposition according to which recorded and control speeches originate from the same source versus the alternative proposition that they originate from different sources.

Example 3.3 (Firearm Examination)

Consider a case involving a questioned bullet. During comparison with a reference bullet, the examiner counts four CMS , i.e., Δ(x, y) = 4. Suppose that the assumptions made in Bunch (2000) are suitable for the case here so that for bullets fired from the same gun (proposition H ₁ holds), the weighted average maximum CMS is taken to be equal to 3.91. For bullets fired from different guns (proposition H ₂ holds), the weighted average maximum CMS count is taken to be equal to 1.32. These values are used in the Poisson likelihoods under H ₁ and H ₂, and the likelihood ratio can easily obtained as

The evidence provides weak support in favor of the proposition according to which the recovered bullet passed through the barrel of the seized gun, rather than through the barrel of an unknown gun.

Consider now the Bayesian perspective. Suppose that the available knowledge allows one to set the hyperparameters of the gamma distribution equal to {α ₁ = 125, β ₁ = 32} for the numerator and to {α ₂ = 7, β ₂ = 5} for the denominator. This amounts to using a gamma prior distribution for λ ₁ with mean equal to 3.91 and standard deviation equal to 0.35 and a gamma prior distribution for λ ₂ with mean equal to 1.4 and standard deviation equal to 0.53. The two prior distributions are shown in Fig. 3.1.

Fig. 3.1

Gamma prior for the Poisson parameter λ under H ₁ (solid line) and H ₂ (dashed line)

First, we write a short function poisg that computes the marginal distribution in (3.4)

Next, the Bayes factor can be computed as follows:

Note that the introduction of a prior probability distribution reflecting uncertainty about the population parameters λ ₁ and λ ₂ has slightly lowered the value of the evidence. The result still represents weak evidence in favor of the proposition that the recovered bullet was fired with the seized gun, rather than with an unknown gun.

Note that Example 3.3 involves a non-anchored approach at the numerator. The probability distribution of the score value is solely conditioned on the hypothesis of interest, that is $\hat g(\varDelta (x,y)\mid H_1)$. As mentioned at the beginning of this section, and in Sect. 1.5.2, other anchoring approaches may be considered.

3.2.3.1 Choosing the Parameters of the Gamma Prior

An evaluator who, initially, would like to give the same weight to all possible values of λ may consider to use a non-informative prior distribution , that is

$$\displaystyle \begin{aligned} \begin{array}{rcl} f(\lambda_i)=\lambda_i^{-1/2}; \qquad \qquad \lambda_i>0 \; \text{and} \; i=1,2. \end{array} \end{aligned} $$

The posterior probability distribution given the observations (z ₁, …, z _n) will be of type gamma with shape parameter $\alpha '=\sum _{i=1}^{n}z_i+1/2$ and rate parameter β′ = n. Note that in the type of case considered here, there is only one observation; therefore, α′ = Δ(x, y) + 1∕2 and β′ = 1.

However, the choice of a non-informative prior distribution may be questioned. Take, for instance, the case example discussed earlier in this section (Example 3.3). It is difficult to imagine that no suitable information is available to express prior uncertainty about the unknown weighted average maximum count CMS, and hence that the same non-informative prior distribution should apply under each proposition.

In Example 3.3, an informative prior distribution has been used. This raises the question of how to translate prior knowledge into a prior distribution. As illustrated in Sect. 1.10, one way to elicit prior parameters is to express prior beliefs in terms of a measure of location and a measure of dispersion and then equate these values with the prior moments of the distribution. In the case of a gamma distribution Ga(α, β), this amounts to equate a value for the mean, m, with the prior mean α∕β, and a value for the variance, s ², with the prior variance α∕β ², that is,

$$\displaystyle \begin{aligned} \begin{array}{rcl}{} m=\frac{\alpha}{\beta} \qquad ; \qquad s^2=\frac{\alpha}{\beta^2}. \end{array} \end{aligned} $$

Solving for α and β gives

$$\displaystyle \begin{aligned} \begin{array}{rcl} \alpha& =&\displaystyle \frac{m^2}{s^2} {} \end{array} \end{aligned} $$

(3.6)

$$\displaystyle \begin{aligned} \begin{array}{rcl} \beta& =&\displaystyle \frac{m}{s^2} {}. \end{array} \end{aligned} $$

(3.7)

If the shape of the prior distribution resulting from the choice of α and β as in (3.6) and (3.7) does not reflect one’s prior beliefs suitably, then one should adjust the numerical values of m and s. However, this may not be enough to ensure that the resulting prior distribution is reasonable. One should also inquire about whether the information that is conveyed by the prior is realistically attainable. Consider a random sample of size n _e, providing the same amount of information as conveyed by the elicited prior. The sample mean should have, at least roughly, the same location and the same dispersion as the prior. The equivalent sample size n _e can then be found by matching the moments of the gamma distribution to the corresponding moments characterizing a sample of size n _e from a Poisson distributed random variable located at λ:

$$\displaystyle \begin{aligned} \begin{array}{rcl} \frac{\alpha}{\beta}& =&\displaystyle \lambda\\ \frac{\alpha}{\beta^2}& =&\displaystyle \frac{\lambda}{n_e}. \end{array} \end{aligned} $$

If the mean λ is set equal to the prior mean α∕β, the equivalent sample size n _e is equal to β.

Example 3.4 (Elicitation of a Gamma Prior)

In Example 3.3, a Ga(125, 32) was used for λ ₁ (the weighted average maximum CMS count under proposition H ₁), and a Ga(7, 5) for λ ₂ (the weighted average maximum CMS count under proposition H ₂). For the prior means of λ ₁ and λ ₂, the values 3.91 and 1.4 were used following Bunch (2000). For the dispersion of the two distributions, the values 0.35 and 0.53 have been assigned to the standard deviation under propositions H ₁ and H ₂, respectively. Parameters (α ₁ = 125, β ₁ = 32) and (α ₂ = 7, β ₂ = 5) have then been obtained as in (3.6) and (3.7). This amounts to an equivalent sample size equal to 32 for the prior density of λ ₁, and 5 for λ ₂.

3.2.3.2 Sensitivity to Prior Probabilities of Competing Propositions

It is important to emphasize that the analyses presented here make no direct probabilistic statement about the truth of the propositions put forward by opposing parties at trial. A Bayes factor of approximately 4.25, as obtained in Example 3.3, only means that the evidence is approximately 4 times more probable if proposition H ₁ is true than if the alternative proposition H ₂ is true. As noted earlier, this does not mean that proposition H ₁ is more probable than H ₂. This depends on the prior probabilities of the competing propositions, which can vary considerably among recipients of expert information, and which are beyond the area of competence of scientists.

However, it may be of interest to show the impact of different prior probability assignments on the posterior probability of the competing propositions. To do so, recall that the posterior odds are given by the product of the prior odds and the Bayes factor

$$\displaystyle \begin{aligned} \begin{array}{rcl} \frac{\Pr(H_1\mid\cdot)}{\Pr(H_2\mid\cdot)}=\mbox{BF}\times \frac{\Pr(H_1)}{\Pr(H_2)}. \end{array} \end{aligned} $$

Using this expression, one can then investigate how the posterior probability of proposition H ₁, i.e., α ₁, varies for values of π ₁, i.e., $\Pr (H_1)$, ranging from 0.01 until 0.99, and for a Bayes factor equal to 4.25, as in Example 3.3.

The solid line in Fig. 3.2 shows the value of α ₁, the posterior probability of the proposition H ₁, as a function of the prior probability, π ₁, for BF = 4.25. The plot also shows results for BF = 1 (dashed line) and for BF = 100 (dotted line).

Fig. 3.2

Posterior probability α ₁ of proposition H ₁ for values of prior probabilities π ₁ ranging from 0.01 to 0.99, and a Bayes factor equal to 4.25 (solid line), 1 (dashed line), and 100 (dotted line)

More generally, it can be observed that the higher the value of the Bayes factor, the smaller the impact of the prior probabilities on posterior probabilities.

3.3 Evidence Evaluation for Continuous Data

The previous section considered the evaluation of scientific evidence as given by discrete data. However, for many types of evidence, measurements result in continuous data.

3.3.1 Normal Model with Known Variance

In some applications, the distribution of measurements exhibits enough regularity to be captured by standard parametric models, such as the Normal distribution. One example, introduced earlier in Sect. 1.5.1, is the analysis of magnetism of black toner on printed documents. Due to the wide distribution and availability of printing machines, forensic document examiners are commonly requested to examine documents produced by electrophotographic printing processes that use dry toner. A question that forensic scientists may be asked to help with is whether or not two or more documents were printed with the same laser printer. This task involves the comparison of analytical features of a questioned document with those of control documents. One such analytical feature is the magnetic flux of toner . It is thought to be largely influenced by individual settings of the printing device, so that detectable differences may be expected on documents printed at different instances using the same or different machines (Biedermann et al., 2016a).

Suspected page substitution is a commonly encountered problem in forensic document examination. Imagine a case involving a contract consisting of three pages where the allegation is that the second page has been substituted. It may be of interest, thus, to investigate the extent to which available measurements of magnetic flux can be informative in this case.

Consider the following pair of propositions:

H ₁ : :

Page two has been printed by the device used for printing pages one and three (i.e., the three pages have been printed with the same device).

H ₂ : :

Page two has been printed by a different device.

Denote by y = (y ₁, …, y _n) the measurements of magnetic flux obtained for the questioned page. Measurements are assumed to be normally distributed with unknown mean θ and known variance σ ². The likelihood of the normal random sample (y ₁, …, y _n) can therefore be expressed as

$$\displaystyle \begin{aligned} \begin{array}{rcl}{} f(\mathbf{y}\mid\theta)=\prod_{i=1}^{n}(2\pi\sigma^2)^{-1/2}\exp\left\{-\frac{1}{2\sigma^2}(y_i-\theta)^2\right\}. \end{array} \end{aligned} $$

(3.8)

It can be shown, (e.g., Bolstad and Curran, 2017), that the likelihood of a normal random sample is proportional to the likelihood of the sample mean $\bar y=\frac {1}{n}\sum _{i=1}^{n}y_i$. The sample mean is normally distributed with mean θ and variance σ ²∕n

$$\displaystyle \begin{aligned} \begin{array}{rcl}{} f(\bar{y}\mid\theta)=(2\pi\sigma^2/n)^{-1/2}\exp\left\{-\frac{1}{2\sigma^2/n}(\bar y-\theta)^2\right\}. \end{array} \end{aligned} $$

(3.9)

In other words, it is possible to reduce the problem to one where a single normal observation $\bar y$ is available.

Next, denote the measurements on uncontested pages by {x _l} = (x _lj, j = 1, …, n and l = 1, 2), where the subscript l refers to the page number and j to the number of measurements of magnetic flux obtained for the page l. A normal distribution with mean θ and variance σ ² is assumed for x, analogously to what has been assumed for y. A conjugate normal prior distribution is chosen for θ, say θ ∼N(μ, τ ²). The Bayes factor can be computed as in (1.16):

$$\displaystyle \begin{aligned} \begin{array}{rcl}{} \mbox{BF}& =&\displaystyle \frac{f({\bar y}\mid {\mathbf{x}}_1,{\mathbf{x}}_2,H_1)}{f({\bar y}\mid H_2)}\\ & =&\displaystyle \frac{\int f({\bar y}\mid \theta)f(\theta\mid {\mathbf{x}}_1,{\mathbf{x}}_2,H_1)\mbox{d}\theta}{\int f({\bar y}\mid\theta)f(\theta\mid H_2)\mbox{d}\theta}, \end{array} \end{aligned} $$

(3.10)

where f(θ∣x ₁, x ₂, H ₁) is the posterior distribution of θ, obtained by updating the prior distribution N(μ, τ ²) using the measurements x ₁ and x ₂. This is a normal distribution, $(\theta \mid {\mathbf {x}}_1,{\mathbf {x}}_2)\sim \mbox{N}(\mu _x,\tau ^2_x)$, with posterior mean μ _x and posterior variance $\tau ^2_x$, computed according to the updating rules (2.13) and (2.14). Using the result (1.21), one can easily verify that the density in the numerator is still a normal distribution with mean equal to the posterior mean μ _x and variance equal to the sum of the posterior variance $\tau ^2_x$ and the population variance σ ² divided by the sample size n, i.e., $\tau ^2_x+\sigma ^2/n$. In the same way, invoking (1.22), the density in the denominator is still a normal distribution with mean equal to the prior mean μ and variance equal to the sum of the prior variance τ ² and the population variance σ ² divided by the sample size n, i.e., τ ² + σ ²∕n.

Example 3.5 (Printed Documents)

Consider the case described above where a forensic document examiner measures the magnetic flux on two uncontested pages 1 and 3 (Biedermann et al., 2016a). The results are x ₁ = (16, 15, 15) and x ₂ = (16, 15, 16). The measurements for the contested page 2 are y = (15, 16, 16). Previous experiments allow one to assign the value 0.24 for the population standard deviation σ. Based on the available knowledge regarding the magnetic flux of toner on printed documents, the prior mean μ and the prior variance τ ² for the unknown quantity of magnetic flux are set equal to 17.5 and 3.92², respectively. This means that values of the magnetic flux smaller than 6 and greater than 29 are considered, a priori, to be extremely unlikely.

The posterior distribution f(θ∣x ₁, x ₂) can be obtained by a single application of Bayes theorem with the full set of available measurements (x ₁, x ₂). The posterior parameters μ _x and $\tau ^2_x$ can be calculated using the function post_distr introduced in Sect. 2.3.1.

The two marginal densities in the numerator and denominator of the BF in (3.10) can be calculated at the sample mean $\bar y$. The exact value of the Bayes factor is given by

This value represents moderate support for the proposition of page substitution, compared to the proposition of no page manipulation.

3.3.2 Normal Model with Both Parameters Unknown

So far, the variance of the distribution of the observations has been assumed to be known, though in many practical situations the mean and the variance are both unknown, and it is necessary to choose a prior distribution for the parameter vector (θ, σ ²). The Bayes factor can be computed as in (1.16):

$$\displaystyle \begin{aligned} \begin{array}{rcl}{} \mbox{BF}& =&\displaystyle \frac{f(\mathbf{y}\mid \mathbf{x},H_1)}{f(\mathbf{y}\mid H_2)}\\ & =&\displaystyle \frac{\int f(\mathbf{y}\mid \theta,\sigma^2)f(\theta,\sigma^2\mid \mathbf{x},H_1)\mbox{d}(\theta,\sigma^2)}{\int f(\mathbf{y}\mid\theta,\sigma^2)f(\theta,\sigma^2\mid H_2)\mbox{d}(\theta,\sigma^2)}. \end{array} \end{aligned} $$

(3.11)

Consider the case where a conjugate prior distribution for (θ, σ ²) of the form

$$\displaystyle \begin{aligned} \begin{array}{rcl}{} f(\theta,\sigma^2)=f(\theta\mid\sigma^2)f(\sigma^2) \end{array} \end{aligned} $$

(3.12)

is chosen. In this distribution, prior beliefs about the population mean θ are calibrated by the scale of measurements of the observations.¹ The conditional distribution f(θ∣σ ²) is taken to be normal, centered at μ with variance σ ²∕n ₀, $(\theta \mid \sigma ^2)\sim \mbox{N}(\mu ,\frac {\sigma ^2}{n_0})$. The parameter n ₀ can be thought of as the prior sample size for the distribution of θ. As pointed out in Sect. 2.3.1, it formalizes the size of the sample from a normal population that provides an equivalent amount of information about θ. The distribution f(σ ²) is taken to be an S times inverse chi-squared distribution with k degrees of freedom, σ ² ∼ S ⋅ χ ⁻²(k). It can be shown that this is equivalent to an inverse gamma distribution with shape parameter α = k∕2 and scale parameter β = S∕2, σ ² ∼IG(α = k∕2, β = S∕2). Alternatively, prior uncertainty about dispersion can be formulated in terms of the precision λ ² = 1∕σ ². The prior distribution of λ ² becomes a gamma distribution with shape parameter α = k∕2 and rate parameter β = S∕2, λ ² ∼Ga(α = k∕2, β = S∕2). For further discussion, see e.g. Bernardo and Smith (2000), Bolstad and Curran (2017) and Robert (2001).

Consider now the posterior distribution of the unknown parameter vector (θ, λ ²) once a vector of observations x = (x ₁, …, x _n) becomes available. It takes the form of a normal–gamma distribution

$$\displaystyle \begin{aligned} \begin{array}{rcl} f(\theta,\lambda^2\mid \mathbf{x},H_1)=\mbox{NG}(\mu_n,n',\alpha_n,\beta_n), \end{array} \end{aligned} $$

with

• $$\displaystyle \begin{aligned} \mu_n=\frac{n\bar x+n_0\mu}{n+n_0}\qquad ; \qquad n'=n+n_0 \end{aligned}$$
• $$\displaystyle \begin{aligned} \alpha_n=\alpha+\frac{n}{2}; \end{aligned}$$
• $$\displaystyle \begin{aligned} \beta_n=\beta +\frac 1 2 \left[(n-1)s^2+\frac{n_0n(\bar x-\mu)^2}{n_0+n}\right], \end{aligned}$$

and $s^2=\frac {1}{n-1}\sum _{i=1}^{n}(x_i-\bar x)^2$.

If uncertainty about the two unknown parameters is modeled by means of the conjugate prior distribution in (3.12), the integrations in (3.11) have an analytical solution and the BF can be obtained straightforwardly.

Denote by ${\mathbf y}=(y_1,\dots ,y_{n_y})$ a vector of measurements made on questioned material and consider the sample mean $\bar y=\sum _{i=1}^{n_y}y_i$. It can be proved that the marginal density $f(\bar y\mid \mathbf {x},H_1)$ in the numerator is a Student t distribution with 2α + n degrees of freedom, centered at μ _n, with spread parameter, denoted s _n, equal to

$$\displaystyle \begin{aligned} \begin{array}{rcl} s_n=\frac{n_y(n+n_0)}{n+n_0+n_y}\left(\alpha + \frac{n}{2}\right)\beta_n^{-1}. \end{array} \end{aligned} $$

This can be denoted as $f_1(\bar y\mid \mu _n,s_n,2\alpha +n)$.

The marginal density f(y∣H ₂) in the denominator is a Student t distribution with k degrees of freedom, centered at μ with spread parameter (precision), denoted s _d, equal to

$$\displaystyle \begin{aligned} \begin{array}{rcl} s_d=\frac{n_0n_y}{n_0+n_y}\alpha\beta^{-1} \end{array} \end{aligned} $$

(Bernardo and Smith, 2000). This can be denoted as $f_2(\bar y\mid \mu ,s_d,2\alpha )$.

The Bayes factor can then be computed as

$$\displaystyle \begin{aligned} \begin{array}{rcl}{} \mbox{BF}=\frac{f_1(\bar y\mid\mu_n,s_n,2\alpha+n)}{f_2(\bar y\mid \mu,s_d,2\alpha)}. \end{array} \end{aligned} $$

(3.13)

Choosing the Parameters of the Normal Prior

The use of a conjugate prior distribution for the mean and the variance of a normal distribution raises the question of how to choose the hyperparameters, as the resulting distribution should suitably reflect available prior knowledge. The prior distribution f(θ∣σ ²) requires one to choose a value for μ, the measure of location, and a value for n ₀. The ratio n ₀∕n characterizes the relative precision of the prior distribution compared to the precision of the observations. If this ratio is very small, the less informative will be the prior distribution, and the closest will be the posterior distribution to that obtained using a non-informative prior distribution. In fact, when n ₀∕n approaches zero, the limiting form of the marginal distribution of the population mean θ is $\mbox{N}(\bar x,\sigma ^2/n)$, which corresponds to the posterior distribution that would be obtained using a non-informative prior distribution (Robert, 2001). For more specific prior beliefs (i.e., concentrated on a limited range of values), a higher value of n ₀ should be chosen.

Regarding the prior distribution of σ ², consider a number of degrees of freedom k = 20 so that the prior mass is distributed rather symmetrically. Suppose also that, based on knowledge available from previous experiments, it is considered that values of σ ² greater or smaller than 0.05 are equally plausible, so $\Pr (\sigma ^2>0.05)=0.5$. The parameter S can be elicited by recalling that σ ²∕S ∼ χ ⁻²(k) and, analogously, S ⋅ λ ² ∼ χ ²(k) so

$$\displaystyle \begin{aligned} \begin{array}{rcl} \Pr\left(\sigma^2>0.05\right)=\Pr\left(S\cdot \lambda^2< S\cdot20\right)=0.5, \end{array} \end{aligned} $$

where S ⋅ 20 is the quantile of order 0.5 of a χ ² distributed random variable with k = 20 degrees of freedom.

Parameter S is then equal to

$$\displaystyle \begin{aligned} \begin{array}{rcl} S=19.3374\times0.05\approx 1. \end{array} \end{aligned} $$

The elicited prior distribution for σ ² is $\text{IG}(\frac {20}{2},\frac {1}{2})$ and is shown in Fig. 3.3.

Fig. 3.3

Inverse Gamma prior distribution $\text{IG}( \frac {20}{2}, \frac {1}{2})$ for σ ² in Example 3.6

Example 3.6 (Printed Documents—Continued)

Consider again Example 3.5 where magnetic flux was measured on uncontested and questioned pages. The population variance σ ² was assumed known and equal to 0.0576. Suppose now that a new measuring device is used and that the number of previous experiments (i.e., measurements) conducted with this device is limited. A conjugate prior distribution as in (3.12) is introduced to model prior uncertainty about θ and σ ².

The prior distribution for θ∣σ ² can be centered at μ = 17.5 as in Example 3.5 with n ₀ = 0.004 reflecting a very weak prior belief with respect to the precision of the observations, θ ∼N(17.5, σ ²∕0.004).

The prior distribution about σ ² has been elicited above, with k = 20 degrees of freedom, and S = 1, $\sigma ^2\sim \text{IG}(\frac {20}{2},\frac {1}{2})$, shown in Fig. 3.3.

Note that the function dinvgamma is available in the package extraDistr (Wolodzko, 2020). Measurements are the same as in Example 3.5.

Let us first consider the marginal density in the numerator of the Bayes factor in (3.13). It is a Student t distribution with 2α + n = k + n = 26 degrees of freedom, centered at μ _n = 15.5 with spread parameter s _n = 20.6724.

The marginal density at the denominator of the Bayes factor in (3.13) is a Student t distribution with 2α = k = 20 degrees of freedom, centered at μ = 17.5 with spread parameter s _d = 0.0799.

The density of a non-central Student t distributed random variable can be calculated using the function dstp, available in the package LaplacesDemon (Hall et al., 2020). The Bayes factor can be obtained as

The Bayes factor represents moderate support for the proposition according to which page two has been printed by the same device as the one used for printing pages one and three, compared to the proposition according to which page two has been printed by a different device.

It is worth emphasizing that the BF is highly sensitive to the choice of the prior (see Sect. 1.11). A sensitivity analysis should therefore be conducted.

3.3.3 Normal Model for Inference of Source

Consider again a case as described in Sect. 3.3.1, involving the analysis of toner on printed documents. Magnetic flux was considered as a feature of interest because it is largely influenced by the settings of the printing device. Suppose now that more than one potential source (i.e., printing device) is available for examination. The issue of interest is which of two machines has been used to print a questioned document (e.g., a contested contract). The propositions of interest can be defined as follows:

H ₁ : :

The questioned document has been printed with machine A.

H ₂ : :

The questioned document has been printed with machine B.

The two potential sources, i.e., machines A and B, are used to print documents under controlled conditions. The measurements made on documents printed by the two devices are denoted {x _p} = (x _pi, p = A, B and i = 1, …, m), with x _pi = (x _pi1, …, x _pin) denoting the vector of n measurements for each analyzed page, i = 1, …, m, from each printer p = A, B. Measurements are assumed to be normally distributed with unknown mean θ _p, p = A, B, and variance σ ². The variance is assumed to be known and equal for the two devices. A conjugate normal prior distribution is taken for the unknown mean θ _p, say $\theta _p\sim \mbox{N}(\mu _p,\tau ^2_p)$, p = A, B.

Measurements on the questioned document are denoted by y = (y ₁, …, y _q), with y _j = (y _j1, …, y _jn) denoting the vector of n measurements from each contested page j = 1, …, q. For cases in which q > 1, it is assumed that all pages have been printed with a single device. The distribution of measurements on the questioned document is also taken to be normal. The sample mean $\bar y=\frac {1}{nq}\sum _{j=1}^q\sum _{k=1}^n y_{jk}$ has a normal distribution with mean θ _p and variance σ ²∕nq, $(\bar Y\mid \theta _p,\sigma ^2/nq)\sim \mbox{N}(\theta _p,\sigma ^2/nq)$.

The Bayes factor can be computed as

$$\displaystyle \begin{aligned} \begin{array}{rcl}{} \mbox{BF}& =&\displaystyle \frac{\int f(\bar{y}\mid\theta_A)f(\theta_A\mid{\mathbf{x}}_A)d\theta_A}{\int f(\bar{y}\mid\theta_B)f(\theta_B\mid{\mathbf{x}}_B)d\theta_B}\\ & =&\displaystyle \frac{f(\bar{y}\mid {\mathbf{x}}_A,H_1)}{f(\bar{y}\mid {\mathbf{x}}_B,H_2)}. \end{array} \end{aligned} $$

(3.14)

The marginal probability density in the numerator can be obtained in closed form. It is a normal distribution with mean equal to the posterior mean μ _A,x and variance equal to the sum of the posterior variance $\tau ^2_{A,x}$ and population variance $\sigma ^2_A/nq$ (where nq is the total number of observations), that is, $f(\bar {y}\mid {\mathbf {x}}_A,H_1)=\mbox{N}(\mu _{A,x},\tau ^2_{A,x}+\sigma ^2/nq)$. In the same way, one can obtain the marginal probability density in the denominator, $f(\bar {y}\mid {\mathbf {x}}_B,H_2)=\mbox{N}(\mu _{B,x},\tau ^2_{B,x}+\sigma ^2/nq)$. As observed in Sect. 3.3.1, the numerator and the denominator of (3.14) can be calculated as the densities of two normally distributed random variables, $\mbox{ N}(\mu _{A,x},\tau ^2_{A,x}+\sigma ^2/nq)$ and $\mbox{N}(\mu _{B,x},\tau ^2_{B,x}+\sigma ^2/nq)$, at the sample mean $\bar y$ of the measurements on the questioned document.

Example 3.7 (Printed Documents)

Consider a type of case and propositions as introduced above, and suppose that there is only one contested page, that is, q = 1. Measurements of the magnetic flux lead to the following results: y = (20, 20, 21) (i.e., n = 3 measurements are taken). Two pages are printed with each printing device. The results are as follows (Biedermann et al., 2016a):

			Printer A					Printer B
Page 1			20	20	19				21	20	21
Page 2			20	21	20				21	22	21

The available data thus are

The population standard deviation σ is taken to be equal to 0.24, as in Example 3.5. We also choose the same prior distribution as used in Example 3.5 to describe uncertainty about the magnetic flux of toner printed by the two printing devices. Thus, μ _A = μ _B = 17.5 and $\tau ^2_A=\tau ^2_B=3.92^2$.

The posterior distributions f(θ _A∣x _A) and f(θ _B∣x _B) can be obtained by a single application of Bayes theorem using the full set of available measurements for each printer. The posterior parameters μ _A,x, μ _B,x, $\tau ^2_{A,x}$ and $\tau ^2_{B,x}$ can be calculated using the function post_distr:

The two marginal densities in the numerator and denominator of the BF in (3.14) can be calculated at the observed value $\bar y$. The BF can thus be computed as the ratio of two marginal densities:

This value represents moderately strong support for the proposition according to which the questioned page been printed using device A, rather than using device B.

Consider a “0 − l _i” loss function as in Table 1.4. The optimal decision is to accept the view according to which the questioned page was printed by the device A (as stated by proposition H ₁), rather than by device B, whenever

$$\displaystyle \begin{aligned} \mbox{BF}>\frac{l_1/l_2}{\pi_1/\pi_2}. \end{aligned}$$

If the odds are evens, and a symmetric loss function is felt to be appropriate, the Bayes decision is to accept the view according to which the questioned document has been printed with machine A (B) whenever the BF is greater (smaller) than 1.

When available information is limited, one may choose a non-informative prior distribution for (θ, σ ²) that can be specified as

$$\displaystyle \begin{aligned} \begin{array}{rcl}{} f(\theta,\sigma^2)=\frac{1}{\sigma^2}. \end{array} \end{aligned} $$

(3.15)

In this case, the marginal distribution in the numerator of the BF is proportional to a Student t distribution with n _A − 1 degrees of freedom, centered at the sample mean $\bar x_A$ with spread parameter s _n equal to

$$\displaystyle \begin{aligned}s_n=\frac{n_Anq}{(n_A+nq)s_A^2},\end{aligned}$$

where $s_A=\frac {1}{n_A-1}\sum _{i=1}^{n_A}(x_A-\bar x_A)^2$, n _A is the total number of observations from device A, and nq is the total number of measurements from the q contested pages (i.e., n measurements for each contested page). This can be denoted as $f_1(\bar y\mid \bar x_A,s_n,n_A-1)$.

Vice versa, the marginal distribution in the denominator of the BF is proportional to a Student t distribution with n _B − 1 degrees of freedom, centered at the sample mean $\bar x_B$ with spread parameter s _d equal to

$$\displaystyle \begin{aligned}s_d=\frac{n_Bnq}{(n_B+nq)s_B^2},\end{aligned}$$

where $s_B=\frac {1}{n_B-1}\sum _{i=1}^{n_B}(x_B-\bar x_B)^2$ and n _B is the total number of observations from device B. This can be denoted as $f_2(\bar y\mid \bar x_B,s_d,n_B-1)$.

The Bayes factor can then be obtained as

$$\displaystyle \begin{aligned} \begin{array}{rcl}{} \mbox{BF}=\frac{f_1(\bar y\mid\bar x_A,s_n,n_A-1)}{f_2(\bar y\mid \bar x_B,s_d,n_B-1)}. \end{array} \end{aligned} $$

(3.16)

Example 3.8 (Printed Documents—Continued)

In Example 3.7, a normal prior distribution has been used for (θ, σ ²). Consider now a non-informative prior distribution as in (3.15). In order to compute the Bayes factor, one must first obtain the spread parameters s _n and s _d under the competing propositions.

Note that in this case the number of contested pages q is set equal to 1. The density of a non-central Student t distributed random variable can be obtained using the function dstp available in the package LaplacesDemon (Hall et al., 2020). The Bayes factor can be obtained as follows:

The Bayes factor represents weak support for the proposition according to which the questioned document has been printed with machine A, rather than with machine B.

More Than Two Propositions

Consider now the case where more than two devices are available. As in Sect. 1.6, the question is how to evaluate measurements made on questioned and known items (i.e., documents), as the BF involves pairwise comparisons. A scaled version of the marginal likelihood may be reported as in (1.27).

Example 3.9 (Printed Documents, More Than Two Propositions)

Recall Example 3.7, and assume that a third printer, machine C, is available for comparative examinations. The propositions of interest are therefore:

H ₁ : :

The questioned document has been printed with machine A.

H ₂ : :

The questioned document has been printed with machine B.

H ₃ : :

The questioned document has been printed with machine C.

Two pages are printed with the additional printing device C. All results, including those from machines A and B, are as follows:

			Printer A					Printer B					Printer C
Page 1			20	20	19			21	20	21			21	20	21
Page 2			20	21	20			21	22	21			20	21	20

Let the prior distribution describing uncertainty about the magnetic flux characterizing machine C be the same as introduced previously, that is μ _C = 17.5 and $\tau ^2_C=3.92^2$. First, the posterior distribution f(θ _C∣x _C) is calculated:

Next, consider the marginal likelihoods of the sample mean that can be obtained as

The scaled version of the marginal likelihoods then is

Recall from Sect. 1.6 that this is equivalent to reporting the posterior probability of competing propositions with equal prior probabilities. Therefore, if $\Pr (H_1)=\Pr (H_2)=\Pr (H_3)=\frac 1 3$, then proposition H ₃ has received the greatest evidential support.

Alternatively, the analyst may also consider the possibility of aggregating propositions H ₁ and H ₂ and consider:

H ₁ : :

The questioned document has been printed with machine C.

$\bar {H}_1:$ :

The questioned document has been printed with machine A or B.

Example 3.10 (Printed Documents, More Than Two Propositions—Continued)

When considering a single proposition H ₁ compared to a composite proposition $\bar {H}_1$ as defined above, the Bayes factor can be obtained as in (1.28), with $\Pr (H_1)=1/3$ and $\Pr (\bar {H}_{1})=2/3$.

3.3.4 Score-Based Bayes Factor

As mentioned previously in Sect. 1.5.2, it may not be possible to specify a probability model for some types of forensic evidence and data. An example was given in Sect. 3.2.3 for discrete data regarding consecutive matching striations, used to quantify the extent of agreement between marks on bullets.

Consider now a case where a saliva trace is collected at the crime scene. The salivary microbiome is analyzed as well as that of traces originating from a known source, Mr. X, with the aim of discriminating between the following competing propositions:

H ₁ : :

The saliva trace comes from Mr. X.

H ₂ : :

The saliva trace comes from the twin brother of Mr. X.

Note that the proposition H ₂ represents an extreme case of relatedness. To investigate this type of case, consider the data collected by Scherz (2021). This longitudinal study involving 30 monozygotic twins has shown the potential of salivary microbiome profiles to discriminate between closely related individuals (Scherz et al., 2021). This may represent an alternative method when standard DNA profiling analyses yield no useful results.

In the study by Scherz (2021), four salivary samples have been collected from each participant. The first at the beginning of the study, and the others after 1, 12, and 13 months. Given the complex composition of microbiota, a distance can be calculated to compare microbiota profiles. One possibility is the Jaccard distance, obtained by dividing the number of amplicon sequence variants (AVSs) shared by the two samples by the number of distinct AVSs in the two compared samples. This measure has shown good discriminatory power. Other distances (e.g., Jensen–Shannon) can be calculated (Scherz, 2021).

The intra-individual variability was studied by comparing all four samples of each individual. The intra-pair variability was evaluated by comparing pairs of samples from related individuals (here: homozygous twins). The inter-individual variability was studied by comparing samples of unrelated individuals (Fig. 3.4).

Fig. 3.4

Jaccard distances for salivary microbiota compositions of pairs of samples from individual persons (intra-individual), pairs of related persons (intra-pair), and pairs of unrelated persons (unrelated) [Source of data: (Scherz et al., 2021)]

Let δ(y, x) denote the distance between the analytical features of questioned material (i.e., a saliva trace of unknown origin) and control material (i.e., a saliva sample from Mr. X). A score-based Bayes factor (sBF) can be defined as follows:

$$\displaystyle \begin{aligned} \begin{array}{rcl}{} \mbox{sBF}=\frac{g(\delta(x,y)\mid H_1)}{g(\delta(x,y)\mid H_2)}. \end{array} \end{aligned} $$

(3.17)

To obtain a value for this sBF, it is necessary to study the probability distribution of the calculated score under the competing propositions. However, the limited number of samples per individual, available for pairwise comparison, might make it difficult to assess the numerator, which is specific for a given person of interest. To address this problem, Davis et al. (2012) propose the use of a database of simulated samples to help with the construction of probability distributions for scores.

In the example studied here, a maximum number of 6 intra-volunteer comparisons are available for each participant. A viable alternative is to perform a so-called common-source comparison ,² and use the limited number of items from all participants, provided that one is willing to assume a generic probability distribution for all individuals in the numerator. In the same way, a generic probability distribution is used at the denominator in all cases where a twin is assumed as the alternative source of the salivary trace (Bozza et al., 2022).

Denote by $\{Z_{ij}^1, \, i=1,\dots ,m_1,\, j=1,\dots ,n_1\}$ the intra-individual distances and by $\{Z_{ij}^2, \, i=1,\dots ,m_2,\, j=1,\dots ,n_2\}$ the intra-pair distances, where m ₁ (m ₂) are the number of distinct individuals (couples of twin brothers) and n ₁ (n ₂) are the number of distances calculated for each individual (couple). A normal distribution is used for both the numerator and denominator to model the within-source variation (i.e., the variation between distances characterizing materials originating from the same individual and from the same couple of twins, respectively), $Z_{ij}^p\sim \mbox{N}(\theta _p,\sigma ^2_p)$, where p = {1, 2}. Different distributions can be used to describe the between-source variation (i.e., the variation between distances characterizing materials originating from different individuals and from different couples of twins, respectively). Here, a normal distribution is retained, $\theta _p\sim \mbox{N}(\mu _p,\tau ^2_p)$. The mean vector between sources μ _p, the within-source variance $\sigma ^2_p$, and the between-source variance $\tau ^2_p$ can be estimated from the background data:

$$\displaystyle \begin{aligned} \begin{array}{rcl} \hat\mu_p& =&\displaystyle \bar z_p=\frac{1}{m_p n_p}\sum_{i=1}^{m_p}\sum_{j=1}^{n_p}z_{ij}^p{} \end{array} \end{aligned} $$

(3.18)

$$\displaystyle \begin{aligned} \begin{array}{rcl} \hat\sigma^2_p& =&\displaystyle \frac{1}{m_p(n_p-1)}\sum_{i=1}^{m_p}\sum_{j=1}^{n_p} (z_{ij}^p-\bar{z}_i)^2{} \end{array} \end{aligned} $$

(3.19)

$$\displaystyle \begin{aligned} \begin{array}{rcl} \hat\tau^2_p& =&\displaystyle \frac{1}{m_p-1}\sum_{i=1}^{m_p}(\bar z_i^p-\bar z_p)^2-\frac{\hat{\sigma}^2_p}{n_p},{} \end{array} \end{aligned} $$

(3.20)

where $\bar z_i^p=\sum _{j=1}^{n_p}z_{ij}$.

Example 3.11 (Saliva Traces)

Consider a case where a saliva trace is recovered at a crime scene and a sample is taken from a person of interest for comparative purposes. The Jaccard distance between the microbiota composition of recovered and control sample is equal to 0.51.

The propositions are H ₁, the compared items come from the same source, and H ₂, the compared items come from different sources (twins). Suppose that the estimated means between sources in (3.18) are 0.454 and 0.769; the estimated within-source variances in (3.19) are 0.0057 and 0.00067; the estimated between-source variances in (3.20) are 0.0028 and 0.0024 (Source of data: Scherz (2021)).

The Bayes factor can then be obtained straightforwardly as in (3.17)

The Bayes factor provides very strong support for the proposition that the saliva traces originate from the same individual rather than from two different individuals (twins).

Note that a higher value of the BF is expected whenever the alternative proposition H ₂ involves unrelated individuals. The inspection of Fig. 3.4 highlights that higher distances are recorded in this type of case.

The between-source variability can also be modeled by a kernel density distribution, as presented in Bozza et al. (2022). See also Sect. 3.4.1.2, where a detailed description of the kernel density approach is given for two-level multivariate data.

3.4 Multivariate Data

Forensic scientists encounter multivariate data in contexts where the examined objects and materials can be described by several variables. Examples are glass fragments that are searched and recovered on the clothing of a person of interest and on a crime scene, or seized materials supposed to contain illicit substances. Such materials may be analyzed and compared on the basis of their chemical compounds as well as their physical characteristics. Multivariate data also arise in other forensic science disciplines, such as handwriting examination. Handwritten characters can, in fact, be described by means of several variables, such as the width, the height, the surface, the orientation of the strokes, or by Fourier descriptors (Marquis et al., 2005). In addition, an emerging topic that forensic document examiners nowadays encounter is handwriting (e.g., signatures) on digital tablets. Such electronic devices provide several static (e.g., length of a signature) and dynamic features (e.g., speed) that can be used as variables to describe signatures (Linden et al., 2018). These developments have led to substantial databases that often present a complex dependence structure, a large number of variables, and multiple sources of variation.

3.4.1 Two-Level Models

Denote by p the number of characteristics (variables) observed on items of a particular evidential type. Suppose that continuous measurements of these variables are available on a random sample of m sources with n items from each source. For handwriting evidence, a source is a single writer, with n characters from each writer and p observed characteristics that pertain to the shape of handwritten characters. For glass evidence, a source is a window, with n replicate measurements from a glass fragment originating from each window and p observed characteristics given by concentrations in elemental composition. The background data can be denoted by z _ij = (z _ij1, …, z _ijp), where i = 1, …, m denotes the number of sources (e.g., windows), j = 1, …, n denotes the number of items for each source (e.g., replicate measurements from a glass fragment), and p is the number of variables.

This data structure suggests a two-level hierarchy, accounting for two sources of variation: the variation between replicate measurements within the same source (the so-called within-source variation) and the variation between sources (the so-called between-source variation).

3.4.1.1 Normal Distribution for the Between-Source Variability

In some applications, data exhibit regularity that can reasonably be described using standard probabilistic models. For example, the within-source variability and the between-source variability may be modeled by a normal distribution. A Bayesian statistical model for the evaluation of trace evidence for two-level normally distributed multivariate data was proposed by Aitken and Lucy (2004) in the context of evaluating the elemental composition of glass fragments. To illustrate this model, denote the mean vector within source i by θ _i. Denote by W the matrix of within-source variances and covariances. The distribution of Z _ij for the within-source variation is taken to be normal, Z _ij ∼N(θ _i, W). For the between-source variation, the mean vector between sources is denoted by μ, and the matrix of between-source variances and covariances by B. The distribution of the θ _i is taken to be normal, θ _i ∼N(μ, B).

Measurements are available on items from an unknown source (recovered material) as well as measurements on items from a known source (control material). The examined items may or may not come from the same source. Competing propositions may be formulated as follows:

H ₁ : :

The recovered and the control item originate from the same source.

H ₂ : :

The recovered and the control item originate from different sources.

Denote the measurements on recovered and control items by, respectively, $\mathbf {y}=({\mathbf {y}}_1,\dots ,{\mathbf {y}}_{n_y})$ and $\mathbf {x}=({\mathbf {x}}_1,\dots ,{\mathbf {x}}_{n_x})$, where y _j = (y _j1, …, y _jp), x _j = (x _j1, …, x _jp), j = 1, …, n _y(x). A Bayes factor can be derived as in (1.15):

$$\displaystyle \begin{aligned} \begin{array}{rcl}{} \mbox{BF}=\frac{f(\mathbf{y},\mathbf{x}\mid H_1)}{f(\mathbf{y},\mathbf{x}\mid H_2)}. \end{array} \end{aligned} $$

(3.21)

The distribution of the measurements on the recovered and control materials is taken to be normal , with vector means θ _y and θ _x, and covariance matrices W _y and W _x. Thus,

$$\displaystyle \begin{aligned} \begin{array}{rcl}{} (Y\mid {\boldsymbol\theta}_y,W_y)\sim\mbox{N}({\boldsymbol\theta}_y,W_y)\qquad ; \qquad (X\mid {\boldsymbol\theta}_x,W_x)\sim\mbox{N}({\boldsymbol\theta}_x,W_x). \end{array} \end{aligned} $$

(3.22)

The Bayes factor is the ratio of two probability densities of the form f(y, x∣H _i) = f _i(y, x∣μ, W, B), i = 1, 2. The probability density in the numerator is given by

$$\displaystyle \begin{aligned} \begin{array}{rcl}{} f_1(\mathbf{y},\mathbf{x}\mid\boldsymbol\mu,W,B)=\int_{\boldsymbol\theta}f(\mathbf{y}\mid\boldsymbol\theta,W)f(\mathbf{x}\mid\boldsymbol\theta,W)f(\boldsymbol\theta\mid\boldsymbol\mu,B)d\boldsymbol\theta, \end{array} \end{aligned} $$

(3.23)

where

$$\displaystyle \begin{aligned} \begin{array}{rcl}{} f(\mathbf{y}\mid \boldsymbol\theta,W)=|2\pi|{}^{-pn_y/2}|W|{}^{-n_y/2}\exp\left[-\frac 1 2 \sum_{j=1}^{n_y} \left({\mathbf{y}}_j-\boldsymbol\theta\right)' W^{-1}\left({\mathbf{y}}_j-\boldsymbol\theta\right)\right], \end{array} \end{aligned} $$

(3.24)

f(x∣θ, W) has the same probabilistic structure as f(y∣θ, W), and

$$\displaystyle \begin{aligned} \begin{array}{rcl}{} f({\boldsymbol\theta\mid\boldsymbol\mu,B})=|2\pi|{}^{-p/2}|B|{}^{-1/2}\exp\left[-\frac 1 2\left(\boldsymbol\theta-\boldsymbol\mu\right)'B^{-1}\left(\boldsymbol\theta-\boldsymbol\mu\right)\right]. \end{array} \end{aligned} $$

(3.25)

In the denominator, where y and x are taken to be independent, the probability density is given by

$$\displaystyle \begin{aligned} \begin{array}{rcl}{} f_2(\mathbf{y},\mathbf{x}\mid\boldsymbol\mu,W,B)& =&\displaystyle f_2(\mathbf{y}\mid\boldsymbol{\theta},W,B)\times f_2(\mathbf{x}\mid\boldsymbol{\theta},W,B)\\& =&\displaystyle \int_{\boldsymbol\theta}f(\mathbf{y}\mid\boldsymbol\theta,W)f(\boldsymbol\theta\mid\boldsymbol\mu,B)d\boldsymbol\theta\int_{\boldsymbol\theta}f(\mathbf{x}\mid\boldsymbol\theta,W)f(\boldsymbol\theta\mid\boldsymbol\mu,B)d\boldsymbol\theta. \end{array} \end{aligned} $$

(3.26)

This is equivalent to the algebraic expression of the Bayes factor in (1.23). In the numerator, under proposition H ₁, the source means θ _y and θ _x are assumed equal, say θ _y = θ _x = θ. In the denominator, under proposition H ₂, the source means θ _y and θ _x are assumed to be different.

The integrals in (3.23) and (3.26) have an analytical solution. A proof is given by Aitken and Lucy (2004). The numerator can be shown to be equal to

$$\displaystyle \begin{aligned} f(\mathbf{y},\mathbf{x}\mid H_1)&= \mid 2\pi W\mid ^{-(n_y+n_x)/2}\mid 2\pi B\mid ^{-1/2}\mid 2\pi\left[(n_y+n_x)W^{-1}+B^{-1}\right]^{-1}\mid^{\frac 1 2} \\ & \quad \times\exp \left\{ -\frac 1 2 \left[ F_1+F_2+\text{tr}\left(S_yW^{-1}\right)+\text{tr}\left(S_xW^{-1}\right)\right]\right\}, \end{aligned} $$

(3.27)

where:

• $F_1=(\bar {\mathbf {w}}-\boldsymbol \mu )'\left (\frac {W}{n_y+n_x}+B\right )^{-1}(\bar {\mathbf {w}}-\boldsymbol \mu )$,

• $F_2=\left (\bar {\mathbf {y}}-\bar {\mathbf {x}}\right )'\left (\frac {W}{n_y}+\frac {W}{n_x}\right )^{-1}\left (\bar {\mathbf {y}}-\bar {\mathbf {x}}\right )$,

• $\bar {\mathbf w}=\frac {1}{n_y+n_x}\left (\sum _{j=1}^{n_y}{\mathbf y}_j+\sum _{j=1}^{n_x}{\mathbf x}_j\right )$, $\bar {\mathbf y}=\frac {1}{n_y}\sum _{j=1}^{n_y}{\mathbf y}_j$ and $\bar {\mathbf x}=\frac {1}{n_x}\sum _{j=1}^{n_x}{\mathbf x}_j$ ,

• $S_y=\sum _{j=1}^{n_y}\left ({\mathbf y}_j-\bar {\mathbf y}\right )\left ({\mathbf y}_j-\bar {\mathbf y}\right )'$, $S_x=\sum _{j=1}^{n_x}\left ({\mathbf x}_j-\bar {\mathbf x}\right )\left ({\mathbf x}_j-\bar {\mathbf x}\right )'$.

Consider the first factor in the denominator, f ₂(y∣θ, W, B). It can be obtained as

$$\displaystyle \begin{aligned} f_2(\mathbf{y}\mid \boldsymbol\mu,W,B)&=\mid 2\pi W\mid^{-n_y/2}\mid 2\pi B\mid^{-1/2} \mid 2\pi (n_yW^{-1}+B^{-1})^{-1}\mid^{1/2} \\ & \quad \!\times\! \exp\left\{-\frac 1 2 \left[(\bar{\mathbf{y}}-\boldsymbol\mu)'(n_{y}^{-1}W\!+\!B)^{-1}(\bar{\mathbf{y}}-\boldsymbol\mu) +\text{tr}\left(S_yW^{-1}\right)\right]\right\}. \end{aligned} $$

(3.28)

The second factor f ₂(x∣θ, W, B) can be obtained analogously as

$$\displaystyle \begin{aligned} f_2(\mathbf{x}\mid \boldsymbol\mu,W,B)&=\mid 2\pi W \mid^{-n_x/2}\mid 2\pi B\mid^{-1/2} \mid 2\pi (n_xW^{-1}+B^{-1})^{-1}\mid^{1/2} \\ & \quad \!\times\! \exp\left\{-\frac 1 2 \left[(\bar{\mathbf{x}}-\boldsymbol\mu)'(n_{x}^{-1}W\!+\!B)^{-1}(\bar{\mathbf{x}}-\boldsymbol\mu) +\text{tr}\left(S_xW^{-1}\right)\right]\right\}. \end{aligned} $$

(3.29)

The Bayes factor in (3.21) then is the ratio between (3.27) and the product between (3.28) and (3.29), respectively. After some manipulation, the BF can be obtained as the ratio between

$$\displaystyle \begin{aligned} \begin{array}{rcl}{} \mid 2\pi\left[(n_y+n_x)W^{-1}+B^{-1}\right]^{-1}\mid^{1/2}\exp\left\{-\frac 1 2 \left(F_1+F_2\right)\right\} \end{array} \end{aligned} $$

(3.30)

and

$$\displaystyle \begin{aligned} & \mid 2\pi B\mid^{-1/2}\mid 2\pi (n_yW^{-1}+B^{-1})^{-1}\mid^{1/2}\mid 2\pi (n_xW^{-1}+B^{-1})^{-1}\mid^{1/2} \\ & \quad \times\exp\left\{-\frac 1 2\left(F_3+F_4\right)\right\}, \end{aligned} $$

(3.31)

where:

• $F_3=(\boldsymbol \mu -\boldsymbol \mu ^*)'\left \{\left (\frac {W}{n_y}+B\right )^{-1}+\left (\frac {W}{n_x}+B\right )^{-1}\right \}(\boldsymbol \mu -\boldsymbol \mu ^*)$,

• $F_4=(\bar {\mathbf y}-\bar {\mathbf x})'\left (\frac {W}{n_y}+\frac {W}{n_x}+2B\right )^{-1}(\bar {\mathbf y}-\bar {\mathbf x})$,

• $\boldsymbol \mu ^*=\left \{\left (\frac {W}{n_y}+B\right )^{-1}+\left (\frac {W}{n_x}+B\right )^{-1}\right \}^{-1}\times \left \{\left (\frac {W}{n_y}+B\right )^{-1}\bar {\mathbf y}+\left (\frac {W}{n_x}+B\right )^{-1}\bar {\mathbf x}\right \}$.

The mean vector between sources μ, the within-source covariance matrix W, and the between-source covariance matrix B can be estimated using the available background data:

$$\displaystyle \begin{aligned} \begin{array}{rcl} \hat{\boldsymbol\mu}& =&\displaystyle \bar{\mathbf{z}}=\frac{1}{mn}\sum_{i=1}^{m}\sum_{j=1}^{n}{\mathbf{z}}_{ij}{}, \end{array} \end{aligned} $$

(3.32)

$$\displaystyle \begin{aligned} \begin{array}{rcl} \hat W& =&\displaystyle \frac{1}{m(n-1)}\sum_{i=1}^{m}\sum_{j=1}^{n}({\mathbf{z}}_{ij}-\bar{\mathbf{z}}_i)({\mathbf{z}}_{ij}-\bar{\mathbf{z}}_i)',{} \end{array} \end{aligned} $$

(3.33)

$$\displaystyle \begin{aligned} \begin{array}{rcl} \hat B& =&\displaystyle \frac{1}{m-1}\sum_{i=1}^{m}(\bar{\mathbf{z}}_{i}-\bar{\mathbf{z}})(\bar{\mathbf{z}}_{i}-\bar{\mathbf{z}})' -\frac{\hat W}{n},{} \end{array} \end{aligned} $$

(3.34)

where $\bar {\mathbf {z}}_i=\frac {1}{n}\sum _{j=1}^{n}{\mathbf {z}}_{ij}$.

Example 3.12 (Glass Evidence)

Consider a case in which two glass fragments are recovered on the jacket of an individual who is suspected to be involved in a crime. Two glass fragments are collected at the crime scene for comparative purposes. The competing propositions are:

H ₁ : :

The recovered and known glass fragments originate from the same source (broken window at the crime scene).

H ₂ : :

The recovered and known glass fragments originate from different sources.

For each fragment, three variables are considered: the logarithmic transformation of the ratios Ca∕K, Ca∕Si, and Ca∕Fe (Aitken and Lucy, 2004). Two replicate measurements are available for each fragment. Measurements on the two recovered fragments are

$$\displaystyle \begin{aligned} {\mathbf{y}}_{1}=\left( \begin{array}{c} 3.77379\\ -0.89063\\ 2.62038 \end{array} \right), \; {\mathbf{y}}_{2}=\left( \begin{array}{c} 3.93937\\ -0.89343\\ 2.63860 \end{array} \right). \end{aligned}$$

Measurements on the two control fragments are

$$\displaystyle \begin{aligned} {\mathbf{x}}_{1}=\left( \begin{array}{c} 3.84396\\ -0.91010\\ 2.65437 \end{array} \right), \; {\mathbf{x}}_{2}=\left( \begin{array}{c} 3.72493\\ -0.89811\\ 2.61933 \end{array} \right). \end{aligned}$$

Consider the database named glass-data.txt. This database is part of the supplementary material of Aitken and Lucy (2004) and contains n = 5 replicate measurements of the elemental concentration of glass fragments from several windows (m = 62). The variables of interest (i.e., the logarithmic transformation of the ratios Ca∕K, Ca∕Si, and Ca∕Fe) are displayed in columns 6, 7 and 8, while the object (window) identifier is in column 9.

Measurements from the recovered fragments, y = (y ₁, y ₂), and measurements from the control fragments, x = (x ₁, x ₂), were selected from the available replicate measurements for the first group (window). The first two replicate measurements were selected to act as recovered data, while the last two replicate measurements were selected to act as control data

Data concerning measurements from the first window were then excluded from the database

The database named pop.back will serve as background data and can be used to estimate the model parameters μ, W and B as in (3.32), (3.33), and (3.34) by means of the function two.level.mv.WB contained in the routines file two_level_functions.r. This file is part of the supplementary materials available on the website of this book (on http://link.springer.com/) and can be run in the R console by inserting the command

The mean vector between sources, the within-source covariance matrix, and the between-source covariance matrix can therefore be obtained as follows:

The Bayes factor can be calculated as the ratio between (3.27) and (3.28) using the function two.level.mvn.BF available in the routines file two_level_functions.r. This function is part of the supplementary materials available on the website of this book (on http://link.springer.com/). First, it is necessary to calculate the sample means $\bar {\mathbf {y}}$ and $\bar {\mathbf {x}}$ and to determine the sample size n _y and n _x

The Bayes factor can be obtained as

This Bayes factor represents moderately strong support for the proposition according to which the recovered and the control fragments originate from the same source, rather than from different sources. This is expected because the compared measurements refer to the same fragment.

3.4.1.2 Non-normal Distribution for the Between-Source Variability

The two-level random effect model presented in the previous section is based on the assumption of normality of the between-source variability. However, in many practical applications, observations or measurements do not exhibit (enough) regularity for standard parametric models to be used. For example, a multivariate normal distribution for the mean vector θ may be difficult to justify. It can be replaced by a kernel density estimate, which is sensitive to multimodality and skewness, and which may provide a better representation of the available data.

Starting from a database {z _ij = (z _ij1, …, z _ij1); i = 1, …, m and j = 1, …, n)}, the estimate of the probability density distribution for the between-source variability can be obtained as follows :

$$\displaystyle \begin{aligned} \begin{array}{rcl}{} f({\boldsymbol\theta}\mid\bar{\mathbf{z}}_1,\dots,\bar{\mathbf{z}}_m,B,h)=\frac 1 m \sum_{i=1}^{m}K({\boldsymbol\theta}\mid\bar{\mathbf{z}}_i,B,h), \end{array} \end{aligned} $$

(3.35)

where the kernel density function $K({\boldsymbol \theta }\mid \bar {\mathbf {z}}_i,B,h)$ is taken to be a multivariate normal distribution centered at the group mean $\bar {\mathbf {z}}_i$, with covariance matrix h ² B. The smoothing parameter h can be estimated as

$$\displaystyle \begin{aligned} \begin{array}{rcl}{} \hat h =\left(\frac{4}{2p+1}\right)^{\frac{1}{p+4}}m^{-1/(p+4)}. \end{array} \end{aligned} $$

(3.36)

See also Silverman (1986) and Scott (1992).

We first write a function hopt that computes the estimate of the smoothing parameter.

Thus, if the number p of variables is set equal to 4 and the number of sources m is set equal to 30, the smoothing parameter h can be estimated as in (3.36)

The BF can be obtained as in (3.21), where a multivariate normal distribution is used for the control and the recovered measurements as in (3.22), and a kernel distribution for the between-source variability, as in (3.35). The numerator and the denominator of the BF, f ₁(y, x∣μ, W, B) and f ₂(y, x∣μ, W, B), can be obtained analytically (Aitken and Lucy, 2004). The BF is the ratio between

$$\displaystyle \begin{aligned} & \mid B\mid ^{1/2}mh^p\mid n_yW^{-1} \\ & \quad +n_xW^{-1}+(h^2B)^{-1}\mid^{-1/2}\exp\left\{-\frac 1 2 F_2\right\}\sum_{i=1}^{m}\exp\left\{-\frac 1 2 F_i\right\} \end{aligned} $$

(3.37)

and

$$\displaystyle \begin{aligned} & \mid n_yW^{-1}+(h^2B)^{-1}\mid^{-1/2}\sum_{i=1}^{m}\exp\left\{-\frac 1 2 F_{yi}\right\} \\ & \quad \times \mid n_xW^{-1}+(h^2B)^{-1}\mid^{-1/2}\sum_{i=1}^{m}\exp\left\{-\frac 1 2 F_{xi}\right\}, \end{aligned} $$

(3.38)

where:

• $F_i=({\mathbf {w}}^*-\bar {\mathbf z}_i)'\left \{\left (n_y W^{-1}+n_x W^{-1}\right )^{-1}+\left (h^2 B\right )\right \}^{-1}({\mathbf {w}}^*-\bar {\mathbf z}_i)$,

• ${\mathbf {w}}^*=\left (n_y W^{-1}+n_x W^{-1}\right )^{-1}\left (n_y W^{-1}\bar {\mathbf y}+n_x W^{-1}\bar {\mathbf x}\right )$,

• $F_{yi}=\left (\bar {\mathbf y}-\bar {\mathbf z}_i\right )'\left (\frac {W}{n_y}+h^2B\right )^{-1}\left (\bar {\mathbf y}-\bar {\mathbf z}_i\right )$,

• $F_{xi}=\left (\bar {\mathbf x}-\bar {\mathbf z}_i\right )'\left (\frac {W}{n_x}+h^2B\right )^{-1}\left (\bar {\mathbf x}-\bar {\mathbf z}_i\right )$.

Example 3.13 (Glass Evidence—Continued)

Consider the case examined in Example 3.12, and suppose a kernel distribution is used to model the between-source variability (Aitken and Lucy, 2004). Start from the same database, glass-data.txt, covering n replicate measurements of p variables for each of m = 62 different sources. The smoothing parameter can be estimated using the function hopt, for p = 3.

First, the group means $\bar {\mathbf z}_i$ must be obtained. They are an output of the function two.level.mv.WB, previously used to estimate the model parameters.

Here we show only the first six rows of the (m × p) matrix, where each row represents the means of the measurements $\bar {\mathbf z}_i=\frac 1 n \sum _{i=1}^{n}{\mathbf z}_{ij}$.

The Bayes factor can then be calculated as the ratio between (3.37) and (3.38) using the function two.level.mvk.BF contained in the routines file two_level_functions. This function is part of the supplementary materials available on the website of this book (on http://link.springer.com/).

The Bayes factor represents moderately strong support for the proposition according to which the recovered and the control fragments originate from the same source, rather than from different sources.

A detailed comparison and discussion of the performance of these two multivariate random effect models can be found in Aitken and Lucy (2004). An alternative approach to the kernel density estimation is presented by Franco-Pedroso et al. (2016), modeling the between-source distribution by means of a Gaussian mixture model.

Note that a third level of variability could be considered. In fact, one may wish to model separately the variability between replicate measurements from a given item originating from a given source (e.g., replicate measurements from a glass fragment originating from a given window) and the variability between different items originating from a given source (e.g., different glass fragments originating from the same window). This aspect will be tackled in Sect. 3.4.4 where three-level models will be introduced.

3.4.1.3 Non-constant Within-Source Variability

The two-level random effect models presented in Sects. 3.4.1.1 and 3.4.1.2 are characterized by the assumption of a constant within-source variability. In other words, it was assumed that every single source has the same intra-variability. While for some type of trace evidence this assumption is acceptable (e.g., for measurements of the elemental composition of glass fragments), a constant within-source variation may be more difficult to justify in other forensic domains. Consider, for example, the case of handwriting on questioned documents where it is largely recognized that intra-variability may vary between writers (Marquis et al., 2006).

Suppose that a handwritten document of unknown source is available for comparative examinations. Handwritten items from a person who is suspected to be the writer are collected and analyzed. Multiple characters are analyzed on the questioned document and on the known writings of the person of interest. The following propositions are defined:

H ₁ ::

The person of interest wrote the questioned document.

H ₂ ::

An unknown person wrote the questioned document.

The distribution of the vector of means within group (source) θ _i is treated as explained in Sect. 3.4.1.1, i.e., (θ _i∣μ, B) ∼N(μ, B) . An inverse Wishart distribution is chosen to model the uncertainty about the within-group covariance matrix,

$$\displaystyle \begin{aligned} \begin{array}{rcl}{} (W_i\mid\varOmega,\nu)\sim W^{-1}(\varOmega,\nu), \end{array} \end{aligned} $$

(3.39)

where Ω is the scale matrix and ν are the degrees of freedom (Bozza et al., 2008). The scale matrix Ω is elicited in a way such that the prior mean of W _i is taken to be equal to the within-group covariance matrix estimated from the available background data as in (3.33), while μ is estimated as in (3.32) and the between-group covariance matrix is estimated as

$$\displaystyle \begin{aligned} \begin{array}{rcl} \hat B& =&\displaystyle \frac{1}{m-1}\sum_{i=1}^{m}n(\bar{\mathbf{z}}_{i}-\bar{\mathbf{z}})(\bar{\mathbf{z}}_{i}-\bar{\mathbf{z}})'. \end{array} \end{aligned} $$

A two-level multivariate random effect model with an inverse Wishart distribution, modeling the uncertainty about the within-source covariance matrix, has also been proposed by Ommen et al. (2017).

First, consider the numerator of the Bayes factor in (3.21). If proposition H ₁ holds, then θ _y = θ _x = θ and W _y = W _x = W, and the marginal likelihood is as follows:

$$\displaystyle \begin{aligned} f(\mathbf{y},\mathbf{x}\mid H_1)&=f_1(\mathbf{y},\mathbf{x}\mid \boldsymbol{\mu},B,\varOmega,\nu) \\ &=\int f(\mathbf{y}\mid\boldsymbol\theta,W)f(\mathbf{x}\mid\boldsymbol\theta,W)f(\boldsymbol\theta\mid\boldsymbol\mu,B)f(W\mid\varOmega,\nu)d(\boldsymbol\theta,W), \end{aligned} $$

(3.40)

where f(θ∣μ, B) is as in (3.25), and

$$\displaystyle \begin{aligned} \begin{array}{rcl} f(W\mid\varOmega,\nu)=\frac{c\mid \varOmega\mid^{\nu-p-1}/2}{\mid W\mid^{\nu/2}}\exp\left\{-\frac 1 2 \mbox{tr}(W^{-1}\varOmega)\right\}, \end{array} \end{aligned} $$

where c is the normalizing constant (e.g., Press, 2005).

If proposition H ₂ holds, then θ _y ≠ θ _x and W _y ≠ W _x, and the marginal likelihood takes the following form:

$$\displaystyle \begin{aligned} \begin{array}{rcl}{} f(\mathbf{y},\mathbf{x}\mid H_2)& =&\displaystyle f_2(\mathbf{y},\mathbf{x}\mid \boldsymbol{\mu},B,\varOmega,\nu)\\ & =&\displaystyle \int f(\mathbf{y}\mid\boldsymbol\theta,W)f(\boldsymbol\theta,W\mid\boldsymbol\mu,B,\varOmega,\nu)d(\boldsymbol\theta,W)\\ & &\displaystyle \times\int f(\mathbf{x}\mid\boldsymbol\theta,W)f(\boldsymbol\theta,W\mid\boldsymbol\mu,B,\varOmega,\nu)d(\boldsymbol\theta,W). \end{array} \end{aligned} $$

(3.41)

The Bayes factor is the ratio between the marginal likelihoods in (3.40) and (3.41). However, these distributions are not available in closed form as the integrals do not have an analytical solution. Several approaches are available to deal with this problem. Chib (1995) estimates the marginal likelihood f(y, x∣H _i) by a direct application of Bayes theorem , since the marginal likelihood can be seen as the normalizing constant of the posterior density f(θ, W∣y, x, H _i). The marginal likelihood can therefore be obtained as

$$\displaystyle \begin{aligned} \begin{array}{rcl}{} f(\mathbf{y},\mathbf{x}\mid H_i)=\frac{f( \mathbf{y},\mathbf{x}\mid \boldsymbol\theta,W)f(\boldsymbol\theta,W\mid H_i)}{f(\boldsymbol\theta,W\mid \mathbf{y},\mathbf{x},H_i)}. \end{array} \end{aligned} $$

(3.42)

While the likelihood function f(y, x∣θ, W) and the prior density f(θ, W∣H _i) can be easily evaluated at any parameter point (θ ^∗, W ^∗), this is not the case for the posterior density f(θ, W∣y, x, H _i), which is not known in closed form. A Gibbs sampling algorithm (Sect. 1.8) can be applied to the set of the complete conditional densities f(θ∣W, y, x, H _i) and f(W∣θ, y, x, H _i), and the posterior density f(θ, W∣y, x, H _i) can be approximated from the output of the Gibbs sampling algorithm as $\hat {f}(\boldsymbol \theta ,W\mid \mathbf {y},\mathbf {x},H_i)$ (Chib, 1995; Bozza et al., 2008; Aitken et al., 2021).

The marginal likelihood in (3.42) can be estimated at a given parameter point (θ ^∗, W ^∗) as

$$\displaystyle \begin{aligned} \begin{array}{rcl}{} \hat f(\mathbf{y},\mathbf{x}\mid H_i)=\frac{f( \mathbf{y},\mathbf{x}\mid \boldsymbol\theta^*,W^*)f(\boldsymbol\theta^*,W^*\mid H_i)}{f(\boldsymbol\theta^*,W^*\mid \mathbf{y},\mathbf{x},H_i)}. \end{array} \end{aligned} $$

The Bayes factor is then calculated as

$$\displaystyle \begin{aligned} \begin{array}{rcl}{} \mbox{BF}=\frac{\hat f(\mathbf{y},\mathbf{x}\mid H_1)}{\hat f(\mathbf{y},\mathbf{x}\mid H_2)}. \end{array} \end{aligned} $$

(3.43)

As mentioned in Sect. 1.8, many other approaches are available, and their efficiency should be studied and compared.

Example 3.14 (Handwriting Evidence)

Consider a hypothetical case involving a handwritten document. Handwritten items from a person of interest are available for comparative examinations. The propositions of interest are therefore:

H ₁ : :

The person of interest wrote the questioned document.

H ₂ : :

An unknown person wrote the questioned document.

Suppose that n ₁ = 8 characters of type a are collected from the questioned document and that n ₂ = 8 characters of the same type are extracted from a document originating from the person of interest, taken for comparative purposes. The contour shape of loops of handwritten characters can be described using a methodology based on Fourier analysis (Marquis et al., 2005, 2006). In brief, the contour shape of each handwritten character loop can be described by means of a set of variables representing the surface and a set of harmonics. Each harmonic corresponds to a specific contribution to the shape and is defined by an amplitude and a phase, the Fourier descriptors .

Consider the database named handwriting.txt available on the book’s website. It contains data on p = 9 variables (i.e., the surface, the amplitude and the phase of the first four harmonics), measured on several characters of type a collected from m = 20 writers. The variables of interest are displayed in columns 2 to 10. Column 1 contains the item (writer) identifier

In the current example, measurements y on the questioned document and measurements x on the control document were randomly selected from the available measurements on characters collected from a given writer (i.e., writer no. 1). Starting from a total number of, say, n available characters, 2 × n ₁ characters have been selected: the first n ₁ characters serve as recovered data, while the remaining serve as control data

Data concerning measurements from the selected writer were then excluded from the database

The database pop.back will serve as background data and can be used to estimate the model parameters as in (Bozza et al., 2008) using the function two.level.mv.WB available in the file two_level_functions.r.

The number of degrees of freedom ν of the inverse Wishart distribution is chosen so as to reduce the variability of this distribution, centered at the within-source covariance matrix estimated as in (3.33).

The Gibbs sampling algorithm is run over 10000 iterations with a burn-in of 1000.

The Bayes factor in (3.43) can then be calculated using the function two.level.mvniw.BF that is part of the supplementary materials. Note also that this routine requires other routines that are available in the packages MCMCpack (Martin et al., 2021) and mvtnorm (Genz et al., 2020).

The Bayes factor represents extremely strong support for the proposition according to which the questioned and the recovered handwritten materials originate from the same source, rather than from different sources. A fully documented open-source package (Gaborini, 2019) has been developed by Gaborini (2021).

Note that it is important to critically examine large BF values, such as the one obtained above. For a discussion about extreme values, see Aitken et al. (2021), Hopwood et al. (2012), and Kaye (2009). Moreover, as underlined in Sect. 1.11, the marginal likelihood is highly sensitive to the prior assessments and so is the BF. In particular, while the overall mean vector, the within- and the between-source covariance matrices are estimated from the available background data, the number of degrees of freedom of the inverse Wishart distribution are chosen so as to reduce the dispersion of the prior. A sensitivity analysis may be performed to assess the sensitivity of the BF to different choices of the degrees of freedom ν in (3.39).

The BF may also be sensitive to the MCMC approximation . Figure 3.5 provides an illustration of BF variability. Results are based on 50 realizations of the BF approximation in (3.43).

Fig. 3.5

Histogram of 50 realizations of the BF approximation in (3.43)

The models discussed here rely on the assumption of independence between sources, focusing on the inherent variability of features. In the case of questioned documents (Sect. 3.4.1.3), this amounts to assume that handwritten material has been produced without any intention of reproducing someone else’s writing style. The possibility of forgery and/or disguise breaks the independence assumption made at denominator. Section 3.4.3 will address this complication.

3.4.2 Assessment of Method Performance

The results of the procedures described in the previous sections may be sensitive to changes in the features of recovered and control materials, the available background information, as well as to choices made during probabilistic modeling and prior elicitation. A sensitivity analysis may be conducted in order to gain a better understanding of the properties of the chosen method. It is fundamental to gain an understanding of how well a method performs: if the recovered and control data originate from the same source, the BF is expected to be greater than 1. Vice versa, if the compared items come from different sources, a BF smaller than 1 is expected.

Several methods exist for the assessment of the performance of the methods for evidence evaluation. Commonly encountered measures in this context are rates of false negatives (i.e., cases in which the Bayes factor is smaller than 1, supporting hypothesis H ₂, when hypothesis H ₁ holds) and false positives (i.e., cases in which the Bayes factor is greater than 1, supporting hypothesis H ₁, when hypothesis H ₂ holds). The rate of false negatives is the number of same-source comparisons with a Bayes factor smaller than 1 divided by the total number of same-source comparisons. The false positive rate is the number of different-source comparisons with a Bayes factor greater than 1 divided by the total number of different-source comparisons. Given a database of cases (e.g., measurements on handwriting characters) for which the source is known, it is possible to study the behavior of the Bayes factor as the data pertaining to control and recovered items change.

Consider again the questioned document case discussed in Sect. 3.4.1.3. There is variability in handwriting, and the reported Bayes factor is sensitive to variability of the shape of handwritten characters. This is not surprising as no one writes the same word exactly the same way twice. Consider measurements of features of handwritten characters of a given writer taken from the available database. These measurements are organized into a (n × p) matrix, where n is the number of available handwritten characters and p represents the number of features (variables). Denote this matrix base. Suppose that, among the n characters, we select a certain number 2 × n ₁ < n of characters, forming a group. Repeating this a certain number of times leads to multiple groups. On each member (character) within a group, p variables are measured. Then we take pairs of groups (i.e., measurements on the group members), taken to represent recovered and control data. Then, the Bayes factor is calculated for each couple. Here, each couple represents a same-source comparison.

Example 3.15 (Two-Level Model for Handwriting—Assessment of Model Performance)

Recall Example 3.14 where a total number of 16 characters have been randomly selected from the available characters collected from a given writer (writer no. 1), extracted from the database handwriting.txt. A Bayes factor equal to 5543330 was obtained. If different sets of characters are extracted, the Bayes factor will be influenced (also) by the within-writer variability.

Suppose now that, for the same writer, ns = 50 distinct groups of characters (each of size 16) are drawn and split into groups of size 8 to act as questioned and control data. The Bayes factor is calculated for each of the 50 groups. Clearly, since the sampled measurements originate from the same writer, we expect Bayes factors greater than 1.

Figure 3.6 shows a histogram of the results for the ns = 50 groups of sampled characters. No false negatives have been observed. The range of the BF values obtained is given here below

Fig. 3.6

Histogram of $\log ( \mbox{BF})$ values for 50 groups, each containing 8 handwritten characters, sampled from a given writer to act as questioned and control datasets

There is also variability between writers, as no two writers write exactly alike. Consider now measurements of features of handwritten characters from a different writer, say writer no. 6, drawn from the same database. These measurements are stored in a matrix denoted base2.

We first estimate the population parameters from the background population where both selected writers have been eliminated.

Next, for each of the two writers, take 50 groups of characters (from base and base2). Each group contains 8 members, on each of which p features are measured. Then, take a group from each writer and form a so-called known different-source pair, and do this multiple times. These draws are taken to represent recovered and control data. Then, the Bayes factor is calculated for each couple.

Figure 3.7 shows a histogram of the results. No false positives have been observed. The range of the BF values obtained is

Fig. 3.7

Histogram of $\log ( \mbox{BF})$ values obtained for 50 groups, each containing 8 handwritten characters, sampled from the same couple of writers to act as questioned and control datasets

The variability of BF values for different samples is not surprising because of handwriting variability. However, this should not be understood as there being a Bayes factor distribution. See, e.g., Morrison (2016), Ommen et al. (2016), and Taroni et al. (2016) for a discussion of issues relating to the reporting of the precision of forensic likelihood ratios.

Over the past decade, several other approaches have been proposed in forensic statistics literature for evaluating the performance of statistical procedures, based on a likelihood ratio or a Bayes factor. These methods provide a rigorous approach to assessing and comparing the performance of evaluative methods prior to using them in casework and forensic reporting. See, in particular, Ramos and Gonzalez-Rodriguez (2013) and Ramos et al. (2021) for a methodology to measure calibration of a set of likelihood ratio values and the concept of Empirical Cross-Entropy for representing performance, illustrated using examples from forensic speech analysis. These concepts are also discussed by Meuwly et al. (2017) who present a guideline for the validation of evaluative methods considering source level propositions. Zadora et al. (2014) present performance assessment for physicochemical data in the context of trace evidence (e.g., glass). For a recent review, see also Chapter 8 of Aitken et al. (2021).

3.4.3 On the Assumption of Independence Under H ₂

The models presented in Sect. 3.4.1 are based on the assumption of independence between the questioned and known materials under hypothesis H ₂. This may be reasonable for certain types of evidence and cases, but less for others. In fact, while a physical feature (e.g., the elementary composition of glass fragments) requires external constraint to be altered, a behavioral or biometric feature such as signature can be modified intentionally.

Consider handwriting as an example. When evaluating results of comparative handwriting examination, the case circumstances may be such that there is no issue of handwriting features being disguised or the result of an attempt to imitate the handwriting of another person. The approach suggested in Sect. 3.4.1.3 may thus be applicable. In turn, in case of alleged forgery of signatures, the (unknown) writer specifically intends to reproduce features of a target signature. The allegation, then, is that a signature is either simulated or disguised, rather than presenting a correspondence or similarity with a genuine signature by mere chance alone (Linden et al., 2021). In such cases, the Bayes factors previously developed in Sect. 3.4.1 cannot be used to approach the question of interest here because the assumption of independence between sources at the denominator cannot be maintained. It follows that one must compute

$$\displaystyle \begin{aligned} \begin{array}{rcl}{} \mbox{BF}=\frac{f(\mathbf{y}\mid \mathbf{x},H_1)}{f(\mathbf{y}\mid \mathbf{x},H_2)}, \end{array} \end{aligned} $$

(3.44)

as f(y∣x, H ₂), following the above argument, does not simplify to f(y∣H ₂) (see also Sect. 1.5.1).

Consider the following competing propositions:

H ₁ : :

The person of interest (POI) produced the questioned signature.

H ₂ : :

An unknown person produced the questioned signature, trying to simulate the POI’s signature.

If proposition H ₂ is true, the forensic document examiner has to deal with a signature written by someone who has knowledge of the POI’s signature.

Consider the two-level model in Sect. 3.4.1.3 where the distribution of the measurements on the recovered and control data is taken to be Normal , with vector means θ _y and θ _x, and covariance matrices W _y and W _x

$$\displaystyle \begin{aligned} \begin{array}{rcl} (Y\mid {\boldsymbol\theta}_y,W_y)\sim\mbox{N}({\boldsymbol\theta}_y,W_y)\qquad ; \qquad (X\mid {\boldsymbol\theta}_x,W_x)\sim\mbox{N}({\boldsymbol\theta}_x,W_x). \end{array} \end{aligned} $$

(3.45)

The probability densities at the numerator and denominator of the BF in (3.44) can be obtained as

$$\displaystyle \begin{aligned} f(\mathbf{y},\mathbf{x}\mid H_i)&=f_i(\mathbf{y},\mathbf{x}\mid {\boldsymbol\mu_i},B_i,\varOmega_i,\nu_i) \\ & =\int f(\mathbf{y}\mid{\boldsymbol\theta},W)f(\boldsymbol\theta,W\mid\mathbf{x},{\boldsymbol\mu}_i,B_i,\varOmega_i,\nu_i), \end{aligned} $$

(3.46)

where (μ _i, B _i) and (Ω _i, ν _i) are the hyperparameters of the prior distributions under the competing propositions (i.e., a normal prior and an inverse Wishart prior distribution ). The Bayes factor can thus be calculated as

$$\displaystyle \begin{aligned} \begin{array}{rcl}{} \mbox{BF}=\frac{f_1(\mathbf{y},\mathbf{x}\mid {\boldsymbol\mu_1},B_1,\varOmega_1,\nu_1)}{f_2(\mathbf{y},\mathbf{x}\mid {\boldsymbol\mu_2},B_2,\varOmega_2,\nu_2)}. \end{array} \end{aligned} $$

(3.47)

Two different background databases are needed to inform model parameters under the competing propositions: a database of genuine signatures (z _ij) and a database of imitated signatures (s _ij). Someone who imitates a signature needs to work outside their writing habits and movement patterns. Thus, simulated signatures do not reflect the same movements and writing features as genuine signatures. Model parameter μ _i can be estimated as in (3.32), and B _i as explained in Sect. 3.4.1.3. The scale matrix Ω _i can be chosen so as to center the prior distribution at the within-group covariance matrix W _i that can be estimated as in (3.33).

The probability densities in (3.46) are not available in closed form but can be estimated from the output of a MCMC algorithm following, for example, the ideas described in Sect. 3.4.1.3. A Gibbs sampling algorithm is implemented here. The routine is different from that developed in Sect. 3.4.1.3 because it calculates the BF in (3.47). In this formula, no assumption of independence is made at the denominator, and two different databases are used.

Example 3.16 (Digitally Captured Signatures)

Consider a case involving a questioned signature on a contract signed on a digital tablet. The person of interest denies having signed the contract. Among the multiple features that are captured by the digital tablet, the average speed and writing time are considered here. See Linden et al. (2021) for a detailed description of the experimental conditions. Measurements on the questioned signature are y = (4639, 380.42), while measurements on the control signature are x = (4460, 323.4787). Note that the first value is the average speed and the second is the writing time.

Model parameters under hypothesis H ₁ (i.e., the mean vector μ ₁, the within-group covariance matrix W ₁, and the between-group covariance matrix B ₁) are estimated from an available database of genuine signatures (z _ij) and are given here below.

The trace matrix of the inverse Wishart distribution is then obtained as

In the same way, model parameters under hypothesis H ₂ are estimated from an available database of simulated signatures (s _ij) and are given here below.

A Gibbs sampling algorithm is run over 10000 iterations, with a burn-in of 1000.

The Bayes factor in (3.44) can then be calculated using the function two.level.mvniw2.BF (see supplementary materials).

The BF represents very strong support for the proposition according to which the questioned signature originates from the person of interest rather than from an unknown person who attempted to imitate the target signature.

3.4.4 Three-Level Models

So far, two-level models have been considered, taking into account the within-source and the between-source variability. However, it is not uncommon to encounter situations in which the hierarchical ordering shows an additional level of variability, e.g., in relation to measurement error.

Denote again by p the number of variables observed on items of a given evidential type. Suppose that continuous measurements of these variables are available on a random sample from m sources with s items for each source and n replicate measurements on each of the N = ms items. The background data can be denoted by z _ikj = (z _ikj1, …, z _ikjp)′, where i = 1, …, m denotes the number of sources (e.g., windows, writers), k = 1, …, s denotes the number of items for each source (e.g., glass fragments, handwritten characters), and j = 1, …, n denotes the number of replicate measurements for each item.

A Bayesian statistical model for the evaluation of evidence for three-level normally distributed multivariate data was proposed by Aitken et al. (2006), focusing on the elemental composition of glass fragments. Denote the mean vector within item k in group i as θ _ik and the covariance matrix of replicate measurements as W. For the variability of replicate measurements, the distribution of Z _ikj is taken to be normal, Z _ikj ∼N(θ _ik, W).

Denote by μ _i the mean vector within group i and by V the within-group covariance matrix. The distribution of θ _ik for the within-group variability is taken to be normal, θ _ik ∼N(μ _i, B).

Denote by ϕ the mean vector between groups. Let U denote the between-group covariance matrix. For the between-group variability, the distribution of the μ _i is taken to be normal, μ _i ∼N(ϕ, V ).

Consider the case described in Sect. 3.4.1, where measurements are available on n _y items from an unknown origin as well as measurements on n _x items from a known origin. These two groups of items may or may not come from the same source. Competing propositions may be formulated as follows:

H ₁ : :

The recovered and the control items originate from the same source.

H ₂ : :

The recovered and the control items originate from different sources.

There are n ₁ replicate measurements available on each of the recovered n _y items. Denote the measurement vector by y, where the vector components are denoted by y _kj (for k = 1, …, n _y and j = 1, …, n ₁) and y _kj = (y _kj1, …, y _kjp)′. For each of the n _x control items, n ₂ replicate measurements are available. Denote the measurement vector by x, where the vector components are denoted (x _kj, k = 1, …, n _x and j = 1, …, n ₂) and x _kj = (x _kj1, …, x _kjp)′.

The Bayes factor is the ratio of two probability densities of the form f(y, x∣H _i) = f _i(y, x∣ϕ, W, B, V ), i = 1, 2. The probability density in the numerator is given by

$$\displaystyle \begin{aligned} &f_1(\mathbf{y},\mathbf{x}\mid{\boldsymbol\phi},W,B,V)\\ & \quad =\int\int f(\mathbf{y}\mid\boldsymbol\theta,W)f(\mathbf{x}\mid\boldsymbol\theta,W)f(\boldsymbol\theta\mid\boldsymbol\mu,B)f(\boldsymbol\mu\mid \boldsymbol\phi,V)d\boldsymbol\mu d\boldsymbol\theta, \end{aligned} $$

(3.48)

where all probability densities are multivariate normal.

In the denominator, the probability density is given by

$$\displaystyle \begin{aligned} f_2(\mathbf{y},\mathbf{x}\mid{\boldsymbol\phi},W,B.V)&=\int\int f(\mathbf{y}\mid\boldsymbol\theta,W)f(\boldsymbol\theta\mid\boldsymbol\mu,B)f(\boldsymbol\mu\mid \boldsymbol\phi,V)d\boldsymbol\mu d\boldsymbol\theta \\ & \quad \times \int\int f(\mathbf{x}\mid\boldsymbol\theta,W)f(\boldsymbol\theta\mid\boldsymbol\mu,B)f(\boldsymbol\mu\mid \boldsymbol\phi,V)d\boldsymbol\mu d\boldsymbol\theta, \end{aligned} $$

(3.49)

where all probability densities are multivariate normal.

As shown by Aitken et al. (2006), the value of the evidence is the ratio of

$$\displaystyle \begin{aligned} \begin{array}{rcl}{} & &\displaystyle \mid B+V\mid^{1/2}\mid[(n_yn_1+n_xn_2)W^{-1}\\ & &\displaystyle \quad +(B+V)^{-1}]\mid^{-1/2}\exp\left\{-\frac 1 2 (F_1+F_2)\right\} \end{array} \end{aligned} $$

(3.50)

to

$$\displaystyle \begin{aligned} &\mid(n_yn_1W^{-1}+(B+V)^{-1})\mid^{-1/2}\mid n_xn_2W^{-1}+(B+V)^{-1}\mid^{-1/2}\\ & \quad \times\exp\left\{-\frac 1 2 \left(F_3+F_4\right)\right\},\end{aligned} $$

(3.51)

where:

• $F_1=(\bar {\mathbf {y}}-\bar {\mathbf {x}})'\left (\frac {n_yn_1n_xn_2W^{-1}}{n_yn_1+n_xn_2}\right )(\bar {\mathbf {y}}-\bar {\mathbf {x}})$,

• $F_2=(\bar {\mathbf {w}}-{\boldsymbol \phi })'\left ((n_yn_1+n_xn_2)^{-1}W+B+V\right )^{-1}(\bar {\mathbf {w}}-{\boldsymbol \phi })$,

• $F_3=(\bar {\mathbf {y}}-{\boldsymbol \phi })'\left [(n_yn_1)^{-1}W+B+V\right ]^{-1}(\bar {\mathbf {y}}-{\boldsymbol \phi })$,

• $F_4=(\bar {\mathbf {x}}-{\boldsymbol \phi })'\left [(n_xn_2)^{-1}W+B+V\right ]^{-1}(\bar {\mathbf {x}}-{\boldsymbol \phi })$,

and $\bar {\mathbf {w}}=\frac {n_yn_1\bar {\mathbf {y}}+n_xn_2\bar {\mathbf {x}}}{n_yn_1+n_xn_2}$.

The overall mean ϕ, the measurement error covariance matrix W, the within-group covariance matrix B, and the between-group covariance matrix V can be estimated using the available background data:

$$\displaystyle \begin{aligned} \begin{array}{rcl}{} \hat{\boldsymbol\phi}& =&\displaystyle \frac 1 m \frac 1 s \frac 1 n \sum_{i=1}^{m}\sum_{k=1}^{s}\sum_{j=1}^{n}{\mathbf{z}}_{ikj}, \end{array} \end{aligned} $$

(3.52)

$$\displaystyle \begin{aligned} \begin{array}{rcl} {} \hat{W}& =&\displaystyle \frac{1}{ms(n-1)}\sum_{i=1}^{m}\sum_{k=1}^{s}\sum_{j=1}^{n}({\mathbf{z}}_{ikj}-\bar{\mathbf{z}}_{ik.})({\mathbf{z}}_{ikj}-\bar{\mathbf{z}}_{ik.})', \end{array} \end{aligned} $$

(3.53)

$$\displaystyle \begin{aligned} \begin{array}{rcl} {} \hat{B}& =&\displaystyle \frac{1}{m(s-1)}\sum_{i=1}^{m}\sum_{k=1}^{s}(\bar{\mathbf{z}}_{ik.}-\bar{\mathbf{z}}_{i..})(\bar{\mathbf{z}}_{ik.}-\bar{\mathbf{z}}_{i..})' -\frac{\hat W}{n}, \end{array} \end{aligned} $$

(3.54)

$$\displaystyle \begin{aligned} \begin{array}{rcl} {} \hat{V}& =&\displaystyle \frac{1}{m-1}\sum_{i=1}^{m}(\bar{\mathbf{z}}_{i..}-\bar{\mathbf{z}}_{...})(\bar{\mathbf{z}}_{i..}-\bar{\mathbf{z}}_{...})'-\frac{\hat B}{s}-\frac{\hat W}{sn}, \end{array} \end{aligned} $$

(3.55)

where $\bar {\mathbf {z}}_{ik.}=\frac 1 n \sum _{j=1}^{n}{\mathbf {z}}_{ikj}$, $\bar {\mathbf {z}}_{i..}=\frac 1 s \sum _{k=1}^{s}{\mathbf {z}}_{ik.}$ and $\bar {\mathbf {z}}_{i...}=\frac 1 m \sum _{i=1}^{m} \bar {\mathbf {z}}_{i..}$.

Example 3.17 (Glass Evidence—Continued)

Consider again the case described in Example 3.12 where two glass fragments are recovered on the jacket of an individual who is suspected to be involved in a crime. Two glass fragments are collected at the crime scene for comparative purposes. The competing propositions are:

H ₁ : :

The recovered and known glass fragments originate from the same source (e.g., a broken window).

H ₂ : :

The recovered and known glass fragments originate from different sources.

A database named glass-database.txt is available as part of the supplementary material of Zadora et al. (2014). It contains measurements of the elemental concentration of glass fragments from several windows (m = 200). For each source, there are s = 12 fragments with n = 3 replicate measurements. For each fragment, five variables are considered: the logarithmic transformation of the ratios Na∕O, Mg∕O, Al∕O, Si∕O, Ca∕O. The variables of interest are displayed in columns 3, 4, 5, 6, and 8, while the object (window) identifier is in column 1. The fragment identifier is in column 2.

Three replicate measurements are available for each fragment. Using the notation introduced above

Measurements for the recovered fragments, y, and measurements for the control fragments, x, were selected from the available data for the first and second group (window) and the first two items (fragments) from these windows. Therefore, a BF smaller than 1 is expected.

Next, the means of measurements $\bar {\mathbf {y}}$, $\bar {\mathbf {x}}$, and $\bar {\mathbf {w}}$ are obtained.

Data concerning measurements from the first two windows were then excluded from the database

The database named pop.back will serve as background data. It can be used to estimate the model parameters ϕ, W, B, and V as in (3.52), (3.53), (3.54) and (3.55) by means of the function three.level.mv.WBV contained in the routines file three_level_functions.r. This file is part of the supplementary materials available on the book’s website and can be run in the R console with the command

The overall mean, the measurement error covariance matrix, the within-source covariance matrix, and the between-source covariance matrix can be estimated as follows:

The Bayes factor can be calculated as the ratio between (3.50) and (3.51) using the function three.level.mvn.BF available in the routines file three_level_functions.r. This function is part of the supplementary materials available on the book’s website.

The Bayes factor represents extremely strong support for the proposition according to which the recovered and the control fragments originate from different sources, rather than from the same source.

Note that the above development does not take into account the topic of variable selection. See Aitken et al. (2006) for a proposal for dimensionality reduction based on a probabilistic structure, determined by a graphical model obtained from a scaled inverse covariance matrix.

3.5 Summary of R Functions

The R functions outlined below have been used in this chapter.

Functions Available in the Base Package

• colMeans: Forms column means for numeric arrays (or data frames)

• d <name of distribution>, p <name of distribution> (e.g., dpois, pnorm): Calculate the density and the cumulative probability for many parametric distributions.

• More details can be found in the Help menu, helpstart.

Functions Available in Other Packages

• dinvgamma in package extraDistr: calculates the density of an inverse gamma distribution.

• dstp in package LaplacesDemon: calculates the density of a non-central Student t distribution.

Functions Developed in the Chapter

• hopt: Calculates the estimates $\hat h$ of the smoothing parameter h.

• Usage: hopt(p,m).

• Arguments: p, the number of variables: m, the number of sources.

• Output: A scalar value.

• poisg: Computes the density of a Poisson–gamma distribution Pg(α, β, 1) at x.

• Usage: poisg(a,b,x).

• Arguments: a, the shape parameter α; b, the rate parameter β; x, a scalar value x.

• Output: A scalar value.

• post_distr: Computes the posterior distribution $\mbox{N}(\mu _x,\tau ^2_x)$ of a normal mean θ, with X ∼N(θ, σ ²) and θ ∼N(μ, τ ²).

• Usage: post_distr(sigma,n,barx,pm,pv).

• Arguments: sigma, the variance σ ² of the observations; n, the number of observations; barx, the sample mean $\bar x$ of the observations; pm, the mean μ of the prior distribution N(μ, τ ²); pv, the variance τ ² of the prior distribution N(μ, τ ²).

• Output: A vector of values, the first is the posterior mean μ _x, the second is the posterior variance $\tau ^2_x$.

• two.level.mv.WB: Computes the estimate of the overall mean μ, the group means $\bar {\mathbf {z}}_i$, the within-group covariance matrix W, and the between-group covariance matrix B for the two-level model in Sect. 3.4.1.

• Usage: two.level.mv.WB(population, variables, grouping. variable,nc=FALSE).

• Arguments: population, a data frame with N rows and k columns for measurements on m sources with n items for each source; variables, a vector containing the column indices of the variables to be used; grouping.variable, a scalar specifying the variable that is to be used as the grouping factor. By default (nc = FALSE), the between-group covariance matrix is estimated as in Sect. 3.4.1.1. If nc = TRUE, the between-group covariance matrix is estimated as in Sect. 3.4.1.3.

• Output: The group means $\bar {\mathbf z}_i$, the estimated overall mean $\hat {\boldsymbol \mu }$, the estimated within-group covariance matrix $\hat W$, the estimated between-group covariance matrix $\hat B$.

• two.level.mvn.BF: Computes the BF for a two-level random effect model where both the within-source variability and the between-source variability are normally distributed, and the within-source covariance matrix is constant between sources.

• Usage: two.level.mvn.BF(W,B,mu,xbar,ybar,nx,ny).

• Arguments: W, the within-source covariance matrix; B, the between-source covariance matrix; mu, the mean vector between sources; xbar, the vector of means for the control item; ybar, the vector of means for the recovered item; nx, the number of measurements for the control material; ny, the number of measurements for the recovered material.

• Output: A scalar value.

• two.level.mvk.BF: Computes the BF for a two-level random effect model where the within-source variability is normally distributed, the normal distribution for the between-source variability is replaced by a kernel density distribution, and the within-source covariance matrix is constant between sources.

• Usage: two.level.mvk.BF(xbar,ybar,nx,ny,W,B,group.means,h).

• Arguments: xbar, the vector of means for the control item; ybar, the vector of means for the recovered item; nx, the number of measurements for the control material; ny, the number of measurements for the recovered material; W, the within-source covariance matrix; B, the between-source covariance matrix; group.means, a ( m ×  p) matrix, where each row represents the vector of means $\bar {\mathbf z}_i=\frac {1}{n}\sum _{j=1}^{n}{\mathbf z}_{ij}$; h, the smoothing parameter.

• Output: A scalar value.

• two.level.mvniw.BF: Computes the BF for a two-level random effect model where both the within-source variability and the between-source variability are normally distributed, and the uncertainty about the within-source covariance matrix is modeled by an inverse Wishart distribution.

• Usage: two.level.mvniw.BF(quest,ref,O,B,mu,nw,p,n.iter, burn.in).

• Arguments: quest, a (n × p) matrix containing measurements on the questioned material; ref, a (n × p) matrix containing measurements on the control material; O, the trace matrix of the inverse Wishart distribution; B, the between-source covariance matrix; mu, the mean vector between sources; nw, the number of degrees of freedom of the inverse Wishart distribution; p, the number of variables; n.iter, the number of iterations of the Gibbs sampling algorithm; burn.in, the number of discarded iterations.

• Output: A scalar value.

• two.level.mvniw2.BF: Computes the BF for a two-level random effect model where both the within-source variability and the between-source variability are normally distributed, the uncertainty about the within-source covariance matrix is modeled by an inverse Wishart distribution with no assumption of independence between questioned and known materials at the denominator (i.e., under H ₂).

• Usage: two.level.mvniw2.BF(quest,ref,Og,Bg,mug,Os,Bs,mus, nu,p,n.iter,burn.in).

• Arguments: quest, a (n × p) matrix containing measurements on the questioned material; ref, a (n × p) matrix containing measurements on the control material; Og, the trace matrix of the inverse Wishart distribution from the database of genuine (handwritten) material; Bg, the between-source covariance matrix from the database of genuine (handwritten) material; mug, the mean vector between sources from the database of genuine (handwritten) material; Os, the trace matrix of the inverse Wishart distribution from the database of simulated (handwritten) material; Bs, the between-source covariance matrix from the database of simulated (handwritten) material; mus, the mean vector between sources from the database of simulated (handwritten) material; nw, the number of degrees of freedom of the inverse Wishart distribution; p, the number of variables; n.iter, the number of iterations of the Gibbs sampling algorithm; burn.in, the number of discarded iterations.

• Output: A scalar value.

• three.level.mv.WBV: Computes the estimate of the overall mean ϕ, the measurement error covariance matrix W, the within-group covariance matrix B, and the between-group covariance matrix V for the three-level model presented in Sect. 3.4.4.

• Usage: three.level.mv.WBV(population,variables,grouping. item,grouping.fragment).

• Arguments: population, a data frame with msn rows and k columns collecting measurements on m sources with s items for each source and n replicate measurements for each item; variables, a vector containing the column indices of the variables to be used; grouping.item, a scalar specifying the variable that is to be used as the grouping item; grouping.fragment, a scalar specifying the variable that is to be used for the grouping fragment.

• Output: The estimated overall mean $\hat {\boldsymbol \phi }$, the estimated measurement error covariance matrix $\hat W$, the estimated within-group covariance matrix $\hat B$, the estimated between-group covariance matrix $\hat V$.

• three.level.mvn.BF: Computes the BF for a three-level random effect model where the variation at all three levels is normally distributed.

• Usage: three.level.mvn.BF(bary,barx,barw,ny,nx,n1,n2,psi, W,B,V).

• Arguments: bary, the mean vector of measurements on recovered items; barx, the mean vector of measurements on control items; barw, the mean vector of measurements; ny, the number of recovered items; nx, the number of control items; n1, the number of replicate measurements on each of the recovered items; n2, the number of replicate measurements on each of the control items; psi, the overall mean vector; W, the replicate measurements covariance matrix; B, the within-group covariance matrix; V, the between-source covariance matrix.

• Output: A scalar value.

Published with the support of the Swiss National Science Foundation (Grant no. 10BP12_208532/1).