# Eliciting Dirichlet and Gaussian copula prior distributions for multinomial models

- 2.7k Downloads
- 5 Citations

## Abstract

In this paper, we propose novel methods of quantifying expert opinion about prior distributions for multinomial models. Two different multivariate priors are elicited using median and quartile assessments of the multinomial probabilities. First, we start by eliciting a univariate beta distribution for the probability of each category. Then we elicit the hyperparameters of the Dirichlet distribution, as a tractable conjugate prior, from those of the univariate betas through various forms of reconciliation using least-squares techniques. However, a multivariate copula function will give a more flexible correlation structure between multinomial parameters if it is used as their multivariate prior distribution. So, second, we use beta marginal distributions to construct a Gaussian copula as a multivariate normal distribution function that binds these marginals and expresses the dependence structure between them. The proposed method elicits a positive-definite correlation matrix of this Gaussian copula. The two proposed methods are designed to be used through interactive graphical software written in Java.

## Keywords

Dirichlet distribution Elicitation method Gaussian copula elicitation Interactive graphical software Multinomial model Prior distribution## 1 Introduction

Bayesian statistical methods provide a formal mechanism for taking into account prior knowledge. In many circumstances, prior knowledge is based on historical data that are only recorded in the form of the personal experience of experts. Then expert opinion must be quantified as a prior distribution if the information is to be used. This can be done through an elicitation process which is defined as asking the expert for information that is then used as a basis for encoding the parameters of a subjective prior distribution. The method of quantifying opinion must be designed for the sampling model of interest and the form of prior distribution that is to be used.

One important sampling model that has attracted limited attention in the elicitation context is the multinomial model. This model is used in many scientific and industrial applications. For example, it is frequently applied to voting behavior in political science (e.g. Chamberlin and Featherston 1986), to the compositions of rocks in geology (e.g. Weltje 2002) and to patterns of consumer selection preferences in microeconomics (e.g. Manski 2007). The model is so common that good elicitation methods for quantifying opinion about its parameters are clearly needed. For example, in their recent applied research, Fu et al. (2012) and Conn et al. (2013) stated that an elicited informative prior for the multinomial model would be a possible alternative.

Here we are concerned with the most common case of the multinomial model, where each observation has the same probability of falling into any specified category and observations are independent of each other. Then observations can be modelled to follow a multinomial distribution with, say, probability \(p_i\) that an observation falls in the *i*th category. To quantify an expert’s opinions about the \(p_i\), we assume some distributional form to represent her opinion and uncertainty. Instead of directly asking the expert to assess the parameters of the distribution, we ask her to make assessments that determine appropriate values for the parameters of that distribution, which are referred to as its hyperparameters.

The Dirichlet distribution, which is the conjugate prior distribution for the parameters of a multinomial distribution, is one natural choice for representing an expert’s opinion. A number of elicitation methods that follow this approach have been proposed in the literature. For example, Bunn (1978) encoded a Dirichlet prior from assessed beta marginal distributions through hypothetical future samples. Chaloner and Duncan (1987) based their elicitation method on assessing a sample size and modal values of the Dirichlet variates. Dorp and Mazzuchi (2004) introduced a numerical algorithm for encoding the Dirichlet parameters from a number of quantile assessments that is equal to the number of parameters. Recently, two elicitation methods have been proposed for the Dirichlet distribution by reconciling the assessed parameters of its univariate beta conditional distributions (Elfadaly and Garthwaite 2013) and of its univariate marginal beta distributions (Zapata-Vázquez et al. 2014).

The first method proposed in the current paper elicits the hyperparameters of the Dirichlet distribution from those of its marginal beta distributions through forms of reconciliation that use least-squares techniques. The usage of least-squares techniques for reconciliation has been discussed in Albert et al. (2012). Our proposed method differs from those of Bunn (1978) and Chaloner and Duncan (1987) as we require only medians and quartiles to be assessed. We assess more quartiles than Dorp and Mazzuchi (2004) and then reconcile them to encode hyperparameters. Our method encodes a Dirichlet distribution using its beta marginal distributions rather than its conditional distributions as in Elfadaly and Garthwaite (2013). Our proposed reconciliation methods are different from those that have been independently developed in Zapata-Vázquez et al. (2014) for reconciling beta marginal distributions into a Dirichlet distribution for a set of proportions.

However, while the standard Dirichlet distribution offers tractability and mathematical simplicity, it has been criticized as too inflexible to represent a broad range of prior information about the parameters of multinomial models (Aitchison 1986; O’Hagan and Forster 2004). Specifically, the Dirichlet variates are always negatively correlated and the Dirichlet distribution contains the same number of parameters as the number of categories. With *k* categories, \(k-1\) parameters are required to describe the proportion in each category, leaving only one parameter to describe their dependence structure. This may sometimes be insufficient to give a reasonable representation of an expert’s opinions.

In a Dirichlet prior distribution, the dependence structure that is imposed by only one parameter is a pattern of negative correlations between the multinomial probabilities of all categories. This negative correlation pattern is not appropriate in many practical situations. For example, if the categories represent the political parties in some country, with the corresponding multinomial probabilities reflecting the voting behavior in the next elections, a high probability of voting for a specific radical party might be accompanied by rather high probabilities of voting for other radical parties. A suitable prior in this case should imply a positive correlation pattern between parties of the same political views.

In the case of multinomial models, a particular difficulty is to elicit assessments that satisfy all the constraints of mathematical coherence. For example, the probabilities of each category must be non-negative and sum to one, which we refer to as the unit-sum constraint. Other constraints are implicit and less intuitive. We believe that a well-designed elicitation method should lead to coherent assessments without the expert having to be conscious of coherence constraints. This is not straightforward, especially with dependent variables in multivariate distributions. The expert has no built-in prior distribution to assess directly. Instead, she must be asked to quantify her opinions through questions she can both comprehend and answer. We believe it is important that the expert should be able to focus solely on her opinions when giving assessments.

To date, perhaps due to the difficulty discussed above, elicitation methods for multinomial sampling have mostly been proposed for modelling opinion by a Dirichlet distribution. An exception is the work by Elfadaly and Garthwaite (2013), who model opinion by a Connor–Mosimann distribution. However, this form of generalized Dirichlet distribution provides a structure that is only slightly richer than the standard Dirichlet distribution. Methods that yield a more flexible distribution than the Dirichlet distribution are long overdue. The present paper focuses on elicitation methods that yield a much more flexible distribution than the Dirichlet prior distribution.

Motivated by the deficiencies of the standard Dirichlet distribution, several authors have constructed new families of distributions for proportions that allow more general types of dependence structures. These distributions include, for example, the generalized Dirichlet distribution (Connor and Mosimann 1969), the multivariate normal distribution of log contrasts (Leonard 1975), and the multivariate logistic normal distribution (Aitchison 1986; Elfadaly and Garthwaite 2016).

A possible general multivariate distribution, that can serve as a prior distribution for multinomial models, is constructed through using a multivariate copula function. A copula is a multivariate function that represents a joint multivariate cumulative distribution function (CDF) in terms of one-dimensional marginal CDFs. Hence, it joins marginal distributions into a multivariate distribution that has those marginals. The theoretical foundation of copula functions was introduced in Sklar (1959), where he gave Sklar’s Theorem which states that any joint distribution can be written in a copula form. The marginal distributions can thus be chosen independently from the dependence structure that is represented by the copula function. For an introduction to copulas, see for example Joe (1997), Frees and Valdez (1998) or Nelsen (1999).

*C*. It has to be mentioned here that the arguments of the inversion copula function in (1) are the quantile functions \(F^{-1} _{i}[.]\ (i=1, \ldots , k)\). It is therefore appealing to exploit the inversion copula in our elicitation method in which we ask the expert to assess only medians and quartiles. Experimental work in the literature suggests that the assessment of quantile functions is often the best way to elicit information from experts. See for example Garthwaite et al. (2005) and the references therein. Specifically, we believe that medians and quartiles are easier for an expert to assess than other quantile functions as they can be obtained by the first two steps of equally likely subdivisions using the bisection method (Garthwaite et al. 2005).

*t*-distribution (Demarta and McNeil 2005), or even as a Dirichlet distribution (Lewandowski 2008). Most commonly though, it is selected as a multivariate normal distribution with a joint CDF, \(F_{(1,\ldots ,k)}={\varPhi }_{k,\mathbf{R }}\), and standard normal marginal CDFs, \(F_{i}={\varPhi }\). This gives a Gaussian copula (Clemen and Reilly 1999) that is parameterized by the correlation matrix \(\mathbf{R }\) of a multivariate normal distribution. To elicit \(\mathbf{R }\), Clemen and Reilly (1999) suggest that a pairwise rank-order correlation between each \(X_{i}\) and \(X_{j}\), such as Spearman’s \(\rho _{i,j}\) or Kendall’s \(\tau _{i,j}\), should be assessed. Then properties of the multivariate normal distribution are used to transform them into the product-moment Pearson correlation \(r_{i,j}\) as follows:

Clemen and Reilly (1999) favour eliciting rank-order correlations, not product-moment Pearson correlation, on the grounds that the latter cannot necessarily be transformed through the function \({\varPhi }^{-1} [G_{i} (.)]\). In contrast, rank-order correlations are transformation respecting regardless of the choice of the marginal CDF \(G_{i}(.)\), i.e. they are invariant under strictly monotone increasing transformations of the form \({\varPhi }^{-1} [G_{i} (.)]\), for any \(G_{i}(.)\). To elicit these correlations, Clemen and Reilly (1999) mention three methods that can be used either separately or together. However, none of the methods is guaranteed to yield a positive-definite correlation matrix. On the other hand, the elicitation method of Kadane et al. (1980) has been designed to encode a positive-definite covariance matrix of a multivariate t-distribution as a conjugate prior for the hyperparameters of a normal multiple linear regression model. Their method can be useful in a variety of multivariate elicitation problems that require the assessment of positive-definite matrices (Garthwaite et al. 2005, 2013). These clearly include the problem addressed here, so that we modify their method to form a Gaussian copula.

Interestingly, no other elicitation method for Gaussian copulas seems to encode a positive-definite correlation matrix using quartile assessments. To fill this gap, the second proposed method in this paper elicits a Gaussian copula function with beta marginal distributions as a prior distribution for multinomial models. Our approach simultaneously overcomes two problems of the method of Clemen and Reilly (1999). First, we transform the assessed conditional quartiles of \(X_{i}\) and \(X_{j}\), through \({\varPhi }^{-1} [G_{i} (.)]\), which enables product-moment correlations to be computed on the normal scale without the need of rank-order correlations. Second, the conditional quartiles are assessed according to the structural elicitation procedure of Kadane et al. (1980), which guarantees that the elicited correlation matrix is positive-definite.

The paper is organized as follows. In Sect. 2, we discuss our choice of distribution, the Dirichlet prior and the Gaussian copula prior, for representing expert uncertainty relevant to proportions of a categorical variable. We show how this can be used to specify a prior for the probabilities of a multinomial distribution. In Sect. 3 we describe the assessment tasks that an expert performs, and in Sect. 4 we derive hyperparameters of both forms of prior distributions from the elicited assessments. Sections 3 and 4 also describe an interactive graphical interface that has been developed to implement the two proposed elicitation methods. In Sect. 5, these methods and the implementing software are used by an environmental ecologist to quantify his opinion about the future proportions of different phenotypes of a specific snail species in the UK. Concluding comments are given in Sect. 6.

## 2 Two multivariate prior distributions for a multinomial model

### 2.1 The multinomial likelihood

*k*categories,

*n*trials and a vector of probabilities \(\mathbf{p }=(p_{1}, \ldots , p_{k})\), so that

### 2.2 The Dirichlet prior distribution

*marginal*beta distributions.

### 2.3 Constructing a Gaussian copula prior distribution

Rather than assuming that the marginal beta distributions in (5) stem from a Dirichlet distribution, we would like to allow a more flexible dependence structure via another joint distribution, with the aim of better representing the expert’s opinion. A flexible tool for this task is given by the copula function, as discussed in Sect. 1. This allows us to choose the marginal distributions independently from the dependence structure between them.

*k*-variate normal distribution with zero means, unit variances, and a correlation matrix \(\mathbf{R }\) that reflects the desired dependence structure. The function \({\varPhi }\) is the marginal standard univariate normal CDF, and \(G_{i}\ (i=1,\ldots ,k)\) are any chosen marginal CDFs. The marginal mean and variance of each \(x_{i}\) are expressed through its marginal CDF, \(G_{i} (x_{i})\), hence, no other parameters are required for \({\varPhi }_{k,\mathbf{R }}\) beyond the correlation matrix \(\mathbf{R }\). To be used as a prior distribution, the density function of the Gaussian copula is needed. Fortunately, since \({\varPhi }_{k,\mathbf{R }}\) and \({\varPhi }\) are differentiable, the Gaussian copula density function is easily obtained by differentiating (7) with respect to \(x_{i}\ (i=1, \ldots , k)\); see for example Clemen and Reilly (1999). This gives

*k*.

To construct a Gaussian copula function in the case of a multinomial model, it might seem natural to assume that each marginal distribution of \(p_{i}\ (i=1,\ldots ,k)\) is a beta distribution, as in (5), and try to construct a Gaussian copula function for the multivariate distribution of \(\mathbf{p }\). However, applying the normalizing transformations \(Y_{i}={\varPhi }^{-1} [G_{i} (p_{i})]\) directly on the multinomial probabilities \(p_{i}\), where \(G_{i} (.)\) is the assessed beta CDF, would raise three problems. First, encoding marginal beta distributions directly on the elements of \(\mathbf{p }\) might result in mathematically incoherent marginal distributions with mean values that do not sum to one. Second, the inverse normalizing transformations \(p_{i}=G_{i}^{-1}[{\varPhi } (Y_{i})]\), while preserving the marginal domain of each probability, \(0\le p_i \le 1\), do not necessarily fulfil the unit-sum constraint \(\sum _{i=1}^{k} p_{i} =1\). Third, under the unit-sum constraint on \(\mathbf{p }\), the normally distributed vector \(\mathbf{y }_{k}'=(Y_{1}, \ldots , Y_{k})\) contains a redundant variable and hence leads to a singular multivariate normal distribution.

Using the Gaussian copula function, the dependence structure of the multivariate distribution of \(Z_i\), and consequently of \(p_i\), will have high flexibility rather than the limited dependence structure imposed by the Dirichlet distribution. In the underlying multinomial model, each multinomial parameter \(p_i\) represents the probability that an observation falls in the *i*th category. Hence, \(Z_i\ (i=1, \ldots , k-1)\) can be defined as the probability that an observation falls in category *i*, given that the first \(i-1\) categories have been eliminated and the available categories are only the last \(k-i+1\) categories.

Connor and Mosimann (1969) argued that if \(Z_i\ (i=1, \ldots , k-1)\) are mutually independent and each \(Z_{i}\sim \text {beta}(\alpha _{i}^{\#}, \beta _{i}^{\#})\), then \(\mathbf{p }\) has a generalized Dirichlet distribution with parameters \((\alpha _{1}^{\#}, \ldots , \alpha _{k-1}^{\#}, \beta _{1}^{\#}, \ldots , \beta _{k-1}^{\#})\). Thus the generalized Dirichlet distribution can be regarded as a special case of the Gaussian copula in which \(Z_1, \ldots , Z_{k-1}\) are independent. If, moreover, \(\beta _{i}^{\#}=\alpha _{i+1}^{\#} + \beta _{i+1}^{\#}\), for \(i=1, \ldots , k-2\), then \(\mathbf{p }\) has a standard Dirichlet distribution with parameters \((\alpha _{1}^{\#}, \ldots , \alpha _{k-1}^{\#}, \beta _{k-1}^{\#})\), so the standard Dirichlet distribution is also a special case of the Gaussian copula prior. This confirms that the Gaussian copula function imposes a prior distribution on \(\mathbf{p }\) with a dependence structure that is more flexible than that of the standard and generalized Dirichlet distributions.

## 3 Assessment tasks

With both the Dirichlet and Gaussian copula prior distributions, marginal distributions are each of the beta type. In both cases, the first set of assessments determines the hyperparameters of the marginal beta distributions. For the standard Dirichlet distribution, as detailed in Sect. 3.1 the expert is asked to assess marginal medians and quartiles of the probability of each category. These quantiles are the median and quartiles of the expert’s subjective probability distribution for each \(p_{i}\); the distance between the quartiles reflects her uncertainty about the true value of \(p_{i}\). The marginal median and quartile assessments are obtained in Sect. 3.1.1. They are used to encode the marginal beta distribution of each \(p_i\ (i=1, \ldots , k)\). In the second part of the elicitation process, detailed in Sect. 3.2, we obtain the required assessments to construct a Gaussian copula prior distribution. Specifically, in Sect. 3.2.1, we obtain the marginal median and quartile assessments that are required to encode the marginal beta distributions of \(Z_i\ (i=1, \ldots , k-1)\). Further sets of conditional medians and quartiles are obtained in Sects. 3.2.2 and 3.2.3; they will be used to encode the correlation matrix \(\mathbf{R }\) of the Gaussian copula prior.

To ensure that the expert understands the request to assess her median estimate of a proportion, she is told: “You should consider it equally likely that the true proportion is above the median or below the median. For example, suppose you assess the median as 0.6, you should think it equally likely that the proportion will be above 0.6 as that it will be below 0.6. Hence, if you were asked ‘Would I rather bet on the proportion being above 0.6 or below 0.6?’, you should find it hard to decide.” For the lower (upper) quartile assessment, the expert is told: “The lower (upper) quartile divides the probability below (above) the median in half. Suppose your assessed median is 0.3 and your assessed lower quartile is 0.2. Then your probability that the proportion is below 0.2 should equal your probability that it is between 0.2 and 0.3.”

### 3.1 Assessments required for the standard Dirichlet distribution

#### 3.1.1 Obtaining marginal assessments for \(\mathbf{p }\)

For each category in turn, the expert is asked to assess a lower quartile, a median and an upper quartile for the probability \(p_{i}\), \(i=1,\ldots ,k\), say \({\overline{L}}_{i}\), \({\overline{m}}_{i}\) and \({\overline{U}}_{i}\), respectively. In the practical evolutional example detailed in Sect. 5, the expert assessed his median, lower and upper quartiles for the future proportion of each of the 6 different phenotypes of the most common snail species in the UK, see Table 1. Here we adopt a marginal approach, so that the categories are interchangeable and their order does not matter. In Sect. 4.1, the marginal medians and quartiles assessed here are used to encode the parameters \(\alpha _{i}\) and \(\beta _{i}\) of each marginal beta distribution of \(p_i\) for the standard Dirichlet distribution.

### 3.2 Assessments required for the Gaussian copula distribution

#### 3.2.1 Obtaining marginal assessments for \(\mathbf{z }_{k-1}\)

For category \(i\ (i=2, \ldots , k-1)\), the expert is asked to assume that the first \(i-1\) categories have been eliminated from the analysis and that she should only consider the remaining \(k-i+1\) categories, namely categories \(i,i+1, \ldots , k\). That is, the expert should assume that the observation does not fall in one of the first \(i-1\) categories, but which of the remaining category it falls in is unknown. Given this situation, the expert is asked to assess the median of the probability of each of the categories \(i,i+1, \ldots , k\). Only the assessment for the first of these categories is used and we denote it \(m_{i,0}^{\#}\). The other assessments are elicited to improve internal consistency; assessments of means should sum to one so the assessments of the medians should be close to one.

In fact, these assessments represent the marginal unconditional medians of each \(Z_i=p_i/(1-\sum _{j=1}^{i-1}p_j)\), for \(i=2, \ldots , k-1\). After these medians have been assessed, the expert assesses the marginal unconditional lower and upper quartile of each \(Z_i\). We denote these assessments as \(L_{i,0}^{\#}\) and \(U_{i,0}^{\#}\ (i=2, \ldots ,k-1)\). Clearly, since \(Z_1=p_1\), we have \(L_{1,0}^{\#}={\overline{L}}_{1}\), \(m_{1,0}^{\#}={\overline{m}}_{1}\) and \(U_{1,0}^{\#}={\overline{U}}_{1}\) and these are assessed as in Sect. 3.1.1. Moreover, by definition, \(m_{k,0}^{\#}=1\), since \(Z_k=1\).

If a Gaussian copula prior distribution is to be encoded, we define \(m_{1,0}^{*}\) as the median of \(p_1\), and \(m_{i,0}^{*}\) as the median of \((p_i | p_1=m_{1,0}^{*}, \ldots , p_{i-1}=m_{i-1,0}^{*})\) for \(i=2, \ldots , k\). But we do not directly assess \(m_{i,0}^{*}\ (i=1, \ldots , k)\). Instead, we compute their values from the corresponding marginal medians, \(m_{i,0}^{\#}\), of \(Z_i\ (i=1, \ldots , k-1)\) assessed above. This can be done by utilizing Lemma 1 (below) to obtain \(m_{i,0}^{*}\) in terms of \(m_{i,0}^{\#}\). The former will be used as the conditioning values in Sects. 3.2.2 and 3.2.3.

### **Lemma 1**

See the Appendix for a proof of this lemma.

Assessments obtained in this section are used in Sect. 4.1 to encode the hyperparameters \(\alpha _{i}^{\#}\) and \(\beta _{i}^{\#}\) of the marginal beta distribution of \(Z_i\ (i=1, \ldots , k-1)\) for the Gaussian copula prior distribution.

#### 3.2.2 Assessing conditional quartiles

After finishing the marginal assessment tasks in Sect. 3.2.1, the expert is asked to give conditional quartile assessments for specified categories under the assumption that the initial assessments are the correct values for the other categories. Specifically, the expert is asked to assume that \(p_{1}\) equals its initially assessed median value, i.e. \(p_{1}=m_{1,0}^{*}\), and then gives a lower quartile \(L_{2}^{*}\) and an upper quartile \(U_{2}^{*}\) for \(p_{2}\). For each remaining \(p_{j}\), \(j=3,\ldots , k-1\), the expert assesses the two quartiles \(L_{j}^{*}\) and \(U_{j}^{*}\) conditional on \(p_{1}=m_{1,0}^{*}\), \(\ldots \), \(p_{j-1}=m_{j-1,0}^{*}\).

For the last category, the lower (upper) quartile \(L_{k}^{*}\) (\(U_{k}^{*}\)) of \(p_{k}\) is automatically shown to the expert once she assesses the upper (lower) quartile \(U_{k-1}^{*}\) (\(L_{k-1}^{*}\)) of \(p_{k-1}\). The two quartiles \(L_{k}^{*}\) and \(U_{k}^{*}\) are shown to the expert as a guide to help her choose \(L_{k-1}^{*}\) and \(U_{k-1}^{*}\). When \(p_{1}=m_{1,0}^{*}, \ldots , p_{k-2}=m_{k-2,0}^{*}\), from the unit-sum constraint it follows that \(L_{k-1}^{*}+U_{k}^{*}=U_{k-1}^{*}+L_{k}^{*}=1-m_{1,0}^{*}- \cdots -m_{k-2,0}^{*}\).

In practice, these conditional assessments constitute an easy task for the experts to perform. They normally think of successive category assessments in a conditional way given their already assessed values for the preceding categories. For conditional assessments, the order of the categories is important. We recommend ordering them from the most probable category to the least probable, unless there is a natural order to the categories. This ordering leads to assessment tasks that involve less skewed distributions. See Elfadaly and Garthwaite (2013) for more detail and examples on this recommended category ordering.

#### 3.2.3 Assessing conditional medians

In each successive step *i*, for \(i=2, \ldots , k-2\), the expert is asked to suppose that the true value of \(p_{i}\) is \(m_{i,i}^{*}=m_{i,0}^{*}+\eta _{i}^{*}\) (while her initially calculated median of \(p_{i}\) had been \(m_{i,0}^{*}\)) and that the true values of \(p_{1}\), \(\ldots \), \(p_{i-1}\) are equal to their initially calculated medians, \(m_{1,0}^{*}, \ldots , m_{i-1,0}^{*}\), respectively. Given this information, the expert is asked to revise her initial median values \(m_{j,0}^{*}\) of \(p_{j}\), and the new assessments are denoted by \(m_{j,i}^{*}\ (j=i+1,\ldots , k)\). Again, as explained in the case of \(i=1\), these assessments are also sequentially conditional assessments for \(i=2, \ldots , k-2\), i.e. \(m_{j,i}^{*}\) is assessed under the condition that \(p_{1}=m_{1,0}^{*}, \ldots , p_{i-1}=m_{i-1,0}^{*}, p_i=m_{i,i}^{*}, p_{i+1}=m_{i+1,i}^{*},\ldots , p_{j-1}=m_{j-1,i}^{*}\), for all \(j=i+1, \ldots ,k\).

The assumed conditioning true value \(m_{i,i}^{*}\) should be (a) reasonably different from the initial value \(m_{i,0}^{*}\), so that the expert revises the previously calculated assessments at the other categories by a measurable change, but (b) not too different from \(m_{i,0}^{*}\), so that it is still a plausible value according to the expert’s belief. Moreover, as will be shown later, for mathematical coherence the arbitrary values \(\eta _{i}^{*}\)s have to be chosen such that \(\eta _{i}^{*}\ne 0\ (i=1,\ldots ,k-2)\). To fulfil these constraints, the lower quartile of each category is taken as the value for \(m_{i,i}^{*}\). Hence, we put \(m_{1,1}^{*}=L_{1,0}^{*}\) and \(m_{i,i}^{*}=L_{i}^{*}\), so \(\eta _{1}^{*}=L_{1,0}^{*}-m_{1,0}^{*}\) and \(\eta _{i}^{*}=L_{i}^{*}-m_{i,0}^{*}\ (i=2,\ldots , k-2)\).

The expert uses the software to make her assessments on an interactive graph. An example is illustrated in Fig. 1, where the categories represent 6 phenotypes of snails: yellow-banded/ yellow-unbanded/ pink-banded/ pink-unbanded/ brown-banded/ brown-unbanded. The conditioning set of median values are the three right hand side (red) bars at the first three categories. The expert is asked to assess how her new median values, shown as the three right hand side (one blue and two orange) bars at the last three categories, will change based on the new conditioning set. The initially calculated medians of all categories are shown as the left hand side (grey) bars.

For mathematical coherence, the expert’s assessments must fulfill the first statement of Lemma 2 below. The second statement of Lemma 2 gives the conditional medians of \(Z_j\), say \(m_{j,i}^{\#}\), in terms of the corresponding conditional medians of \(p_{j}\), \(m_{j,i}^{*}\ (j=i+1, \ldots , k)\). The former are used in Sect. 4.3 to estimate the correlation matrix \(\mathbf{R }\) of the Gaussian copula prior distribution.

### **Lemma 2**

- 1.
\( \sum _{j=1}^{i} m_{j,0}^{*}+\eta _{i}^{*}+\sum _{j=i+1}^{k} m_{j,i}^{*}=1\),

- 2.
\(m_{j,i}^{\#}=m_{j,i}^{*}\big /\big (1-\sum _{r=1}^{i} m_{r,0}^{*}-\eta _{i}^{*}-\sum _{r=i+1}^{j-1} m_{r,i}^{*}\big )\), for \(j=i+1, \ldots , k\).

See the Appendix for a proof of this lemma.

By scaling their values, the software normalizes the values of \(m_{i+1,i}^{*}\), \(\ldots \), \(m_{k,i}^{*}\) so that they satisfy Lemma 2. Specifically, the normalized value of \(m_{j,i}^{*}\) is \([(1-\sum _{r=1}^{i} m_{r,0}^{*}-\eta _{i}^{*})/\sum _{r=i+1}^{k} m_{r,i}^{*}] m_{j,i}^{*}\), for \(j=i+1,\ldots , k\). The expert is told that the conditional medians must satisfy a mathematical equation and that a good representation of her assessments has been calculated. The software displays the normalized conditional medians as the thinner (yellow) bars in Fig. 1. The expert has the option of changing these medians until she feels that the suggested normalized set \(m_{i+1,i}^{*}\), \(\ldots \), \(m_{k,i}^{*}\) gives an acceptable representation of her opinion. In Fig. 1, the expert’s conditional assessments are the three right-most (blue and orange) bars for the last three categories. These are reasonably coherent, so the suggested values in the thinner (yellow) bars are almost equal to these assessments.

The current assessment task stops at step \(k-2\), as we do not ask for any conditional assessments for the last remaining category \(p_{k}\). Since the condition of summing to one should always be fulfilled, conditioning on specific values of all \(p_{1},\ldots ,p_{k-1}\) gives a fixed value for \(p_{k}\).

## 4 Encoding the hyperparameters

### 4.1 Encoding the hyperparameters of the beta marginal distributions

To encode a standard Dirichlet prior distribution, the expert has already assessed a lower quartile, \({\overline{L}}_{i}\), a median, \({\overline{m}}_{i}\), and an upper quartile, \({\overline{U}}_{i}\), for \(p_{i}\ (i=1,\ldots ,k)\), as discussed in Sect. 3.1.1. But, to encode a Gaussian copula prior distribution, the expert has already assessed a lower quartile, \(L_{i,0}^{\#}\), a median, \(m_{i,0}^{\#}\), and an upper quartile, \(U_{i,0}^{\#}\), for \(Z_{i}\ (i=1,\ldots ,k-1)\), as discussed in Sect. 3.2.1. The quartile assessments \({\overline{L}}_{i}\), \({\overline{m}}_{i}\) and \({\overline{U}}_{i}\) are used here to encode the hyperparameters \(\alpha _i\) and \(\beta _i\) of the marginal beta distribution of each \(p_{i}\ (i=1,\ldots ,k)\). Similarly, \(L_{i,0}^{\#}\), \(m_{i,0}^{\#}\) and \(U_{i,0}^{\#}\) are used to encode the hyperparameters \(\alpha _{i}^{\#}\) and \(\beta _{i}^{\#}\) of the marginal beta distribution of each \(Z_{i}\ (i=1,\ldots ,k-1)\). The same method of encoding beta hyperparameters is used in both cases, so we denote a general set of quartile assessments by \({\tilde{L}}_{i,0}\), \({\tilde{m}}_{i,0}\) and \({\tilde{U}}_{i,0}\) and show how we use them to encode the beta hyperparameters denoted by \({\tilde{\alpha }}_i\) and \({\tilde{\beta }}_i\).

*q*. The least-squares optimization enables fast and accurate computation. See, for example, Albert et al. (2012) and the references therein.

### 4.2 Encoding the Dirichlet hyperparameters

After beta marginal distributions have been assessed for each category’s probability \(p_i\) (c.f. Sect. 4.1), the beta parameters are then adjusted to estimate the Dirichlet hyperparameter vector as follows. First, note that the system of equations in (6) does not guarantee a consistent solution for \(\mathbf{a }\). This is because the expert will seldom be fully consistent and give marginal assessments that are exactly mathematically coherent. From (6), each marginal step of the elicitation process provide two estimates for \(a_{i}\) and \(N_{i}\), namely, for \(i=1,\ldots ,k\), we get \(a_{i}=\alpha _{i}\) and \(N_{i} = \alpha _{i}+\beta _{i}=\sum _{j =1}^{k} a_{j}\). Moreover, the estimated hyperparameters must fulfill the unit-sum constraint on the expected values of the probabilities. Specifically, if the expected values are denoted by \(\mu _{i}\) (where \(\mu _{i}=a_{i}/N_{i}\), for \(i=1,\ldots , k\)) they must satisfy \(\sum _{i =1}^{k} \mu _{i}=1\). If the assessments are not coherent and do not satisfy these constraints, which is usually the case, initial assessments will need to be reconciled. Even if the expert were very consistent and gave reasonably coherent assessments, reconciliation would be needed for correcting small rounding errors.

*N*into mathematically coherent estimates \(\mu _{i}^{*}, N^{*}\), respectively.

- (I)
Normalize \(\mu _{i}\)’s into \(\mu _{i}^{*}\)’s that add up to one, i.e. put \(\mu _{i}^{*}=\mu _{i}/\sum _{j =1}^{k} \mu _{j}\), for \(i=1, \ldots , k\).

- (II)Minimize the sum of squares of differences between \(\mu _{i}^{*}\) and \(\mu _{i}\), subject to the constraints \(0<\mu _{i}^{*}<1\) and \(\sum _{i =1}^{k} \mu _{i}^{*}=1\). So, the \(\mu _{i}^{*}\) are taken as the values that minimizewhere \(\lambda \) is a Lagrange multiplier.$$\begin{aligned} S_{1}=\sum _{i =1}^{k} (\mu _{i}^{*}-\mu _{i})^{2}+\lambda \left( \sum _{i =1}^{k} \mu _{i}^{*}-1\right) , \end{aligned}$$
- (III)The precision of each \(p_{i}\), i.e. the inverse of its variance, can be used as a weight to reflect the expert’s confidence about each of her assessments (Lindley et al. 1979). These weights are used in a weighted least-squares procedure. Hence we determine the \(\mu _{i}^{*}\)s that minimizewhere \(w_{i}=\left[ \alpha _{i}\beta _{i}/(\alpha _{i}+\beta _{i}+1)(\alpha _{i}+\beta _{i})^{2}\right] ^{-1}\), for \(i=1, \ldots , k\).$$\begin{aligned} S_{2}=\sum _{i =1}^{k} w_{i}\big (\mu _{i}^{*}-\mu _{i}\big )^{2}+\lambda \left( \sum _{i =1}^{k} \mu _{i}^{*}-1\right) , \end{aligned}$$

The software elicits three hyperparameter vectors of the Dirichlet distribution, one vector for each of the above options. Each vector is then used to compute the corresponding pairs of marginal beta parameters as given in (6). Three quartiles of each beta marginal distribution are computed numerically for each Dirichlet hyperparameter vector. The three sets of quartiles are then displayed to the expert and she is asked to select the set of quartiles that best represents her opinion. See Fig. 2: the middle three button groups at the bottom of the screen offer the options that the expert can choose for her quantile assessments. (The other buttons are for navigation to the previous screen or the next screen.) The vector with the selected set of quartiles will be taken as the final elicited hyperparameter vector of the Dirichlet prior.

The expert is still able, however, to modify any or all of the selected set of quartiles, in which case beta parameters are computed again and the final Dirichlet hyperparameter vector is computed through averaging, as in option (I) above.

### 4.3 Encoding the Gaussian copula hyperparameters

The normalizing one-to-one functions in (12) are used to transform the assessed conditional quartiles of \(\mathbf{z }_{k-1}\) into conditional quartiles of \(\mathbf{y }_{k-1}\), yielding conditional expectations, variances and covariances of the multivariate normal variables. To form a positive-definite correlation matrix \(\mathbf{R }\), let \(\mathbf{y }_{i}=(Y_{1}, \ldots , Y_{i})\) and \(\mathbf{R }_{i}=\text {Var}(\mathbf{y }_{i})\), for \(i=1,\ldots , k-1\), where \(\mathbf{R }_{1}=\text {Var}(Y_{1})=1\) and the final matrix \(\mathbf{R }=\mathbf{R }_{k-1}\). Supposing that \(\mathbf{R }_{i-1}\) has been estimated as a positive-definite matrix, an encoding method, based on that of Kadane et al. (1980), is used to form a positive-definite correlation matrix \(\mathbf{R }_{i}\ (i=2, \ldots , k-1)\). The mathematical details of the method are shown in the Appendix. Since \(\mathbf{R }_{1}=1>0\), by mathematical induction the full correlation matrix \(\mathbf{R }=\mathbf{R }_{k-1}\) is guaranteed to be positive-definite.

We have to note that, according to this method of elicitation, the variances on the main diagonal of \(\mathbf{R }\), say \(r_{i,i}\), \(i=1, \ldots , k-1\), cannot be guaranteed to each equal one, except for the first element \(r_{1,1}\). It is easy, however, to transform \(\mathbf{R }\) into \(\mathbf{R }^{*}= \mathbf{A }\mathbf{R }\mathbf{A }\), where \(\mathbf{A }\) is a diagonal matrix with \(a_{1,1}=1\) and \(a_{i,i}=1/\displaystyle \sqrt{r_{i,i}}\), for \(i=2, \ldots ,k-1\). Then \(\mathbf{R }^{*}\) can be used as the correlation matrix in the Gaussian copula function and satisfies both the unit variances and positive-definiteness requirements. The unit variances in the correlation matrix \(\mathbf{R }^{*}\) guarantee that each marginal distribution \(G_{i}(Z_{i})\) is still a beta distribution with the same hyperparameters \(\alpha _{i}^{\#}\) and \(\beta _{i}^{\#}\) that were encoded, for \(i=1, \ldots , k-1\), in Sect. 4.1.

### 4.4 Evaluating the encoded priors

A good elicitation method encodes a prior distribution that captures the expert’s knowledge and beliefs faithfully and as closely as possible. Thus, a reasonable way to compare the performance of the two encoded prior distributions is to check which of them produces values that are closer to the expert’s original assessments. The aim of the Gaussian copula prior is to capture the correlation structure better than the Dirichlet prior, so the conditional medians computed from each encoded prior distribution can be compared to the expert’s original assessments of the conditional medians to check whether this aim has been achieved. We implement this approach at the end of the evolution example in Sect. 5 to evaluate the encoded priors.

A different quality of probability assessments is *calibration*, which measures the agreement between an expert’s assessed probabilities and reality. Events that are given a prior probability of occurring of, say, 0.3 should ideally occur about 30 % of the time. Similarly, a set of subjective 50 % prediction intervals should contain the target event half the time. A body of psychological research has examined the calibration of probability assessments in varied contexts; see (O’Hagan et al. 2006, chapter 4) for a review. However, as noted by (O’Hagan et al. 2006, p. 161), “comparing the elicited probabilities or distributions with the true values of the events or variables in question does not evaluate the success of elicitation; it does not compare the elicited probabilities with the expert’s underlying opinions.”

## 5 Example: Evolution of the snails in the UK

Environmental scientists are interested in the effect of climate change on the evolutionary responses of different species. The brown-lipped banded snail (*Cepaea nemoralis*) is known to have a short generation time and tends to sensitively adapt its shell colour and banding pattern according to its thermal environment. Silvertown et al. (2011) tested the evolutionary changes in this type of snail by comparing a recent dataset with historical datasets from the last century. Data on the frequency of each phenotype of this species were collected by volunteers in 15 European countries through the citizen science project Evolution MegaLab (Worthington et al. 2012). Thousands of public volunteers contributed data to the project by recording the shell colour and banding pattern of the snails they had observed in each country.

An environmental expert who has worked with these data is interested in the modelling of ecosystems under future climate change. In this example, the expert (who is male) used the elicitation software to quantify his opinion about the expected proportions of each phenotype of Cepaea nemoralis snails in the UK in 2050 under the assumption that only a little is done to reduce climate change (a scenario called Representative Climate Forcing 6, under which greenhouse gas emissions continue to rise without strong mitigation). He advised that three main phenotypes can be considered: yellow, pink or brown; each of them can be banded or unbanded. Hence, any single snail can be categorized in one of six different phenotypes, so the problem can be formulated as a multinomial model with six categories. The probabilities \(p_{1}, \ldots , p_{6}\) are the proportions of each phenotype that might be observed in the UK. Our method and software were used to quantify the expert’s opinion about his uncertainty regarding the parameters of this multinomial model, representing his opinion by both a Dirichlet prior distribution and a Gaussian copula prior.

Expert’s assessments of medians and quartiles

Phenotype | Lower quartile (\({\overline{L}}_{i}\)) | Median (\({\overline{m}}_{i}\)) | Upper quartile (\({\overline{U}}_{i}\)) | |
---|---|---|---|---|

Yellow-banded | \(p_{1}\) | 0.351 | 0.449 | 0.551 |

Yellow-unbanded | \(p_{2}\) | 0.154 | 0.198 | 0.240 |

Pink-banded | \(p_{3}\) | 0.127 | 0.146 | 0.182 |

Pink-unbanded | \(p_{4}\) | 0.048 | 0.069 | 0.092 |

Brown-banded | \(p_{5}\) | 0.056 | 0.092 | 0.140 |

Brown-unbanded | \(p_{6}\) | 0.025 | 0.042 | 0.058 |

The encoded hyperparameters of the Dirichlet prior distribution

\(a_{1}^{*}\) | \(a_{2}^{*}\) | \(a_{3}^{*}\) | \(a_{4}^{*}\) | \(a_{5}^{*}\) | \(a_{6}^{*}\) | \(N^{*}\) |
---|---|---|---|---|---|---|

18.33 | 8.12 | 6.27 | 2.96 | 4.12 | 1.81 | 41.60 |

*N*. Specifically, they give \(N_{i}=\alpha _{i}+\beta _{i}\ (i=1, \ldots , 6)\) that range between 11.75 and 73.27. The variability of these values can be used as a diagnostic check to determine whether the Dirichlet prior distribution might be a reasonable representation of the expert’s knowledge. As concluded in Elfadaly and Garthwaite (2013), a substantial variation in the values of \(N_{i}\ (i=1,\ldots ,k)\) indicates that a Dirichlet prior does not capture the expert’s opinions well. This suggests that here the Dirichlet prior may not be an adequate representation of the expert’s opinion, in which case the Gaussian copula prior is to be preferred.

The encoded hyperparameters of the revised Dirichlet prior distribution

\(a_{1}^{*}\) | \(a_{2}^{*}\) | \(a_{3}^{*}\) | \(a_{4}^{*}\) | \(a_{5}^{*}\) | \(a_{6}^{*}\) | \(N^{*}\) |
---|---|---|---|---|---|---|

51.46 | 22.63 | 17.06 | 7.70 | 10.89 | 4.29 | 114.03 |

Medians and quartiles of the revised marginal beta distributions

Phenotype | Lower quartile | Median | Upper quartile | |
---|---|---|---|---|

Yellow-banded | \(p_{1}\) | 0.401 | 0.451 | 0.501 |

Yellow-unbanded | \(p_{2}\) | 0.156 | 0.194 | 0.236 |

Pink-banded | \(p_{3}\) | 0.111 | 0.144 | 0.182 |

Pink-unbanded | \(p_{4}\) | 0.050 | 0.065 | 0.082 |

Brown-banded | \(p_{5}\) | 0.083 | 0.095 | 0.107 |

Brown-unbanded | \(p_{6}\) | 0.027 | 0.036 | 0.046 |

The expert gave further assessments to encode a Gaussian copula prior as well, which gains much greater flexibility in the correlation structure of the parameters of his multinomial model. To encode a Gaussian copula prior, the expert was asked to assess a median value for the proportion of the yellow-banded phenotype of snails in the UK in 2050. He assessed this as \(m_{1,0}^{\#}=0.446\), the median of \(Z_{1}=p_{1}\). As mentioned before in Sect. 3.2.1, \(m_{1,0}^{\#}\simeq {\overline{m}}_{1}=0.449\). To improve the internal consistency between this assessment and the forthcoming ones, the expert was also asked at this stage to assess the median values of the proportions of all other phenotypes. These were assessed as 0.198, 0.147, 0.076, 0.091, 0.041. Clearly, these assessments are quite consistent with the experts’s assessments \({\overline{m}}_{i}\) in Table 1.

Expert’s assessments for the medians and quartiles of the \(Z_{i}\) variables

Phenotype | Lower quartile (\(L_{i,0}^{\#}\)) | Median (\(m_{i,0}^{\#}\)) | Upper quartile (\(U_{i,0}^{\#}\)) | |
---|---|---|---|---|

Yellow-banded | \(Z_{1}\) | 0.349 | 0.446 | 0.545 |

Yellow-unbanded | \(Z_{2}\) | 0.253 | 0.358 | 0.449 |

Pink-banded | \(Z_{3}\) | 0.330 | 0.413 | 0.495 |

Pink-unbanded | \(Z_{4}\) | 0.296 | 0.366 | 0.438 |

Brown-banded | \(Z_{5}\) | 0.557 | 0.688 | 0.802 |

Then at each step \(i\ (i=3,4,5)\), assuming that a selected snail does not belong to the first \(i-1\) phenotypes, the expert accepted the suggested value of \(m_{i,0}^{\#}\) as the median of \(Z_{i}=p_{i}/(1-\sum _{j=1}^{i-1} m_{j,0}^{\#})\), together with the remaining median values at each step.

*i*th phenotype given that the snail could not be categorized in the first \(i-1\) categories. The full set of median and quartile assessments obtained at this stage are shown in Table 5. These assessments were used in the optimization equation (13) in Sect. 4.1 to encode the hyperparameters \(\alpha _{i}^{\#}\) and \(\beta _{i}^{\#}\) of the beta distribution of each \(Z_{i}\ (i=1, \ldots ,5)\). The values of these hyperparameters are given in Table 6.

The encoded hyperparameters of marginal beta distributions

\(Z_{1}\) | \(Z_{2}\) | \(Z_{3}\) | \(Z_{4}\) | \(Z_{5}\) | |
---|---|---|---|---|---|

\(\alpha ^{\#}\) | 5.353 | 3.906 | 6.819 | 7.838 | 4.531 |

\(\beta ^{\#}\) | 6.577 | 6.934 | 9.607 | 13.344 | 2.227 |

*i*th \((i=2, \ldots , 5)\) category, under the condition that the assessed medians for the previous \(j\ (j=1, \ldots , i-1)\) categories were actually their true values. The expert’s five pairs of assessments for the conditional lower and upper quartiles are given in Table 7. The quartiles for the last category in Table 7 were automatically computed by the software when the expert assessed two quartiles for the fifth category. This is also illustrated in Fig. 5.

Expert’s assessments of conditional quartiles

\(p_{2}\) | \(p_{3}\) | \(p_{4}\) | \(p_{5}\) | \(p_{6}\) | |
---|---|---|---|---|---|

Conditional lower quartile (\(L_{i}^{*}\)) | 0.163 | 0.132 | 0.067 | 0.081 | 0.032 |

Conditional upper quartile (\(U_{i}^{*}\)) | 0.230 | 0.163 | 0.086 | 0.100 | 0.051 |

Expert’s assessments of conditional medians. The conditional median assessment in the *i*th row and *j*th column is \(m_{j,i}^{*}\ (i=1,\ldots ,4; j=i+1,\ldots ,6)\)

\(p_{1}\) | \(p_{2}\) | \(p_{3}\) | \(p_{4}\) | \(p_{5}\) | \(p_{6}\) |
---|---|---|---|---|---|

0.349 | 0.279 | 0.162 | 0.084 | 0.088 | 0.039 |

0.446 | 0.163 | 0.167 | 0.090 | 0.092 | 0.043 |

0.446 | 0.198 | 0.132 | 0.093 | 0.089 | 0.041 |

0.446 | 0.198 | 0.147 | 0.067 | 0.094 | 0.048 |

For example, Fig. 1 shows the expert’s assessments of conditional medians \(m_{1,4}^{*}\), \(m_{1,5}^{*}\) and \(m_{1,6}^{*}\) for the last three categories, given that \(p_{1}=m_{1,1}^{*}=0.349\), where the conditional median assessments \(m_{1,2}^{*}=0.279\) and \(m_{1,3}^{*}=0.162\) are also assumed to be the true values of \(p_{2}\) and \(p_{3}\), respectively. This is one of the sub-stages required to obtain the assessments in the first row of Table 8. However, the expert chose not to change his conditional median assessments at different sub-stages. He only revised his assessments when the underlined conditioning values were changed, i.e. from one row to the next in Table 8.

The encoded correlation matrix, \(\mathbf{R }^{*}\), of the Gaussian copula prior

\(Y_{1}\) | \(Y_{2}\) | \(Y_{3}\) | \(Y_{4}\) | \(Y_{5}\) | |
---|---|---|---|---|---|

\(Y_{1}\) | 1.000 | \(-0.742\) | \(-0.416\) | \(-0.386\) | \(-0.079\) |

\(Y_{2}\) | \(-0.742\) | 1.000 | 0.145 | 0.031 | 0.099 |

\(Y_{3}\) | \(-0.416\) | 0.145 | 1.000 | \(-0.296\) | 0.076 |

\(Y_{4}\) | \(-0.386\) | 0.031 | \(-0.296\) | 1.000 | 0.145 |

\(Y_{5}\) | \(-0.079\) | 0.099 | 0.076 | 0.145 | 1.000 |

Conditional medians as assessed by the expert (denoted by A), as computed from the encoded Dirichlet distribution (denoted by D) and as computed from the encoded Gaussian copula prior distribution (denoted by C)

\(p_{1}\) | \(p_{2}\) | \(p_{3}\) | \(p_{4}\) | \(p_{5}\) | \(p_{6}\) | |
---|---|---|---|---|---|---|

A | 0.349 | 0.279 | 0.162 | 0.084 | 0.088 | 0.039 |

D | 0.234 (0.16 %) | 0.177 (0.09%) | 0.079 (0.06%) | 0.116 (0.32%) | 0.044 (0.13%) | |

C | 0.278 (0.00 %) | 0.166 (0.02 %) | 0.082 (0.02 %) | 0.093 (0.06 %) | 0.038 (0.03 %) | |

A | 0.446 | 0.163 | 0.167 | 0.09 | 0.092 | 0.043 |

D | 0.166 (0.01 %) | 0.074 (0.18 %) | 0.109 (0.18 %) | 0.041 (0.05 %) | ||

C | 0.168 (0.01 %) | 0.087 (0.03 %) | 0.094 (0.02 %) | 0.043 (0.00 %) | ||

A | 0.446 | 0.198 | 0.132 | 0.093 | 0.089 | 0.041 |

D | 0.074 (0.20 %) | 0.109 (0.22 %) | 0.041(0.00 %) | |||

C | 0.087 (0.06 %) | 0.096 (0.08 %) | 0.043 (0.05 %) | |||

A | 0.446 | 0.198 | 0.147 | 0.067 | 0.094 | 0.048 |

D | 0.103 (0.10 %) | 0.039 (0.19 %) | ||||

C | 0.094 (0.00 %) | 0.046 (0.04 %) |

The encoded matrix in Table 9 does not give covariances between the beta distributed variables, \(Z_{1}, \ldots , Z_{5}\). Instead, it gives the covariances between the transformed normal variates, \(Y_{1}, \ldots , Y_{5}\). The Gaussian copula multivariate distribution is parameterized by both the marginal beta parameters in Table 6 and the correlation matrix in Table 9. These determine the experts’s Gaussian copula prior distribution. The software produces a file containing the WinBUGS model (Spiegelhalter et al. 2007) that uses the Gaussian copula prior distribution. This is given at the end of the Appendix.

To examine the assessed prior distributions, we compute the conditional medians from each of the two encoded prior distributions and compare them to the expert’s assessments in Table 8. The aim is to check whether the Gaussian copula has captured the correlation structure of the multinomial probabilities better than the Dirichlet distribution. Table 10 gives the conditional medians computed from the Dirichlet distribution, with the encoded hyperparameters in Table 3, together with the corresponding conditional medians computed from the Gaussian copula prior, with the encoded hyperparameters in Tables 6 and 9. The original conditional medians assessed by the expert are given again in Table 10 to facilitate the comparison. For both prior distributions, the discrepancies of the computed conditional medians from the expert’s assessments are also presented in the same table as percentages of the assessments.

Comparing the conditional median values with the corresponding expert’s original assessments it can be seen that the conditional medians computed from the proposed Gaussian copula prior are much closer to the expert’s assessments than are those computed from the Dirichlet distribution. This illustrates the potential advantage from using the Gaussian copula prior rather than the Dirichlet prior distribution.

The elicitation process took about an hour to complete. The expert stressed the importance of the convenient order of categories when giving conditional assessments, commenting that ordering the categories in a suitable sequence made it easier for him to think about the conditions.

## 6 Concluding comments

Our aim in this paper was to propose methods for eliciting expert opinion about the hyperparameters of both the Dirichlet distribution and the Gaussian copula distribution. So as to simplify the separate tasks that an assessor must perform, the elicitation process for a multivariate prior has been decomposed into a number of processes for eliciting univariate beta distributions. The hyperparameters of these beta distributions are estimated from marginal median and quartile assessments obtained from a field expert. Two novel methods have been proposed for reconciling the beta marginal distributions into a multivariate prior distribution. The first method reconciles the beta distributions as marginals of the well-known standard conjugate Dirichlet distribution, giving a prior distribution that is easily applied in practice.

In the second method, instead of assuming a Dirichlet prior, we use a Gaussian copula function with beta marginals to model the joint prior distribution of multinomial probabilities. The proposed Gaussian copula prior assumes that the dependence structure between the transformed probabilities can be represented by a multivariate normal distribution, where the marginal prior distribution of each of them is still expressed as a beta distribution. This requires further conditional quartile assessments to describe the correlation structure between these probabilities. The monotonicity of the Gaussian copula transformation allows conditional quartiles of the multinomial probabilities to be transformed into normal quartiles. The latter are used to obtain product-moment correlations for normal variates. This powerful technique of transforming quartiles avoids the difficulties encountered when transforming product-moment correlations. Structural assessments of the conditional quartiles are used to ensure that the encoded correlation matrix is positive-definite. Although the encoding of a Gaussian copula requires more assessments than the Dirichlet distribution, it gives a more flexible multivariate prior distribution. A practical example where the methods are used was given and the two encoded prior distributions compared.

The two methods have been implemented in a single user-friendly software package that is freely available at http://statistics.open.ac.uk/elicitation. The user has the option of eliciting either of the two priors or both of them simultaneously. For the Dirichlet prior distribution, the available software outputs the encoded hyperparameter vector \(\mathbf{a }^{*}\). The expectations and variances of each \(p_{i}\) are also given. For the Gaussian Copula prior, the output includes the encoded pairs of beta parameters \(\alpha _{i}^{\#}\) and \(\beta _{i}^{\#}\ (i=1, \ldots , k-1)\), together with the encoded correlation matrix \(\mathbf{R }^{*}\).

## Notes

### Acknowledgments

The work reported here was supported by a studentship from The Open University. We are very grateful to Dr. Yoseph Araya (a plant ecologist and a lecturer in Environmental Geography at Birkbeck College, University of London, UK) whose opinion was quantified in the example of the phenotypes of Cepaea nemoralis snails.

## References

- Aitchison, J.: The Statistical Analysis of Compositional Data. Chapman and Hall, London (1986)CrossRefzbMATHGoogle Scholar
- Albert, I., Donnet, S., Guihenneuc-Jouyaux, C., Low-Choy, S., Mengersen, K., Rousseau, J.: Combining expert opinions in prior elicitation. Bayesian Anal.
**7**, 503–532 (2012)MathSciNetCrossRefzbMATHGoogle Scholar - Bunn, D.W.: Estimation of a Dirichlet prior distribution. Omega
**6**, 371–373 (1978)CrossRefGoogle Scholar - Chaloner, K., Duncan, G.T.: Some properties of the Dirichlet-multinomial distribution and its use in prior elicitation. Commun. Stat.-Theory. Methods
**16**, 511–523 (1987)zbMATHGoogle Scholar - Chamberlin, J.R., Featherston, F.: Selecting a voting system. J. Polit.
**48**, 347–369 (1986)CrossRefGoogle Scholar - Clemen, R.C., Reilly, T.: Correlations and copulas for decision and risk analysis. Manag. Sci.
**45**, 208–224 (1999)CrossRefzbMATHGoogle Scholar - Conn, P.B., McClintock, B.T., Cameron, M.F., Johnson, D., Moreland, E.E., Boveng, P.L.: Accommodating species identification errors in transect surveys. Ecology
**94**, 26072618 (2013)CrossRefGoogle Scholar - Connor, R.J., Mosimann, J.E.: Concepts of independence for proportions with a generalization of the Dirichlet distribution. J. Am. Stat. Assoc.
**64**, 194–206 (1969)MathSciNetCrossRefzbMATHGoogle Scholar - Cook, L., Cameron, R., Dodd, M., McConway, K., Worthington, J., Skelton, P., Anton, C., Bossdor, O., Baur, B., Schilthuizen, M., Fontaine, B., Sattmann, H., Bertorelle, G., Correia, M., Oliveira, C., Pokryszko, B., Ożgo, M.: Stalaz̆s, A., Gill, E., Üllar, Rammu, Sólymos, P., Féher, Z., Juan, X.: Citizen science reveals unexpected continental-scale evolutionary change in a model organism. PLoS One
**6**, e18927 (2011)CrossRefGoogle Scholar - Demarta, S., McNeil, A.J.: The t copula and related copulas. Int. Stat. Rev.
**73**, 111–129 (2005)CrossRefzbMATHGoogle Scholar - Elfadaly, F.G., Garthwaite, P.H.: Eliciting Dirichlet and Connor-Mosimann prior distributions for multinomial models. Test
**22**, 628–646 (2013)MathSciNetCrossRefzbMATHGoogle Scholar - Elfadaly, F.G., Garthwaite, P.H.: On eliciting logistic normal priors for multinomial models. In preparation. http://mcs-brains.open.ac.uk/elicitation/Logistic%20Paper.pdf (2016)
- Frees, E.W., Valdez, E.A.: Understanding relations using copulas. North Am. Actuar. J.
**2**, 125 (1998)CrossRefzbMATHGoogle Scholar - Fu, H., Qu, Y., Zhu, B., Huster, W.: A Bayesian approach to the statistical analysis of device preference studies. Pharm. Stat.
**11**, 149–156 (2012)CrossRefGoogle Scholar - Garthwaite, P.H., Kadane, J.B., O’Hagan, A.: Statistical methods for eliciting probability distributions. J. Am. Stat. Assoc.
**100**, 680–701 (2005)MathSciNetCrossRefzbMATHGoogle Scholar - Garthwaite, P.H., Al-Awadhi, S.A., Elfadaly, F.G., Jenkinson, D.J.: Prior distribution elicitation for generalized linear and piecewise-linear models. J. Appl. Stat.
**40**, 59–75 (2013)MathSciNetCrossRefGoogle Scholar - Joe, H.: Multivariate Models and Dependence Concepts. Chapman & Hall, London (1997)CrossRefzbMATHGoogle Scholar
- Jouini, M.N., Clemen, R.T.: Copula models for aggregating expert opinions. Oper. Res.
**44**, 444–457 (1996)CrossRefzbMATHGoogle Scholar - Kadane, J.B., Dickey, J.M., Winkler, R., Smith, W., Peters, S.: Interactive elicitation of opinion for a normal linear model. J. Am. Stat. Assoc.
**75**, 845–854 (1980)MathSciNetCrossRefGoogle Scholar - Kurowicka, D., Cooke, R.: Uncertainty Analysis with High Dimensional Dependence Modelling. Wiley Series in Probability and Statistics. Wiley, Chichester (2006)CrossRefzbMATHGoogle Scholar
- Leonard, T.: Bayesian estimation methods for two-way contingency tables. J. R. Stat. Soc. Ser. B
**37**, 23–37 (1975)MathSciNetzbMATHGoogle Scholar - Lewandowski, D.: High Dimensional Dependence: Copulae, Sensitivity, Sampling. Ph.D. thesis, Delft University of Technology, Netherlands (2008)Google Scholar
- Lindley, D.V., Tversky, A., Brown, R.V.: On the reconciliation of probability assessments (with discussion). J. R. Stat. Soc. Ser. A
**142**, 146–180 (1979)MathSciNetCrossRefzbMATHGoogle Scholar - Manski, C.F.: Partial identification of counterfactual choice probabilities. Int. Econ. Rev.
**48**, 1393–1410 (2007)MathSciNetCrossRefGoogle Scholar - Nelsen, R.B.: An Introduction to Copulas. Lecture Notes in Statistics, vol. 139. Springer, New York (1999)CrossRefGoogle Scholar
- O’Hagan, A., Forster, J.: Bayesian Inference, volume 2B of Kendall’s Advanced Theory of Statistics, 2nd edn. Arnold, London (2004)Google Scholar
- O’Hagan, A., Buck, C.E., Daneshkhah, A., Eiser, J.R., Garthwaite, P.H., Jenkinson, D.J., Oakley, J.E., Rakow, T.: Uncertain Judgements: Eliciting Expert Probabilities. Wiley, Chichester (2006)CrossRefzbMATHGoogle Scholar
- Sklar, A.: Fonctions de répartition à n dimensions et leurs marges. Publ. Inst. Stat. Univ. Paris
**8**, 229–231 (1959)zbMATHGoogle Scholar - Spiegelhalter, D., Thomas, A., Best, N., Lunn, D.: WinBUGS Version 1.4.3 User Manual. Medical Research Council Biostatistics Unit, Cambridge (2007)Google Scholar
- van Dorp, J.R., Mazzuchi, T.A.: Parameter specification of the beta distribution and its Dirichlet extensions utilizing quantiles. In: Gupta, A.K., Nadarajah, S. (eds.) Handbook of Beta Distribution and Its Applications. Marcel Dekker Inc, New York (2004)Google Scholar
- Weltje, G.J.: Quantitative analysis of detrital modes: statistically rigorous confidence regions in ternary diagrams and their use in sedimentary petrology. Earth-Sci. Rev.
**57**, 211–253 (2002)CrossRefGoogle Scholar - Wilks, S.S.: Mathematical Statistics. Wiley, New York (1962)zbMATHGoogle Scholar
- Worthington, J.P., Silvertown, J., Cook, L., Cameron, R., Dodd, M., Greenwood, R.M., McConway, K., Skelton, P.: Evolution megalab: a case study in citizen science methods. Methods Ecol. Evol.
**3**, 303–309 (2012)CrossRefGoogle Scholar - Zapata-Vázquez, R.E., O’Hagan, A., Bastos, L.S.: Eliciting expert judgements about a set of proportions. J. Appl. Stat.
**41**, 1919–1933 (2014)MathSciNetCrossRefzbMATHGoogle Scholar

## Copyright information

**Open Access**This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.