Introduction

Among the quantitative methods for eliciting stated preferences, choice experiments have been increasingly advocated in recent years. Human resources, health, marketing, economics, tourism, and policy-making are just a few of the disciplines that utilise choice experiments in their applications. Choice experiments are a useful tool for understanding the intricacies of decision-making by individuals. The process entails the generation of a set of hypothetical scenarios, referred to as choice sets, which describe a product or service in terms of various features or attributes. Participants are then asked to make decisions and trade-offs in these realistic yet hypothetical situations, revealing their preferences and priorities (Louviere et al. 2000). As a result, constructing an optimal and efficient design of choice experiments has become a subject of significant interest in recent years.

A choice experiment was first introduced by Thurstone (1927) under the context of the “discrete choice experiment”. Since then, scholars have suggested several various strategies from different perspectives aiming at obtaining well-designed choice sets that are either optimal or near-optimal. Of particular interest are the designs with a small number of choice sets; see for example, the review paper by Alamri et al. (2023) and the references therein in the case of models with only the main effect. In the presence of higher-order interactions, researchers have investigated optimal designs as discussed in the works of Grasshoff et al. (2003), Großmann et al. (2012), Chai et al. (2018) and Street and Viney (2019). Others have suggested the use of full and/or partial profiles for DCE designs, the interested reader is referred to the work of Großmann (2017), Nyarko and Schwabe (2019), Nyarko and Doku-Amponsah (2022) and Nyarko (2023) for further insights on this topic. These studies contribute valuable insights for researchers interested in constructing optimal or near-optimal choice experiments.

Although practitioners tend to use efficient designs with prior-required parameters in choice experiment studies (Soekhai et al. 2019), the use of optimal designs that assume all alternatives in a choice set have the same probability (referred to as the indifference assumption) still continues to be used in choice experiment studies. This is because the necessary priors for efficient designs are often not easily obtainable (Bliemer and Rose, 2010). Several algorithms, which are quite common in commercial use, have been proposed for identifying efficient designs. These algorithms include the Modified Fedorov algorithm (Cook and Nachtrheim, 1980), the Coordinate Exchange algorithm (Meyer and Nachtsheim, 1995) or the Relabelling-Swapping-Cycling algorithm (Huber and Zwerina, 1996; Sándor and Wedel, 2002). Applying any of the above will usually result in the construction of designs with high D-efficiencies. A common drawback of such algorithms is that they can not ensure D-optimality over the whole search space, leaving the possibility that a better design may exist (Street et al. 2005). A further aspect that must be taken into account in constructing efficient designs is the potential lack of prior knowledge regarding the extent and direction of population preference in certain situations (Norman et al. 2019). Therefore, when no prior parameters are assumed (e.g., in pilot studies), optimal designs are the most efficient designs known (Norman et al. 2019; Rose and Bliemer, 2009). Moreover, optimal designs are usually suitable to be applied in cases where the choice experiments are unlabelled, i.e. all attributes have the same generic coefficient across all alternatives (Louviere et al. 2000).

Paired choice experiments are an important special case in which participants have to select between alternatives that are shown in pairs, based on their perceived utility. Optimal paired choice designs based on D-optimality, which estimate only the main effects, have been developed by scholars such Demirkale et al. (2013); Graßhoff et al. (2004). These designs are also optimal under other criteria such as A-, E- and MS-optimal (Singh et al. 2015). Despite the fact that their findings are comprehensive, their designs for binary attributes are limited to cases where N is a multiple of 4 (that is, N ≡ 0 mod 4), since orthogonal designs are only possible for two-level attributes under such conditions. Therefore, Singh et al. (2015) suggested several constructions in which practitioners can construct optimal paired choice designs for any number of choice sets N. However, the Singh et al. (2015) techniques do not provide the optimal design when the number of attributes, k, is equal to the number of choice sets, N. Thus, we will introduce here some new methods that can be used to obtain optimal designs when the number of attributes is equal to the number of runs. These will be designs with choice sets of sizes that are not necessarily multiples of 4, i.e. N ≢ 0mod4.

Orthogonality is one of the desired structural properties in designing D-optimal choice experiments (Huber and Zwerina, 1996). Several phenomenon approaches have been proposed to construct small optimal orthogonal designs. However, as the number of attributes or attribute levels moderately increases, the number of choice sets in the design grows rapidly, particularly in paired choice designs. An orthogonal design seeks to minimise the correlation between attribute levels within the choice sets and thereby ensures that the design is statistically independent (Street and Burgess, 2007). However, for certain dimensions, an orthogonal design may be unavailable or unknown. For example, two-level orthogonal designs can only be constructed when the number of experimental runs is a multiple of four. In certain cases, furthermore, an orthogonal design may require a considerably higher number of option scenarios than the theoretical minimum number of rows, and may also fail to balance the attribute levels (Rose and Bliemer, 2009). Controversies still exist in the literature concerning problems with the use of orthogonal designs in practice, and interested readers may consult the works of (Rose and Bliemer, 2009).

When an orthogonal design is not available and a D-optimal design is desired for a practitioner, there are only two options: (1) go through the literature for construction that precisely matches the desired sample size, number of attributes, and attribute levels, or (2) use a computer-aided construction for an approximated D-optimal design that relies on algorithms. These algorithms, which are quite common in commercial use, can provide a design that is highly D-efficient using many random starting designs but cannot ensure D-optimality of the resulting design over the whole search space (Cuervo et al. 2016). Since the 1950s, however, the literature has presented direct construction methods for generating D-optimal designs, often referred to as “weighing designs”. These designs were initially developed to determine the unknown weights of a given number of P objects using a specified number of weighings N (P ≤ N), using either a single spring balance or a chemical balance with two pans (Graczyk, 2013). A “spring balance design” is a design in which a selection of the P objects would be placed in a single pan together, hence the entries of the incidence matrix would be 1 if an object is presented, otherwise 0. However, a “chemical balance design” is a design where two groups are selected from the P objects and placed in each of the two pans, hence the entries of the incidence matrix are typically + 1 if the object is put on the left pan, − 1 if the object is put on the right pan, and 0 entry if the object is not selected. Throughout this paper, we propose the construction of weighing designs using “spring balance” to construct D-optimal symmetric paired choice designs for main effects models. The term “Spring balance” refers to a type of design where two-level factors are used and each factor always has one level selected (i.e., no zero entries). This can be seen as a special case of weighing designs. The formulations of the weighing design matrices included in this article are original contributions from King et al. (2020).

The subsequent sections of this paper will be organised as follows; In Section Preliminary Definitions and Notation, definitions and notations pertinent to the study will be presented. In Construction of D-optimal choice design, we present four different methods for constructing D-optimal paired choice designs for attributes at two levels, main effect models, and equal choice probabilities. These methods rely on the assumption that the number of attributes, k, is equal to the number of choice sets, N, as this is the minimum number of runs for which we can obtain a D-optimal design. While (Singh et al. 2015) constructed D-optimal two-level choice designs for all existing possible combinations of attributes and runs, their constructions cannot provide the D-optimal when the number of attributes, k, is equal to the number of choice sets, N. Therefore, the methods we develop here improve the designs for all practical cases that have equal or fewer than 12 attributes. In DCEs with more than two-levels symmetric attributes, we extend these methods to be applied for attributes with more than two levels. Notably, this extension yields a pronounced reduction in the number of runs for the designs generated in this paper compared to the existing designs found in the literature. This has practical applications and implementations as one would like the respondents to maintain their interest in the survey. In Case Study, a simulation study is conducted to verify the advantages of the methodology proposed in this paper. Ultimately, Section Discussion concludes this paper by providing a concise analysis of the construction methods presented earlier, comparing them to the designs proposed by Graßhoff et al. (2004), and highlighting potential avenues for improvement and future research.

Preliminary definitions and notation

In this article, we consider choice experiments with N choice sets, where each choice set is composed of two alternatives, resulting in a total of m = 2 options per choice scenario. It is noteworthy that in the context of choice experiments, designs where each choice set only includes two alternatives are sometimes referred to as “a paired comparison design” (Graßhoff et al. 2004). In each choice set n, the model should not have the same two alternatives, n1 ≠ n2, where ni is the ith alternative in the nth choice set, n = 1, 2, . . . , N and i = 1 and 2. In this context, each alternative is defined by a set of symmetric attributes k (the same number of levels in all attributes), where these attributes may have two or more levels, denoted by ℓ. Thus, there are L = ℓk possible options in total. It is assumed that all attributes appear in the utility function of each alternative, each attribute takes one of the ℓ levels, and these levels are represented numerically by x, where x ∈ {0, 1, . . . , ℓ − 1}. Note that the attribute levels of the alternatives are chosen to be as different as possible to maximise the information (utility) obtained from the choice experiment.

The multinomial logit (MNL) model is a statistical model that is often used for analysing DCEs (choice experiments). Throughout this paper, the DCE designs are considered under the MNL model for estimating the main effects. The probability Prnj that a particular alternative j is chosen from the paired alternatives by the respondent r in the set of choices n is expressed as the relationship between the parameter for the chosen alternative and the sum of the parameters corresponding to all the options in the set of options,

$${P}_{rnj}=\frac{exp({V}_{rnj})}{\mathop{\sum }\limits_{i=1}^{2}exp({V}_{rni})}$$
(1)

where \({V}_{rnj}=\mathop{\sum }\nolimits_{k = 1}^{K}{\beta }_{k}{x}_{rnjk}\); x represents the observed attribute levels of each alternative and β represents the effect of the attribute levels on the utility using maximum likelihood estimation to be estimated. Thus, the utility of alternative j preferred by the respondent r in the set of choices n is Urnj = Vrnj + ϵrnj, which consists of an observed component Vrnj and an unobserved component ϵrnj. Typically, these parameters β are unknown and are estimated from the data collected during the experiment. In our case, we assume that the parameter of each βk is treated as a generic attribute across all alternatives. Note that the matrix x is associated with a generated choice design. However, It is commonly assumed that the unobserved component ϵrnj follows a type 1 extreme value (EV1) distribution that is independently and identically distributed (IID), which results in the MNL model (see McFadden, 1974).

The purpose here is to generate optimal paired choice experiments that provide the optimal k-tuple of attribute levels per task, which meet a specified criterion for a given total number of choice sets. This criteria is generally related to the information matrix of the design, also called C-matrix, which is computed as \(C=B{{\Lambda }}{B}^{{\prime} }\) for estimating the main effects with no interactions under the orthonormal coding. Recently, Das and Singh (2020) proved that deriving the information matrix of choice experiments using linear effects coding, as mentioned in (Huber and Zwerina, 1996), or orthonormal coding, as mentioned in (Street and Burgess, 2007), is equivalent when seeking optimal designs. Thus, the matrix Λ = [Λ(r, s)] is a square matrix of order L with rows and columns indexed by 0, 1, . . . , L − 1. The elements of this matrix, Λ(r, s), is given by

$${{\Lambda }}(r,s)=\left\{\begin{array}{ll}\frac{1}{4N}{\eta }_{r},\quad &\,{{\mbox{if}}}\,\,r=s\,\,{{\mbox{so-called diagonal entries}}}\,\\ -\frac{1}{4N}{\eta }_{r,s},\quad &\,{{\mbox{if}}}\,\,r\,\ne\, s\,\,{{\mbox{so-called off-diagonal entries}}}\,\end{array}\right.,$$
(2)

where ηr represents the frequency of option r appearing in the design, while ηr,s represents the frequency of options r and s appearing together in the same choice set of the design. Equation (2) is a simplified form of the Lambda matrix, that was given in Burgess and Street (2005), since in this paper we only consider pairs of choices, i.e. m = 2. The matrix B is a normalised contrast matrix of order k(ℓ − 1) × L in which the rows of B represent a set of orthogonal contrasts that correspond to the main effects and satisfies \(BB{\prime} ={I}_{k(\ell -1)}\). To know more about how to define the contrast matrix B, see the review paper by Alamri et al. (2023) and the example explained there.

In this paper, we focus on comparing designs using D-optimality since the D-optimal criterion is invariant to reparameterization. Let Cℓ,opt denote the D-optimal form of the information matrix for any given choice design with ℓ levels. Burgess and Street (2005) identified the D-optimal choice design with a fixed choice set size m. Only paired choice designs (m = 2) with symmetric attributes (the same number of levels in all attributes) will be covered in this paper. Therefore, the result of Burgess and Street (2005) is simplified and we have that number of levels will be equal to or greater than the size of the choice set (ℓ ≥ m). In this case, the number of possible pairs for m = 2 is always one, S = m(m − 1)/2 = 1, ∀ ℓ ≥ 2. Thus, the largest corresponding determinant, det(Cℓ,opt), for paired choice designs, is given in Theorem 2.1. The interested reader may consult the works of Demirkale et al. (2013) for information on generalisations of this theorem.

Theorem 2.1

Under the assumption of equal choice probabilities and for only testing the main effects, the information matrix of a D-optimal paired choice design is a block-diagonal matrix with the qth block equal to [ℓ/(2L(ℓ − 1))]Iℓ−1, ∀ 1 ≤ q ≤ k; that is, the corresponding determinant is

$$det({C}_{\ell ,opt})={\left[\frac{\ell }{2L(\ell -1)}\right]}^{k(\ell -1)}.$$
(3)

As a practical matter, D-efficiency in which measuring the quality of any exact design, for only testing the main effects model, is calculated as,

$${eff}_{D}={\left[\frac{det(C)}{det({C}_{\ell ,opt})}\right]}^{\frac{1}{k(\ell -1)}},$$
(4)

It is worth noting that other researchers, such as Graßhoff et al. (2004) and Dey et al. (2017), also identified the D-optimal designs under a different model called the linear paired comparison model. Others have suggested the use of A-optimality in linearised models for DCE designs, see for example Singh et al. (2021). However, Großmann and Schwabe (2015) showed that when assuming equal choice probabilities and only estimating main effects, “the information matrix under the linear paired comparison model is identical to the one under the MNL model for designs with only paired choice sets”. Consequently, any optimal design under the MNL model is also optimal under the linear paired comparison model and vice versa.

Let Dζ,ℓ represent the set of all connected choice sets of size two that involve k attributes, each with ℓ levels. To construct optimal pair-choice sets, it is desirable to start with the Wr,k matrix; that is, an D-optimal design derived directly from the theory of weighing designs. Throughout this article, we consider only saturated weighing designs, that is, a design has as many runs as there are parameters r = k, with two-level attributes coded as − 1 and 1 for each attribute. Thus, we have an optimal paired choice design with parameters N = r(ℓ − 1)ℓ/2 and k, for a particular level ℓ. Note that for two-level designs (ℓ = 2), we have N = r.

Lemma 2.1

None of the symmetric choice designs in pairs is D-optimal unless each of the levels for each attribute q, 1 ≤ q ≤ k, appears exactly r(ℓ − 1) times, which is known as a level balance.

Construction of D-optimal choice design

This section showcases the use of D-optimal designs in building DCEs with two-level symmetric attributes for the sole purpose of identifying the main effects. It includes the construction of designs with smaller run sizes and increased efficiency for cases where the number of choice sets, denoted as N, is not divisible by four. Additionally, it outlines the potential extension of this methodology for DCEs with symmetric attributes that have more than two levels.

DCEs with two-level symmetric attributes

Here, we will present some direct constructions of D-optimal designs for main-effect models, two-level pairwise choice sets and equal choice probabilities. These methods can be grouped into four cases, each case corresponding to one of the four possible remainders when N is divided by 4 (N mod 4). Here, the modular notation, N ≡ c mod 4, will be used, that is, c is the remaining of dividing N by 4 (i.e., N = c + 4), where c = 0, 1, 2, 3 separately. We base our approach on Theorem 1 of Singh et al. (2015), which established that the search for an optimal paired choice design Dζ,2 is equivalent to locating an optimal design matrix, here denoted as Wr, with elements ± 1 and 0.

We base our approach on Theorem 1 of Singh et al. (2015), which established that the search for an optimal paired choice design Dζ,2 is equivalent to locating an optimal design matrix Wr with elements ± 1 and 0, we present the optimal form of C-matrix for binary-attributes (2k) in paired choice designs, in which each choice set consists of a pair of alternatives and each attribute has two levels ℓ = 2. Specifically, we provide the optimal format of paired choice experiments for each case and show how to construct an optimal choice set. As many of these approaches are rooted in the field of discrete mathematics, we provide additional concepts as needed.

Let \({C}_{c,k}^{* }\) denote the optimal form of the C-matrix for the paired choice design, where k indicates the size of C, and c ∈ {0, 1, 2, 3} is an identifier for the case. In some cases, it is possible to have a design that is D-optimal, even though the corresponding C-matrix does not match the optimal form \({C}_{c,k}^{* }\) for the given run size of choice sets. This occurs when the number of attributes for these designs does not satisfy the specific criteria required to attain the ideal \({C}_{c,k}^{* }\). Through suitable modifications to the constructions in the weighing designs, new D-optimal paired choice designs in Dζ,2 are constructed, as demonstrated in construction 3.1.

Construction 3.1

Suppose that there is a Wr weighing design of two levels and r rows (and columns). The first alternative in the paired choice sets is P1 = (Wr + 1)/2, and the second alternative in the choice sets will be P2 = (1 − Wr)/2. This gives the D optimal pair choice design Dζ,2 = [P1, P2] with N = r choice sets for each of the k = r attributes.

Case 0

This is the case, N is a multiple of 4 (N ≡ 0 mod 4), where the designs are D-optimal with 100% efficiency for which orthogonal designs exist “the most notable being the family designs of Plackett and Burman (1946)”. The optimal form of the C-matrix for this case was proved by Street and Burgess (2007) and is given by

$${C}_{0,k}^{* }=\left(\begin{array}{ccccc}\frac{1}{{2}^{k}}&0&0&\cdots \,&0\\ 0&\frac{1}{{2}^{k}}&0&\cdots \,&0\\ \vdots &\vdots &\vdots &\ddots &\vdots \\ 0&0&0&\cdots \,&\frac{1}{{2}^{k}}.\end{array}\right)$$
(5)

Its determinant is

$$det({C}_{0,k}^{* })={\left[\frac{1}{{2}^{k}}\right]}^{k}.$$
(6)

In this case, the matrix Wr is simply referred to as a Hadamard matrix of order r, hereafter denoted as Hr, which is an r × r square matrix with all elements equal to ± 1 and satisfies \({H}_{r}{H}_{r}^{{\prime} }={H}_{r}^{{\prime} }{H}_{r}=r{I}_{r}\). Typically, these matrices are standardised to have the form with all 1’s in the first row and column. Note that this class of matrix has the largest determinant among all matrices created from binary matrices of size r(Hadamard, 1893). Sylvester (1867) is credited with inventing the earliest construction method, which is a recursive construction with run sizes that are powers of 2:

Sylvester construction for Hadamard matrices

$$\begin{array}{cc}{H}_{1}=\left[\begin{array}{c}1\end{array}\right],&{H}_{{2}^{k}}=\left[\begin{array}{cc}{H}_{{2}^{k-1}}&{H}_{{2}^{k-1}}\\ {H}_{{2}^{k-1}}&-{H}_{{2}^{k-1}}\end{array}\right]fork=1,2,\cdots \,.\end{array}$$

Example 3.1

$$\begin{array}{cccc}{{{{\bf{H}}}}}_{{{{\bf{1}}}}}=\left[\begin{array}{c}1\end{array}\right],&{{{{\bf{H}}}}}_{{{{\bf{2}}}}}=\left[\begin{array}{cc}1&1\\ 1&-1\end{array}\right],&{{{{\bf{H}}}}}_{{{{\bf{4}}}}}=\left[\begin{array}{cccc}1&1&1&1\\ 1&-1&1&-1\\ 1&1&-1&-1\\ 1&-1&-1&1\end{array}\right],&{{{{\bf{H}}}}}_{{{{\bf{8}}}}}=\left[\begin{array}{cccccccc}1&1&1&1&1&1&1&1\\ 1&-1&1&-1&1&-1&1&-1\\ 1&1&-1&-1&1&1&-1&-1\\ 1&-1&-1&1&1&-1&-1&1\\ 1&1&1&1&-1&-1&-1&-1\\ 1&-1&1&-1&-1&1&-1&1\\ 1&1&-1&-1&-1&-1&1&1\\ 1&-1&-1&1&-1&1&1&-1\end{array}\right]\end{array}$$

Goethals and Seidel (1967) extended the work of Williamson (1944) in the construction of a Hadamard matrix of order 4r using cyclic (or “circular”) convolution. Details on how to construct the Goethals-Seidel construction for Hadamard matrices can be found in King et al. (2020).

It is worth noting that each sub-design of a Hadamard matrix is an orthogonal main-effect plan since a Hadamard matrix has orthogonal rows and columns. Therefore, optimal designs with less number of attributes than runs (k < r) can be constructed from a Hadamard matrix, donated as Hr,k, for which the runs r were 2 or a multiple of 4 (King et al. 2020). However, choice design usually requires that all rows need to be distinct; therefore, no two rows in the Wr,k matrix should have the same entries of ± 1. Note that one can randomly delete from a Hadamard matrix as a maximum of r/2 − 1 columns, but more than that, the columns need to be carefully deleted to ensure that all rows are distinct (Singh et al. 2015). As established in Graßhoff et al. (2004), for k ≤ r, the choice design \({C}_{0,k}^{* }\) in which a Hadamard matrix (Hr,k) used to construct a paired choice design is a D-optimal design, for which a Hadamard matrix of order r is known. However, for which a Hadamard matrix of order r is unknown, still in some cases a D-optimal paired choice design can be constructed even for k≤r. Singh et al. (2015) identified, under the main effects model, optimal two-level choice designs for any number of choice pairs. However, in the saturated case where k = r and orthogonal designs do not exist, we propose alternative types of matrices that yield higher D-efficiencies compared to those reported by Singh et al. (2015). Details are proposed in the following sections.

Case 1

This is the case in which N is odd and one more than a multiple of 4 (N ≡ 1 mod 4), where it is not possible to construct orthogonal designs. After modifying the optimal form of the information matrices for the D-optimal design matrix, which was first determined by Barba (1933), we present the optimal form of the C-matrix for this case, up to the absolute value of the off-diagonal terms, as shown in matrix (7). The formulation of \({C}_{1,k}^{* }\) for this case is an original contribution from the authors of this paper.

$${C}_{1,k}^{* }=\frac{1}{{2}^{k}k}\left(\begin{array}{ccccc}k&1&1&\cdots \,&1\\ 1&k&1&\cdots \,&1\\ \vdots &\vdots &\vdots &\ddots &\vdots \\ 1&1&1&\cdots \,&k.\end{array}\right)$$
(7)

The matrix Wr can be generated from the incidence matrix of a Balanced Incomplete Block Design (BIBD) with r treatments in r blocks and \((r+\sqrt{2r-1})/2\) treatments per block. However, such a Wr matrix exists only if \(\sqrt{2r-1}\) is an integer (Raghavarao, 1959). To generate the Wr design matrix, the 0’s in the incidence matrix are replaced with − 1’s. This technique gives a D-efficiency of 94.09% for choice pairs when only testing the main effects. However, all practical cases that we cover here involve equal or less than a 12 number of attributes, so only a BIBD design with r = 5 can be used to construct the matrix Wr since \(\sqrt{(2\times 5)-1}=\sqrt{9}=3\) is an integer. It should be noted that this design achieves the optimal form of \({C}_{1,k}^{* }\).

Although D-optimal saturated designs do exist for r values that do not satisfy the condition earlier. To date, a widely general technique for generating D-optimal designs in such scenarios has not yet been established. King et al. (2020) highlighted some of the procedures that have been the most successful general methods for constructing highly D-efficient saturated designs, such as Farmakis and Kounias (1987); Orrick and Solomon (2007). However, for certain small run sizes, r = 9, 17, 21, and37, a D-optimal saturated design has been confirmed using more specialised procedures. Only one design will be presented here for run sizes r = 9, while other run sizes can be found in the supplementary files of King et al. (2020). Thus, the Wr matrix can be expressed as

$${W}_{r}=\left[\begin{array}{ccc}1&1&{1}_{7}^{{\prime} }\\ 1&1&-{1}_{7}^{{\prime} }\\ {1}_{7}^{{\prime} }&-{1}_{7}^{{\prime} }&{B}_{7,7,3}\end{array}\right],where\,\,{B}_{7,7,3}=\left[\begin{array}{ccccccc}1&1&-1&1&-1&-1&-1\\ -1&1&1&-1&1&-1&-1\\ -1&-1&1&1&-1&1&-1\\ -1&-1&-1&1&1&-1&1\\ 1&-1&-1&-1&1&1&-1\\ -1&1&-1&-1&-1&1&1\\ 1&-1&1&-1&-1&-1&1\end{array}\right]$$

B7,7,3 is the incidence matrix of a BIBD with 7 treatments in 7 blocks and 3 treatments per block, with the 0 entries replaced with − 1’s. This design was first constructed by Ehlich (1964b). Using the W9 design, constructed from B7,7,3 as shown above, in construction 3.1, gives a D-efficiency of 93.20% for choice pairs when only testing the main effects. This design is the highest efficiency in the literature for a DCE design with this parameter (9 attributes and 9 choice sets), and also achieves the optimal form of \({C}_{1,k}^{* }\).

Case 2

This is the case in which N is even but not a multiple of 4 (N ≡ 2 mod 4), where it is not possible to construct orthogonal designs. After modifying the optimal form of the information matrices for the D-optimal design matrix, which had been determined independently by Barba (1933); Ehlich (1964b), we present the optimal form of the C-matrix for this case as shown in matrix (8). The formulation of \({C}_{2,k}^{* }\) for this case is an original contribution from the authors of this paper.

$${C}_{2,k}^{* }=\frac{1}{{2}^{k}k}\left(\begin{array}{cc}A&0\\ 0&A\end{array}\right),$$
(8)

where

$$A=\left(\begin{array}{ccccc}k&2&2&\cdots \,&2\\ 2&k&2&\cdots \,&2\\ \vdots &\vdots &\vdots &\ddots &\vdots \\ 2&2&2&\cdots \,&k\end{array}\right)$$

Ehlich (1964b) identified the D-optimal saturated designs, however, such a Wr matrix exists only if r − 2 is a sum of two squares. Thus, the Wr matrix can be expressed as:

$${W}_{r}=\left[\begin{array}{cc}P&Q\\ -{Q}^{{\prime} }&{P}^{{\prime} }\end{array}\right]$$

where P and Q are both circulant matrices. A circulant matrix is a type of square matrix in linear algebra, characterised by the property that each of its row vectors consists of the same elements as the others, but shifted cyclically by one position relative to its predecessor. This circular nature means that circulant matrices can be uniquely identified by specifying only their first row. An example for practical use includes the cases with k = 6, 10. If k = 6, then P = circ(1, 1, − 1) and Q = circ(1, 1, 1), while if k = 10, then P = Q = circ(1, 1, 1, 1, − 1). The D-efficiencies of k = 6 and 10 are 90.48% and 94.09%, respectively. It is important to note that this design achieves the optimal form of \({C}_{2,k}^{* }\), which improves the D-efficiencies of the DCE designs known in the literature. For example, k = 6 and 10 the DCE designs from Singh et al. (2015) have D-efficiency that is approximately 84%.

Case 3

This is the case, in which N is odd and three more than a multiple of 4 (N ≡ 3mod4), where it is not possible to construct orthogonal designs. After modifying the optimal form of the information matrices for the D-optimal design matrix, which had been determined independently by Ehlich (1964a), we present the optimal form of the C-matrix for this case as shown in matrix (9). The formulation of \({C}_{3,k}^{* }\) for this case is an original contribution from the authors of this paper.

$${C}_{3,k}^{* }=\frac{1}{{2}^{k}k}\left(\begin{array}{cc}{A}_{u}&-1\\ -1&{A}_{v}\end{array}\right),$$
(9)

where

$${A}_{j}=\left(\begin{array}{ccccc}{A}_{{r}_{1}}&-1&-1&\cdots \,&-1\\ -1&{A}_{{r}_{2}}&-1&\cdots \,&-1\\ \vdots &\vdots &\vdots &\ddots &\vdots \\ -1&-1&-1&\cdots \,&{A}_{{r}_{j}}\end{array}\right)$$

and

$${A}_{{r}_{k}}=\left(\begin{array}{ccccc}k&3&3&\cdots \,&3\\ 3&k&3&\cdots \,&3\\ \vdots &\vdots &\vdots &\ddots &\vdots \\ 3&3&3&\cdots \,&k\end{array}\right),$$

where rk is the size of \({A}_{{r}_{k}},j=u,v\) is the size of Aj, and s = u + v is the total number of \({A}_{{r}_{k}}\) blocks.

As is commonly understood, the construction methods for this particular case are considered to be the most challenging to obtain among the four. Therefore, they are more fragmented than the previous three cases. So far, no other than K = N = 3 saturated designs have been found that can achieve the optimal form. Note that, for some of these combinations, there may be two optimal values of s for a given scenario, and constructs that can yield designs for these values have been developed. While there are many highly D-efficient saturated designs, only for certain small run sizes, r = 7, 11, 15, and19, D-optimal saturated designs have been confirmed using more specialised procedures. Only two designs are presented here, for run sizes r = 7and11, where other run sizes can be found in the supplementary files of King et al. (2020). Thus, the Wr matrix for r = 7 can be expressed as:

$${W}_{r}=\left[\begin{array}{cc}P{B}_{6,6,3,3;2,1}&-{1}_{6}\\ -{1}_{6}^{{\prime} }&1\end{array}\right],where\,\,P{B}_{6,6,3,3;2,1}=\left[\begin{array}{cccccc}-1&-1&1&-1&1&1\\ 1&-1&-1&1&-1&1\\ 1&1&-1&-1&1&-1\\ -1&1&1&-1&-1&1\\ 1&-1&1&1&-1&-1\\ -1&1&-1&1&1&-1\end{array}\right]\quad$$

PB6,6,3,3;2,1 is the incidence matrix of a Partially Balanced Incomplete Block Design PBIBD with 6 treatments in 6 blocks and 3 treatments per block with each treatment occurring in 3 blocks with λ1 = 2 for the 1st association and λ2 = 1 for the 2nd association, with the 0 entries replaced with − 1’s. Details on how to construct the necessary PBIBD matrices can be found in Street and Street (2006); Toutenburg et al. (2009). This design was first constructed by Williamson (1946), and it gives a D-efficiency of 87.82% for choice pairs when only testing the main effects. This is the highest efficiency in the literature for a DCE design with this parameter (7 attributes and 7 choice sets). While the C-matrix is not in the optimal form of \({C}_{3,k}^{* }\), it still exhibits a block form similar to the optimal form.

For the case of r = 11, three different constructions exist in which D-optimal saturated designs have been confirmed all with the same determinant. The design given here achieves the optimal form of \({C}_{3,k}^{* }\) with the smallest sum of squares of off-diagonal terms (King et al. 2020), while the other two designs can be found at this website http://www.indiana.edu/~maxdet/fullPage.shtmltableTop. Thus, the Wr matrix for r = 11 can be expressed as:

where H2 is the Hadamard matrix of order 2, I2 is the 2 × 2 identity matrix, J2 is the 2 × 2 matrix of all 1’s, and 12 is the 2 × 1 vector of all 1’s. This design was first constructed by Ehlich (1964a), and it gives a D-efficiency of 91.20% for choice pairs when only testing the main effects. This is the highest efficiency in the literature for a DCE design with this parameter (11 attributes and 11 choice sets).

DCEs with more than two-levels symmetric attributes

In the previous section, we discussed four direct constructions of D-optimal two-level paired choice designs for main effects models. This section aims to extend those constructions to be able to generate paired choice designs with more than two-level symmetric attributes and main effects models. One significant issue with the current optimal paired choice designs is the rapid increase in the number of choice sets as the number of attributes k or levels ℓ increases. Such large designs are not really appealing to a practitioner. Alternatively, researchers such as Street and Burgess (2007) and Dey et al. (2017) suggested several optimal design construction methods with manageable numbers of pairs with reasonably high D-efficiencies for the estimation of the main effects. For instance, an orthogonal array of strength two plus a collection of sets of generators given by Burgess and Street (2005), denoted as OA + g, can be used in this case to construct D-optimal symmetric paired choice designs. In particular, it is not always easy to find appropriate generators, especially for practitioners (Großmann and Schwabe, 2015).

Graßhoff et al. (2004) also used a Hr matrix to construct D-optimal symmetric paired choice designs by replacing the coded levels ± 1 in each row with a group of combinations, which leads to a matrix of pairs where each row represents one pair of the choice set in the optimal design. However, their approach, for N ≢ 0 mod 4, generates a matrix with an excessive number of columns to the optimal design. In this paper, those proposed construction methods, mentioned in Section 3 for N ≢ 0 mod 4, are used instead of only Hr matrix. Those methods yield highly D-efficient paired choice designs with fewer choice pairs than those of Graßhoff et al. (2004). Next, we present a straightforward technique for extending those methods to generate paired choice designs with symmetric attributes, as shown in Construction 3.2.

Construction 3.2

For ℓ levels, there are (ℓ − 1)ℓ/2 combinations in pairs (i, j). First, construct a D-optimal paired choice design Dζ,2 = [P1, P2] with N = r choice sets each of k attributes as shown in Construction 3.1. Then, for each pair (i, j), i < j, i, j = 0, 1, …, ℓ − 1, the first choice is T = jP1 + iP2 and the second choice set is F = iP1 + jP2. This gives highly D-efficient paired choice design Dζ,ℓ = [T, F] with N = r(ℓ − 1)ℓ/2 choice sets each of k attributes.

Example 3.2

Assume there are five attributes (k = 5), each with four levels (ℓq = 4), and the levels are represented by 0, 1, 2, and 3. We will demonstrate how our suggested construction methodology can generate a D-optimal paired choice design Dζ,3 for five attributes (k = 5), each with four levels (ℓq = 4) and run size equal to 30. Note that this is a new design and is presented here for the first time.

Following the Construction 3.2, for ℓ = 4 levels, there are (ℓ − 1)ℓ/2 = (4 − 1)4/2 = 6 combinations in pairs {(0, 1), (0, 2), (0, 3), (1, 2), (1, 3)and(2, 3)}. From case 3.1.2, the size of runs for two-level is five r = 5 with five columns. Therefore, the total number N of choice designs in pairs that form the optimal design is equal to the size of weighing design r times the number of combinations in pairs (i, j), calculate as follows:

$$N=r(\ell -1)\ell /2=5(4-1)4/2=30.$$

Table 1, displayed below, represents the D-optimal paired choice design Dζ,3 for five attributes (k = 5), each with four levels (ℓq = 4).

Table 1 Construction of optimal design Dζ for main effects only.
Full size table

This new design has produced a DCE with fewer distinct choice sets than in a DCE constructed using Graßhoff et al. (2004), Street and Burgess (2007), or Demirkale et al. (2013) with 48 choice sets. Our proposed design results in a reduction of 37.5% in the number of choice pairs, while experiencing a 10% decrease in D-efficiency compared to the optimal design.

Case study

We conduct a simulation study and build two DCE designs using two different techniques under the same circumstance to illustrate the advantage of our designs in identifying the main effects with smaller standard errors. The first design, noted as DCE I, corresponds to the methods discussed in Construction of D-optimal choice design, and the other one, noted as DCE II, corresponds to the methods discussed in Singh et al. (2015). To ensure consistency in our case study, we restrict our analysis to designs that involve only two alternatives and consist of six attributes, each of which has only two levels. We report for each design D-efficiencies (denoted by effD), Dz-error (assumed zero priors for all parameters), and Dp-error (assumed non-zero priors for all parameters). Also, we analyse the results of each DCE using the MNL model to compare their parameter estimates. Two models were considered as the true models, with six main effects, as follows:

$${\mu }_{1}=-0.3{x}_{1}-0.2{x}_{2}+0.3{x}_{3}+0.2{x}_{4}+0.2{x}_{5}-0.3{x}_{6}.$$
(10)
$${\mu }_{2}=-0.5{x}_{1}+0.5{x}_{2}-0.5{x}_{3}+0.5{x}_{4}-0.5{x}_{5}+0.5{x}_{6}.$$
(11)

The MNL probability (1) for each choice within each choice set was then calculated. After that, we duplicated each DCE design 250 times to represent 250 respondents. For each of the two DCEs, we then simulated a response choice based on the multinomial distribution using these probabilities.

As an example, Table 2 illustrates a single duplicated simulation for DCE I using the first true model (10). The first column of Table 2 displays the identification number of the choice set, and is referred to as the “choice set" column. The second column displays the identification number of the alternative within each choice set, and each choice set contains two alternatives. The following six columns correspond to the six main effects of DCE I. The probability (1) of each alternative is shown in the second to last column. The last column displays the choice selected, that is, 1 identifies the preferred alternative. The six main effects were incorporated into the model for each of the two DCEs utilising the simulated responses, that is, the choice column in Table 2. We used the ‘mlogit’ package in R (Croissant et al. 2012) to estimate the parameter values. The estimated parameter values along with the corresponding standard errors of the estimates were tabulated in Table 3, based on the actual models.

Table 2 Simulated discrete choice experiment I.
Full size table
Table 3 Comparison of DCE designs.
Full size table

The findings in Table 3 indicate that both DCE designs can effectively identify all the parameters present in both true models, namely (10) and (11). However, the DCE I outperformed all tests, as it gave better results in terms of effD, Dz-error, and Dp-error. The DCE I produces effD of 0.9048, Dz-error of 0.7368 and Dp-error of 0.7883, while DCE II produces effD of 0.8399, Dz-error of 1 and Dp-error of 1.0937. In addition, DCE I gives closer estimates of the true model with smaller standard errors than DCE II for all the parameters. It is worth noting that with an increase in sample size, the parameter estimates are expected to be more accurate, and this is demonstrated in Figure 1, where the standard errors for DCE I and DCE II converge as the sample size increases.

Fig. 1: Standard Errors of the Designs.
figure 1

Comparison of Standard Error for the two different 2-level designs with Fixed Choices. The standard errors are provided for all the designs variables x1 to x6.

Full size image

This simulation study shows that DCE I improve the D-efficient, Dz-error and Dp-error over DCE II, that is, we can have better results in terms of the parameter estimates. As is commonly understood, DCE designs that possess higher D-efficiency aim to maximise differences between attribute levels, whereas DCE designs that exhibit higher Dz-error or Dp-error strive to minimise elements that are likely to appear within the AVC matrices of models that are estimated from data obtained using DCE design.

Discussion

The paper proposes a new method for constructing efficient paired choice designs for main effects models using a “spring balance" approach. This method addresses the issue of generating D-optimal designs when orthogonal designs are not available and practitioners desire optimal designs. Two direct construction methods are presented, both of which begin with the Wr matrix, an D-optimal design derived from the weighing design theory. The methods are particularly useful when designing for situations where orthogonal designs are not available and attributes have only two levels. The methods can also be extended to symmetric attributes with higher levels, resulting in choice designs with fewer possible pairs and high D-efficiencies for main effect estimates.

It is worth noting that construction 3.1 described in this paper for ℓ = 2, known as “foldover”, followed the method of Street and Burgess (2004). However, Street and Burgess (2004) used a regular fractional factorial design of resolution III or higher for only ℓ = 2 to generate choice pairs. Simultaneously, Graßhoff et al. (2004) proposed a different technique using Hadamard matrices, as a case 3.1.1 outlined in this paper where N ≡ 0 mod 4, which is applicable for general ℓ. However, instead of only using Hadamard matrices to construct an optimal paired choice design, we considered other types of the Wr weighing matrices that give fewer possible choice pairs. These types of matrices were grouped into four cases, for each case, we presented the best-known construction methods that construct D-optimal saturated designs for attributes at two levels and main effects models.

As is known, including very large values of k and/or ℓ into the designs does not appear to be very practical. As a result, we limit ourselves to the same k and ℓ values mentioned in Table 2 of Demirkale et al. (2013), whereas construction 3.2 outlined in this paper enables one to construct at larger levels. Note that construction 3.4 of Demirkale et al. (2013) yields choice pairs with the same parameters as those found in Theorem 3 of Graßhoff et al. (2004). The table presented below, named Table 4, shows a collection of D-efficient designs and the corresponding reduction percentage in sample size compared to the optimal design generated by the Hadamard approach, as stated in Theorem 3 of Graßhoff et al. (2004). Such a finding holds significant practical implications, particularly in the context of survey research, where maintaining respondent interest and minimising the number of questions they need to answer are key considerations. By achieving a design with fewer runs while preserving its efficiency, we can ensure that survey respondents remain engaged and motivated, resulting in a more efficient and satisfactory survey experience for all involved parties. Practitioners with the specific goal of testing for variations across respondents or samples have a distinct need for using consistent choice sets across all participants. Consequently, they tend to favour designs that feature a reduced number of choice sets, as this ensures that respondents can efficiently complete all the questions within a reasonable time-frame.

Table 4 A percentage reduction in simple size using weighing designs compared to D-optimal designs with Hadamard structure.
Full size table

As mentioned earlier in this paper, Singh et al. (2015) constructed optimal two-level paired choice designs with distinct choice sets. These are optimal under the main effects model and feasible for any number of choice sets. Their method was based on Hadamard matrices. Their suggestion was to remove columns and rows as needed to get the desirable optimal designs. Their constructions work more effectively in the unsaturated case, where N ≠ K. However, for the saturated case of N = K, where orthogonal designs do not exist, an alternative design may be applied using the same methodology. For example, one can generate the desirable D-optimal design by a different procedure that is known as “spring balance designs”. In general, both constructions define a two-level choice design with the same number of choice sets, however, using this procedure outlined in this paper for the same set of parameters gives better D-efficiency than those found by Singh et al. (2015). For instance, for parameter sets (ℓ = 2, k = 7), Singh et al. (2015) construction produces a design with 84.91% D-efficiency, while using the approach described in this paper produces a design with 87.82% D-efficiency. Also, for some saturated cases, Singh et al. (2015) did not provide such as N = K = 5 and N = K = 9.

For ℓ = 2, Graßhoff et al. (2004) already highlighted that choice experiments can be directly derived from the theory of weighing designs. To our knowledge, there has been no real-world application of these techniques, and no software currently incorporates them directly. This may be due to a lack of summarising of these techniques for runs that are not multiples of four, as well as limited knowledge of these techniques by practitioners since many of them rely on the construction of Hadamard matrices. Therefore, we have collected and presented techniques that can produce D-optimal paired choice designs by modifying the constructions of weighing designs. The designs presented here result in the form of \({C}_{c,k}^{* }\) with a minimal sum of squares of off-diagonal terms.

In addition, we found that extending these techniques to be applied for attributes with larger levels gives the same D-efficiencies as in the two levels. We also noted that even though \({C}_{c,k}^{* }\) is not in the optimal form, it still exhibits a block form similar to the optimal form. Colour cells, shown in Table 4, represent that designs in which the C-matrix is in a block form. This could possibly be due to the fact that all the levels in the two alternatives appear the same number of times in each attribute. Comparing the method proposed in this paper with Graßhoff et al. (2004), there are, on average, 8.33–37.5% fewer choice pairs in the design with reasonably high D-efficiencies of at least 90% except for k = 7, and those designs were 87.82% efficient. In addition, there are only some parameter sets, such (ℓ = 3, k = 7, 10there are, on average, 8.33 or 11) and (ℓ = 5, k = 6 or 11), where the OA + g construction produces an optimal design with a smaller number of choice sets than the number of choice sets constructed in this paper. One important contribution of this paper, in comparison with the results of Dey et al. (2017), is that it is often possible to obtain designs with higher D-efficiencies and at the same time with a less or equal number of choice sets. However, their designs do not have at all a block diagonal information matrix.

Designs with N ≡ 0 mod 4 choice sets are both optimal and orthogonal and that is why a lot of researchers suggest only using this case when searching for good designs. While this is correct, we can still argue for the need for these constructions. First, our design allows up to a 37% reduction in the number of choice pairs at the cost of a maximum 10% loss in D-efficiency relative to an optimal design. Also, as designers, we should not force practitioners to fall into the N ≡ 0 mod 4 case. King et al. (2020) pointed out that “the whole field of optimal design theory is predicated on the belief that experimental designs should tailor to the needs of the practitioner and not the other way around”. An objection that could be raised against this work is that there are many software applications that are able to generate the needed choice sets for N ≢ 0 mod 4 using sophisticated search algorithms, such as the SAS macros and Ngene. As discussed earlier, however, these algorithms do not guarantee optimality over the whole search space, and obtaining a good design may take a very long time. Thus, we suggest that in cases where an optimal choice design exists, as the approach, we have presented here, they can simply incorporate into software to either enhance or even replace algorithms. To illustrate that, we can start with weighing designs instead of random starting designs, this would allow the software to be quicker and more accurate in constructing D-optimal choice designs, especially at a large number of levels and/or attributes. As is known, the quality of the design generated by search algorithms may be influenced by the choice of starting design, such as coordinate exchange algorithm Kuhfeld (2005). We noted that the constructions proposed in this study will, for specific parameter sets, provide D-optimal DCEs with a smaller number of choice sets, as compared to construction methods that have been known in the literature.

Conclusion

Driven by the recognition of the paramount importance of the method of choice experiments in various fields as a powerful tool for understanding people’s preferences and aiding decision-makers in making informed choices, this study sought to construct new paired choice designs under the MNL model for estimating the main effects. Throughout this paper, the constructions were provided for binary attributes using weighing designs and were compared using the D-optimality criterion under the assumption that all alternatives per choice set are equally attractive.

The proposed design techniques can be extended to be applied to situations where the attributes of DCEs have a higher number of levels (ℓ > 2), resulting in designs with the same improved D-efficiencies and sufficiently small sample sizes. We believe that the methods described in this article give small, optimal designs that are at least easy to construct, i.e. can be systematically obtained even without the help of a computer, and have scientific significance and potential application prospects.

A simulation study was also conducted to verify the feasibility of the proposed design in comparison to existing ones. This affirms its potential as a valuable alternative for academics and practitioners in their work. By providing a robust and efficient approach, this study contributes to the growing body of research on paired choice experiments and equips decision-makers with an additional tool to optimise their decision-making processes, while enriching the understanding of human preferences in diverse contexts.