Substitutes for the Non-existent Square Lattice Designs for 36 Varieties

Abstract

Square lattice designs are often used in trials of new varieties of various agricultural crops. However, there are no square lattice designs for 36 varieties in blocks of size six for four or more replicates. Here, we use three different approaches to construct designs for up to eight replicates. All the designs perform well in terms of giving a low average variance of variety contrasts.

Supplementary materials accompanying this paper appear online.

1 Introduction

In variety-testing programmes, later-stage trials can involve multiple replications of up to 100 varieties: see Patterson et al. (1978). Even at a well-run testing centre, variation across the experimental area makes it desirable to group the plots (experimental units) into homogeneous blocks, usually too small to contain all the varieties. As R. A. Fisher wrote in a letter in 1938, “... on any given field agricultural operations, at least for centuries, have followed one of two directions”, so that variability among the plots is well captured by blocking in one or both of these directions, with no need for more complicated spatial correlations: see Fisher et al. (1990, p. 270). Thus, on land which has been farmed for centuries, or where plots cannot be conveniently arranged to allow blocking in two directions (rows and columns), it is reasonable to assume the following model for the yield $Y_\omega$ on plot $\omega$:

$$\begin{aligned} Y_\omega = \tau _{V(\omega )} + \beta _{B(\omega )} + \varepsilon _\omega . \end{aligned}$$

(1)

Here, $V(\omega )$ denotes the variety planted on $\omega$ and $B(\omega )$ denotes the block containing $\omega$. The variety constants $\tau _i$ are the unknown parameters of interest, and the block constants $\beta _j$ are unknown nuisance parameters. The quantities $\varepsilon _\omega$ are independent identically distributed random variables with zero mean and common (unknown) variance $\sigma ^2$. Denote the number of varieties by v.

For management reasons, it is often convenient if the blocks can themselves be grouped into replicates, in such a way that each variety occurs exactly once in each replicate. Such a block design is called resolvable. Let r be the number of replicates.

Yates (1936, 1937) introduced square lattice designs for this purpose. In these, $v=n^2$ for some positive integer n, and each replicate consists of n blocks of n plots. The design is constructed by first listing the varieties in an abstract $n \times n$ square array ${\mathcal {S}}$. The rows of ${\mathcal {S}}$ form the blocks of the first replicate, and the columns of ${\mathcal {S}}$ form the blocks of the second replicate.

If $r>2$, then $r-2$ mutually orthogonal Latin squares ${\mathcal {L}}_1$, ..., ${\mathcal {L}}_{r-2}$ of order n are needed. This is not possible unless $r\le n+1$: see Street and Street (1987, Chapter 6). For replicate i, where $i>2$, superimpose Latin square ${\mathcal {L}}_{i-2}$ on the array ${\mathcal {S}}$: then the n positions where any given letter of ${\mathcal {L}}_{i-2}$ occurs give the set of varieties in one block. Orthogonality implies that each block has one variety in common with each block in each other replicate. Thus, these designs belong to the class of affine resolvable designs defined by Bose (1942), and this construction is a special case of that given by Bailey et al. (1995). Moreover, all pairwise variety concurrences are in $\{0,1\}$, where the concurrence of varieties i and j is the number of blocks in which varieties i and j both occur. If $r=n+1$, then all pairwise concurrences are equal to 1 and so the design is balanced.

Equireplicate incomplete block designs are typically assessed using the A-criterion: see Shah and Sinha (1989). Denote by $\varvec{\Lambda }$ the $v \times v$ concurrence matrix: its (i, j)-entry is equal to the concurrence of varieties i and j, which is r when $i=j$. The scaled information matrix is $\mathbf {I}-(rk)^{-1}\varvec{\Lambda }$, where $\mathbf {I}$ is the identity matrix and k is the block size. The constant vectors are in the null space of this matrix. The eigenvalues for the other eigenvectors, counting multiplicities, are the canonical efficiency factors. Denote their harmonic mean as A. (John and Williams (1995) call this E, but many authors, including several cited in Sect. 5, use E to denote the smallest canonical efficiency factor.) Under model (1), the average variance of the estimator of a difference $\tau _i -\tau _j$ between two distinct varieties is $2\sigma ^2/(rA)$. If the variance in an ideal design with the same number of plots but no need for blocking is $\sigma _0^2$, then this average variance would be $2\sigma _0^2/r$. Hence, $A\le 1$, and a design maximizing A, for given values of v, r and k, is called A-optimal.

Cheng and Bailey (1991) showed that, if $r\le n+1$, then square lattice designs are A-optimal (even over non-resolvable designs).

The class of square lattice designs also has some practical advantages. Adding or removing a replicate gives another square lattice design, which permits last-minute changes in the planning stage. It also means that, if a whole replicate is lost (for example, if heavy rain during harvest flattens the plants in the last replicate), then the remaining design is A-optimal for its size.

If $n\in \{2,3,4,5,7,8,9\}$, then there is a complete set of $n-1$ mutually orthogonal Latin squares of order n: see Street and Street (1987, Chapter 6). These give square lattice designs for $n^2$ varieties in rn blocks of size n for $r\in \{2, \ldots , n+1\}$. However, there is not even a pair of mutually orthogonal Latin squares of order 6, so square lattice designs for 36 varieties are available for two or three replicates only. This gap in the catalogue of good resolvable block designs is pointed out in many books: for example, Cochran and Cox (1957) and John and Williams (1995).

Patterson and Williams (1976a) used computer search to find an efficient resolvable design for 36 varieties in four replicates of blocks of size six. All pairwise variety concurrences are in $\{0,1,2\}$. It has $A =0.836$, which compares well with the unachievable upper bound of 0.840 for the non-existent square lattice design.

In this paper, we present three new methods of constructing efficient resolvable block designs for 36 varieties in 6r blocks of size six, for $r\in \{4, \ldots , 8\}$. These methods are in Sects. 3–5. In each case, Supplementary Material gives the design as a plain text file, which can easily be imported into a spreadsheet or statistical software.

The concurrence graph of an incomplete block design has a vertex for each variety. The number of edges between vertices i and j is equal to the concurrence of varieties i and j. Although the methods in Sects. 3–5 are very different, their designs for eight replicates all have the same concurrence graph, which we describe in Sect. 2. The final sections compare the new designs and discuss further work.

2 The Sylvester Graph

The Sylvester graph $\Sigma$ is a graph on 36 vertices with valency 5. See Brouwer et al. (1989) and Bailey (2019). Here, we give enough information about it to show how the designs in Sect. 3 are constructed, using the approach in Cameron and van Lint (1991, Chapter 6).

Consider the complete graph $K_6$. It has a set $\mathcal {A}$ of six vertices, labelled 1–6. There is an edge between every pair of distinct vertices. Let ${\mathcal {B}}$ be this set of edges. A 1-factor is a partition of $\mathcal {A}$ into three edges (subsets of size two). For example, one 1-factor consists of the pairs $\{1,2\}$, $\{3,6\}$ and $\{4,5\}$. For brevity, we write this in the slightly non-standard way 12|36|45 in Table 1. Let ${\mathcal {C}}$ be the set of 1-factors. A 1-factorization is a set of five elements of ${\mathcal {C}}$ with the property that each edge is contained in just one of them. For example, $d_1$ in Table 1 is a 1-factorization; here, we use the symbol || to separate the five 1-factors in $d_1$. Let ${\mathcal {D}}$ be the set of 1-factorizations.

It was shown by Sylvester (1844) that

• (a) $\left| \mathcal {A} \right| = \left| {\mathcal {D}} \right| = 6$ and $\left| {\mathcal {B}} \right| = \left| {\mathcal {C}} \right| = 15$;

• (b) any two elements of ${\mathcal {D}}$ share exactly one element of ${\mathcal {C}}$.

(Sylvester used the terms duads, synthemes and synthematic totals for edges, 1-factors and 1-factorizations respectively.)

Table 1 The set ${\mathcal {D}}$ of six 1-factorizations

Table 1 shows the six 1-factorizations, labelled as $d_1$ to $d_6$. Each of these can be considered as a schedule for a tournament involving six teams which takes place over five weekends so that each pair of teams meets exactly once.

Now we construct the Sylvester graph $\Sigma$ as follows. The vertex set consists of the cells of the $6\times 6$ square array ${\mathcal {S}}$ with rows labelled by the elements of $\mathcal {A}$ and columns labelled by the elements of ${\mathcal {D}}$. Given distinct $d_i$ and $d_j$ in ${\mathcal {D}}$, the unique 1-factor they have in common defines six edges in $\Sigma$, each joining a vertex in column $d_i$ to a vertex in column $d_j$. For example, $d_3$ and $d_4$ have the one-factor 12|34|56 in common, so we put edges between the vertices (1, 3) and (2, 4), (2, 3) and (1, 4), (3, 3) and (4, 4), ..., and (6, 3) and (5, 4), as shown in Fig. 1a.

Fig. 1

Some edges in the Sylvester graph

Thus, each vertex in $\Sigma$ is joined to five other vertices, one in each other row and one in each other column. Figure 1b shows the five edges at the vertex $a=(3,d_3)$. We shall call this set of six vertices the starfish centred at a.

It can be shown that the graph $\Sigma$ has no triangles or quadrilaterals. One consequence of this is that, given any vertex, the vertices at distances one and two from it in the graph are precisely all the other vertices in different rows and different columns.

Denote by $\mathop {\mathbf {Adj}}(\Sigma )$ the adjacency matrix of the graph $\Sigma$. We call a block design for 36 varieties in 48 blocks of size six a Sylvester design if there is a permutation of the varieties that takes the concurrence matrix for the design to $7\mathbf {I} + \mathbf {J} + \mathop {\mathbf {Adj}}(\Sigma )$, where $\mathbf {J}$ is the all-1 matrix. This means that the concurrences are 2 on each edge of $\Sigma$ and 1 for every other pair of varieties. In particular, a Sylvester design is a regular graph design, as defined by John and Mitchell (1977), where the graph is the Sylvester graph $\Sigma$.

Two block designs (for the same set of varieties) are isomorphic if one can be converted into the other by a permutation of varieties and a permutation of blocks. An isomorphism from a block design to itself is called an automorphism.

If two block designs are isomorphic, then their canonical efficiency factors are the same and their automorphism groups have the same order, but neither converse need be true. In particular, all Sylvester designs have the same canonical efficiency factors and hence the same value of the A-criterion, but they are not all isomorphic. Of the three that we construct in this paper, no two are isomorphic, as we discuss in Sect. 6.

3 New Designs Constructed from the Sylvester Graph

3.1 The New Designs

Figure 1a shows that, if we choose two different vertices in the same column, then their starfish will not overlap. Thus, each column gives what we call a galaxy of six starfish, which together include every vertex just once. In other words, we can think of a galaxy as a Latin square of order 6. Figure 2 shows the galaxy of starfish centred on vertices in column $d_3$. Just as with a square lattice design, we can identify the varieties with the 36 vertices and use this Latin square to construct a single replicate of six blocks of size six.

Fig. 2

The galaxy of six starfish defined by column $d_3$: the centre of each starfish is marked $*$ and the Latin letters show vertices in the same starfish

However, unlike in a square lattice design, the Latin squares defined by different columns are not orthogonal to each other. If two vertices are joined by an edge, then they both occur in the two starfish which they define. Thus, if we use galaxies of starfish from two or more columns, then some variety concurrences will be bigger than one. On the other hand, a consequence of the lack of triangles and quadrilaterals is that, if two or more galaxies are used as replicates, then there is no other way that two varieties can concur in more than one block.

We therefore propose the following resolvable designs. The design $\Gamma _r$ consists of the galaxies of starfish from r columns, where $0\le r\le 6$; $\Gamma _0$ (a design with no blocks) and $\Gamma _1$ (a disconnected design) are used in the following constructions, but are not themselves suitable designs. For $1\le r \le 7$, the design $\Gamma ^{\mathrm {R}}_r$ consists of $\Gamma _{r-1}$ together with another replicate whose blocks are the rows of ${\mathcal {S}}$, while the design $\Gamma ^{\mathrm {C}}_r$ consists of $\Gamma _{r-1}$ together with another replicate whose blocks are the columns of ${\mathcal {S}}$. The design $\Gamma ^{\mathrm {C}}_7$ was used by Bailey et al. (2018). For $2\le r \le 8$, the design $\Gamma ^{\mathrm {RC}}_r$ consists of $\Gamma ^{\mathrm {R}}_{r-1}$ together with another replicate whose blocks are the columns of ${\mathcal {S}}$. In particular, $\Gamma ^{\mathrm {RC}}_2$ is the square lattice design whose blocks are the rows and columns.

The automorphisms of $\Sigma$ consist of the symmetric group $S_6$ acting simultaneously on rows and columns of the array, as well as a further involution transposing it. It follows that, for a design consisting of m galaxies (possibly with rows, and possibly with columns), it does not matter which m galaxies we choose.

When $r=2$, then $\Gamma ^{\mathrm {RC}}_2$, $\Gamma ^{\mathrm {R}}_2$ and $\Gamma ^{\mathrm {C}}_2$ are square lattice designs and hence A-optimal, but $\Gamma _2$ is not. When $r=3$, then $\Gamma ^{\mathrm {RC}}_3$ is a square lattice design and hence A-optimal, but none of the others is. When $r\ge 4$, then none of the designs is a square lattice design, so we need to calculate the canonical efficiency factors, and hence A.

As discussed in Sect. 6.1, for each value of r the designs $\Gamma ^{\mathrm {R}}_r$ and $\Gamma ^{\mathrm {C}}_r$ have the same canonical efficiency factors, so we do not include $\Gamma ^{\mathrm {R}}_r$ in further comparisons.

Another useful consequence of the lack of triangles and quadrilaterals in $\Sigma$ is that the relations ‘same row’, ‘same column’, ‘joined in the graph’ and ‘other’ form a 4-class association scheme on the set of vertices. It follows that $\Gamma _6$, $\Gamma ^{\mathrm {C}}_7$ and $\Gamma ^{\mathrm {RC}}_8$ are partially balanced with respect to this association scheme, and so their canonical efficiency factors can be calculated using the methods in Bailey (2004). Table 2 shows the results. In fact, $\Gamma _6$ and $\Gamma ^{\mathrm {RC}}_8$ are also partially balanced with respect to the 3-class association scheme obtained by merging the classes ‘same row’ and ‘same column’. Moreover, $\Gamma ^{\mathrm {RC}}_8$ is a Sylvester design.

Table 2 Canonical efficiency factors and values of the A-criterion for the partially balanced designs

For all the other designs, we calculated A as an exact rational number by using the DESIGN package (Soicher 2019) in GAP (The GAP Group 2019). The method used for such exact calculation of block design efficiency measures is described in Appendix B of Soicher (2013a). These results were verified by using GAP to find the exact Moore–Penrose inverse of the scaled information matrix, calculate its trace, divide this by 35 and then invert this as an exact rational number.

Table 3 shows the results to four decimal places. This shows that, apart from the square lattice designs $\Gamma ^{\mathrm {RC}}_2$ and $\Gamma ^{\mathrm {C}}_2$, the design $\Gamma ^{\mathrm {RC}}_r$ always beats $\Gamma ^{\mathrm {C}}_r$ and $\Gamma _r$. Moreover, $\Gamma ^{\mathrm {RC}}_4$ does very slightly better than the design found by Patterson and Williams (1976a).

The final column of Table 3 shows the value of A for a square lattice design. This exists only for $r=2$ and $r=3$, when $\Gamma ^{\mathrm {RC}}_r$ is an example. For $4 \le r \le 7$, there is no square lattice design, so the value shown gives an unachievable upper bound; in every case, A for $\Gamma ^{\mathrm {RC}}_r$ comes very close to this.

Table 3 Values of the A-criterion for the designs in Sects. 3–5

3.2 Using These New Designs

Figure 3 shows the design $\Gamma ^{\mathrm {RC}}_8$, starting with the replicates defined by columns and rows. The varieties are numbered 1 to 6 in row 1 of Fig. 1a, then 7 to 12 in row 2 and so on. For a design with r replicates, use the first two replicates here and any $r-2$ of the others. A plain text version of $\Gamma ^{\mathrm {RC}}_8$ is available in Supplementary Material.

Fig. 3

The design $\Gamma ^\mathrm {RC}_8$: columns are blocks

4 New Designs Found by Computer Search

The computer search algorithm used by Patterson and Williams (1976a) to obtain the efficient design for 36 varieties has been extensively developed, both in the range of design types that can be constructed and in the algorithmic approach. Significant improvements in computer speed have also facilitated search procedures. CycDesigN Version 6.0 (VSNI 2016) is a computer package for the generation of optimal or near-optimal experimental designs, as measured by the A-criterion. The package has been written in Visual C++ and uses simulated annealing in the design search process. CycDesigN can be used to construct efficient resolvable block designs for 36 varieties in blocks of size six with a range of values for r. Hence, running CycDesigN separately for $r=3$ through 8 gives designs $\Theta _r$ with the results in Table 3. For $r=3$, ..., 7, the design $\Theta _r$ has pairwise variety concurrences in $\{0,1,2\}$, while $\Theta _8$ has concurrences in $\{1,2\}$. In fact, $\Theta _8$ is a Sylvester design. For $r=4$, the improvement from Patterson and Williams (1976a), namely $A=0.836$ to that in Table 3 ($A=0.839$), is representative of overall developments in computer speed and search methods throughout the years.

Because $\Theta _r$ is not constructed by simply omitting a replicate from $\Theta _{r+1}$, we do not show all these designs here. Figure 4 shows $\Theta _8$. Plain text versions for $r=4$, ..., 8 are available in Supplementary Material.

Fig. 4

The design $\Theta _8$: columns are blocks

5 New Designs Constructed from Semi-Latin Squares

Let $\Delta$ be a resolvable block design for 36 varieties in 6r blocks of size 6. Its dual design $\Delta '$ is obtained by interchanging the roles of blocks and varieties, so it has 6r varieties in 36 blocks of size r. If the varieties of $\Delta$ are identified with cells of the $6\times 6$ square array ${\mathcal {S}}$, then the blocks of $\Delta '$ also form a $6 \times 6$ square array. When each variety in $\Delta '$ occurs exactly once in each row and once in each column, then $\Delta '$ is called a $(6\times 6)/r$ semi-Latin square: see Yates (1935) and Preece and Freeman (1983). The term orthogonal multi-array is also used: see Brickell (1984). One way of constructing such a semi-Latin square is to superpose r Latin squares with disjoint alphabets. Not all semi-Latin squares arise in this way, but the resolvability of $\Delta$ forces the r replicates to be a collection of r Latin squares when $\Delta '$ is a semi-Latin square.

Let $A'$ be the A-criterion for $\Delta '$. Roy (1958) proved that

$$\begin{aligned} \frac{35}{A} = 6(6-r) + \frac{(6r-1)}{A'}. \end{aligned}$$

(2)

Hence, $\Delta$ is A-optimal if and only if $\Delta '$ is A-optimal, as Patterson and Williams (1976b) showed.

Highly efficient $(6\times 6)/r$ semi-Latin squares have been found by Brickell (1984), Bailey (1990, 1997), Bailey and Royle (1997) and Soicher (2012a, b, 2013a, b). In most cases, their duals are not resolvable. However, Soicher (2013a, Section 6) gives an efficient $(6\times 6)/6$ semi-Latin square made by superposing six Latin squares labelled $L_1$,..., $L_6$. These Latin squares can be used to construct our designs, just as the galaxies in Sect. 3.

For $0\le r \le 6$, denote by $\Delta _r$ the design for 36 varieties in 6r blocks of size six given by the Latin squares $L_1, \ldots , L_r$; $\Delta _0$ is a design with no blocks, but, for $0 < r\le 6$, $\Delta _r$ is the dual of the design called $X_r$ by Soicher (2013a). Table 2 of Soicher (2013a) gives $A'$ for $X_2$, ..., $X_6$: from this, A can be calculated from Eq. (2).

As in Sect. 3, we can add to $\Delta _{r-1}$ another replicate whose blocks are the rows of ${\mathcal {S}}$, to obtain $\Delta _r^\mathrm {R}$, or we can add another replicate whose blocks are the columns of ${\mathcal {S}}$, to obtain $\Delta _r^\mathrm {C}$. Adding both of these extra replicates to $\Delta _{r-2}$ gives $\Delta _r^{\mathrm {RC}}$.

Unlike the situation in Sect. 3, choosing a different r-subset of $\{L_1,\ldots ,L_6\}$ may give a design with a value of A different from that for $\Delta _r$, but computation shows that, in every case, the highest value of the A-criterion arises from taking $\{L_1,\ldots ,L_r\}$ as our r-subset.

Table 3 shows the values of the A-criterion for $\Delta _r$, $\Delta ^\mathrm {C}_r$ and $\Delta ^\mathrm {RC}_r$, calculated exactly using the DESIGN package and rounded to four decimal places. We found that, for each $r=2,\ldots ,7$, the canonical efficiency factors of $\Delta _r^\mathrm {R}$ are the same as those of $\Delta _r^\mathrm {C}$, and so their A-values are the same. Note that, just as in Sect. 3, $\Delta _r^{\mathrm {RC}}$ always beats $\Delta _r^\mathrm {R}$, $\Delta _r^\mathrm {C}$ and $\Delta _r$, apart from the fact that $\Delta ^\mathrm {RC}_2$, $\Delta ^\mathrm {R}_2$ and $\Delta ^\mathrm {C}_2$ are all square lattice designs. Note also that, to four decimal places of the A-criterion, $\Delta ^\mathrm {RC}_r$ is always at least as good as $\Gamma ^\mathrm {RC}_r$, and sometimes better.

Figure 5 shows the design $\Delta ^{\mathrm {RC}}_8$, with the replicates defined in the order columns, rows, $L_1$, ..., $L_6$. The varieties are numbered 1 to 6 in row 1, then 7 to 12 in row 2 and so on. A plain text version of this design is available in Supplementary Material. For a design with r replicates, use the first r of the given replicates.

Fig. 5

The design $\Delta ^\mathrm {RC}_8$: columns are blocks

Although the design $X_6$ of Soicher (2013a) was not constructed using the Sylvester graph, it turns out that $\Delta ^\mathrm {RC}_8$ is a Sylvester design and so has the same canonical efficiency factors as $\Gamma _8^\mathrm {RC}$ and $\Theta _8$.

6 Comparison of Designs

6.1 Isomorphism

We can determine the automorphism group of a block design and check block design isomorphism using the DESIGN package (Soicher 2019). We can also use this package to check whether two block designs have the same canonical efficiency factors. Extended examples of the use of the DESIGN package for the construction, classification and analysis of block designs are given in Soicher (2013b).

For $r=8$, we have checked that the designs given in Sects. 3–5 are all Sylvester designs, so they all have the same canonical efficiency factors. However, $\Gamma ^{\mathrm {RC}}_8$, $\Theta _8$ and $\Delta ^{\mathrm {RC}}_8$ have automorphism groups of order 1440, 1 and 144, respectively, so no two of these designs are isomorphic.

Now, the square lattice design $\Gamma ^{\mathrm {R}}_2$ is isomorphic to $\Gamma ^{\mathrm {C}}_2$, and we found, using the DESIGN package, that $\Gamma ^{\mathrm {R}}_7$ is isomorphic to $\Gamma ^{\mathrm {C}}_7$. For $3 \le r \le 6$, it turns out that the designs $\Gamma ^{\mathrm {R}}_r$ and $\Gamma ^{\mathrm {C}}_r$ are not isomorphic, but they have the same canonical efficiency factors.

We found that, for $r=2,3,5,7$, $\Delta ^{\mathrm {R}}_r$ is isomorphic to $\Delta ^{\mathrm {C}}_r$. For $r=4,6$, the designs $\Delta ^{\mathrm {R}}_r$ and $\Delta ^{\mathrm {C}}_r$ are not isomorphic, but they have the same canonical efficiency factors. We do not yet know a theoretical reason for this.

Further, although they are the same up to four decimal places, we found that, to seven decimal places, the value of the A-criterion for $\Gamma _5$ is 0.8382815, but for $\Delta _5$ this value is 0.8382679. Similarly, up to seven decimal places, the A-value for $\Gamma _6^\mathrm {C}$ is 0.8472622, but for $\Delta _6^\mathrm {C}$ this value is 0.8472563. For $\Gamma _7^\mathrm {RC}$, the A-value to seven decimal places is 0.8527641, but for $\Delta _7^\mathrm {RC}$ this value is 0.8527611. Designs $\Gamma _6$ and $\Delta _6$ are not isomorphic, but have the same canonical efficiency factors. The same holds for $\Gamma _7^\mathrm {C}$ and $\Delta _7^\mathrm {C}$, and, as we have already noted, for $\Gamma _8^\mathrm {RC}$ and $\Delta _8^\mathrm {RC}$.

We also found that $\Theta _4$ and $\Delta _4^\mathrm {RC}$ have the same canonical efficiency factors, but there is no permutation of varieties taking one concurrence matrix into the other.

6.2 Robustness

If a replicate is lost from one of the designs in Sect. 3, then the remaining design is also in Table 3. For example, if the original design is $\Gamma ^{\mathrm {RC}}_5$ then the loss of a replicate leaves $\Gamma ^{\mathrm {RC}}_4$, with $A=0.8380$, in three cases out of five; the other two cases leave $\Gamma ^{\mathrm {R}}_4$ or $\Gamma ^{\mathrm {C}}_4$, both with $A=0.8341$. The average efficiency of the remaining design is 0.8364, while the worst case is 0.8341.

If a replicate is lost from $\Gamma ^{\mathrm {RC}}_8$, then $A=0.8528$ in six cases and $A=0.8507$ in two cases: the average is 0.8522390 and the worst case is 0.8507. Losing a replicate from $\Theta _8$ gives eight different values for A, with exactly the same maximum and minimum as those just given; now the average is 0.8522389.

If a replicate is lost from one of the designs in Sects. 4 and 5, the remaining design need not be in Table 3. However, for $4\le r \le 8$ we have calculated the worst-case value and the average value of the A-criterion when a single replicate is lost from $\Gamma _r^{\mathrm {RC}}$, $\Theta _r$ or $\Delta _r^{\mathrm {RC}}$. Table 4 shows the results, using sufficient decimal places in each column to show when two values are different. For $r=4$, the worst case for $\Theta _4$ is obtained by deleting the first or fourth replicate, and the worst case for $\Delta ^{\mathrm {RC}}_4$ is obtained by deleting the first or second replicate: all four resulting designs are pairwise isomorphic.

Table 4 Worst-case and average values of the A-criterion if a replicate is lost from the design shown

7 Discussion

The authors were very surprised to find that, for $r=8$, the three very different approaches all produced Sylvester designs. This leads us to conjecture that Sylvester designs are A-optimal. Further evidence for this is that, while the first method started from the Sylvester graph, neither of the others did; indeed, the second method used numerical optimization.

CycDesigN also calculates upper bounds for the A-criterion. For $r\le 7$, these are same as those shown in the final column of Table 3. For $r=8$, it gives an upper bound of 0.854931, compared to the A-criterion for all the Sylvester designs, which is equal to 0.854929 to six decimal places. The proximity of these values gives further support to the conjecture that the Sylvester designs are A-optimal.

For a given value of r, which design should be used in practice? Table 3 shows that the A-values for $\Gamma _r^{\mathrm {RC}}$, $\Theta _r$ and $\Delta _r^{\mathrm {RC}}$ are extremely close, but $\Theta _r$  is always at least as good as the other two. If the user is concerned about the possible loss of one replicate, then Table 4 shows that $\Theta _r$ is still at least as good as the other two when $r\le 7$, but $\Gamma _8^{\mathrm {RC}}$ might be preferred to $\Theta _8$.

If the 36 varieties are replaced by 36 treatments consisting of all combinations of two factors with six levels each, and $r\le 6$, then levels of these two factors can be identified with the blocks of any two replicates which together form a square lattice design. For the designs in Sects. 3 and 5, these are either rows and columns, or one of rows and columns combined with any other replicate. Table 3 shows that the second possibility gives higher values for the A-criterion.