Models and partition of variance for quantitative trait loci with epistasis and linkage disequilibrium

Abstract

Background

A genetic model about quantitative trait loci (QTL) provides a basis to interpret the genetic basis of quantitative traits in a study population, such as additive, dominance and epistatic effects of QTL and the partition of genetic variance. The standard quantitative genetics model is based on the least squares partition of genetic effects and also genetic variance in an equilibrium population. However, over years many specialized QTL models have also been proposed for applications in some specific populations. How are these models related? How to analyze and partition a QTL model and genetic variance when both epistasis and linkage disequilibrium are considered?

Results

Starting from the classical description of Cockerham genetic model, we first represent the model in a multiple regression setting by using indicator variables to describe the segregation of QTL alleles. In this setting, the definition of additive, dominance and epistatic effects of QTL and the basis for the partition of genetic variance are elaborated. We then build the connection between this general genetic model and a few specialized models (a haploid model, a diploid F₂ model and a general two-allele model), and derive the genetic effects and partition of genetic variance for multiple QTL with epistasis and linkage disequilibrium for these specialized models.

Conclusion

In this paper, we study extensively the composition and property of the genetic model parameters, such as genetic effects and partition of genetic variance, when both epistasis and linkage disequilibrium are considered. This is the first time that both epistasis and linkage disequilibrium are considered in modeling multiple QTL. This analysis would help us to understand the structure of genetic parameters and relationship of various genetic quantities, such as allelic frequencies and linkage disequilibrium, on the definition of genetic effects, and will also help us to understand and properly interpret estimates of the genetic effects and variance components in a QTL mapping experiment.

Background

Modeling quantitative trait loci (QTL) started with Yule [1, 2] and Pearson [3] (see [4, 5] for the early history of quantitative genetics). However, it was Fisher [6] who laid the firm foundation for quantitative genetics. Fisher defined gene effects (additive, dominance and epistatic effects) based on the partition of genetic variance. He partitioned the genetic variance into a portion due to additive effects (averaged allelic substitution effects), a portion due to dominance effects (allelic interactions), and a portion due to epistatic effects (non-allelic interactions) of genes. He then studied the correlation between relatives using the model. Cockerham [7] used the orthogonal contrasts to redefine the additive and dominance effects of QTL and, by extending the contrasts to include epistatic effects, he partitioned the epistatic variance of two loci into those due to additive × additive, additive × dominance, dominance × additive and dominance × dominance effects of QTL. Cockerham then generalized the model to multiple loci. This was further generalized by Kempthorne [8, 9] to multiple alleles. This model has been used as the basis for studying quantitative genetics ever since.

However, over years, many specialized models have also been proposed. Some are just special cases of the general genetic model and some are simplified variants tailored for particular applications or interpretations. With the propagation of numerous quantitative genetic models, there have also been some confusions in literature on the definition and interpretation of additive, dominance and epistatic effects of QTL and their relationship to the partition of genetic variance. Also, there has never been a study that considers both epistasis and linkage disequilibrium in the partition of genetic variance for multiple QTL. In this paper, we try to build the connection between the general genetic model and a few other commonly used genetic models to clarify the basis for the interpretation of different genetic models.

We start with an introduction of the genetic model as expressed in [7] in the context of variance components. Then by introducing an indicator variable for each QTL allele, we represent the model in a multiple regression setting and examine the definition and meaning of the genetic effects (additive, dominance and epistatic effects) of QTL and partition of the genetic variance in an equilibrium population and also in a disequilibrium population. Most of previous studies on modeling QTL discuss epistasis only in reference to an equilibrium population. An examination of properties of a model with both epistasis and linkage disequilibrium is important for QTL analysis in both experimental and natural populations. This is another goal of the paper and is studied in great detail here. We discuss a few reduced models used for QTL analysis, such as backcross model (essentially a haploid model) and F₂ model. We also give details for a general two-allele model which may be useful for studying the genetic architecture in a natural population using single nucleotide polymorphisms (SNPs).

Previously, in [10], we compared F₂ model and the general two-allele model with another commonly used genetic model, called F_∞ model. By specifying the basis of definition for each model, we compared the properties of these models in the estimation and interpretation of QTL effects including epistasis and discussed a few potential problems of using F_∞ model in a segregating population for QTL analysis. Similarly, we also compared these models with another model proposed by Cheverud [11, 12].

An important result of [10] is that the genetic effects defined in reference to an equilibrium population also apply to a disequilibrium population. The partial regression coefficients, that define the genetic effects in a disequilibrium population, equal to the simple regression coefficients in a corresponding equilibrium population – the usual basis to define and interpret a genetic effect including an epistatic effect. Hardy-Weinberg and linkage disequilibria only introduce covariances between different genetic effects. With this result, in this paper our discussion on epistasis and linkage disequilibrium is focused on the partition and composition of genetic variances and covariances between different genetic effects in different populations.

Results

The genetic model

A general genetic model for the partition of genetic variance (particularly epistatic variance) in a random mating population was first given by Cockerham [7, 13] and extended to multiple alleles by Kempthorne [8, 9], following the basic genetic model formulated by Fisher [6]. The model for two loci A and B with multiple alleles was expressed as follows

$\begin{array}{c}{G}_{jl}^{ik}=\mu +{\alpha }^i+{\alpha }_j+{\delta }_j^i+{\beta }^k+{\beta }_l+{\gamma }_l^k+({\alpha }^i{\beta }^k)\\ +({\alpha }^i{\beta }_l)+({\alpha }_j{\beta }^k)+({\alpha }_j{\beta }_l)+({\alpha }^i{\gamma }_l^k)\\ +({\alpha }_j{\gamma }_l^k)+({\delta }_j^i{\beta }^k)+({\delta }_j^i{\beta }_l)+({\delta }_j^i{\gamma }_l^k)\end{array}\left(1\right)$

where the genotypic value $G_{jl}^{ik}$ is the expected phenotype of an individual carrying alleles A _i , A _j , B _k , and B _l with phased genotype A _i B _k /A _j B _l formed by the union of a paternal gamete A _i B _k and a maternal gamete A _j B _l . The model partitions the total genotypic value into a number of genetic effects which include additive effects of each allele (α's and β's), dominance effects between two alleles at each locus (δ's and γ's), additive × additive interactions between two alleles at two loci ((αβ)'s), additive × dominance interactions involving three alleles ((αγ)'s and (δβ)'s), and dominance × dominance interaction involving all four alleles ((δγ)'s).

As an ANOVA model, it is known that not all the parameters in model (1) are estimable. A number of constraint conditions on these parameters are therefore needed. Let pⁱ, q^kdenote allelic frequencies for alleles on paternal gametes, and p _j , q _l allelic frequencies for alleles on maternal gametes. It is usually assumed that a weighted summation of genetic effects is zero over any index for each genetic component as a deviation from the mean. Some examples are

$\begin{array}{l}{\displaystyle \sum _i{p}^i{\alpha }^i=0,{\displaystyle \sum _i{p}^i{\delta }_j^i=0,{\displaystyle \sum _j{p}_j{\delta }_j^i=0,}}}\hfill \\ {\displaystyle \sum _i{p}^i({\alpha }^i{\beta }^k)=0,}{\displaystyle \sum _k{p}^k({\alpha }^i{\beta }^k)=0,}\hfill \\ {\displaystyle \sum _j{p}_j({\delta }_j^i{\beta }^k)=0,{\displaystyle \sum _k{q}^k({\delta }_j^i{\beta }^k)=0,}}\cdots \cdots \hfill \end{array}\left(2\right)$

Under the assumption of random mating and linkage equilibrium and allowing for different allelic frequencies in paternal and maternal gametes, the mean and genetic effects can be expressed as follows based on the least squares principle:

$\begin{array}{l}\mu ={\displaystyle \sum _{i,j,k,l}{p}^i{p}_j{q}^k{q}_l{G}_{jl}^{ik}}\left(3\right)\\ {\alpha }^i=G_{\mathrm{..}}^{i.}-G_{\mathrm{..}}^{\mathrm{..}},{\beta }^k=G_{\mathrm{..}}^{.k}-G_{\mathrm{..}}^{\mathrm{..}},\\ {\alpha }_j=G_{j.}^{\mathrm{..}}-G_{\mathrm{..}}^{\mathrm{..}},{\beta }_l=G_{.l}^{\mathrm{..}}-G_{\mathrm{..}}^{\mathrm{..}},\\ {\delta }_j^i=G_{j.}^{i.}-G_{\mathrm{..}}^{i.}-G_{j.}^{\mathrm{..}}+G_{\mathrm{..}}^{\mathrm{..}},\\ {\gamma }_l^k=G_{.l}^{.k}-G_{\mathrm{..}}^{.k}-G_{.l}^{\mathrm{..}}+G_{\mathrm{..}}^{\mathrm{..}},\\ ({\alpha }^i{\beta }^k)=G_{\mathrm{..}}^{ik}-G_{\mathrm{..}}^{i.}-G_{\mathrm{..}}^{.k}+G_{\mathrm{..}}^{\mathrm{..}},\\ ({\alpha }^i{\beta }_l)=G_{.l}^{i.}-G_{\mathrm{..}}^{i.}-G_{.l}^{\mathrm{..}}+G_{\mathrm{..}}^{\mathrm{..}},\\ ({\alpha }_j{\beta }^k)=G_{j.}^{.k}-G_{j.}^{\mathrm{..}}-G_{\mathrm{..}}^{.k}+G_{\mathrm{..}}^{\mathrm{..}},\\ ({\alpha }_j{\beta }_l)=G_{jl}^{\mathrm{..}}-G_{j.}^{\mathrm{..}}-G_{.l}^{\mathrm{..}}+G_{\mathrm{..}}^{\mathrm{..}},\\ ({\alpha }^i{\gamma }_l^k)=G_{.l}^{ik}-G_{\mathrm{..}}^{ik}-G_{.l}^{i.}-G_{.l}^{.k}+G_{\mathrm{..}}^{i.}+G_{\mathrm{..}}^{.k}\\ +G_{.l}^{\mathrm{..}}-G_{\mathrm{..}}^{\mathrm{..}},\\ ({\alpha }_j{\gamma }_l^k)=G_{jl}^{.k}-G_{j.}^{.k}-G_{.l}^{.k}-G_{jl}^{\mathrm{..}}+G_{j.}^{\mathrm{..}}+G_{\mathrm{..}}^{.k}\\ +G_{.l}^{\mathrm{..}}-G_{\mathrm{..}}^{\mathrm{..}},\\ ({\delta }_j^i{\beta }^k)=G_{j.}^{ik}-G_{\mathrm{..}}^{ik}-G_{j.}^{i.}-G_{j.}^{.k}+G_{\mathrm{..}}^{i.}+G_{\mathrm{..}}^{.k}\\ +G_{j.}^{\mathrm{..}}-G_{\mathrm{..}}^{\mathrm{..}},\\ ({\delta }_j^i{\beta }_l)=G_{jl}^{i.}-G_{j.}^{i.}-G_{.l}^{i.}-G_{jl}^{\mathrm{..}}+G_{\mathrm{..}}^{i.}+G_{j.}^{\mathrm{..}}\\ +G_{.l}^{\mathrm{..}}-G_{\mathrm{..}}^{\mathrm{..}},\\ ({\delta }_j^i{\gamma }_l^k)=G_{jl}^{ik}-G_{j.}^{ik}-G_{.l}^{ik}-G_{jl}^{i.}+G_{jl}^{.k}+G_{\mathrm{..}}^{ik}\\ +G_{jl}^{\mathrm{..}}+G_{j.}^{i.}+G_{.l}^{.k}+G_{j.}^{.k}+G_{.l}^{i.}-G_{\mathrm{..}}^{i.}\\ -G_{\mathrm{..}}^{.k}-G_{j.}^{\mathrm{..}}-G_{.j}^{\mathrm{..}}+G_{\mathrm{..}}^{\mathrm{..}}\end{array}$

where $G_{\mathrm{..}}^{\mathrm{..}}={\displaystyle {\sum }_{i,j,k,l}{p}^i{p}_j{q}^k{q}_l{G}_{jl}^{ik}}$, $G_{\mathrm{..}}^{i.}={\displaystyle {\sum }_{j,k,l}{p}_j{q}^k{q}_l{G}_{jl}^{ik}}$, and so on. The total genetic variance is $V_G={\displaystyle {\sum }_{i,j,k,l}{p}^i{p}_j{p}^k{p}_l{(G_{jl}^{ik}-\mu )}^2}$, and has an orthogonal partition under random mating and linkage equilibrium

$\begin{array}{l}{V}_G=V_{A_1}+V_{A_2}+V_{D_1}+V_{D_2}+V_{A_1{A}_2}+V_{A_1{D}_2}\left(4\right)\\ +V_{D_1{A}_2}+V_{D_1{D}_2}\end{array}$

with

$\begin{array}{l}{V}_{A_1}={\displaystyle \sum _i{p}^i{({\alpha }^i)}^2}+{\displaystyle \sum _j{p}_j{({\alpha }_j)}^2}\\ V_{D_1}={\displaystyle \sum _{i,j}{p}^i{p}_j{({\delta }_j^i)}^2}\\ V_{A_2}={\displaystyle \sum _k{q}^k{({\beta }^k)}^2}+{\displaystyle \sum _l{q}_l{({\beta }_l)}^2}\\ V_{D_2}={\displaystyle \sum _{k,l}{q}^k{q}_l{({\gamma }_l^k)}^2}\\ V_{A_1{A}_2}={\displaystyle \sum _{i,k}{p}^i{q}^k{({\alpha }^i{\beta }^k)}^2}+{\displaystyle \sum _{j,l}{p}_j{q}_l{({\alpha }_j{\beta }_l)}^2}\\ +{\displaystyle \sum _{i,l}{p}^i{q}_l{({\alpha }^i{\beta }_l)}^2}+{\displaystyle \sum _{j,k}{p}_j{q}^k{({\alpha }_j{\beta }^k)}^2}\\ V_{A_1{D}_2}={\displaystyle \sum _{i,k,l}{p}^i{q}^k{q}_l{({\alpha }^i{\gamma }_l^k)}^2}+{\displaystyle \sum _{j,k,l}{p}_j{q}^k{q}_l{({\alpha }_j{\gamma }_l^k)}^2}\\ V_{D_1{A}_2}={\displaystyle \sum _{i,j,k}{p}^i{p}_j{q}^k{({\delta }_j^i{\beta }^k)}^2}+{\displaystyle \sum _{i,j,l}{p}^i{p}_j{q}_l{({\delta }_j^i{\beta }_l)}^2}\\ V_{D_1{D}_2}={\displaystyle \sum _{i,j,k,l}{p}^i{p}_j{q}^k{q}_l{({\delta }_j^i{\gamma }_l^k)}^2}\end{array}$

Using indicator variables, we can represent model (1) in another form. Assume that the two loci A and B have alleles A _i , i = 1, 2, ..., n₁; and B _k , i = 1, 2, ..., n₂, respectively. We define the following indicator variables to represent the segregation of alleles in a population.

$\begin{array}{l}{z}_{M_i}^{(1)}=\{\begin{array}{ll}1,\hfill & \text{for }{A}_i\text{ allele from paternal gamete}\hfill \\ 0,\hfill & \text{otherwise}.\hfill \end{array}\\ z_{F_j}^{(1)}=\{\begin{array}{ll}1,\hfill & \text{for }{A}_j\text{ allele from maternal gamete}\hfill \\ 0,\hfill & \text{otherwise}.\hfill \end{array}\end{array}$

for i, j = 1, 2, ..., n₁ at locus A, and

$\begin{array}{l}{z}_{M_k}^{(2)}=\{\begin{array}{ll}1,\hfill & \text{for }{B}_k\text{ allele from paternal gamete}\hfill \\ 0,\hfill & \text{otherwise}.\hfill \end{array}\\ z_{F_l}^{(2)}=\{\begin{array}{ll}1,\hfill & \text{for }{B}_l\text{ allele from maternal gamete}\hfill \\ 0,\hfill & \text{otherwise}.\hfill \end{array}\end{array}$

for k, l = 1, 2, ..., n₂ at locus B. In terms of these indicator variables, we have the following.

• Hardy-Weinberg equilibrium (HWE) implies that {$z_{M_i}^{(1)}$, i = 1, 2, ..., n₁} are independent of {$z_{F_j}^{(1)}$, j = 1, 2, ..., n₁}, and {$z_{M_k}^{(2)}$, k = 1, 2, ..., n₂} are independent of {$z_{F_l}^{(2)}$, l = 1, 2, ..., n₂}.

• Linkage equilibrium (LE) implies that {$z_{M_i}^{(1)}$, i = 1, 2, ..., n₁} are independent of {$z_{M_k}^{(2)}$, k = 1, 2, ..., n₂}, and {$z_{F_j}^{(1)}$, j = 1, 2, ..., n₁} are independent of {$z_{F_l}^{(2)}$, l = 1, 2, ..., n₂}.

• There is another type of disequilibrium; i.e., the so-called genotypic disequilibrium [14] for two alleles on different gametes and at different loci. So, the genotypic equilibrium (GE) here means that {$z_{M_i}^{(1)}$, i = 1, 2,..., n₁} are independent of {$z_{F_l}^{(2)}$, l = 1, 2, ..., n₂}, and {$z_{F_j}^{(1)}$, j = 1, 2, ..., n₁} are independent of {$z_{M_k}^{(2)}$, k = 1, 2, ..., n₂}.

It is known that under random mating we have both HWE and GE, which together are called gametic phase equilibrium. Now, let G denote the genotypic value of a progeny drawn randomly from the current population. Based on Cockerham model, G can be expressed as

$\begin{array}{l}G=\mu +{\displaystyle \sum _{i=1}^{n_1}{\alpha }^i{z}_{M_i}^{(1)}}+{\displaystyle \sum _{j=1}^{n_1}{\alpha }_j{z}_{F_j}^{(1)}}+{\displaystyle \sum _{i,j}{\delta }_j^i{z}_{M_i}^{(1)}{z}_{F_j}^{(1)}}\\ +{\displaystyle \sum _{k=1}^{n_2}{\beta }^k{z}_{M_k}^{(2)}}+{\displaystyle \sum _{l=1}^{n_2}{\beta }_l{z}_{F_l}^{(2)}}+{\displaystyle \sum _{k,l}{\gamma }_l^k{z}_{M_k}^{(2)}{z}_{F_l}^{(2)}}\\ +[{\displaystyle \sum _{i,k}({\alpha }^i{\beta }^k)z_{M_i}^{(1)}{z}_{M_k}^{(2)}}+{\displaystyle \sum _{i,l}({\alpha }^i{\beta }_l)z_{M_i}^{(1)}{z}_{F_l}^{(2)}}\\ +{\displaystyle \sum _{j,k}({\alpha }_j{\beta }^k)z_{F_j}^{(1)}{z}_{M_k}^{(2)}}+{\displaystyle \sum _{j,l}({\alpha }_j{\beta }_l)z_{F_j}^{(1)}{z}_{F_l}^{(2)}}]\\ +[{\displaystyle \sum _{i,k,l}({\alpha }^i{\gamma }_l^k)z_{M_i}^{(1)}{z}_{M_k}^{(2)}{z}_{F_l}^{(2)}}+{\displaystyle \sum _{j,k,l}({\alpha }_j{\gamma }_l^k)z_{F_j}^{(1)}{z}_{M_k}^{(2)}{z}_{F_l}^{(2)}}]\\ +[{\displaystyle \sum _{i,j,k}({\delta }_j^i{\beta }^k)z_{M_i}^{(1)}{z}_{F_j}^{(1)}{z}_{M_k}^{(2)}}+{\displaystyle \sum _{i,j,l}({\delta }_j^i{\beta }_l)z_{M_i}^{(1)}{z}_{F_j}^{(1)}{z}_{F_l}^{(2)}}]\\ +{\displaystyle \sum _{i,j,k,l}({\delta }_j^i{\gamma }_l^k)z_{M_i}^{(1)}{z}_{F_j}^{(1)}{z}_{M_k}^{(2)}{z}_{F_l}^{(2)}}\left(5\right)\end{array}$

This is simply a different presentation of Cockerham model with the same constraint conditions applied on the coefficient parameters. For a given individual with genotype A _i B _k /A _j B _l , G will take the same value of $G_{jl}^{ik}$ as before. However, this expression is helpful for us to understand some details about each component of genetic effects. We can see this more clearly in the examination of some reduced models later.

In general, the genetic effects can be defined separately for alleles that are paternally and maternally transmitted to account for possible biological differences. As a fully parameterized model for $G_{jl}^{ik}$, model (3) may give $G_{jl}^{ik}\ne G_{jk}^{il}\ne G_{il}^{jk}\ne G_{ik}^{jl}$ depending on how genetic effects are defined. If locus A has n₁ alleles, and locus B has n₂ alleles, there are N = $n_1^2{n}_2^2$ possible phased genotypes in total with the partition of the degrees of freedom given in Table 1.

Table 1 Partition of degrees of freedom for two loci with number of alleles n₁ and n₂ (a general case)

If we assume that the union of paternal gamete A _i B _k with maternal gamete A _j B _l have the same mean effect as that of paternal gamete A _j B _l with maternal gamete A _i B _k (i.e., $G_{jl}^{ik}=G_{ik}^{jl}$), the coupling and repulsion heterozygotes have the same genotypic value (i.e., $G_{jl}^{ik}=G_{jk}^{il}$), and paternal and maternal gametes have the same gametic frequency distribution, we do not need to distinguish paternal and maternal effects. In this case, the two loci can be regarded as 2 factors and each factor has $n_i^2$ (i = 1, 2) levels produced by the allelic combinations of n _i alleles (cf. [7]). The total number of genotypes is N = n₁(n₁ + 1)n₂(n₂ + 1)/4 and the partition of degrees of freedom is shown in Table 2. Since in this case, αⁱ= α _i , β^k= β _k , ..., and so on, the model can also be expressed as follows

$\begin{array}{l}G=\mu +{\displaystyle \sum _{i=1}^{n_1}{\alpha }_i(z_{M_i}^{(1)}+z_{F_i}^{(1)})}+{\displaystyle \sum _{i,j}{\delta }_j^i{z}_{M_i}^{(1)}{z}_{F_j}^{(1)}}\\ +{\displaystyle \sum _{k=1}^{n_2}{\beta }_k(z_{M_k}^{(2)}+z_{F_k}^{(2)})}+{\displaystyle \sum _{k,l}{\gamma }_l^k{z}_{M_k}^{(2)}{z}_{F_l}^{(2)}}\\ +{\displaystyle \sum _{i,k}({\alpha }_i{\beta }_k)}(z_{M_i}^{(1)}+z_{F_i}^{(1)})(z_{M_k}^{(2)}+z_{F_k}^{(2)})\\ +{\displaystyle \sum _{i,k,l}({\alpha }_i{\gamma }_l^k)}(z_{M_i}^{(1)}+z_{F_i}^{(1)})z_{M_k}^{(2)}{z}_{F_l}^{(2)}\\ +{\displaystyle \sum _{i,j,k}({\delta }_j^i{\beta }_k)}{z}_{M_i}^{(1)}{z}_{F_j}^{(1)}(z_{M_k}^{(2)}+z_{F_k}^{(2)})\\ +{\displaystyle \sum _{i,j,k,l}({\delta }_j^i{\gamma }_l^k)}{z}_{M_i}^{(1)}{z}_{F_j}^{(1)}{z}_{M_k}^{(2)}{z}_{F_l}^{(2)})\left(6\right)\end{array}$

Table 2 Partition of degrees of freedom for two loci with number of alleles n₁ and n₂ (a simplified case without distinguishing the paternal and maternal origins)

For the case of an arbitrary number of loci, the situation will become more complicated. In addition to the additive and dominance effects at each locus and two locus interactions (additive × additive, additive × dominance, dominance × additive, dominance × dominance, with a total number of 2² terms), there are 3 locus interactions (additive × additive × additive, additive × additive × dominance, ..., with a total number of 2³ terms), 4 locus interactions (additive × additive × additive × additive, ..., with a total number of 2⁴ terms), and so on. Though the extension is straightforward, the total number of terms will increase dramatically. We will show some models with multiple loci in later examples by ignoring trigenic and higher order epistasis.

Effects and variance components

Let pⁱ, p _j (i, j = 1, 2, ..., n₁) be allelic frequencies of paternal and maternal gametes at locus A, respectively. Let also q^k, q _l (k, l = 1, 2, ..., n₂) denote allelic frequencies of paternal and maternal gametes at locus B, respectively. In the analysis of variance for the model, it is convenient to use deviations of the indicator variables $z_{M_i}^{(1)}$, $z_{F_i}^{(1)}$, $z_{M_j}^{(2)}$ and $z_{F_j}^{(2)}$ from their expected values. That is

$\begin{array}{l}{x}_{M_i}^{(1)}=z_{M_i}^{(1)}-E(z_{M_i}^{(1)})=z_{M_i}^{(1)}-p^i\\ =\{\begin{array}{ll}1-p^i,\hfill & \text{for }{A}_i\text{ allele from paternal gamete}\hfill \\ -p^i,\hfill & \text{otherwise}\hfill \end{array}\end{array}$

Similarly, define

$\begin{array}{l}{x}_{F_j}^{(1)}=z_{F_j}^{(1)}-E(z_{F_j}^{(1)})=z_{F_j}^{(1)}-p_j\\ x_{M_k}^{(2)}=z_{M_k}^{(2)}-E(z_{M_k}^{(2)})=z_{M_k}^{(2)}-q^k\\ x_{F_l}^{(2)}=z_{F_l}^{(2)}-E(z_{F_l}^{(2)})=z_{F_l}^{(2)}-q_l\end{array}$

Taking the constraint conditions on the genetic effects into account, we can show that,

$\begin{array}{c}{\displaystyle \sum _{i=1}^{n_1}{\alpha }^i{x}_{M_i}^{(1)}}={\displaystyle \sum _{i=1}^{n_1}{\alpha }^i{z}_{M_i}^{(1)}}\\ {\displaystyle \sum _{i,j}{\delta }_j^i{x}_{M_i}^{(1)}{x}_{F_j}^{(1)}}={\displaystyle \sum _{i,j}{\delta }_j^i{z}_{M_i}^{(1)}{z}_{F_j}^{(1)}}\\ {\displaystyle \sum _{i,k,l}({\alpha }^i{\gamma }_l^k)x_{M_i}^{(1)}{x}_{M_k}^{(2)}{x}_{F_l}^{(2)}}={\displaystyle \sum _{i,k,l}({\alpha }^i{\gamma }_l^k)z_{M_i}^{(1)}{z}_{M_k}^{(2)}{z}_{F_l}^{(2)}}\\ {\displaystyle \sum _{i,j,k,l}({\delta }_j^i{\gamma }_l^k)x_{M_i}^{(1)}{x}_{F_j}^{(1)}{x}_{M_k}^{(2)}{x}_{F_l}^{(2)}}={\displaystyle \sum _{i,j,k,l}({\delta }_j^i{\gamma }_l^k)z_{M_i}^{(1)}{z}_{F_j}^{(1)}{z}_{M_k}^{(2)}{z}_{F_l}^{(2)}}\end{array}$

and so on. For example,

$\begin{array}{l}{\displaystyle \sum _{i=1}^{n_1}{\alpha }^i{x}_{M_i}^{(1)}}={\displaystyle \sum _{i=1}^{n_1}{\alpha }^i(z_{M_i}^{(1)}-p^i)}\\ ={\displaystyle \sum _{i=1}^{n_1}{\alpha }^i{z}_{M_i}^{(1)}}-{\displaystyle \sum _{i=1}^{n_1}{\alpha }^i{p}^i={\displaystyle \sum _{i=1}^{n_1}{\alpha }^i{z}_{M_i}^{(1)}}}\end{array}$

as ${\displaystyle {\sum }_{i=1}^{n_1}{\alpha }^i{p}^i}=0$ by the constrain condition (2). Therefore, model (5) can be rewritten as

$\begin{array}{l}G=\mu +{\displaystyle \sum _{i=1}^{n_1}{\alpha }^i{x}_{M_i}^{(1)}}+{\displaystyle \sum _{j=1}^{n_1}{\alpha }^i{x}_{F_j}^{(1)}}+{\displaystyle \sum _{i,j}{\delta }_j^i{x}_{M_i}^{(1)}{x}_{F_j}^{(1)}}\\ +{\displaystyle \sum _{k=1}^{n_2}{\beta }^k{x}_{M_k}^{(2)}}+{\displaystyle \sum _{l=1}^{n_2}{\beta }_l{x}_{F_l}^{(2)}}+{\displaystyle \sum _{k,l}{\gamma }_l^k{x}_{M_k}^{(2)}{x}_{F_l}^{(2)}}\\ +[{\displaystyle \sum _{i,k}({\alpha }^i{\beta }^k)x_{M_i}^{(1)}{x}_{M_k}^{(2)}}+{\displaystyle \sum _{i,l}({\alpha }^i{\beta }_l)x_{M_i}^{(1)}{x}_{F_l}^{(2)}}\\ +{\displaystyle \sum _{j,k}({\alpha }_j{\beta }^k)x_{F_j}^{(1)}{x}_{M_k}^{(2)}}+{\displaystyle \sum _{j,l}({\alpha }_j{\beta }_l)x_{F_j}^{(1)}{x}_{F_l}^{(2)}]}\\ +[{\displaystyle \sum _{i,k,l}({\alpha }^i{\gamma }_l^k)x_{M_i}^{(1)}{x}_{M_k}^{(2)}}{x}_{F_l}^{(2)}+{\displaystyle \sum _{j,k,l}({\alpha }_j{\gamma }_l^k)x_{F_j}^{(1)}{x}_{M_k}^{(2)}}{x}_{F_l}^{(2)}]\\ +[{\displaystyle \sum _{i,j,k}({\delta }_j^i{\beta }^k)x_{M_i}^{(1)}{x}_{F_j}^{(1)}}{x}_{M_k}^{(2)}+{\displaystyle \sum _{i,j,l}({\delta }_j^i{\beta }_l)x_{M_i}^{(1)}{x}_{F_j}^{(1)}}{x}_{F_l}^{(2)}]\\ +{\displaystyle \sum _{i,j,k,l}({\delta }_j^i{\gamma }_l^k)x_{M_i}^{(1)}{x}_{F_j}^{(1)}}{x}_{M_k}^{(2)}{x}_{F_l}^{(2)}\left(7\right)\end{array}$

When we deal with some reduced models in the next section, we will find that the model of this form is especially helpful as it makes the model parameter constraints built into regression variables which is suited for genetic interpretation. The form of model (7) can also facilitate the demonstration that under Hardy-Weinberg, linkage and genotypic equilibria, the regression coefficients (genetic effects) are Cockerham's least squares effects (3) (Appendix A), and the genotypic variance V _G has the orthogonal partition (4) (Appendix B).

Now we discuss the properties of model in a disequilibrium situation. As stated in [14], there are three types of disequilibria

• Typel: between alleles on the same gametes but at different loci

• Type2: between alleles at the same locus but on different gametes

• Type3: between alleles on different gametes and at different loci.

If we denote $P_{jl}^{ik}$ as the genotypic frequency of A _i B _k /A _j B _l , $P_{j.}^{i.}$ as the genotypic frequency of A _i /A _j , and so on, following [14], the digenic disequilibria can be written as

$\begin{array}{l}{D}_{\mathrm{..}}^{ik}=\text{Cov}(z_{M_i}^{(1)},z_{M_k}^{(2)})=E(x_{M_i}^{(1)}{x}_{M_k}^{(2)})=P_{\mathrm{..}}^{ik}-p^i{q}^k\\ D_{jl}^{\mathrm{..}}=\text{Cov}(z_{F_j}^{(1)},z_{F_l}^{(2)})=E(x_{F_j}^{(1)}{x}_{F_l}^{(2)})=P_{jl}^{\mathrm{..}}-p_j{q}_l\\ D_{j.}^{i.}=E(x_{M_i}^{(1)}{x}_{F_j}^{(1)})=P_{j.}^{i.}-p^i{p}_j\\ D_{.l}^{.k}=E(x_{M_k}^{(2)}{x}_{F_l}^{(2)})=P_{.l}^{.k}-q^k{q}_l\\ D_{.j}^{i.}=E(x_{M_i}^{(1)}{x}_{F_l}^{(2)})=P_{.l}^{i.}-p^i{q}_l\\ D_{j.}^{.k}=E(x_{F_j}^{(1)}{x}_{M_k}^{(2)})=P_{j.}^{.k}-p_j{q}^k.\end{array}$

And the trigenic disequilibria

$\begin{array}{c}{D}_{j.}^{ik}=E(x_{M_i}^{(1)}{x}_{F_j}^{(1)}{x}_{M_k}^{(2)})\\ =P_{j.}^{ik}-p^i{P}_{j.}^{.k}-q^k{P}_{j.}^{i.}-p_j{P}_{\mathrm{..}}^{ik}+2p^i{p}_j{q}^k\\ D_{.l}^{ik}=E(x_{M_i}^{(1)}{x}_{M_k}^{(2)}{x}_{F_l}^{(2)})\\ =P_{.l}^{ik}-p^i{P}_{.l}^{.k}-q^k{P}_{.l}^{i.}-q_l{P}_{\mathrm{..}}^{ik}+2p^i{q}^k{q}_l\\ D_{jl}^{i.}=E(x_{M_i}^{(1)}{x}_{F_j}^{(1)}{x}_{F_l}^{(2)})\\ =P_{jl}^{i.}-p^i{P}_{jl}^{\mathrm{..}}-p_j{P}_{\mathrm{..}}^{i.}-q_l{P}_{j.}^{i.}+2p^i{p}_j{q}_l\\ D_{jl}^{.k}=E(x_{F_j}^{(1)}{x}_{M_k}^{(2)}{x}_{F_l}^{(2)})\\ =P_{jl}^{.k}-p_j{P}_{.l}^{.k}-q^k{P}_{j.}^{.l}-q_l{P}_{j.}^{.k}+2p_j{q}^k{q}_l\end{array}$

Similarly for the quadrigenic disequilibrium, we may define

$\begin{array}{c}{D}_{jl}^{ik}=E(x_{M_i}^{(1)}{x}_{F_j}^{(1)}{x}_{M_k}^{(2)}{x}_{F_l}^{(2)})\\ =P_{jl}^{ik}-p^i{P}_{jl}^{.k}-p_j{P}_{.l}^{ik}-q^k{P}_{jl}^{i.}-q_l{P}_{j.}^{ik}\\ +p^i{p}_j{P}_{.l}^{.k}-p^i{q}^k{P}_{jl}^{\mathrm{..}}+p^i{q}_l{P}_{j.}^{.k}+p_j{q}^k{P}_{.l}^{i.}\\ +p_j{q}_l{P}_{\mathrm{..}}^{ik}+q^k{q}_l{P}_{j.}^{i.}-3p^i{p}_j{q}^k{q}_l.\end{array}$

If we express $D_{jl}^{ik}$ as a function of lower-order linkage disequilibria, we have

$\begin{array}{c}{D}_{jl}^{ik}=P_{jl}^{ik}-p^i{D}_{jl}^{.k}-p_j{D}_{.l}^{ik}-q^k{D}_{jl}^{i.}-q_l{D}_{j.}^{ik}\\ -p^i{p}_j{D}_{.l}^{.k}-p^i{q}^k{D}_{jl}^{\mathrm{..}}-p^i{q}_l{D}_{j.}^{.k}-p_j{q}^k{D}_{.l}^{i.}\\ -p_j{q}_l{D}_{\mathrm{..}}^{ik}-q^k{q}_l{D}_{j.}^{i.}-p^i{p}_j{q}^k{q}_l.\end{array}$

This definition is the same as that given by [15, 16]. Note that ${\displaystyle {\sum }_i{z}_{M_i}^{(1)}}=1$. Then, we have

$\begin{array}{l}{\displaystyle \sum _i{D}_{jl}^{ik}}=E[({\displaystyle \sum _i{x}_{M_i}^{(1)})x_{F_j}^{(1)}}{x}_{M_k}^{(2)}{x}_{F_l}^{(2)}]\\ =E[{\displaystyle \sum _i(z_{M_i}^{(1)}-p^i)x_{F_j}^{(1)}}{x}_{M_k}^{(2)}{x}_{F_l}^{(2)}=0\end{array}$

In general, $D_{jl}^{ik}$ is summed to zero over any allele involved, so are $D_{j.}^{ik}$ and other disequilibrium measurements.

With Hardy-Weinberg and genotypic equilibria but linkage disequilibrium, model (7) leads to the following expression for the overall mean

$\begin{array}{c}E(G)=\mu +{\displaystyle \sum _{i,k}({\alpha }^i{\beta }^k)D_{\mathrm{..}}^{ik}}+{\displaystyle \sum _{j,l}({\alpha }_j{\beta }_l)D_{jl}^{\mathrm{..}}}\\ +{\displaystyle \sum _{i,j,k,l}({\delta }_j^i{\gamma }_l^k)D^{ik}{D}_{jl}}\end{array}$

where μ is the mean genotypic value under linkage equilibrium, and ${\Delta }_{\mu }={\displaystyle {\sum }_{i,k}({\alpha }^i{\beta }^k)}{D}_{\mathrm{..}}^{ik}+{\displaystyle {\sum }_{j,l}({\alpha }_j{\beta }_l)}{D}_{jl}^{\mathrm{..}}+{\displaystyle {\sum }_{i,j,k,l}({\delta }_j^i{\gamma }_l^k)}{D}_{\mathrm{..}}^{ik}{D}_{jl}^{\mathrm{..}}$ represents the departure from μ due to linkage disequilibrium and epistasis. If there is no epistasis, linkage disequilibrium does not affect the mean genotypic value. Similar results were given by [17]. Note that for marginal means of the genotypic values, we have

$\begin{array}{l}{G}_{\mathrm{..}}^{\mathrm{..}}={\displaystyle \sum _{i,j,k,l}{P}_{\mathrm{..}}^{ik}{P}_{jl}^{\mathrm{..}}{G}_{jl}^{ik},G_{\mathrm{..}}^{i.}=\frac{1}{p^i}{\displaystyle \sum _{j,k,l}{P}_{\mathrm{..}}^{ik}{P}_{jl}^{\mathrm{..}}{G}_{jl}^{ik}}},\\ G_{\mathrm{..}}^{ik}={\displaystyle \sum _{j,l}{P}_{jl}^{\mathrm{..}}{G}_{jl}^{ik},G_{j.}^{i.}=\frac{1}{p^i{p}_j}{\displaystyle \sum _{k,l}{P}_{jl}^{\mathrm{..}}{P}_{\mathrm{..}}^{ik}{G}_{jl}^{ik},}}\\ G_{j.}^{ik}=\frac{1}{p_j}{\displaystyle \sum _l{P}_{jl}^{\mathrm{..}}{G}_{jl}^{ik},\mathrm{...}}\end{array}$

and so on.

Then the question is what the genetic effects are in a disequilibrium population. Do Hardy-Weinberg and linkage disequilibria change the definition and values of genetic effects? The short answer to this question is "no" in a fully characterized model, but "yes" in a model that ignores some QTL or genetic effects. This is proved and discussed in [10]. With Hardy-Weinberg and linkage disequilibria, the genetic effects no longer correspond to the deviations from marginal means of genotypic values in a disequilibrium population. In a multiple regression model (7), the genetic effects are partial regression coefficients. These partial regression coefficients correspond to the simple regression coefficients, or deviations from marginal means of genotypic values, only in an equilibrium population. In a disequilibrium population, a direct analysis on the partial regression coefficients can be very complex (see the appendix of [10] for a relatively simple example). However, in a full model which includes all relevant loci and genetic effects, the model parameters depend only on how the regressors, i.e. x variables in (7), are defined and are independent of correlations between x variables, i.e. Hardy-Weinberg and linkage disequilibria. So, the genetic effects are still the same as those defined in the equilibrium population, although the population mean and marginal means of genotypic values are changed in a disequilibrium population.

Hardy-Weinberg and linkage disequilibria introduce correlation between x variables, thus covariances between different genetic effect components. Define

$\begin{array}{l}{A}_1={\displaystyle \sum _{i=1}^{n_1}{\alpha }^i{x}_{M_i}^{(1)}}+{\displaystyle \sum _{j=1}^{n_1}{\alpha }_j{x}_{F_j}^{(1)}}\\ D_1={\displaystyle \sum _{i,j}{\delta }_j^i{x}_{M_i}^{(1)}{x}_{F_j}^{(1)}}\\ A_2={\displaystyle \sum _{k=1}^{n_2}{\beta }^k{x}_{M_k}^{(2)}}+{\displaystyle \sum _{l=1}^{n_2}{\beta }_l{x}_{F_l}^{(2)}}\\ D_2={\displaystyle \sum _{k,l}{\gamma }_l^k{x}_{M_k}^{(2)}{x}_{F_l}^{(2)}}\\ A_1{A}_2={\displaystyle \sum _{i,k}({\alpha }^i{\beta }^k)x_{M_i}^{(1)}{x}_{M_k}^{(2)}}+{\displaystyle \sum _{i,l}({\alpha }^i{\beta }_l)x_{M_i}^{(1)}{x}_{F_l}^{(2)}}\\ +{\displaystyle \sum _{j,k}(}{\alpha }_j{\beta }^k)x_{F_j}^{(1)}{x}_{M_k}^{(2)}+{\displaystyle \sum _{j,l}(}{\alpha }_j{\beta }_l)x_{F_j}^{(1)}{x}_{F_l}^{(2)}\\ A_1{D}_2={\displaystyle \sum _{i,k,l}({\alpha }^i{\gamma }_l^k})x_{M_i}^{(1)}{x}_{M_k}^{(2)}{x}_{F_l}^{(2)}\\ +{\displaystyle \sum _{j,k,l}({\alpha }_j{\gamma }_l^k})x_{F_j}^{(1)}{x}_{M_k}^{(2)}{x}_{F_l}^{(2)}\\ A_2{D}_1={\displaystyle \sum _{i,j,k}({\delta }_j^i{\beta }^k})x_{M_i}^{(1)}{x}_{F_j}^{(1)}{x}_{M_k}^{(2)}\\ +{\displaystyle \sum _{i,j,l}({\delta }_j^i{\beta }_l})x_{M_i}^{(1)}{x}_{F_j}^{(1)}{x}_{F_l}^{(2)}\\ D_1{D}_2={\displaystyle \sum _{i,j,k,l}({\delta }_j^i{\gamma }_l^k})x_{M_i}^{(1)}{x}_{F_j}^{(1)}{x}_{M_k}^{(2)}{x}_{F_l}^{(2)}\end{array}$

Then we can write

G = μ + A₁ + A₂ + D₁ + D₂ + A₁A₂ + A₁D₂ + A₂D₁ + D₁D₂

In a disequilibrium population, the partition of the genotypic variance becomes

$V_G={\displaystyle \sum _{i=1}^8{\displaystyle \sum _{j=1}^8{V}_{ij}}}=1^{T}V1$

where

V = (V _ij )_{8 × 8}

It is a symmetric matrix. In Appendix C, we give the detailed result for each component of the matrix with linkage disequilibrium, but assuming Hardy-Weinberg equilibrium.

For the rest of paper, when we discuss disequilibrium, we mainly discuss linkage disequilibrium and assume Hardy-Weinberg and genotypic equilibria which can be achieved by random mating in one generation. Hardy-Weinberg disequilibrium can be taken into account which will make results more complex and is thus omitted.

Reduced models

In many genetic applications, experimental population has some regular genetic structure by design. In these cases, the genetic model can be further simplified to reflect the experimental design structure. Also sometimes we may want to simplify the genetic model by imposing certain constrains or assumptions, such as the number of alleles, to increase the feasibility of analysis. In this section, we give a few reduced genetic models that are relevant to many genetic applications.

1. Backcross population or recombinant inbred population (haploid model)

Backcross population or recombinant inbred population is a common experimental design for QTL mapping study. By crossing two inbred lines, we can create a F₁ population. If we randomly backcross F₁ to one of the inbred lines, we have a backcross population. Let us assume that the cross is AA (paternal) × Aa (maternal). In a random-mating backcross population, there are only two possible genotypes at each segregating locus A _r A _r or A _r a _r , for r = 1, 2, ..., m, where m is the number of QTL. Since for the paternal gametes,

$z_{M_1}^{(r)}=\{\begin{array}{ll}1,\hfill & \text{for }{A}_r\text{ allele from paternal gamete}\hfill \\ 0,\hfill & \text{otherwise }\hfill \end{array}=1,$

and

$z_{M_2}^{(r)}=\{\begin{array}{ll}1,\hfill & \text{for }{a}_r\text{ allele from paternal gamete}\hfill \\ 0,\hfill & \text{otherwise }\hfill \end{array}=0,$

thus $x_{M_1}^{(r)}=z_{M_1}^{(r)}-1=0$ and $x_{M_2}^{(r)}=z_{M_2}^{(r)}=0$ for r = 1, 2, ..., m. For maternal gametes however,

$\begin{array}{c}{x}_{F_1}^{(r)}=\{\begin{array}{l}1/2,\text{ for }{A}_r\text{ from maternal gamete}\hfill \\ -1/2,\text{ otherwise}\hfill \end{array}\\ =-x_{F_2}^{(r)},\text{ for }r=1,2,\cdot \cdot \cdot ,m.\end{array}$

Thus the model becomes

$\begin{array}{c}G=\mu +{\displaystyle \sum _{r=1}^m{a}_r{x}_{F_1}^{(r)}+{\displaystyle \sum _{r<s}{b}_{rs}(x_{F_1}^{(r)}{x}_{F_1}^{(s)})}}\\ +{\displaystyle \sum _{r<s<t}{c}_{rst}(x_{F_1}^{(r)}{x}_{F_1}^{(s)}{x}_{F_1}^{(t)})+\cdots }\end{array}\left(8\right)$

where $a_r={\alpha }_1^{(r)}-{\alpha }_2^{(r)}$ is the substitution effect between homozygote genotype A _r A _r and heterozygote genotype $A_r{a}_r,b_{rs}=({\alpha }_1^{(r)}{\alpha }_1^{(s)})-({\alpha }_1^{(r)}{\alpha }_2^{(s)})-({\alpha }_2^{(r)}{\alpha }_1^{(s)})+({\alpha }_2^{(r)}{\alpha }_2^{(s)})$ is the interaction effect between loci r and $s,c_{rst}=({\alpha }_1^{(r)}{\alpha }_1^{(s)}{\alpha }_1^{(t)})-({\alpha }_1^{(r)}{\alpha }_1^{(s)}{\alpha }_2^{(t)})-({\alpha }_1^{(r)}{\alpha }_2^{(s)}{\alpha }_1^{(t)})+({\alpha }_1^{(r)}{\alpha }_2^{(s)}{\alpha }_2^{(t)})-({\alpha }_2^{(r)}{\alpha }_1^{(s)}{\alpha }_1^{(t)})+({\alpha }_2^{(r)}{\alpha }_1^{(s)}{\alpha }_2^{(t)})+({\alpha }_2^{(r)}{\alpha }_2^{(s)}{\alpha }_1^{(t)})-({\alpha }_2^{(r)}{\alpha }_2^{(s)}{\alpha }_2^{(t)})$, ..., and so on. Taking constraint conditions into account, we have α₁ = -α₂, β₁ = -β₂, and so on. Then, a _r = $2{\alpha }_1^{(r)}$, b _rs = 4(${\alpha }_1^{(r)}{\alpha }_1^{(s)}$), and c _rst = 8(${\alpha }_1^{(r)}{\alpha }_1^{(s)}{\alpha }_1^{(t)}$), and so on. With linkage equilibrium, the genetic effects as the partial regression coefficients of the model correspond to the simple regression coefficients. For example, for the substitution effect of locus r, a _r , it is the covariance between genotypic value G and substitution effect design variable $x_{F_1}^{(r)}$ divided by the variance of $x_{F_1}^{(r)}$. So in general, we have

$\begin{array}{c}{a}_r=E[(G-\mu )x_{F_1}^{(r)}]/E(x_{F_1}^{(r)2})\\ =E[G(z_{F_1}^{(r)}-1/2)]/(1/4)\\ =2[E(G|z_{F_1}^{(r)}=1)-E(G)]\\ b_{rs}=E[(G-\mu )x_{F_1}^{(r)}{x}_{F_1}^{(s)}]/[E(x_{F_1}^{(r)2})E(x_{F_1}^{(s)2})]\\ \begin{array}{l}=4[E(G|z_{F_1}^{(r)}=z_{F_1}^{(s)}=1)-E(G|z_{F_1}^{(r)}=1)\\ -E(G|z_{F_1}^{(s)}=1)+E(G)]\end{array}\\ c_{rst}=E[(G-\mu )x_{F_1}^{(r)}{x}_{F_1}^{(s)}{x}_{F_1}^{(t)}]/\\ [E(x_{F_1}^{(r)2})E(x_{F_1}^{(s)2})E(x_{F_1}^{(t)2})]\\ \begin{array}{l}=8[E(G|z_{F_1}^{(r)}=z_{F_1}^{(s)}=z_{F_1}^{(t)}=1)\\ -E(G|z_{F_1}^{(r)}=z_{F_1}^{(s)}=1)\\ -E(G|z_{F_1}^{(r)}=z_{F_1}^{(t)}=1)\\ -E(G|z_{F_1}^{(s)}=z_{F_1}^{(t)}=1)\\ +E(G|z_{F_1}^{(r)}=1)+E(G|z_{F_1}^{(s)}=1)\\ +E(G|z_{F_1}^{(t)}=1)-E(G)]\end{array}\end{array}$

The orthogonal partition of the genotypic variance in an equilibrium population is

$V_G=\frac{1}{4}{\displaystyle \sum _{r=1}^l{a}_r^2}+\frac{1}{4^2}{\displaystyle \sum _{r<s}{b}_{rs}^2}+\frac{1}{4^3}{\displaystyle \sum _{r<s<t}{c}_{rst}^2}+\cdots \left(9\right)$

As noted above, linkage disequilibrium does not change the values of genetic effects in a full model. The model parameters are still the same as those defined in the equilibrium population. However, in this case there is a simple relationship between the substitution effects at multiple loci and marginal means of genotypic values in a disequilibrium population [18]. This is noted here. Let $P_{rs}=P\{z_{F_1}^{(r)}=z_{F_1}^{(s)}=1\}$, and the digenic linkage disequilibrium be defined as

$D_{rs}=\text{Cov(}{z}_{F_1}^{(r)},z_{F_1}^{(s)})=E(x_{F_1}^{(r)}{x}_{F_1}^{(s)})$

Ignoring trigenic and higher order linkage disequilibria, we have

$E\left[(G-\mu )x_{F_1}^{(r)}\right]=\frac{1}{4}{a}_r+{\displaystyle \sum _{r\text{'}\ne r}{D}_{rr\text{'}}{a}_{r\text{'}}}$

$E[(G-\mu )x_{F_1}^{(r)}{x}_{F_1}^{(s)}]=\frac{1}{4^2}{b}_{rs}$

Therefore, the digenic interaction effects can be expressed as

$\begin{array}{l}{b}_{rs}=4^2[P_{rs}E(G|z_{F_1}^{(r)}=z_{F_1}^{(s)}=1)-D_{rs}\mu ]\\ -4[E(G|z_{F_1}^{(r)}=1)+E(G|z_{F_1}^{(s)}=1)-E(G)]\end{array}$

Then the substitution effects can be expressed as a function of marginal means in the disequilibrium population as

$\left(\begin{array}{c}{a}_1\\ a_2\\ ⋮\\ a_l\end{array}\right)={(I+4D)}^{-1}(2q)$

where I is a m × m identity matrix, D = (D _ij )_{m × m}with all diagonal elements being zeros;

q = (q₁, q₂, ..., q _m ,)^T, with q _i = E(G|$z_{F_1}^{(i)}$ = 1) - E(G), for i = 1, 2, ..., m.

The partition of genetic variance with linkage disequilibrium is complex. Here we give details of the partition of genotypic variance for the following model

$G=\mu +{\displaystyle \sum _{r=1}^m{a}_r{x}_{F_1}^{(r)}}+{\displaystyle \sum _{r<s}{b}_{rs}}(x_{F_1}^{(r)}{x}_{F_1}^{(s)})\left(10\right)$

Let x _r = $x_{F_1}^{(r)}$ and x _s = $x_{F_1}^{(s)}$ to simplify the notation here. The genotypic variance is

$\begin{array}{l}{V}_G=V({\displaystyle \sum _r{a}_r}{x}_r)+2\text{Cov(}{\displaystyle \sum _r{a}_r{x}_r,{\displaystyle \sum _{r<s}{b}_{rs}{x}_r{x}_s)}}\\ +\text{V}({\displaystyle \sum _{r<s}{b}_{rs}{x}_r{x}_s)}\\ ={\displaystyle \sum _r{a}_r^2{p}_r(1-p_r)}+2{\displaystyle \sum _{r<s}{a}_r{a}_s{D}_{rs}}\\ +2{\displaystyle \sum _{r<s}[a_r{b}_{rs}(1-2p_r)D_{rs}+a_s{b}_{rs}(1-2p_s)D_{rs}]}\\ +2{\displaystyle \sum _{r<s<t}(a_r{b}_{st}+a_s{b}_{rt}+a_t{b}_{rs})D_{rst}}\\ +{\displaystyle \sum _{r<s}{b}_{rs}^2}[(1-2p_r)(1-2p_s)D_{rs}\\ +p_r(1-p_r)p_s(1-p_s)-D_{rs}^2]\\ +2{\displaystyle \sum _{r<s<t}\{b_{rs}{b}_{rt}[(1-2p_r)D_{rst}}\\ -p_r(1-p_r)D_{st}-D_{rs}{D}_{rt}]\\ +b_{rs}{b}_{st}[(1-2p_s)D_{rst}+p_s(1-p_s)D_{rt}-D_{rs}{D}_{st}]\\ +b_{rt}{b}_{st}[(1-2p_t)D_{rst}\\ +p_t(1-p_t)D_{rs}-D_{rt}{D}_{st}]\}\\ +2{\displaystyle \sum _{r<s<t<u}[b_{rt}{b}_{su}(D_{rstu}-D_{rt}}{D}_{su})\\ +b_{ru}{b}_{st}(D_{rstu}-D_{ru}{D}_{st})\\ +b_{rs}{b}_{tu}(D_{rstu}-D_{rs}{D}_{tu})]\left(11\right)\end{array}$

where

D _rst = E(x _r x _s x _t ) and D _rstu = E(x _r x _s x _t x _u )

are three locus and four locus linkage disequilibria. This is a general partition of genetic variance for a haploid model.

For the backcross population, it can be shown that D _rst = 0 (see Appendix D for both backcross and F₂ populations) and D _rstu = D _rs D _tu for loci r, s, t and u in this order under the assumption of no crossing-over interference. Also with this assumption, D _rt = 4D _rs D _st and D _rs = (1 - 2λ _rs )/4, where λ _rs is the recombination frequency between loci r and s. Since, p _r = p _s = 1/2, the variance becomes

$\begin{array}{ll}{V}_G=\hfill & \frac{1}{4}{\displaystyle \sum _r{a}_r^2+2{\displaystyle \sum _{r<s}{a}_r{a}_s{D}_{rs}+}}\hfill \\ \frac{1}{16}{\displaystyle \sum _{r<s}{b}_{rs}^2(1-16D_{rs}^2)}\hfill \\ +\frac{1}{2}{\displaystyle \sum _{r<s<t}[b_{rs}{b}_{rt}(1-16D_{rs}^2)D_{st}}\hfill \\ +b_{rt}{b}_{st}(1-16D_{st}^2)D_{rs}]\hfill \\ +2{\displaystyle \sum _{r<s<t<u}(b_{rt}{b}_{su}+b_{ru}{b}_{st})}\hfill \\ (1-16D_{st}^2)D_{rs}{D}_{tu}\hfill & \left(12\right)\hfill \end{array}$

In this partition of variance, the first summation term is the genetic variance due to the substitution effect of each QTL, the second summation term is the covariance between substitution effects of QTL pairs due to linkage disequilibrium, the third summation term is the genetic variance due to epistatic effects of QTL, and the fourth and fifth summation terms are the covariance between different epistatic effects of QTL due to linkage disequilibrium. There is no covariance between the main substitution effects and epistatic effects (see also [19]).

For a backcross population, the genetic interpretation of the substitution effect a _r depends on which parental line is backcrossed. In one backcross AA × Aa, the substitution effect is traditionally defined as the difference between the additive effect and dominance effects, and in the other backcross Aa × aa, it is the sum of the additive and dominance effects. Only with both backcrosses, can one estimate both additive and dominance effects separately (for example [20]).

The same model also applies to a recombinant inbred population which is another very popular experimental design for QTL mapping study. For a recombinant inbred population, the substitution effects of QTL are the additive effects and the epistatic effects are the additive × additive interaction effects. Statistical methods to map QTL and to estimate various components of the genetic variance due to QTL including epistasis has been developed through the maximum likelihood approach [19, 21]. In a few cases where the method was applied, we estimated, for the first time, how the quantitative genetic variance was partitioned into various components in designed experimental populations. For example, Weber et al. [22] reported the result of QTL mapping for wing shape on the third chromosome of Drosophila melanogaster from a cross of divergent selection lines. From 519 recombinant inbred lines, 11 QTL were mapped on the third chromosome. Nine QTL pairs showed significant epistatic effects. The total genetic variance amounts to 95.5% of the phenotypic variance in the recombinant inbred lines with phenotypes measured and averaged over 50 male flies for each recombinant inbred line. The partition of the genetic variance is as follows (see Table 6 and 7 of [22]): 27.4% due to the variances of additive effects (equivalent to the first summation term of (12)); 67.3% due to the covariances between additive effects (the second summation term); 7.2% due to the variances of epistatic effects (the third summation term); and -6.0% due to the covariances between epistatic effects (the fourth and fifth summation terms). The covariances between additive and epistatic effects, expected to be 0, account for -0.4% due to sampling. Similar kind of partition of the genetic variance is also observed in a group of 701 second chromosome recombinant inbred lines from a cross of the same divergent selection lines (see Table 4 and 5 of [23]). See also [20] for another example.

2. F₂ population

F₂ is created from a cross between pairs of F₁ individuals. It is also a very popular experimental design for QTL mapping study. The advantage of this design is that both additive and dominance effects of a QTL can be estimated as well as various epistatic effects. The design also has more statistical power for QTL detection as compared to a backcross population. In a random-mating F₂ population, there are only two alleles at each segregating locus and allelic frequencies are expected to be one half if there is no segregation distortion.

Let us consider only two loci first. Let A and a denote the two alleles at locus 1, and B and b at locus 2. In this case, $x_{M_1}^{(1)}=-x_{M_2}^{(1)}$ and $x_{F_1}^{(1)}=-x_{F_2}^{(1)}$. Assuming $G_{jl}^{ij}=G_{ik}^{jl}=G_{jk}^{il}=G_{il}^{jk}$, it also holds that α¹ = α₁ = -α² = -α₂, ${\delta }_1^1={\delta }_2^2=-{\delta }_1^2$, and so on. The additive term for locus 1 then becomes

$A_1={\alpha }^1(x_{M_1}^{(1)}-x_{M_2}^{(1)})+{\alpha }_1(x_{F_1}^{(1)}-x_{F_2}^{(1)})=2{\alpha }_1{w}_1$

with

$w_1=x_{M_1}^{(1)}+x_{F_1}^{(1)}=\{\begin{array}{l}1,\text{ for }AA\text{ at locus 1}\hfill \\ \text{0, for }Aa\text{ at locus 1}\hfill \\ -1,\text{ for }aa\text{ at locus 1}\hfill \end{array}\left(13\right)$

and the dominance term is

$D_1=2({\delta }_1^1)(x_{M_1}^{(1)}{x}_{F_1}^{(1)}+x_{M_1}^{(1)}{x}_{F_1}^{(1)})=(-2)({\delta }_1^1)v_1$

with

$v_1=(-2)x_{M_1}^{(1)}{x}_{F_1}^{(1)}=\{\begin{array}{c}-1/2,\text{ for }AA\text{ at locus 1}\\ 1/2,\text{ for }Aa\text{ at locus 1}\\ -1/2,\text{ for }aa\text{ at locus 1}\end{array}\left(14\right)$

Note that the v variable in this section for F₂ differ, by a factor -2, from the v variable in the next section for a general two-allele model to conform to the usual definition for the F₂ model. Similarly, for locus 2

A₂ = 2β₁w₂ and D₂ = (-2)(${\gamma }_{\text{1}}^{\text{2}}$)v₂

with

$w_2=\{\begin{array}{c}1,\text{ for }BB\text{ at locus 2}\\ 0,\text{ for }Bb\text{ at locus 2}\\ -1,\text{ for }bb\text{ at locus 2}\end{array}$

and

$v_2=\{\begin{array}{c}-1/2,\text{ for }BB\text{ at locus 2}\\ 1/2,\text{ for }Bb\text{ at locus 2}\\ -1/2,\text{ for }bb\text{ at locus 2}\end{array}\left(15\right)$

The model can then be written as

$\begin{array}{c}G=\mu +a_1{w}_1+d_1{v}_1+a_2{w}_2+d_2{v}_2\\ +{(aa)}_{12}(w_1{w}_2)+{(ad)}_{12}(w_1{v}_2)\\ +{(da)}_{12}(v_1{w}_2)+{(dd)}_{12}(v_1{v}_2)\end{array}\left(16\right)$

where the parameters are related as a₁ = 2α₁, a₂ = 2β₂, d₁ = - $-2{\delta }_1^1$, d₂ = -$-2{\gamma }_1^1$, (aa)₁₂ = 4(α₁β₁), (ad)₁₂ = 8(${\alpha }_1{\gamma }_1^1$), (da)₁₂ = 8(${\delta }_1^1{\beta }_1$), (dd)₁₂ = 16(${\delta }_1^1{\gamma }_1^1$). With random mating and linkage equilibrium, we have

$\begin{array}{c}{a}_1=E[G-\mu )w_1]/E(w_1^2)=2(G_{1.}^{\mathrm{..}}-G_{\mathrm{..}}^{\mathrm{..}})\\ d_1=E[(G-\mu )v_1]/E(v_2^1)\\ =(-2)(G_{1.}^{1.}-2G_{1.}^{\mathrm{..}}+G_{\mathrm{..}}^{\mathrm{..}})\\ a_2=2(G_{.1}^{\mathrm{..}}-G_{\mathrm{..}}^{\mathrm{..}})\\ d_2=(-2)(G_{.1}^{.1}-2G_{.1}^{\mathrm{..}}+G_{\mathrm{..}}^{\mathrm{..}})\\ (aa{)}_{12}=E[(G-\mu )w_1{w}_2]/E(w_1^2{w}_2^2)\\ =4(G_{\mathrm{..}}^{11}-G_{1.}^{\mathrm{..}}-G_{.1}^{\mathrm{..}}+G_{\mathrm{..}}^{\mathrm{..}}\\ (ad{)}_{12}=E[(G-\mu )w_1{v}_2]/E(w_1^2{v}_2^2)\\ =(-4)(G_{11}^{.1}-2G_{11}^{\mathrm{..}}-G_{.1}^{.1}+G_{1.}^{\mathrm{..}}+2G_{.1}^{\mathrm{..}}-G_{\mathrm{..}}^{\mathrm{..}})\\ (da{)}_{12}=E[(G-\mu )v_1{w}_2]/E(v_1^2{w}_2^2)\\ =(-4)(G_{11}^{1.}-2G_{11}^{\mathrm{..}}-G_{1.}^{1.}+G_{.1}^{\mathrm{..}}+2G_{1.}^{\mathrm{..}}-G_{\mathrm{..}}^{\mathrm{..}})\\ (dd{)}_{12}=E[(G-\mu )v_1{v}_2]/E(v_1^2{v}_2^2)\\ =4(G_{11}^{11}-2G_{11}^{1.}-2G_{11}^{.1}+G_{1.}^{1.}+G_{.1}^{.1}+4G_{11}^{\mathrm{..}}\\ -2G_{1.}^{\mathrm{..}}-2G_{.1}^{\mathrm{..}}+G_{\mathrm{..}}^{\mathrm{..}})\end{array}$

The orthogonal partition of the genotypic variance is

$\begin{array}{c}{V}_G=\frac{1}{2}{a}_1^2+\frac{1}{4}{d}_1^2+\frac{1}{2}{a}_2^2+\frac{1}{4}{d}_2^2+\frac{1}{4}{(aa)}_{12}^2\\ +\frac{1}{8}{(ad)}_{12}^2+\frac{1}{8}{(da)}_{12}^2+\frac{1}{16}{(dd)}_{12}^2.& \left(17\right)\end{array}$

Recently, Kao and Zeng [18] have examined many genetic and statistical issues of the above F₂ model and the effects of linkage disequilibrium. As we have shown here, the F₂ model is a special case of Cockerham model with two alleles at each locus and all allelic frequencies being 1/2.

Now we give the partition of genetic variance for m loci with epistasis and linkage disequilibrium in the F₂ population. Generalizing model (16) to m loci and ignoring the trigenic and higher order epistasis, we have the following model

$\begin{array}{c}G=\mu +{\displaystyle \sum _{r=1}^m{a}_r{w}_r+{\displaystyle \sum _{r=1}^m{d}_r{v}_r+{\displaystyle \sum _{r<s}{(aa)}_{rs}(w_r{w}_s)}}}\\ +{\displaystyle \sum _{r\ne s}{(ad)}_{rs}(w_r{v}_s)+{\displaystyle \sum _{r<s}{(dd)}_{rs}(v_r{v}_s)}}\\ =\mu +A+D+AA+Ad+DD& \left(18\right)\end{array}$

The partition of genetic variance for this model under the assumption of Hardy-Weinberg equilibrium is

$\begin{array}{l}{V}_G=V_A+V_D+V_{AA}+V_{AD}+V_{DD}\hfill \\ +2\text{Cov}(A,D)+2\text{Cov}(A,AA)+2\text{Cov}(A,AD)\hfill \\ +2\text{Cov(A,DD)+2Cov}(D,AA)+2\text{Cov}(D,AD)\hfill \\ +2\text{Cov(}D,DD)+2Cov(AA,AD)\hfill \\ +2\text{Cov(}AA,DD)+2\text{Cov}(AD,DD).\hfill & \left(19\right)\hfill \end{array}$

The detail of each component is presented in Appendix D.

The F₂ model is a special case of the general two-allele model with p _r = 1/2. Note the difference on the v variable used for the F₂ model and for the general two-allele model below. This partition of genetic variance can provide a basis for the interpretation of genetic variance estimation by multiple interval mapping in a F₂ population [19, 21].

3. A general two-allele model

Here, we provide details of a general two-allele model for multiple loci. This model is probably useful for studying genetic architecture of a quantitative trait in natural populations. Let the two alleles at locus r be A _r and a _r for r = 1, 2, ..., m with m the number of QTL. Assume that the frequencies and genetic effects of alleles are the same for both paternal and maternal gametes. Let p _r denote the frequency of allele A _r at locus r. Note that in this case $z_{M_1}^{(r)}=1-z_{M_2}^{(r)}$, r = 1, 2, ..., m. Also $x_{M_1}^{(r)}=z_{M_1}^{(r)}-E[z_{M_1}^{(r)}]=(1-z_{M_2}^{(r)})-E(1-z_{M_2}^{(r)})=-x_{M_2}^{(r)}$. Similarly, $x_{F_2}^{(r)}=-x_{F_1}^{(r)}$ Ignoring higher order epistasis involving at least three loci, we can define a two-allele model as

$\begin{array}{c}G=\mu +{\displaystyle \sum _{r=1}^m{a}_r{w}_r+{\displaystyle \sum _{r=1}^m{d}_r{v}_r+{\displaystyle \sum _{r<s}{(aa)}_{rs}(w_r{w}_s)}}}\\ +{\displaystyle \sum _{r\ne s}{(ad)}_{rs}(w_r{v}_s)+{\displaystyle \sum _{r<s}{(dd)}_{rs}(v_r{v}_s)}}& \left(20\right)\end{array}$

where

$\begin{array}{c}{w}_r=x_{M_1}^{(r)}+x_{F_1}^{(r)}\\ =\{\begin{array}{cc}(1-p_r)& \text{for }{A}_r{A}_r\text{ at locus }r\\ 1-2p_r& \text{for }{A}_r{a}_r\text{ at locus }r\\ -2p_r& \text{for }{a}_r{a}_r\text{ at locus }r\end{array}\left(21\right)\end{array}$

$\begin{array}{c}{v}_r=x_{M_1}^{(x)}{x}_{F_1}^{(x)}\\ =\{\begin{array}{ll}{(1-p_r)}^2\hfill & \text{for }{A}_r{A}_r\text{ at locus }r\hfill \\ -p_r(1-p_r)\hfill & \text{for }{A}_r{a}_r\text{ at locus }r\hfill \\ p_r^2\hfill & \text{for }{a}_r{a}_r\text{ at locus }r\hfill \end{array}\left(22\right)\end{array}$

for r = 1, 2, ..., m. The coefficients are associated with the original parameters in Cockerham model as follows.

$\begin{array}{cc}{a}_r=& {\alpha }_1^{(r)}-{\alpha }_2^{(r)}\\ d_r=& {\delta }_1^{1(r)}-{\delta }_2^{1(r)}-{\delta }_1^{2(r)}+{\delta }_2^{2(r)}\\ {(aa)}_{rs}=& ({\alpha }_1^{(r)}{\alpha }_1^{(s)})-({\alpha }_1^{(r)}{\alpha }_2^{(s)})-({\alpha }_2^{(r)}{\alpha }_1^{(s)})\\ +({\alpha }_2{\alpha }_2^{(s)})\\ {(ad)}_{rs}=& ({\alpha }_1^{(r)}{\delta }_1^{1(s)})-({\alpha }_1^{(r)}{\delta }_2^{1(s)})-({\alpha }_1^{(r)}{\delta }_1^{2(s)})\\ +({\alpha }_1^{(r)}{\delta }_2^{2(s)})-({\alpha }_2^{(r)}{\delta }_1^{1(s)})+({\alpha }_2^{(r)}{\delta }_2^{1(s)})\\ +({\alpha }_2^{(r)}{\delta }_1^{2(s)})-({\alpha }_2^{(r)}{\delta }_2^{2(s)})\\ {(dd)}_{rs}=& ({\delta }_1^{1(r)}{\delta }_1^{1(s)})-({\delta }_1^{1(r)}{\delta }_2^{1(s)})-({\delta }_1^{1(r)}{\delta }_1^{2(s)})\\ +({\delta }_1^{1(r)}{\delta }_2^{2(s)})-({\delta }_2^{1(r)}{\delta }_1^{1(s)})+({\delta }_2^{1(r)}{\delta }_2^{1(s)})\\ +({\delta }_2^{1(r)}{\delta }_1^{2(s)})-({\delta }_2^{1(r)}{\delta }_2^{2(s)})-({\delta }_1^{2(r)}{\delta }_1^{1(s)})\\ +({\delta }_1^{2(r)}{\delta }_2^{1(s)})+({\delta }_1^{2(r)}{\delta }_1^{2(s)})-({\delta }_1^{2(r)}{\delta }_2^{2(s)})\\ \begin{array}{l}+({\delta }_2^{2(r)}{\delta }_1^{1(s)})-({\delta }_2^{2(r)}{\delta }_2^{1(s)})-({\delta }_2^{2(r)}{\delta }_1^{2(s)})\\ +({\delta }_2^{2(r)}{\delta }_2^{2(s)})\end{array}\end{array}$

The constraint conditions further lead to

$\begin{array}{c}{a}_r={\alpha }_1^{(r)}-\frac{(-p_r)}{(1-p_r)}{\alpha }_1^{(r)}=\frac{1}{(1-p_r)}{\alpha }_1^{(r)}\\ d_r=\frac{1}{{(1-p_r)}^2}{\delta }_1^{1(r)}\\ (aa{)}_{rs}=\frac{1}{(1-p_r)(1-p_s)}({\alpha }_1^{(r)}{\alpha }_1^{(s)})\\ (ad{)}_{rs}=\frac{1}{(1-p_r){(1-p_s)}^2}({\alpha }_1^{(r)}{\delta }_1^{1(s)})\\ (dd{)}_{rs}=\frac{1}{{(1-p_r)}^2{(1-p_s)}^2}({\delta }_1^{1(r)}{\delta }_1^{1(s)})\end{array}$

With Hardy-Weinberg, linkage and genotypic equilibria, the partial regression coefficients in the above model correspond to the simple regression coefficients

$\begin{array}{lll}{a}_r\hfill & =\hfill & \frac{1}{(1-p_r)}[E(G|z_{M_1}^{(r)}=1)-E(G)]\hfill \\ d_r\hfill & =\hfill & \frac{1}{{(1-p_r)}^2}[E(G|z_{M_1}^{(r)}=z_{F_1}^{(r)}=1)\hfill \\ -2E(G|z_{M_1}^{(r)}=1)+E(G)]\hfill \\ {(aa)}_{rs}\hfill & =\hfill & \frac{1}{(1-p_r)(1-p_s)}[E(G|z_{M_1}^{(r)}=z_{M_1}^{(s)}=1)\hfill \\ -E(G|z_{M_1}^{(r)}=1)-E(G|z_{M_1}^{(s)}=1)+E(G)]\hfill \\ {(ad)}_{rs}\hfill & =\hfill & \frac{1}{(1-p_r){(1-p_s)}^2}[E(G|z_{M_1}^{(r)}=z_{M_1}^{(s)}=z_{F_1}^{(s)}=1)\hfill \\ -2E(G|z_{M_1}^{(r)}=z_{M_1}^{(s)}=1)\hfill \\ -E(G|z_{M_1}^{(s)}=z_{F_1}^{(s)}=1)+E(G|z_{M_1}^{(r)}=1)\hfill \\ +2E(G|z_{M_1}^{(s)}=1)-E(G)]\hfill \\ {(dd)}_{rs}\hfill & =\hfill & \frac{1}{{(1-p_r)}^2{(1-p_s)}^2}\hfill \\ [E(G|z_{M_1}^{(r)}=z_{F_1}^{(r)}=z_{M_1}^{(s)}=z_{F_1}^{(s)}=1)\hfill \\ -2E(G|z_{M_1}^{(r)}=z_{F_1}^{(r)}=z_{M_1}^{(s)}=1)\hfill \\ -2E(G|z_{M_1}^{(r)}=z_{M_1}^{(s)}=z_{F_1}^{(s)}=1)\hfill \\ -E(G|z_{M_1}^{(r)}=z_{F_1}^{(r)}=1)\hfill \\ +E(G|z_{M_1}^{(s)}=z_{F_1}^{(r)}=1)\hfill \\ +4E(G|z_{M_1}^{(r)}=z_{F_1}^{(s)}=1)\hfill \\ -2E(G|z_{M_1}^{(r)}=1)-2E(G|z_{M_1}^{(s)}=1)\hfill \\ +E(G)]\hfill \end{array}$

Note that in this case the genetic effects in the original model are

$\begin{array}{lll}{\alpha }_1^{(r)}\hfill & =\hfill & E(G|z_{M_1}^{(r)}=1)-E(G)\hfill \\ {\delta }_1^1(r)\hfill & =\hfill & E(G|z_{M_1}^{(r)}=z_{F_1}^{(r)}=1)\hfill \\ -2E(G|z_{M_1}^{(r)}=1)+E(G)\hfill \\ ({\alpha }_1^{(r)}{\alpha }_1^{(s)})\hfill & =\hfill & E(G|z_{M_1}^{(r)}=z_{M_1}^{(s)}=1)\hfill \\ -E(G|z_{M_1}^{(r)}=1)-E(G|z_{M_1}^{(s)}=1)\hfill \\ +E(G)\hfill \\ ({\alpha }_1^{(r)}{\delta }_1^{1(s)})\hfill & =\hfill & E(G|z_{M_1}^{(r)}=z_{M_1}^{(s)}=z_{F_1}^{(s)}=1)\hfill \\ -2E(G|z_{M_1}^{(r)}=z_{M_1}^{(s)}=1)\hfill \\ -E(G|z_{M_1}^{(s)}=z_{F_1}^{(s)}=1)\hfill \\ +E(G|z_{M_1}^{(r)}=1)+2E(G|z_{M_1}^{(s)}=1)\hfill \\ -E(G)\hfill \\ ({\delta }_1^{1(r)}{\delta }_1^{1(s)})\hfill & =\hfill & E(G|z_{M_1}^{(r)}=z_{F_1}^{(r)}=z_{M_1}^{(s)}=z_{F_1}^{(s)}=1)\hfill \\ -2E(G|z_{M_1}^{(r)}=z_{F_1}^{(r)}=z_{M_1}^{(s)}=1)\hfill \\ -2E(G|z_{M_1}^{(r)}=z_{M_1}^{(s)}=z_{F_1}^{(s)}=1)\hfill \\ +E(G|z_{M_1}^{(r)}=z_{F_1}^{(r)}=1]\hfill \\ +E(G|z_{M_1}^{(s)}=z_{F_1}^{(s)}=1)\hfill \\ +4E(G|z_{M_1}^{(r)}=z_{F_1}^{(s)}=1)\hfill \\ -2E(G|z_{M_1}^{(r)}=1)-2E(G|z_{M_1}^{(s)}=1)\hfill \\ +E(G)\hfill \end{array}$

They are the same as the least squares definition.

Yet another form of this result is shown in Table 1 of [10]. Zeng et al. [10] also show that linkage disequilibrium does not change the values of genetic effects in a full model. This means that the partial regression coefficients in a disequilibrium population equal to the simple regression coefficients in a corresponding equilibrium population with the same allelic frequency configuration.

The partition of genotypic variance in an equilibrium population is

$\begin{array}{c}{V}_G=2{\displaystyle \sum _{r=1}^m{p}_r(1-p_r)a_r^2+{\displaystyle \sum _{r=1}^m{p}_r^2{(1-p_r)}^2{d}_r^2}}\\ +4{\displaystyle \sum _{r<s}{p}_r{p}_s(1-p_r)(1-p_s){(aa)}_{rs}^2}\\ +2{\displaystyle \sum _{r\ne s}{p}_r(1-p_r)p_s^2{(1-p_s)}^2{(ad)}_{rs}^2}\\ +{\displaystyle \sum _{r<s}{p}_r^2{p}_s^2{(1-p_r)}^2{(1-p_s)}^2{(dd)}_{rs}^2}& \left(23\right)\end{array}$

or

$\begin{array}{c}{V}_G=\frac{2p_r}{(1-p_r)}{\displaystyle \sum _{r=1}^m{({\alpha }_1^{(r)})}^2+\frac{p_r^2}{{(1-p_r)}^2}}{\displaystyle \sum _{r=1}^m{({\delta }_1^{1(r)})}^2}\\ +\frac{4p_r{p}_s}{(1-p_r)(1-p_s)}{\displaystyle \sum _{r<s}{({\alpha }_1^{(r)}{\alpha }_1^{(s)})}^2}\\ +\frac{2p_r{p}_s^2}{(1-p_r){(1-p_s)}^2}{\displaystyle \sum _{r\ne s}{({\alpha }_1^{(r)}{\delta }_1^{1(s)})}^2}\\ +\frac{p_r^2{p}_s^2}{{(1-p_r)}^2{(1-p_s)}^2}{\displaystyle \sum _{r<s}{({\delta }_1^{1(r)}{\delta }_1^{1(s)})}^2}\end{array}\left(24\right)$

The partition of the genetic variance with epistasis and linkage disequilibrium is complex. We give the result with trigenic and quadrigenic linkage disequilibria included as well. The partition of variance has a similar form as (19). The detail of each component is presented in Appendix E.

Discussion

In this paper we explore various properties of the standard quantitative genetic model with multiple interacting loci in linkage equilibrium and disequilibrium. Starting from the traditional least squares model, we represent it in the setting of multiple regression with standardized allelic indicator variables and their products as the independent variables and the trait value as the dependent variable. Then the partial regression coefficients associated with these indicator variables define the additive, dominance and epistatic effects for QTL. This is the original definition of QTL effects introduced by Fisher [6] and extended to epistasis by Cockerham [7]. We examine the properties and meaning of these QTL effects in an equilibrium population and also in a disequilibrium population. We show details of the partition of genetic variance for both equilibrium and disequilibrium populations in terms of QTL effects, allelic frequencies and disequilibrium measures. Moreover, we relate this general model to several reduced models used for QTL mapping analysis in cross populations from inbred lines, such as F₂, backcross and recombinant inbred lines. The detailed partition of genetic variance in these populations can provide a basis for the interpretation of genetic variance component estimates from multiple interval mapping [21].

The purpose of modeling QTL is to provide a meaningful and convenient framework and basis to infer and interpret relative significance of each QTL and intricate inter-relationship among QTL on a set of quantitative traits in an experimental or natural population for genetic study. The linear model provides a framework to study the effects of QTL on the mean and variance of the distribution of a trait or multiple traits in a population. With the assumption of a normal distribution for both genotypic and phenotypic values of a quantitative trait or multiple traits, this analysis on the first and second order statistics is sufficient to characterize the relationship between QTL and trait (s). Otherwise, it is an approximation on the relationship. In this model, the model parameters are partitioned into two parts: one is the effects of QTL (additive, dominance and epistatic effects), and the other is frequencies and correlations (Hardy-Weinberg and linkage disequilibria) of QTL alleles. Together they characterize the genetic architecture of quantitative traits in a population.

This linear model also provides a framework for statistical inference of genetic model parameters. If QTL genotypes are known and directly observed, a regression analysis of trait phenotype on QTL genotypes would provide a direct estimation of the genetic model parameters. However, if QTL genotypes are unknown and are only indirectly observed through molecular markers, the statistical inference of QTL and model parameters becomes more complicated. Statistically, we can regard QTL genotypes as missing data with trait phenotypes and marker genotypes as observed data and use a mixture model through the maximum likelihood analysis to infer the conditional distribution of missing data and through that to infer QTL parameters which also include the number and genomic position of QTL [19, 24, 25]. The likely positions of QTL are searched in the whole genome if data permit and the number of significant QTL positions can be estimated through some model selection procedure.

On modeling QTL, the consistence of model parameters in a multiple-locus setting is an important consideration. It is important for a model to be multiple-locus consistent, and the relationship within and between loci can be clearly and readily analyzed, estimated and interpreted. Here the consistence means that the effect of a QTL is consistently defined in a reference equilibrium population for one, two or multiple loci. In statistics, this is the property of orthogonality. This property is particularly important for the study of epistasis. With that the additive, dominance and epistatic effects can be independently and consistently estimated for one, two, three or multiple loci in the reference population where the model is defined and interpreted. Thus, if the number of QTL is incorrectly identified which seems to be always the case in practice, the parameter values for those identified QTL can still be consistently estimated. However, the situation would certainly be different and complicated if the population is not at equilibrium, for example for QTL in linkage disequilibrium. Linkage disequilibrium would complicate the partition of genetic variance, and could certainly bias the estimation of parameter values for those identified QTL if the QTL model (number and genomic position of QTL) is miss-identified.

In this paper, we study extensively the composition and property of the genetic model parameters, such as genetic effects and partition of genetic variance, when both epistasis and linkage disequilibrium are considered. This would help us to understand the relationship of various genetic quantities, such as allelic frequencies and linkage disequilibrium, on the definition of genetic effects. It would also help us to understand and properly interpret estimates of the genetic effects and variance components in a QTL mapping experiment. It is important to emphasize that modeling QTL is inherently population based as it defines the variation of QTL in reference to a population, either a study population, cross population or natural population. The very basic concept of additive effect of a QTL is a population concept and is population dependent. It depends on the genotypes at other loci and depends on the genetic structure of the population (allelic frequencies, Hardy-Weinberg and linkage disequilibria).

We also clarify the connection between the general genetic model and some reduced models. By restricting the number of alleles at each locus to two and setting allelic frequencies to half, the general genetic model is reduced to the F₂ model. This simplification reduces the partition of genetic variance enormously.

Another property for this F₂ population is that, if there is no crossing-over interference, the three-locus linkage disequilibrium is expected to be zero regardlessly whether the loci are linked. Also the four-locus disequilibrium is reduced to the product of the two-locus disequilibria for the two non-adjacent locus pairs. If there is crossing-over interference, the three-locus linkage disequilibrium would be a good measure of the interference. As many QTL mapping experiments are performed in a F₂ population, this reduced model is very relevant to QTL mapping analysis for the interpretation of genetic architecture in a F₂ population. Another reduced model is the backcross model which is essentially a haploid model.

We give many details for a general two-allele model with epistasis and linkage disequilibrium. Research on QTL mapping analysis has been shifted in recent years from inbred line crosses to natural populations. With the availability of very dense SNP markers, it is now possible to use SNP for fine mapping of QTL in a natural population. Currently most QTL fine mapping studies are concentrated on candidate genes. It will be increasingly possible to have genome-wide SNP data for a sample of individuals from a natural population. The general two-allele model can be used as a framework to interpret and estimate the genome-wide genetic architecture for a quantitative trait in a natural population. The model can be extended to multiple alleles to take haplotypes into account if needed.

Appendix

A. Cockerham least squares estimates

In this appendix, we show that the regression coefficients (genetic effects) in model (7) are Cockerham least squares estimates under Hardy-Weinberg, linkage and genotypic equilibria. First, note that at each locus only one allele is present on a gamete. That is if an individual inherits an allele A _i from a parental gamete, $z_{M_i}^{(1)}$ = 1 and the other $z_{M_j}^{(1)}$ = 0 for j ≠ i Therefore, when i ≠ j, we have E($z_{M_i}^{(1)}{z}_{M_j}^{(1)}$) = 0 or $E(x_{M_i}^{(1)}{x}_{M_j}^{(1)})=E(z_{M_i}^{(1)}{z}_{M_j}^{(1)})-E(z_{M_i}^{(1)})E(z_{M_j}^{(1)})=-p^i{p}^j$. Using these relationships, we can show the following for model (7).

• Additive effects: For i = 1, 2, ..., n₁, we can show

$\begin{array}{l}E[(G-\mu )x_{M_i}^{(1)}]=E[({\displaystyle \sum _{i\text{'}}{\alpha }^{i\text{'}}{x}_{M_{i\text{'}}}^{(1)})x_{M_i}^{(1)}]={\alpha }^{i}E(x_{M_i}^{(1)2})+{\displaystyle \sum _{i\text{'}\ne i}{\alpha }^{i\text{'}}}E(x_{M_i^{\text{'}}}^{(1)}{x}_{M_i}^{(1)}})\\ ={\alpha }^i{p}^i(1-p^i)+{\displaystyle \sum _{i\text{'}\ne i}{\alpha }^{i\text{'}}}(-p^{i\text{'}}{p}^i)={\alpha }^i{p}^i+(-p^i){\displaystyle \sum _{i\text{'}}{\alpha }^{i\text{'}}}{p}^{i\text{'}}={\alpha }^i{p}^i,\end{array}$

On the other hand,

$\begin{array}{l}E[(G-\mu )x_{M_i}^{(1)}]=E(G{x}_{M_i}^{(1)})=E[G(z_{M_i}^{(1)}-p^i)]=E(G{z}_{M_i}^{(1)})-p^{i}E(G)\\ =E[E(G{z}_{M_i}^{(1)}|z_{M_i}^{(1)})]-p^i{G}_{\mathrm{..}}^{\mathrm{..}}=p^{i}E(G|z_{M_i}^{(1)}=1)-p^i{G}_{\mathrm{..}}^{\mathrm{..}}=p^i(G_{\mathrm{..}}^{i.}-G_{\mathrm{..}}^{\mathrm{..}}).\end{array}$

Therefore, ${\alpha }^i=G_{\mathrm{..}}^{i.}-G_{\mathrm{..}}^{\mathrm{..}}$ for i = 1, 2, ..., n₁. Similarly, we can show that ${\alpha }_j=G_{j.}^{\mathrm{..}}-G_{\mathrm{..}}^{\mathrm{..}}$ for j = 1, 2, ..., n₁, and ${\beta }^k=G_{\mathrm{..}}^{.k}-G_{\mathrm{..}}^{\mathrm{..}}$, ${\beta }_l=G_{.l}^{\mathrm{..}}-G_{\mathrm{..}}^{\mathrm{..}}$ for k, l = 1, 2, ..., n₂.

• Dominance effects: For locus 1, we have

$\begin{array}{l}E[(G-\mu )x_{M_i}^{(1)}{x}_{F_j}^{(1)}]=E[({\displaystyle \sum _{i\text{'},j\text{'}}{\delta }_{j\text{'}}^{i\text{'}}{x}_{M_{i\text{'}}}^{(1)}{x}_{F_{j\text{'}}}^{(1)})x_{M_i}^{(1)}{x}_{F_j}^{(1)}]}\\ ={\displaystyle \sum _{j\text{'}}{\delta }_{j\text{'}}^{i}E(x_{M_i}^{(1)2})E(x_{F_j}^{(1)}{x}_{F_{j\text{'}}}^{(1)})+{\displaystyle \sum _{i\text{'}\ne i}{\displaystyle \sum _{j\text{'}}{\delta }_{j\text{'}}^{i\text{'}}E(x_{M_i}^{(1)}{x}_{M_{i\text{'}}}^{(1)})E(x_{F_j}^{(1)}{x}_{F_{j\text{'}}}^{(1)})}}}\\ =p^i(1-p^i){\displaystyle \sum _{j\text{'}}{\delta }_{j\text{'}}^{i}E(x_{F_j}^{(1)}{x}_{F_{j\text{'}}}^{(1)})+{\displaystyle \sum _{i\text{'}\ne i}(-p^i{p}^{i\text{'}}){\displaystyle \sum _{j\text{'}}{\delta }_{j\text{'}}^{i\text{'}}E(x_{F_j}^{(1)}{x}_{F_{j\text{'}}}^{(1)}}})}\\ =p^i{\displaystyle \sum _{j\text{'}}{\delta }_{j\text{'}}^{i}E(x_{F_j}^{(1)}{x}_{F_j\text{'}}^{(1)})-{\displaystyle \sum _{i\text{'}}{p}^i{p}^{i\text{'}}}{\displaystyle \sum _{j\text{'}}{\delta }_{j\text{'}}^{i}E(x_{F_j}^{(1)}{x}_{F_{j\text{'}}}^{(1)})}}\\ =p^i(p_j{\delta }_j^i+{\displaystyle \sum _{j\text{'}}{\delta }_{j\text{'}}^i{p}_{j\text{'}})-p^i{\displaystyle \sum _{j\text{'}}E(x_{F_j}^{(1)}{x}_{F_{j\text{'}}}^{(1)}){\displaystyle \sum _{i\text{'}}{p}^{i\text{'}}{\delta }_{j\text{'}}^{i\text{'}}}}}\\ =p^i{p}_j{\delta }_j^i\text{ for }i,j=1,2,\dots ,n_1.\end{array}$

On the other hand,

$\begin{array}{l}E[(G-\mu )x_{M_i}^{(1)}{x}_{F_j}^{(1)}]=E[G(z_{M_i}^{(1)}-p^i)(z_{F_j}^{(1)}-p_j)]\\ =E(G{z}_{M_i}^{(1)}{z}_{F_j}^{(1)}-p^{i}E(G{z}_{F_j}^{(1)})-p_{j}E(G{z}_{M_i}^{(1)})+p^i{p}_{j}E(G)\\ =E[E(G{z}_{M_i}^{(1)}{z}_{F_j}^{(1)}|z_{M_i}^{(1)},z_{F_j}^{(1)})]-p^{i}E[E(G{z}_{F_j}^{(1)}|z_{F_j}^{(1)}])-p_{j}E[E(G{z}_{M_i}^{(1)}|z_{M_i}^{(1)})]+p^i{p}_j{G}_{\mathrm{..}}^{\mathrm{..}}\\ =p^i{p}_j(G_{j.}^{i.}-G_{\mathrm{..}}^{i.}-G_{j.}^{\mathrm{..}}+G_{\mathrm{..}}^{\mathrm{..}})\end{array}$

Therefore, ${\delta }_j^i=G_{j.}^{i.}-G_{\mathrm{..}}^{i.}-G_{j.}^{\mathrm{..}}+G_{\mathrm{..}}^{\mathrm{..}}$ for i, j = 1, 2, ..., n₁. Similar results can be derived for other dominance terms at locus 2.

• Additive × additive effects: Note that

$\begin{array}{l}E[(G-\mu )x_{M_i}^{(1)}{x}_{M_k}^{(2)}]={\displaystyle \sum _{i^{\prime },k^{\prime }}({\alpha }^{i^{\prime }}{\beta }^{k^{\prime }})E(x_{M_i}^{(1)}{x}_{M_{i^{\prime }}}^{(1)})E(x_{M_k}^{(2)}{x}_{M_{k^{\prime }}}^{(2)}})\\ ={\displaystyle \sum _{k^{\text{'}}}({\alpha }^i{\beta }^{k^{\prime }})E(x_{M_i}^{(1)2})E(x_{M_k}^{(2)}{x}_{M_{k^{\prime }}}^{(2)})}+{\displaystyle \sum _{i^{\prime }\ne i}{\displaystyle \sum _{k^{\prime }}({\alpha }^{i^{\prime }}{\beta }^{k^{\prime }})}(-p^i{p}^{i^{\prime }})E(x_{M_k}^{(2)}{x}_{M_{k^{\prime }}}^{(2)})}\\ =p^i{\displaystyle \sum _{k^{\prime }}({\alpha }^i{\beta }^{k^{\prime }})}E(x_{M_k}^{(2)}{x}_{M_{k^{\prime }}}^{(2)})-p^i{\displaystyle \sum _{i^{\prime }}{\displaystyle \sum _{k^{\prime }}({\alpha }^{i^{\prime }}{\beta }^{k^{\prime }})}{p}^{i^{\prime }}E(x_{M_k}^{(2)}{x}_{M_{k^{\prime }}}^{(2)})}\\ =p^i{q}^k({\alpha }^i{\beta }^{k^{\prime }})\end{array}$

and

$E[(G-\mu )x_{M_i}^{(1)}{x}_{M_k}^{(2)}]=E(G{x}_{M_i}^{(1)}{x}_{M_k}^{(2)})=E[G(z_{M_i}^{(1)}-p^i)(z_{M_k}^{(2)}-q^k)]=p^i{q}^k(G_{\mathrm{..}}^{ik}-G_{\mathrm{..}}^{i.}-G_{\mathrm{..}}^{.k}+G_{\mathrm{..}}^{\mathrm{..}})$

We have $({\alpha }^i{\beta }^k)=G_{\mathrm{..}}^{ik}-G_{\mathrm{..}}^{i.}-G_{\mathrm{..}}^{.k}+G_{\mathrm{..}}^{\mathrm{..}}$ for i = 1, 2, ..., n₁ and k = 1, 2, ..., n₂. Similar results can be derived for other additive by additive terms.

• Additive × dominance effects: Note that

$\begin{array}{l}E[(G-\mu )x_{M_i}^{(1)}{x}_{M_k}^{(2)}{x}_{F_l}^{(2)}]=E[x_{M_i}^{(1)}{x}_{M_k}^{(2)}{x}_{F_l}^{(2)}{\displaystyle \sum _{i^{\prime },k^{\prime },l^{\prime }}({\alpha }^{i^{\prime }}{\gamma }_{l^{\prime }}^{k^{\prime }})x_{M_{i^{\prime }}}^{(1)}{x}_{M_{k^{\prime }}}^{(2)}{x}_{F_{l^{\prime }}}^{(2)}}]\\ ={\displaystyle \sum _{k^{\prime },l^{\prime }}({\alpha }^i{\gamma }_{l^{\prime }}^{k^{\prime }})E}(x_{M_i}^{(1)2})E(x_{M_k}^{(2)}{x}_{M_{k^{\prime }}}^{(2)})E(x_{F_l}^{(2)}{x}_{F_{l^{\prime }}}^{(2)})=p^i{\displaystyle \sum _{k^{\prime },l^{\prime }}({\alpha }^i{\gamma }_{l^{\prime }}^{k^{\prime }})E}(x_{M_k}^{(2)}{x}_{M_{k^{\prime }}}^{(2)})E(x_{F_l}^{(2)}{x}_{F_{l^{\prime }}}^{(2)})=p^i{q}^k{q}_l({\alpha }^i{\gamma }_l^k)\end{array}$

and

$\begin{array}{l}E[(G-\mu )x_{M_i}^{(1)}{x}_{M_k}^{(2)}{x}_{F_l}^{(2)}]=E[G(z_{M_i}^{(1)}-p^i)(z_{M_k}^{(2)}-q^k)(z_{F_l}^{(2)}-q_l)]\\ =p^i{q}^k{q}_l(G_{.l}^{ik}-G_{\mathrm{..}}^{ik}-G_{.l}^{i.}-G_{.l}^{.k}+G_{\mathrm{..}}^{i.}+G_{\mathrm{..}}^{.k}+G_{.l}^{\mathrm{..}}-G_{\mathrm{..}}^{\mathrm{..}})\end{array}$

For i = 1, 2, ..., n₁ and k, l = 1, 2, ..., n₂, we have

$({\alpha }^i{\gamma }_l^k)=G_{.l}^{ik}-G_{\mathrm{..}}^{ik}-G_{.l}^{i.}-G_{.l}^{.k}+G_{\mathrm{..}}^{i.}+G_{\mathrm{..}}^{.k}+G_{.l}^{\mathrm{..}}-G_{\mathrm{..}}^{\mathrm{..}}$

Similar results can be derived for dominance by additive terms.

• Dominance × dominance effects: Note that

$\begin{array}{l}E[(G-\mu )x_{M_i}^{(1)}{x}_{F_j}^{(1)}{x}_{M_k}^{(2)}{x}_{F_l}^{(2)}]=E[x_{M_i}^{(1)}{x}_{F_j}^{(1)}{x}_{M_k}^{(2)}{x}_{F_l}^{(2)}{\displaystyle \sum _{i^{\prime },j^{\prime },k^{\prime },l^{\prime }}({\delta }_{j^{\prime }}^{i^{\prime }}{\gamma }_{l^{\prime }}^{k^{\prime }})}{x}_{M_{i^{\prime }}}^{(1)}{x}_{F_{j^{\prime }}}^{(1)}{x}_{M_{k^{\prime }}}^{(2)}{x}_{F_{l^{\prime }}}^{(2)}]\\ ={\displaystyle \sum _{i^{\prime },j^{\prime }}E(x_{M_i}^{(1)}{x}_{M_{i^{\prime }}}^{(1)})E(x_{F_j}^{(1)}{x}_{F_{j^{\prime }}}^{(1)})}{\displaystyle \sum _{k^{\prime },l^{\prime }}({\delta }_{j^{\prime }}^{i^{\prime }}{\gamma }_{l^{\prime }}^{k^{\prime }})E(x_{M_k}^{(2)}{x}_{M_{k^{\prime }}}^{(2)})}E(x_{F_l}^{(2)}{x}_{F_{l^{\prime }}}^{(2)})\\ ={\displaystyle \sum _{i^{\prime },j^{\prime }}E(x_{M_i}^{(1)}{x}_{M_{i^{\prime }}}^{(1)})E(x_{F_j}^{(1)}{x}_{F_{j^{\prime }}}^{(1)})}({\delta }_{j^{\prime }}^{i^{\prime }}{\gamma }_l^k)q^k{q}_l=({\delta }_j^i{\gamma }_l^k)p^i{p}_j{q}^k{q}_l\end{array}$

On the other hand

$\begin{array}{l}E[(G-\mu )x_{M_i}^{(1)}{x}_{F_j}^{(1)}{x}_{M_k}^{(2)}{x}_{F_l}^{(2)}]=E[G(z_{M_i}^{(1)}-p^i)(x_{F_j}^{(1)}-p_j)(z_{M_k}^{(2)}-q^k)(z_{F_l}^{(2)}-q_l)]\\ =p^i{p}_j{q}^k{q}_l(G_{jl}^{ik}-G_{j.}^{ik}-G_{.l}^{ik}-G_{jl}^{i.}-G_{jl}^{.k}+G_{\mathrm{..}}^{ik}+G_{jl}^{\mathrm{..}}\\ +G_{j.}^{i.}+G_{.l}^{.k}+G_{j.}^{.k}+G_{.l}^{i.}-G_{\mathrm{..}}^{i.}-G_{\mathrm{..}}^{.k}-G_{j.}^{\mathrm{..}}-G_{.l}^{\mathrm{..}}+G_{\mathrm{..}}^{\mathrm{..}})\end{array}$

For i, j = 1, 2, ..., n₁ and k, l = 1, 2, ..., n₂, we have

$({\delta }_j^i{\gamma }_l^k)=G_{jl}^{ik}-G_{j.}^{ik}-G_{.l}^{ik}-G_{jl}^{i.}-G_{jl}^{.k}+G_{\mathrm{..}}^{ik}+G_{jl}^{\mathrm{..}}+G_{j.}^{i.}+G_{.l}^{.k}+G_{j.}^{.k}+G_{.l}^{i.}-G_{\mathrm{..}}^{i.}-G_{\mathrm{..}}^{.k}-G_{j.}^{\mathrm{..}}-G_{.l}^{\mathrm{..}}+G_{\mathrm{..}}^{\mathrm{..}}$

B. Partition of genotypic variance in linkage equilibrium

Here we show that the genotypic variance V _G of model (7) has the orthogonal partition (4) under Hardy-Weinberg, linkage and genotypic equilibria. First, note that the index variables $x_{M_i}^{(1)}$, $x_{F_j}^{(1)}$, $x_{M_k}^{(2)}$, $x_{F_l}^{(2)}$ have expectation zero. Second, the assumption of Hardy-Weinberg, linkage and genotypic equilibria mean that all alleles in different gametes and loci are independent so that, for example,

$Cov(x_{M_i}^{(r)},x_{F_j}^{(r)})=E(x_{M_i}^{(r)}{x}_{F_j}^{(r)})=E(x_{M_i}^{(r)})E(x_{F_j}^{(r)})=0$

Thus the additive and dominance effects within a locus are orthogonal to each other because

$Cov(x_{M_k}^{(r)},x_{M_i}^{(r)},x_{F_j}^{(r)})=E(x_{M_k}^{(r)}{x}_{M_i}^{(r)}{x}_{F_j}^{(r)})=E(x_{M_k}^{(r)})E(x_{M_i}^{(r)})E(x_{F_j}^{(r)})=0$

for any 1 ≤ i, j, k ≤ n₁ and locus r = 1 or 2. Similarly, the epistatic effects between loci are orthogonal to additive and dominance effects and also to other epistatic effects. Therefore, the total genotypic variance V _G can be partitioned as

$V_G=V_{A_1}+V_{A_2}+V_{D_1}+V_{D_2}+V_{A_1{A}_2}+V_{A_1{D}_2}+V_{D_1{A}_2}+V_{D_1{D}_2}$

with each component analyzed below.

• The additive variance: For locus 1,

as ${\displaystyle \sum _{i=1}^{n_1}{\alpha }^i{p}^i=0}$ by the constrain condition (2). Similarly, for locus 2, we have $V_{A_2}={\displaystyle \sum _{k=1}^{n_2}{({\beta }^k)}^2{q}^k+}{\displaystyle \sum _{j=1}^{n_2}{({\beta }_l)}^2{q}_l}$.

• The dominance variance: For locus 1,

Similarly, for locus 2, we have $V_{D_2}={\displaystyle \sum _{k,l}{({\gamma }_l^k)}^2{q}^k{q}_l}$.

• The additive × additive variance:

$\begin{array}{c}{V}_{A_1{A}_2}=Var({\displaystyle \sum _{i,k}({\alpha }^i{\beta }^k)x_{M_i}^{(1)}{x}_{M_k}^{(2)})}+Var({\displaystyle \sum _{i,l}({\alpha }^i{\beta }_l)x_{M_i}^{(1)}{x}_{F_l}^{(2)})}\\ +Var({\displaystyle \sum _{j,k}({\alpha }_j{\beta }^k)x_{F_j}^{(1)}{x}_{M_k}^{(2)})}+Var({\displaystyle \sum _{j,l}({\alpha }_j{\beta }_l)x_{F_j}^{(1)}{x}_{F_l}^{(2)})}\\ ={\displaystyle \sum _{i,k}{p}^i{q}^k{({\alpha }^i{\beta }^k)}^2+{\displaystyle \sum _{j,l}{p}_j{q}_l{({\alpha }_j{\beta }_l)}^2}+}{\displaystyle \sum _{i,l}{p}^i{q}_l{({\alpha }^i{\beta }_l)}^2+}{\displaystyle \sum _{j,k}{p}_j{q}^k{({\alpha }_j{\beta }^k)}^2}\end{array}$

• The additive × dominance variance:

$\begin{array}{c}{V}_{A_1{D}_2}=Var({\displaystyle \sum _{i,k,l}({\alpha }^i{\gamma }_l^k)}{x}_{M_i}^{(1)}{x}_{M_k}^{(2)}{x}_{F_l}^{(2)})+Var({\displaystyle \sum _{j,k,l}({\alpha }_j{\gamma }_l^k)}{x}_{F_j}^{(1)}{x}_{M_k}^{(2)}{x}_{F_l}^{(2)})\\ ={\displaystyle \sum _{i,k,l}{p}^i{q}^k{q}_l{({\alpha }^i{\gamma }_l^k)}^2}+{\displaystyle \sum _{j,k,l}{p}_j{q}^k{q}_l{({\alpha }_j{\gamma }_l^k)}^2}\end{array}$

Similarly, the dominance × additive variance is $V_{D_1{A}_2}={\displaystyle \sum _{i,j,k}{p}^i{p}_j{q}^k{({\delta }_j^i{\beta }^k)}^2}+{\displaystyle \sum _{i,j,l}{p}^i{p}_j{q}_l{({\delta }_j^i{\beta }_l)}^2}$.

• The dominance × dominance variance:

C. Partition of genotypic variance in linkage disequilibrium

We present the partition of the genotypic variance based on model (7) under the assumption of Hardy-Weinberg equilibrium but linkage disequilibrium.

• The additive variance:

$V_{A_1}={\displaystyle \sum _{i=1}^{n_1}{p}^i{({\alpha }^i)}^2+}{\displaystyle \sum _{j=1}^{n_1}{p}_j{({\alpha }_j)}^2,V_{A_2}={\displaystyle \sum _{k=1}^{n_2}{q}^k{({\beta }^k)}^2+}{\displaystyle \sum _{j=1}^{n_2}{q}_l{({\beta }_l)}^2}}$

• The dominance variance:

$V_{D_1}={\displaystyle \sum _{i,j}{({\delta }_j^i)}^2{p}^i{p}_j,V_{D_2}=}{\displaystyle \sum _{k,l}{({\gamma }_l^k)}^2{q}^k{q}_l}$

• The additive × additive variance:

$\begin{array}{ll}{V}_{A_1{A}_2}=\hfill & {\displaystyle \sum _{i,k}(p^i{q}^k+D_{\mathrm{..}}^{ik}){({\alpha }^i{\beta }^k)}^2-[{\displaystyle \sum _{i,k}({\alpha }^i{\beta }^k)}}{D}_{\mathrm{..}}^{ik}{]}^2+{\displaystyle \sum _{j,l}(p_j{q}_l+D_{jl}^{\mathrm{..}}){({\alpha }_j{\beta }_l)}^2}\hfill \\ -[{\displaystyle \sum _{j,l}({\alpha }_j{\beta }_l)D_{jl}^{\mathrm{..}}{]}^2+{\displaystyle \sum _{i,l}{p}^i{q}_l{({\alpha }^i{\beta }_l)}^2+{\displaystyle \sum _{j,k}{p}_j{q}^k{({\alpha }_j{\beta }^k)}^2+2}{\displaystyle \sum _{i,j,k,l}({\alpha }^i{\beta }^k)({\alpha }_j{\beta }_l)D_{jl}^{\mathrm{..}}{D}_{\mathrm{..}}^{ik}}}}\hfill \\ +2{\displaystyle \sum _{i,j,k,l}({\alpha }^i{\beta }_l)({\alpha }_j{\beta }^k)D_{jl}^{\mathrm{..}}{D}_{\mathrm{..}}^{ik}}+2({\displaystyle \sum _{i,k}({\alpha }^i{\beta }^k)D_{\mathrm{..}}^{ik})}({\displaystyle \sum _{j,l}({\alpha }_j{\beta }_l)D_{jl}^{\mathrm{..}})}\hfill \end{array}$

• The additive × dominance variance:

$V_{A_1{D}_2}={\displaystyle \sum _{i,k,l}{q}_l(p^i{q}^k+D_{\mathrm{..}}^{ik})}{({\alpha }^i{\gamma }_l^k)}^2+{\displaystyle \sum _{j,k,l}{q}^k(p_j{q}_l+D_{jl}^{\mathrm{..}})}{({\alpha }_j{\gamma }_l^k)}^2+2{\displaystyle \sum _{i,j,k,l}({\alpha }^i{\gamma }_l^k)}({\alpha }_j{\gamma }_l^k)D_{jl}^{\mathrm{..}}{D}_{\mathrm{..}}^{ik}$

• The dominance × additive variance:

$V_{D_1{A}_2}={\displaystyle \sum _{i,j,k}{p}_j(p^i{q}^k+D_{\mathrm{..}}^{ik}){({\delta }_j^i{\beta }^k)}^2+}{\displaystyle \sum _{i,j,l}{p}^i(p_j{q}_l+D_{jl}^{\mathrm{..}}){({\delta }_j^i{\beta }_l)}^2+2}{\displaystyle \sum _{i,j,k,l}({\delta }_j^i{\beta }^k)}({\delta }_j^i{\beta }_l)D_{jl}^{\mathrm{..}}{D}_{\mathrm{..}}^{ik}$

• The dominance × dominance variance:

$V_{D_1{D}_2}={\displaystyle \sum _{i,j,k,l}{({\delta }_j^i{\gamma }_l^k)}^2(D_{\mathrm{..}}^{ik}+p^i{q}^k)(D_{jl}^{\mathrm{..}}+p_j{q}_l)}$

• The covariances related to additive and dominance effects:

$\begin{array}{l}Cov(A_1,A_2)={\displaystyle \sum _{i,k}({\alpha }^i{\beta }^k)D_{\mathrm{..}}^{ik}}+{\displaystyle \sum _{j,l}({\alpha }_j{\beta }_l)D_{jl}^{\mathrm{..}}}\\ Cov(D_1,D_2)={\displaystyle \sum _{i,j,k,l}({\delta }_j^i)}({\gamma }_l^k)D_{jl}^{\mathrm{..}}{D}_{\mathrm{..}}^{ik}\\ Cov(A_1,D_1)=Cov(A_1,D_2)=Cov(A_2,D_1)=Cov(A_2,D_2)=0\end{array}$

• The covariances related to additive × additive effects:

$\begin{array}{l}Cov(A_1,A_1{A}_2)={\displaystyle \sum _{i,k}({\alpha }^i)({\alpha }^i{\beta }^k)D_{\mathrm{..}}^{ik}+{\displaystyle \sum _{j,l}({\alpha }_j)({\alpha }_j}{\beta }_l)D_{jl}^{\mathrm{..}}}\\ Cov(A_2,A_1{A}_2)={\displaystyle \sum _{i,k}({\beta }^k)({\alpha }^i{\beta }^k)D_{\mathrm{..}}^{ik}+{\displaystyle \sum _{j,l}({\beta }_l)({\alpha }_j}{\beta }_l)D_{jl}^{\mathrm{..}}}\\ Cov(D_1,A_1{A}_2)={\displaystyle \sum _{i,j,k}{p}_j({\delta }_j^i)({\alpha }_j{\beta }^k)D_{\mathrm{..}}^{ik}+{\displaystyle \sum _{i,j,l}{p}^i({\delta }_j^i)({\alpha }^i}{\beta }_l)D_{jl}^{\mathrm{..}}}\\ Cov(D_2,A_1{A}_2)={\displaystyle \sum _{i,k,l}{q}_l({\alpha }^i{\beta }_l)({\gamma }_l^k)D_{\mathrm{..}}^{ik}+{\displaystyle \sum _{j,k,l}{q}^k({\alpha }_j}{\beta }^k)({\gamma }_l^k)D_{jl}^{\mathrm{..}}}\end{array}$

The covariances related to additive × dominance and dominance × additive effects:

$\begin{array}{ccc}Cov(A_1,A_1{D}_2)& =& {\displaystyle \sum _{i,j,k,l}[({\alpha }^i)({\alpha }_j{\gamma }_l^k)+({\alpha }_j)({\alpha }^i{\gamma }_l^k)]D_{jl}^{\mathrm{..}}}{D}_{\mathrm{..}}^{ik}\\ Cov(A_2,A_1{D}_2)& =& {\displaystyle \sum _{i,k,l}{q}_l({\beta }_l)({\alpha }^i{\gamma }_l^k)D_{\mathrm{..}}^{ik}+{\displaystyle \sum _{j,k,l}{q}^k({\beta }^k)({\alpha }_j{\gamma }_l^k)}{D}_{jl}^{\mathrm{..}}}\\ Cov(A_1,D_1{A}_2)& =& {\displaystyle \sum _{i,j,k}{p}_j({\alpha }_j)({\delta }_j^i{\beta }^k)D_{\mathrm{..}}^{ik}+{\displaystyle \sum _{i,j,l}{p}^i({\alpha }^i)({\delta }_j^i{\beta }_l)}{D}_{jl}^{\mathrm{..}}}\\ Cov(A_2,D_1{A}_2)& =& {\displaystyle \sum _{i,j,k,l}[({\beta }^k)({\delta }_j^i{\beta }_l)+({\beta }_l)({\delta }_j^i{\beta }^k)]D_{jl}^{\mathrm{..}}}{D}_{\mathrm{..}}^{ik}\\ Cov(D_1,A_1{D}_2)& =& {\displaystyle \sum _{i,j,k,l}({\delta }_j^i)[({\alpha }^i{\gamma }_l^k)+({\alpha }_j{\gamma }_l^k)]D_{jl}^{\mathrm{..}}}{D}_{\mathrm{..}}^{ik}\\ Cov(D_2,A_1{D}_2)& =& {\displaystyle \sum _{i,k,l}{q}_l({\gamma }_l^k)({\alpha }^i{\gamma }_l^k)}{D}_{\mathrm{..}}^{ik}+{\displaystyle \sum _{j,k,l}{q}^k({\gamma }_l^k)({\alpha }_j{\gamma }_l^k)}{D}_{jl}^{\mathrm{..}}\\ Cov(D_1,D_1{A}_2)& =& {\displaystyle \sum _{i,j,k}{p}_j({\delta }_j^i)({\delta }_j^i{\beta }^k)}{D}_{\mathrm{..}}^{ik}+{\displaystyle \sum _{i,j,l}{p}^i({\delta }_j^i)({\delta }_j^i{\beta }_l)}{D}_{jl}^{\mathrm{..}}\\ Cov(D_2,D_1{A}_2)& =& {\displaystyle \sum _{i,j,k,l}({\gamma }_l^k)[({\delta }_j^i{\beta }^k)+}({\delta }_j^i{\beta }_l)]D_{jl}^{\mathrm{..}}{D}_{\mathrm{..}}^{ik}\\ Cov(A_1{A}_2,A_1{D}_2)& =& {\displaystyle \sum _{i,k,l}{q}_l({\alpha }^i{\beta }_l)({\alpha }^i{\gamma }_l^k)}{D}_{\mathrm{..}}^{ik}+{\displaystyle \sum _{j,k,l}{q}^k({\alpha }_j{\beta }^k)({\alpha }_j{\gamma }_l^k)}{D}_{jl}^{\mathrm{..}}\\ +{\displaystyle \sum _{i,j,k,l}[({\alpha }^i{\beta }^k)({\alpha }_j{\gamma }_l^k)}+({\alpha }^i{\beta }_l)({\alpha }_j{\gamma }_l^k)+({\alpha }_j{\beta }^k)({\alpha }^i{\gamma }_l^k)+({\alpha }_j{\beta }_l)({\alpha }^i{\gamma }_l^k)]D_{jl}^{\mathrm{..}}{D}_{\mathrm{..}}^{ik}\\ \text{Cov}(A_1,A_2,D_1{A}_2)& =& {\displaystyle \sum _{i,j,k}{p}_j({\alpha }_j}{\beta }^k)({\delta }_j^i{\beta }^k)D_{\mathrm{..}}^{ik}+{\displaystyle \sum _{i,j,l}{p}^i(}{\alpha }^i{\beta }_l)({\delta }_j^i{\beta }_l)D_{jl}^{\mathrm{..}}\\ +{\displaystyle \sum _{i,j,k,l}[(}{\alpha }^i{\beta }^k)({\delta }_j^i\beta l)+({\alpha }_j{\beta }^k)({\delta }_j^i{\beta }_l)+({\alpha }^i{\beta }_l)({\delta }_j^i{\beta }^k)+({\alpha }_j{\beta }_l)({\delta }_j^i{\beta }^k)]D_{jl}^{\mathrm{..}}{D}_{\mathrm{..}}^{ik}\\ Cov(A_1{D}_2,D_1{A}_2)& =& {\displaystyle \sum _{i,j,k,l}[({\alpha }^i{\gamma }_l^k)}({\delta }_j^i{\beta }^k)(D_{\mathrm{..}}^{ik}+p^i{q}^k)D_{jl}^{\mathrm{..}}+({\alpha }^i{\gamma }_l^k)({\delta }_j^i{\beta }_l)D_{jl}^{\mathrm{..}}{D}_{\mathrm{..}}^{ik}\\ +({\alpha }_j{\gamma }_l^k)({\delta }_j^i{\beta }^k)D_{jl}^{\mathrm{..}}{D}_{\mathrm{..}}^{ik}+({\alpha }_j{\gamma }_l^k)({\delta }_j^i{\beta }_l)(D_{jl}^{\mathrm{..}}+p_j{q}_l)D_{\mathrm{..}}^{ik}]\end{array}$

• The covariances related to dominance × dominance effects:

$\begin{array}{c}Cov(A_1,D_1{D}_2)={\displaystyle \sum _{i,j,k,l}({\delta }_j^i{\gamma }_l^k)({\alpha }^i+{\alpha }_j)D_{jl}^{\mathrm{..}}{D}_{\mathrm{..}}^{ik}}\\ Cov(A_2,D_1{D}_2)={\displaystyle \sum _{i,j,k,l}({\delta }_j^i{\gamma }_l^k)({\beta }^k+{\beta }_l)D_{jl}^{\mathrm{..}}{D}_{\mathrm{..}}^{ik}}\\ Cov(D_1,D_1{D}_2)={\displaystyle \sum _{i,j,k,l}({\delta }_j^i{\gamma }_l^k)({\delta }_j^i)D_{jl}^{\mathrm{..}}{D}_{\mathrm{..}}^{ik}}\\ Cov(D_2,D_1{D}_2)={\displaystyle \sum _{i,j,k,l}({\delta }_j^i{\gamma }_l^k)({\gamma }_l^k)D_{jl}^{\mathrm{..}}{D}_{\mathrm{..}}^{ik}}\\ Cov(A_1{A}_2,D_1{D}_2)=-\left({\displaystyle \sum _{i,k}({\alpha }^i{\beta }^k)D_{\mathrm{..}}^{ik}+{\displaystyle \sum _{j,l}({\alpha }_j{\beta }_l)D_{jl}^{\mathrm{..}}}}\right)\left({\displaystyle \sum _{i,j,k,l}({\delta }_j^i{\gamma }_l^k)D_{jl}^{\mathrm{..}}{D}_{\mathrm{..}}^{ik}}\right)+{\displaystyle \sum _{i,j,k,l}({\delta }_j^i{\gamma }_l^k)}\{[({\alpha }^i{\beta }^k)\\ +({\alpha }^i{\beta }_l)+({\alpha }_j{\beta }^k)+({\alpha }_j{\beta }_l)]D_{jl}^{\mathrm{..}}{D}_{\mathrm{..}}^{ik}+({\alpha }^i{\beta }^k)p^i{q}^k{D}_{jl}^{\mathrm{..}}+({\alpha }_j{\beta }_l)p_j{q}_l{D}_{\mathrm{..}}^{ik}\}\\ Cov(A_1{D}_2,D_1{D}_2)={\displaystyle \sum _{i,j,k,l}({\delta }_j^i{\gamma }_l^k)[}({\alpha }^i{\gamma }_l^k)(D_{\mathrm{..}}^{ik}+p^i{q}^k)D_{jl}^{\mathrm{..}}+({\alpha }_j{\gamma }_l^k)(D_{jl}^{\mathrm{..}}+p_j{q}_l)D_{\mathrm{..}}^{ik}]\\ Cov(D_1{A}_2,D_1{D}_2)={\displaystyle \sum _{i,j,k,l}({\delta }_j^i{\gamma }_l^k)[}({\delta }_j^i{\beta }^k)(D_{\mathrm{..}}^{ik}+p^i{q}^k)D_{jl}^{\mathrm{..}}+({\delta }_j^i{\beta }_l)(D_{jl}^{\mathrm{..}}+p_j{q}_l)D_{\mathrm{..}}^{ik}]\end{array}$

D. Partition of genotypic variance in F₂ population with linkage disequilibrium

Detail of each component of equation (19) for F₂ population is presented here.

• The additive variance:

$V_A=\frac{1}{2}{\displaystyle \sum _{r=1}^m{a}_r^2+2{\displaystyle \sum _{r\ne s}{a}_r{a}_s{D}_{rs}}}$

• The dominance variance:

$V_D=\frac{1}{4}{\displaystyle \sum _{r=1}^m{d}_r^2+4{\displaystyle \sum _{r\ne s}{d}_r{d}_s{D}_{rs}^2}}$

• The additive × additive variance:

$\begin{array}{c}{V}_{AA}=\frac{1}{4}{\displaystyle \sum _{r<s}{(aa)}_{rs}^2+{\displaystyle \sum _{r\ne s\ne s^{\prime }}{(aa)}_{rs}{(aa)}_{r{s}^{\prime }}{D}_{s{s}^{\prime }}}}\\ +\frac{1}{2}{\displaystyle \sum _{r\ne s\ne r^{\prime }\ne s^{\prime }}{(aa)}_{rs}{(aa)}_{r^{\prime }{s}^{\prime }}(D_{r{r}^{\prime }s{s}^{\prime }}+D_{r{s}^{\prime }}{D}_{r^{\prime }s}}+D_{r{r}^{\prime }}{D}_{s{s}^{\prime }}-D_{rs}{D}_{r^{\prime }{s}^{\prime }})\end{array}$

• The additive × dominance variance:

$\begin{array}{c}{V}_{AD}=\frac{1}{8}{\displaystyle \sum _{r\ne s}{(ad)}_{rs}^2}+\frac{1}{2}{\displaystyle \sum _{r\ne s}{(ad)}_{rs}{(ad)}_{sr}{D}_{rs}}+2{\displaystyle \sum _{r\ne s\ne s^{\prime }}{(ad)}_{rs}}[{(ad)}_{r{s}^{\prime }}{D}_{s{s}^{\prime }}^2+{(ad)}_{s{s}^{\prime }}{D}_{s{s}^{\prime }}{D}_{r{s}^{\prime }}\\ +{(ad)}_{s^{\prime }r}{D}_{rs}{D}_{s{s}^{\prime }}+\frac{1}{4}{(ad)}_{s^{\prime }s}{D}_{r{s}^{\prime }}]+8{\displaystyle \sum _{r\ne s\ne r^{\prime }\ne s^{\prime }}{(ad)}_{rs}{(ad)}_{r^{\prime }{s}^{\prime }}{D}_{rs{r}^{\prime }{s}^{\prime }}{D}_{s{s}^{\prime }}}\end{array}$

• The dominance × dominance variance:

$\begin{array}{l}{V}_{DD}={\displaystyle \sum _{r<s}{(dd)}_{rs}^2(\frac{1}{16}-16D_{rs}^4)}+{\displaystyle \sum _{r\ne s\ne s\text{'}}{(dd)}_{rs}{(dd)}_{rs\text{'}}(D_{ss\text{'}}^2-16D_{rs}^2{D}_{rs\text{'}}^2)}\\ +4{\displaystyle \sum _{r\ne s\ne r\text{'}\ne s\text{'}}{(dd)}_{rs}{(dd)}_{r\text{'}s\text{'}}(D_{rsr\text{'}s\text{'}}^2}-D_{rs}^2{D}_{r\text{'}s\text{'}}^2)\end{array}$

• The covariances: Cov(A, D) = Cov(A, AA) = Cov(A, DD) = Cov(D, AD) = Cov(D, DD) = Cov(AA, AD) = Cov(AD, DD) = 0, and

$\begin{array}{c}\text{Cov}(A,AD)=-{\displaystyle \sum _{r\ne s}{a}_r{D}_{rs}}\left[4{(ad)}_{rs}{D}_{rs}+{(ad)}_{sr}\right]-4{\displaystyle \sum _{r\ne s\ne r\text{'}}{a}_{r\text{'}}{(ad)}_{rs}{D}_{rs}{D}_{r\text{'}s}}\\ \text{Cov}(D,AA)=-{\displaystyle \sum _{r<s}(d_r+d_s){(aa)}_{rs}{D}_{rs}-}4{\displaystyle \sum _{r\ne s\ne r\text{'}}{d}_{r\text{'}}{(aa)}_{rs}{D}_{rr\text{'}}{D}_{r\text{'}s}}\\ \text{Cov}(AA,DD)={\displaystyle \sum _{r<s}{(aa)}_{rs}}{(dd)}_{rs}(\frac{1}{2}{D}_{rs}-8D_{rs}^3)+{\displaystyle \sum _{r\ne s\ne s\text{'}}{(aa)}_{rs}}{(dd)}_{rs\text{'}}(2D_{rs\text{'}}{D}_{ss\text{'}}-8D_{rs}{D}_{rs\text{'}}^2)\\ +2{\displaystyle \sum _{r\ne s\ne r\text{'}\ne s\text{'}}{(aa)}_{rs}}(dd{)}_{r\text{'}s\text{'}}(D_{rsr\text{'}s\text{'}}{D}_{r\text{'}s\text{'}}-D_{rs}{D}_{r\text{'}s\text{'}}^2)\end{array}$

where (aa) _sr = (aa) _rs and (dd) _sr = (dd) _rs for r <s.

In this presentation, we utilized the assumption of no crossing-over interference which results in the third order linkage disequilibrium of three loci being zero, i.e. D_rss'= 0. It may be instructive to show this result.

$\begin{array}{c}{D}_{rst}=E(x_r{x}_s{x}_t)=E((z_r-p_r)(z_s-p_s)(z_t-p_t))\\ =E(z_r{z}_s{z}_t-p_r{z}_s{z}_t-z_r{p}_s{z}_t-z_r{z}_s{p}_t+p_r{p}_s{z}_t+p_r{z}_s{p}_t+z_r{p}_s{p}_t-p_r{p}_s{p}_t)\\ =E(z_r{z}_s{z}_t-\frac{1}{2}{z}_s{z}_t-\frac{1}{2}{z}_r{z}_t-\frac{1}{2}{z}_r{z}_s+\frac{1}{4})\end{array}$

Let r₁ be the recombination frequency between loci r and s, r₂ be that between s and t, and r₁₂ be that between r and t. Under the assumption of no crossing-over interference, for loci r, s, t in this order, we have E(z _r z _s z _t ) = $\frac{1}{2}$(1 - r₁)(1 - r₂), E(z _r z _s ) = $\frac{1}{2}$(1 - r₁), E(z _s z _t ) = $\frac{1}{2}$(1 - r₂), and E(z _r z _t ) = $\frac{1}{2}$(1 - r₁₂). Since the assumption of no crossing-over interferene implies r₁₂ = r₁ + r₂ - 2r₁r₂, thus D _rst = 0.

E. Partition of genotypic variance for the general two-allele model with linkage disequilibrium

Detail of each component of equation (19) for the general two-allele model is presented here.

• The additive variance:

$V_A=2{\displaystyle \sum _{r=1}^m{p}_r(1-p_r)a_r^2}+2{\displaystyle \sum _{r\ne s}{a}_r{a}_s{D}_{rs}}$

• The dominance variance:

$V_D={\displaystyle \sum _{r=1}^m{p}_r^2{(1-p_r)}^2{d}_r^2}+{\displaystyle \sum _{r\ne s}{d}_r{d}_s{D}_{rs}^2}$

• The additive × additive variance:

$\begin{array}{c}{V}_{AA}=2{\displaystyle \sum _{r<s}{(aa)}_{rs}^2}\left[(1-2p_r)(1-2p_s)D_{rs}+2p_r(1-p_r)p_s(1-p_s)\right]\\ +2{\displaystyle \sum _{r\ne s\ne s\text{'}}{(aa)}_{rs}{(aa)}_{rs\text{'}}}\left[(1-2p_r)D_{rss\text{'}}+2p_r(1-p_r)D_{ss\text{'}}\right]\\ +\frac{1}{2}{\displaystyle \sum _{r\ne s\ne r\text{'}\ne s\text{'}}{(aa)}_{rs}{(aa)}_{r\text{'}s\text{'}}}(D_{rr\text{'}ss\text{'}}+D_{rs\text{'}}{D}_{r\text{'}s}+D_{rr\text{'}}{D}_{ss\text{'}}-D_{rs}{D}_{r\text{'}s\text{'}})\end{array}$

The additive × dominance variance:

$\begin{array}{l}{V}_{AD}=2{\displaystyle \sum _{r\ne s}{(ad)}_{rs}^2\left[{(1-2p_s)}^2{D}_{rs}^2+(1-2p_r)(1-2p_s)p_s(1-p_s)D_{rs}+p_r(1-p_r)p_s^2{(1-p_s)}^2\right]}\\ +2{\displaystyle \sum _{r\ne s}{(ad)}_{rs}{(ad)}_{sr}{D}_{rs}}\left[2(1-2p_r)(1-2p_s)D_{rs}+p_r(1-p_r)p_s(1-p_s)\right]\\ +2{\displaystyle \sum _{r\ne s\ne s\text{'}}{(ad)}_{rs}}\{{(ad)}_{rs\text{'}}\left[D_{rss\text{'}}^2+(1-2p_r)D_{rss\text{'}}{D}_{ss\text{'}}+p_r(1-p_r)D_{ss\text{'}}^2\right]\\ +(ad{)}_{ss\text{'}}\left[2(1-2p_s)D_{rss\text{'}}{D}_{ss\text{'}}+p_s(1-p_s)D_{ss\text{'}}{D}_{rs\text{'}}\right]\\ +(ad{)}_{s\text{'}r}\left[2(1-2p_r)D_{rss\text{'}}{D}_{rs}+p_r(1-p_r)D_{rs}{D}_{ss\text{'}}\right]\\ +{(ad)}_{s\text{'}s}\left[{(1-2p_s)}^2{D}_{rs}{D}_{ss\text{'}}+(1-2p_s)p_s(1-p_s)D_{rss\text{'}}+p_s^2{(1-p_s)}^2{D}_{rs\text{'}}\right]\}\\ +2{\displaystyle \sum _{r\ne s\ne r\text{'}\ne s\text{'}}{(ad)}_{rs}{(ad)}_{r\text{'}s\text{'}}(D_{rsr\text{'}s\text{'}}{D}_{ss\text{'}}+D_{rss\text{'}}{D}_{sr\text{'}s\text{'}})}\end{array}$

• The dominance × dominance variance:

$\begin{array}{lll}{V}_{DD}\hfill & =\hfill & {\displaystyle \sum _{r<s}{(dd)}_{rs}^2}\left\{{\left[(1-2p_r)(1-2p_s)D_{rs}+p_r(1-p_r)p_s(1-p_s)\right]}^2-D_{rs}^4\right\}\hfill \\ +{\displaystyle \sum _{r\ne s\ne s\text{'}}{(dd)}_{rs}{(dd)}_{rs\text{'}}}\left\{{\left[(1-2p_r)D_{rss\text{'}}+p_r(1-p_r)D_{ss\text{'}}\right]}^2-D_{rs}^2{D}_{rs\text{'}}^2\right\}\hfill \\ +\frac{1}{4}{\displaystyle \sum _{r\ne s\ne r\text{'}\ne s\text{'}}{(dd)}_{rs}{(dd)}_{r\text{'}s\text{'}}}(D_{rsr\text{'}s\text{'}}^2-D_{rs}^2{D}_{r\text{'}s\text{'}}^2)\hfill \end{array}$

• The covariances:

$\begin{array}{lll}\text{Cov}(A,D)\hfill & =\hfill & 0\hfill \\ \text{Cov}(A,AA)\hfill & =\hfill & 2{\displaystyle \sum _{r<s}{(aa)}_{rs}{D}_{rs}}\left[(1-2p_r)a_r+(1-2p_s)a_s\right]+{\displaystyle \sum _{r\ne s\ne r\text{'}}{a}_{r\text{'}}{(aa)}_{rs}{D}_{r\text{'}rs}}\hfill \\ \text{Cov}(A,AD)\hfill & =\hfill & 2{\displaystyle \sum _{r\ne s}{a}_r{D}_{rs}\left[{(ad)}_{rs}{D}_{rs}+p_r(1-p_r){(ad)}_{sr}\right]}+2{\displaystyle \sum _{r\ne s\ne r\text{'}}{a}_{r\text{'}}{(ad)}_{rs}{D}_{rs}{D}_{r\text{'}s}}\hfill \\ \text{Cov}(A,DD)\hfill & =\hfill & 2{\displaystyle \sum _{r<s}{(dd)}_{rs}{D}_{rs}^2\left[(1-2p_r)a_r+(1-2p_s)a_s\right]}+2{\displaystyle \sum _{r\ne s\ne r\text{'}}{a}_{r\text{'}}{(dd)}_{rs}{D}_{rsr\text{'}}{D}_{rs}}\hfill \\ \text{Cov}(D,AA)\hfill & =\hfill & 2{\displaystyle \sum _{r<s}{(aa)}_{rs}{D}_{rs}}\left[p_r(1-p_r)d_r+p_s(1-p_s)d_s\right]+2{\displaystyle \sum _{r\ne s\ne r\text{'}}{d}_{r\text{'}}{(aa)}_{rs}{D}_{rr\text{'}}{D}_{r\text{'}s}}\hfill \\ \text{Cov}(D,AD)\hfill & =\hfill & 2{\displaystyle \sum _{r\ne s}{d}_r{D}_{rs}}(1-2p_r)\left[D_{rs}{(ad)}_{rs}+p_r(1-p_r){(ad)}_{sr}\right]+2{\displaystyle \sum _{r\ne s\ne r\text{'}}{d}_{r\text{'}}{(ad)}_{rs}{D}_{r\text{'}rs}{D}_{r\text{'}s}}\hfill \\ Cov(D,DD)\hfill & =\hfill & {\displaystyle \sum _{r<s}{(dd)}_{rs}{D}_{rs}^2}\left[{(1-2p_r)}^2{d}_r+{(1-2p_s)}^2{d}_s\right]+2{\displaystyle \sum _{r\ne s\ne r\text{'}}{d}_{r\text{'}}{(dd)}_{rs}{D}_{r\text{'}rs}^2}\hfill \\ \text{Cov}(AA,AD)\hfill & =\hfill & 2{\displaystyle \sum _{r<s}{(aa)}_{rs}{D}_{rs}}\{{(ad)}_{rs}\left[(1-2p_r)p_s(1-p_s)+2(1-2p_s)D_{rs}\right]\hfill \\ +{(ad)}_{sr}\left[(1-2p_s)p_r(1-p_r)+2(1-2p_r)D_{rs}\right]\}\hfill \\ +2{\displaystyle \sum _{r\ne s\ne s\text{'}}{(aa)}_{rs}}\{{(ad)}_{ss\text{'}}\left[(1-2p_s)D_{ss\text{'}}{D}_{rs\text{'}}+2D_{rss\text{'}}{D}_{ss\text{'}}\right]\hfill \\ +{(ad)}_{s\text{'}s}\left[2(1-2p_s)D_{rs}{D}_{ss\text{'}}+p_s(1-p_s)D_{rss\text{'}}\right]\}\hfill \\ +{\displaystyle \sum _{r\ne s\ne r\text{'}\ne s\text{'}}{(aa)}_{rs}{(ad)}_{r\text{'}s\text{'}}(D_{rss\text{'}}{D}_{r\text{'}s}+D_{rr\text{'}s\text{'}}{D}_{ss\text{'}}+D_{r\text{'}ss\text{'}}{D}_{rs\text{'}})}\hfill \\ \text{Cov}(AA,DD)\hfill & =\hfill & 2{\displaystyle \sum _{r<s}{(aa)}_{rs}{(dd)}_{rs}}\left[2(1-2p_r)(1-2p_s)D_{rs}^2+p_r(1-p_r)p_s(1-p_s)D_{rs}-D_{rs}^3\right]\hfill \\ +2{\displaystyle \sum _{r\ne s\ne s\text{'}}{(aa)}_{rs}{(dd)}_{rs\text{'}}}\left[2(1-2p_r)D_{rss\text{'}}{D}_{rs\text{'}}+p_r(1-p_r)D_{rs\text{'}}{D}_{ss\text{'}}-D_{rs}{D}_{rs\text{'}}^2\right]\hfill \\ +\frac{1}{2}{\displaystyle \sum _{r\ne s\ne r\text{'}\ne s\text{'}}{(aa)}_{rs}{(dd)}_{r\text{'}s\text{'}}(D_{rsr\text{'}s\text{'}}{D}_{r\text{'}s\text{'}}+D_{rr\text{'}s\text{'}}{D}_{sr\text{'}s\text{'}}-D_{rs}{D}_{r\text{'}s\text{'}}^2)}\hfill \\ \text{Cov}(AD,DD)\hfill & =\hfill & 2{\displaystyle \sum _{r\ne s}{(ad)}_{rs}{(dd)}_{rs}{D}_{rs}(1-2p_s)\left[(1-2p_r)(1-2p_s)D_{rs}+p_r(1-p_r)p_s(1-p_s)\right]}\hfill \\ +2{\displaystyle \sum _{r\ne s\ne s\text{'}}{(ad)}_{rs}\{{(dd)}_{rs\text{'}}{D}_{rss\text{'}}\left[(1-2p_r)D_{rss\text{'}}+p_r(1-p_r)D_{ss\text{'}}\right]}\hfill \\ +{(dd)}_{ss\text{'}}{D}_{ss\text{'}}(1-2p_s)\left[(1-2p_s)D_{rss\text{'}}+p_s(1-p_s)D_{rs\text{'}}\right]\}\hfill \\ +{\displaystyle \sum _{r\ne s\ne r\text{'}\ne s\text{'}}{(ad)}_{rs}{(dd)}_{r\text{'}s\text{'}}{D}_{rss\text{'}}{D}_{rsr\text{'}s\text{'}}}\hfill \end{array}$

where (aa) _sr = (aa) _rs and (dd) _sr = (dd) _rs for r <s.

The result in Appendix D for the F₂ model with p _r = 1/2 for r = 1, ..., m and also assuming D_rss'= 0 is a special case of the results presented here. There is a difference, by a factor -2, on the specification of v variable for dominance effect for the F₂ model and the general two-allele model, which carries over to the comparison of results in Appendix D and E.