Linear Discrimination, Ordination, and the Visualization of Selection Gradients in Modern Morphometrics

Abstract

Linear discriminant analysis (LDA) is a multivariate classification technique frequently applied to morphometric data in various biomedical disciplines. Canonical variate analysis (CVA), the generalization of LDA for multiple groups, is often used in the exploratory style of an ordination technique (a low-dimensional representation of the data). In the rare case when all groups have the same covariance matrix, maximum likelihood classification can be based on these linear functions. Both LDA and CVA require full-rank covariance matrices, which is usually not the case in modern morphometrics. When the number of variables is close to the number of individuals, groups appear separated in a CVA plot even if they are samples from the same population. Hence, reliable classification and assessment of group separation require many more organisms than variables. A simple alternative to CVA is the projection of the data onto the principal components of the group averages (between-group PCA). In contrast to CVA, these axes are orthogonal and can be computed even when the data are not of full rank, such as for Procrustes shape coordinates arising in samples of any size, and when covariance matrices are heterogeneous. In evolutionary quantitative genetics, the selection gradient is identical to the coefficient vector of a linear discriminant function between the populations before vs. after selection. When the measured variables are Procrustes shape coordinates, discriminant functions and selection gradients are vectors in shape space and can be visualized as shape deformations. Except for applications in quantitative genetics and in classification, however, discriminant functions typically offer no interpretation as biological factors.

Appendix

Figure 2 shows that the log likelihood surface for two groups A, B with the same covariance matrix ${\varvec{\Upsigma}}_A={\varvec{\Upsigma}}_B \equiv {\varvec{\Upsigma}}$ is a flat plane with a gradient vector ${\varvec{\Upsigma}}^{-1}({\varvec{\mu}}_A-{\varvec{\mu}}_B)$. To prove this, at least informally, consider ${\bf y}_i={\varvec{\Upsigma}}^{-1/2}{\bf x}_i$, so that the within-group covariance matrix of these transformed variables y is equal to the identity matrix. Then, still assuming equal covariance matrices, the log likelihood ratio, i.e., the exponent of (1), is half the difference between the squared Euclidean distances of x _i to the two group means. This, in turn, is equal to the distance of x _i from the midpoint of the group means ${{1}\over {2}}({\varvec{\mu}}_A+{\varvec{\mu}}_B)$, projected onto the vector of group mean differences ${\varvec{\mu}}_A-{\varvec{\mu}}_B$:

$$ \frac{1}{2}(({\bf x}_i-{\varvec{\mu}}_B)^{\prime}({\bf x}_i-{\varvec{\mu}}_B) - ({\bf x}_i-{\varvec{\mu}}_A)^{\prime}({\bf x}_i-{\varvec{\mu}}_A))=({\varvec{\mu}}_A-{\varvec{\mu}}_B)^{\prime}\left({\bf x} _i- \frac{1}{2}({\varvec{\mu}}_A+{\varvec{\mu}}_B)\right) $$

Apparently, likelihood ratios are then constant along lines orthogonal to the vector of group mean differences, and lines of constant log likelihood ratio are regularly spaced, constituting a flat log likelihood ratio surface with the gradient vector ${\varvec{\mu}}_A-{\varvec{\mu}}_B$. Since the multiplication by ${\varvec{\Upsigma}}^{1/2}$ is a linear transformation that preserves linear structures, the surface must be flat for the untransformed variables ${\bf x}_i={\varvec{\Upsigma}}^{1/2}{\bf y}_i$, too.

Let X be a mean-centered n × p data matrix for a sample consisting of two distinct groups with the group averages $\bar{{\bf x}}_1, \bar{{\bf x}}_2$ and sample sizes $n_1, n_2$ so that $n=n_1+n_2$. The total sum of squares T = X′X can be decomposed into the pooled within-group sum of squares W and the between-group sum of squares B. The discriminant vector a usually is computed as the vector of group mean differences premultiplied by the inverse of W:

$$ {\bf a}={\bf W}^{-1}(\bar{{\bf x}}_1-\bar{{\bf x}}_2). $$

Consider the n-dimensional vector d with elements $d_i=1/n_1$ if the ith specimen belongs to group 1 and $d_i=-1/n_2$ if it belongs to group 2, so that ${\bf X}^{\prime}{\bf d}= \bar{\bf x}_1-\bar{\bf x}_2$ is the group mean difference. The multiple regression of this dummy variable on shape

$$ ({\bf X}^{\prime}{\bf X})^{-1}{\bf X}^{\prime}{\bf d} = {\bf T}^{-1}(\bar{{\bf x}}_1-\bar{{\bf x}}_2) \propto {\bf a} $$

(6)

gives a vector that is proportional to the linear discriminant vector. This implies that the discriminant function can also be computed by premultiplying the vector of group mean differences ${\bf g}=\bar{{\bf x}}_1-\bar{{\bf x}}_2$ by the inverse of the total sum of squares matrix T rather than the within-group sum of squares matrix W. To prove this, consider first that the between-group sum of squares matrix B has only one non-zero eigenvalue. The corresponding eigenvector is ${\bf e}={\bf g}/\|{\bf g}\|$ and the eigenvalue λ is the trace of B or $\| \bf g\|^2/2$. Multiplying B with the vector of group mean differences thus gives Bg = λ g. Furthermore, the inverse of the sum of two matrices can be decomposed in the following way:

$$ ({\bf A+BCD})^{-1}{\bf BC}={\bf A}^{-1}{\bf B}({\bf C}^{-1}+{\bf DA}^{-1}{\bf B})^{-1}, $$

where A and C are invertible and B, C are arbitrary (rectangular) matrices (Henderson and Searle 1981). Let ${\bf A=W, B=g, C=}{\varvec{\lambda}}$, and D = g′, then

$$ \begin{aligned} ({\bf W}+{\bf B})^{-1}{\bf g}&=({\bf W}+{\bf e} \lambda {\bf e}^{\prime})^{-1}{\bf e}\lambda (\|\bf g\|/2)^{-1} \\ &={\bf W}^{-1}{\bf e}(\lambda^{-1}+{\bf e}^{\prime}{\bf W}^{-1}{\bf e})^{-1}(\|\bf g\|/2)^{-1} \\ &={\bf W}^{-1}{\bf g}\xi,\\ \end{aligned} $$

where ξ is a scalar. Another proof is given by Fisher (1938).

Discriminant functions are invariant to affine transformations of the variables, including separate changes of the scale of each variable. Let A be any regular p × p matrix; then XA is a linear transformation of the original variables X. The linear discriminant function for these transformed data is identical to the discriminant function of the original variables:

$$ \begin{aligned} {\bf XA}({\bf A}^{\prime}{\bf WA})^{-1}{\bf A}^{\prime}(\bar{{\bf x}}_1-\bar{{\bf x}}_2)&={\bf XAA}^{-1}{\bf W}^{-1}({\bf A}^{\prime})^{-1}{\bf A}^{\prime}(\bar{{\bf x}}_1-\bar{{\bf x}}_2) \\ &={\bf XW}^{-1}(\bar{{\bf x}}_1-\bar{{\bf x}}_2). \end{aligned} $$

Several authors suggested visualizing the regression of shape on the discriminant scores Xa, that is

$$ ({\bf a}^{\prime}{\bf X}^{\prime}{\bf Xa})^{-1}({\bf Xa})^{\prime}{\bf X} . $$

(7)

As the length of this vector is irrelevant, we can ignore the scalar quantity (a′X′Xa)⁻¹ and replace (7) by the term

$$ ({\bf Xa})^{\prime}{\bf X}, $$

(8)

which is proportional to the vector of covariances between the shape variables and the discriminant score. When substituting a in (8) by (6) we get

$$ \begin{aligned} ({\bf Xa})^{\prime}{\bf X}&={\bf a}^{\prime}{\bf X}^{\prime}{\bf X}\\ &=\zeta{\bf d}^{\prime}{\bf X}({\bf X}^{\prime}{\bf X})^{-1}{\bf X}^{\prime}{\bf X} \\ &=\zeta{\bf d}^{\prime}{\bf X} \\ &=\zeta(\bar{{\bf x}}_1-\bar{{\bf x}}_2)^{\prime},\\ \end{aligned} $$

(9)

where $\zeta$ is a scalar value. In words, the vector of coefficients for the regression of X on the discriminant function is proportional to the mean difference vector.

Similarly, instead of a direct visualization of selection gradients, Klingenberg and Monteiro (2005) suggested visualizing the multivariate regression of shape on the scores along the selection gradient ${\varvec{\beta}}$. This vector of regression coefficients is proportional to the vector $({\bf X}{\varvec{\beta}})^{\prime}{\bf X}$. Klingenberg and Monteiro noticed that this leads to a visualization of the scaled selection differential s. A proof follows from (9) when taking a as the selection gradient ${\varvec{\beta}}$ and $\bar{{\bf x}}_1, \bar{{\bf x}}_2$ as the group means after and before selection.