Canonical Analysis: Ranks, Ratios and Fits
Measurements of p variables for n samples are collected into a n×p matrix X, where the samples belong to one of k groups. The group means are separated by Mahalanobis distances. CVA optimally represents the group means of X in an r-dimensional space. This can be done by maximizing a ratio criterion (basically one- dimensional) or, more flexibly, by minimizing a rank-constrained least-squares fitting criterion (which is not confined to being one-dimensional but depends on defining an appropriate Mahalanobis metric). In modern n < p problems, where W is not of full rank, the ratio criterion is shown not to be coherent but the fit criterion, with an attention to associated metrics, readily generalizes. In this context we give a unified generalization of CVA, introducing two metrics, one in the range space of W and the other in the null space of W, that have links with Mahalanobis distance. This generalization is computationally efficient, since it requires only the spectral decomposition of a n×n matrix.
KeywordsCanonical analysis Ratio form Fit form Mahalonobis distance Discriminant analysis
Unable to display preview. Download preview PDF.
- GOWER, J.C. (1976), “Growth-Free Canonical Variates and Generalised Inverses”, Bulletin of the Geological Institute of the University of Uppsala 7, 1–10.Google Scholar
- KRZANOWSKI, W.J., JONATHAN, P., MCCARTHY, W.V., and THOMAS, M.R. (1995), “Discriminant Analysis with Singular Covariance Matrices: Methods and Applications to Spectroscopic Data”, Journal of the Royal Statistical Society, Series C (Applied Statistics), 44, 101–115.Google Scholar
- MARDIA, K.V. (1977), “Mahalanobis Distances and Angles”, in Multivariate Analysis (IV), ed. P.R. Krishnaiah, Amsterdam: North-Holland, pp. 495–511.Google Scholar
- RAO, C.R. (1949), “Representation of p-Dimensional Data in Lower Dimensions”, in Anthropometric Survey of the United Provinces: A Statistical Study, 1941, by Mahalanobis, P.C., Majumdar, D.N. and Rao, C.R., in Sankhya, The Indian Journal of Statistics, 9, 90–324.Google Scholar
- C.J. Albers and J.C. Gower YE, J. (2005), “Characterization of a Family of Algorithms for Generalized Discriminant Analysis on Undersampled Problems”, Journal of Machine Learning Research, 6, 483-502.Google Scholar