Abstract
Measurements of p variables for n samples are collected into a n×p matrix X, where the samples belong to one of k groups. The group means are separated by Mahalanobis distances. CVA optimally represents the group means of X in an r-dimensional space. This can be done by maximizing a ratio criterion (basically one- dimensional) or, more flexibly, by minimizing a rank-constrained least-squares fitting criterion (which is not confined to being one-dimensional but depends on defining an appropriate Mahalanobis metric). In modern n < p problems, where W is not of full rank, the ratio criterion is shown not to be coherent but the fit criterion, with an attention to associated metrics, readily generalizes. In this context we give a unified generalization of CVA, introducing two metrics, one in the range space of W and the other in the null space of W, that have links with Mahalanobis distance. This generalization is computationally efficient, since it requires only the spectral decomposition of a n×n matrix.
Similar content being viewed by others
References
ALBERS, C.J., CRITCHLEY, F., and GOWER, J.C. (2011a), “Quadratic Minimisation Problems in Statistics”, Journal of Multivariate Analysis, 102, 698–713.
ALBERS, C.J., CRITCHLEY, F., and GOWER, J.C. (2011b), “Applications of Quadratic Minimisation Problems in Statistics”, Journal of Multivariate Analysis, 102, 714-722.
CHEN, L., LIAO, H., KO, M., LIN, J., and YU, G. (2000), “A New LDA-Based Face Recognition System Which Can Solve the Small Sample Size Problem”, Pattern Recognition, 33, 1713–1726.
ECKART, C., and YOUNG, G. (1936), “The Approximation of One Matrix by Another of Lower Rank”, Psychometrika, 1, 211–218.
GOWER, J.C., and ALBERS, C.J. (2011), “Between Group Metrics”, Journal of Classification. 28, 315–326.
GOWER, J.C. (1976), “Growth-Free Canonical Variates and Generalised Inverses”, Bulletin of the Geological Institute of the University of Uppsala 7, 1–10.
GOWER, J.C. (1998), “The Role of Constraints in Determining Optimal Scores”, Statistics in Medicine, 17, 2709–2721.
HEALY, M.J.R., and GOLDSTEIN, H. (1976), “An Approach to the Scaling of Categorised Attributes”, Biometrika, 63, 219–229.
KRZANOWSKI, W.J., JONATHAN, P., MCCARTHY, W.V., and THOMAS, M.R. (1995), “Discriminant Analysis with Singular Covariance Matrices: Methods and Applications to Spectroscopic Data”, Journal of the Royal Statistical Society, Series C (Applied Statistics), 44, 101–115.
LUBBE-GARDNER, D.L., LE ROUX, N.J., and GOWER, J.C. (2008), “Measures of Fit in Principal Component and Canonical Variate Analysis”, Journal of Applied Statistics, 35 , 947–965.
MARDIA, K.V. (1977), “Mahalanobis Distances and Angles”, in Multivariate Analysis (IV), ed. P.R. Krishnaiah, Amsterdam: North-Holland, pp. 495–511.
QUEEN, C.M., WRIGHT, B., and ALBERS, C.J. (2007), “Eliciting a Directed Acyclic Graph for a Multivariate Time Series of Vehicle Counts in a Traffic Network”, Australian and New Zealand Journal of Statistics, 49(3), 1–19.
QUEEN, C.M., and ALBERS, C.J. (2009), “Intervention and Causality: Forecasting Traffic Flows Using a Dynamic Bayesian Network”, Journal of the American Statistical Association, 104(486), 669–681.
RAO, C.R. (1949), “Representation of p-Dimensional Data in Lower Dimensions”, in Anthropometric Survey of the United Provinces: A Statistical Study, 1941, by Mahalanobis, P.C., Majumdar, D.N. and Rao, C.R., in Sankhya, The Indian Journal of Statistics, 9, 90–324.
RAO, C.R. (1967), “Calculus of Generalised Inverses of Matrices: Part 1 - General Theory”, Sankhya, Series A, 29, 317–342.
RAO, C.R., and YANAI, H. (1979), “General Definition and Decomposition of Projectors and Some Applications to Statistical Problems”, Journal of Statistical Planning and Inference 3, 1–17.
C.J. Albers and J.C. Gower YE, J. (2005), “Characterization of a Family of Algorithms for Generalized Discriminant Analysis on Undersampled Problems”, Journal of Machine Learning Research, 6, 483-502.
YE, J., and XIONG, T. (2006), “Computational and Theoretical Analysis of Null Space and Orthogonal Linear Discriminant Analysis”, Journal of Machine Learning Research, 7, 1183–1204.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Albers, C.J., Gower, J.C. Canonical Analysis: Ranks, Ratios and Fits. J Classif 31, 2–27 (2014). https://doi.org/10.1007/s00357-014-9146-y
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00357-014-9146-y