Journal of Classification

, Volume 31, Issue 1, pp 2–27 | Cite as

Canonical Analysis: Ranks, Ratios and Fits

  • Casper J. AlbersEmail author
  • John C. Gower


Measurements of p variables for n samples are collected into a n×p matrix X, where the samples belong to one of k groups. The group means are separated by Mahalanobis distances. CVA optimally represents the group means of X in an r-dimensional space. This can be done by maximizing a ratio criterion (basically one- dimensional) or, more flexibly, by minimizing a rank-constrained least-squares fitting criterion (which is not confined to being one-dimensional but depends on defining an appropriate Mahalanobis metric). In modern n < p problems, where W is not of full rank, the ratio criterion is shown not to be coherent but the fit criterion, with an attention to associated metrics, readily generalizes. In this context we give a unified generalization of CVA, introducing two metrics, one in the range space of W and the other in the null space of W, that have links with Mahalanobis distance. This generalization is computationally efficient, since it requires only the spectral decomposition of a n×n matrix.


Canonical analysis Ratio form Fit form Mahalonobis distance Discriminant analysis 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. ALBERS, C.J., CRITCHLEY, F., and GOWER, J.C. (2011a), “Quadratic Minimisation Problems in Statistics”, Journal of Multivariate Analysis, 102, 698–713.CrossRefzbMATHMathSciNetGoogle Scholar
  2. ALBERS, C.J., CRITCHLEY, F., and GOWER, J.C. (2011b), “Applications of Quadratic Minimisation Problems in Statistics”, Journal of Multivariate Analysis, 102, 714-722.CrossRefzbMATHMathSciNetGoogle Scholar
  3. CHEN, L., LIAO, H., KO, M., LIN, J., and YU, G. (2000), “A New LDA-Based Face Recognition System Which Can Solve the Small Sample Size Problem”, Pattern Recognition, 33, 1713–1726.CrossRefGoogle Scholar
  4. ECKART, C., and YOUNG, G. (1936), “The Approximation of One Matrix by Another of Lower Rank”, Psychometrika, 1, 211–218.CrossRefzbMATHGoogle Scholar
  5. GOWER, J.C., and ALBERS, C.J. (2011), “Between Group Metrics”, Journal of Classification. 28, 315–326.CrossRefMathSciNetGoogle Scholar
  6. GOWER, J.C. (1976), “Growth-Free Canonical Variates and Generalised Inverses”, Bulletin of the Geological Institute of the University of Uppsala 7, 1–10.Google Scholar
  7. GOWER, J.C. (1998), “The Role of Constraints in Determining Optimal Scores”, Statistics in Medicine, 17, 2709–2721.CrossRefGoogle Scholar
  8. HEALY, M.J.R., and GOLDSTEIN, H. (1976), “An Approach to the Scaling of Categorised Attributes”, Biometrika, 63, 219–229.CrossRefzbMATHGoogle Scholar
  9. KRZANOWSKI, W.J., JONATHAN, P., MCCARTHY, W.V., and THOMAS, M.R. (1995), “Discriminant Analysis with Singular Covariance Matrices: Methods and Applications to Spectroscopic Data”, Journal of the Royal Statistical Society, Series C (Applied Statistics), 44, 101–115.Google Scholar
  10. LUBBE-GARDNER, D.L., LE ROUX, N.J., and GOWER, J.C. (2008), “Measures of Fit in Principal Component and Canonical Variate Analysis”, Journal of Applied Statistics, 35 , 947965.CrossRefMathSciNetGoogle Scholar
  11. MARDIA, K.V. (1977), “Mahalanobis Distances and Angles”, in Multivariate Analysis (IV), ed. P.R. Krishnaiah, Amsterdam: North-Holland, pp. 495–511.Google Scholar
  12. QUEEN, C.M., WRIGHT, B., and ALBERS, C.J. (2007), “Eliciting a Directed Acyclic Graph for a Multivariate Time Series of Vehicle Counts in a Traffic Network”, Australian and New Zealand Journal of Statistics, 49(3), 1–19.CrossRefMathSciNetGoogle Scholar
  13. QUEEN, C.M., and ALBERS, C.J. (2009), “Intervention and Causality: Forecasting Traffic Flows Using a Dynamic Bayesian Network”, Journal of the American Statistical Association, 104(486), 669–681.CrossRefMathSciNetGoogle Scholar
  14. RAO, C.R. (1949), “Representation of p-Dimensional Data in Lower Dimensions”, in Anthropometric Survey of the United Provinces: A Statistical Study, 1941, by Mahalanobis, P.C., Majumdar, D.N. and Rao, C.R., in Sankhya, The Indian Journal of Statistics, 9, 90–324.Google Scholar
  15. RAO, C.R. (1967), “Calculus of Generalised Inverses of Matrices: Part 1 - General Theory”, Sankhya, Series A, 29, 317–342.zbMATHGoogle Scholar
  16. RAO, C.R., and YANAI, H. (1979), “General Definition and Decomposition of Projectors and Some Applications to Statistical Problems”, Journal of Statistical Planning and Inference 3, 1–17.CrossRefzbMATHMathSciNetGoogle Scholar
  17. C.J. Albers and J.C. Gower YE, J. (2005), “Characterization of a Family of Algorithms for Generalized Discriminant Analysis on Undersampled Problems”, Journal of Machine Learning Research, 6, 483-502.Google Scholar
  18. YE, J., and XIONG, T. (2006), “Computational and Theoretical Analysis of Null Space and Orthogonal Linear Discriminant Analysis”, Journal of Machine Learning Research, 7, 1183–1204.zbMATHMathSciNetGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2014

Authors and Affiliations

  1. 1.University of GroningenGroningenThe Netherlands
  2. 2.The Open UniversityBuckinghamshireUK

Personalised recommendations