Skip to main content
Log in

Canonical Analysis: Ranks, Ratios and Fits

  • Published:
Journal of Classification Aims and scope Submit manuscript

Abstract

Measurements of p variables for n samples are collected into a n×p matrix X, where the samples belong to one of k groups. The group means are separated by Mahalanobis distances. CVA optimally represents the group means of X in an r-dimensional space. This can be done by maximizing a ratio criterion (basically one- dimensional) or, more flexibly, by minimizing a rank-constrained least-squares fitting criterion (which is not confined to being one-dimensional but depends on defining an appropriate Mahalanobis metric). In modern n < p problems, where W is not of full rank, the ratio criterion is shown not to be coherent but the fit criterion, with an attention to associated metrics, readily generalizes. In this context we give a unified generalization of CVA, introducing two metrics, one in the range space of W and the other in the null space of W, that have links with Mahalanobis distance. This generalization is computationally efficient, since it requires only the spectral decomposition of a n×n matrix.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • ALBERS, C.J., CRITCHLEY, F., and GOWER, J.C. (2011a), “Quadratic Minimisation Problems in Statistics”, Journal of Multivariate Analysis, 102, 698–713.

    Article  MATH  MathSciNet  Google Scholar 

  • ALBERS, C.J., CRITCHLEY, F., and GOWER, J.C. (2011b), “Applications of Quadratic Minimisation Problems in Statistics”, Journal of Multivariate Analysis, 102, 714-722.

    Article  MATH  MathSciNet  Google Scholar 

  • CHEN, L., LIAO, H., KO, M., LIN, J., and YU, G. (2000), “A New LDA-Based Face Recognition System Which Can Solve the Small Sample Size Problem”, Pattern Recognition, 33, 1713–1726.

    Article  Google Scholar 

  • ECKART, C., and YOUNG, G. (1936), “The Approximation of One Matrix by Another of Lower Rank”, Psychometrika, 1, 211–218.

    Article  MATH  Google Scholar 

  • GOWER, J.C., and ALBERS, C.J. (2011), “Between Group Metrics”, Journal of Classification. 28, 315–326.

    Article  MathSciNet  Google Scholar 

  • GOWER, J.C. (1976), “Growth-Free Canonical Variates and Generalised Inverses”, Bulletin of the Geological Institute of the University of Uppsala 7, 1–10.

  • GOWER, J.C. (1998), “The Role of Constraints in Determining Optimal Scores”, Statistics in Medicine, 17, 2709–2721.

    Article  Google Scholar 

  • HEALY, M.J.R., and GOLDSTEIN, H. (1976), “An Approach to the Scaling of Categorised Attributes”, Biometrika, 63, 219–229.

    Article  MATH  Google Scholar 

  • KRZANOWSKI, W.J., JONATHAN, P., MCCARTHY, W.V., and THOMAS, M.R. (1995), “Discriminant Analysis with Singular Covariance Matrices: Methods and Applications to Spectroscopic Data”, Journal of the Royal Statistical Society, Series C (Applied Statistics), 44, 101–115.

    Google Scholar 

  • LUBBE-GARDNER, D.L., LE ROUX, N.J., and GOWER, J.C. (2008), “Measures of Fit in Principal Component and Canonical Variate Analysis”, Journal of Applied Statistics, 35 , 947965.

    Article  MathSciNet  Google Scholar 

  • MARDIA, K.V. (1977), “Mahalanobis Distances and Angles”, in Multivariate Analysis (IV), ed. P.R. Krishnaiah, Amsterdam: North-Holland, pp. 495–511.

  • QUEEN, C.M., WRIGHT, B., and ALBERS, C.J. (2007), “Eliciting a Directed Acyclic Graph for a Multivariate Time Series of Vehicle Counts in a Traffic Network”, Australian and New Zealand Journal of Statistics, 49(3), 1–19.

    Article  MathSciNet  Google Scholar 

  • QUEEN, C.M., and ALBERS, C.J. (2009), “Intervention and Causality: Forecasting Traffic Flows Using a Dynamic Bayesian Network”, Journal of the American Statistical Association, 104(486), 669–681.

    Article  MathSciNet  Google Scholar 

  • RAO, C.R. (1949), “Representation of p-Dimensional Data in Lower Dimensions”, in Anthropometric Survey of the United Provinces: A Statistical Study, 1941, by Mahalanobis, P.C., Majumdar, D.N. and Rao, C.R., in Sankhya, The Indian Journal of Statistics, 9, 90–324.

    Google Scholar 

  • RAO, C.R. (1967), “Calculus of Generalised Inverses of Matrices: Part 1 - General Theory”, Sankhya, Series A, 29, 317–342.

    MATH  Google Scholar 

  • RAO, C.R., and YANAI, H. (1979), “General Definition and Decomposition of Projectors and Some Applications to Statistical Problems”, Journal of Statistical Planning and Inference 3, 1–17.

    Article  MATH  MathSciNet  Google Scholar 

  • C.J. Albers and J.C. Gower YE, J. (2005), “Characterization of a Family of Algorithms for Generalized Discriminant Analysis on Undersampled Problems”, Journal of Machine Learning Research, 6, 483-502.

    Google Scholar 

  • YE, J., and XIONG, T. (2006), “Computational and Theoretical Analysis of Null Space and Orthogonal Linear Discriminant Analysis”, Journal of Machine Learning Research, 7, 1183–1204.

    MATH  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Casper J. Albers.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Albers, C.J., Gower, J.C. Canonical Analysis: Ranks, Ratios and Fits. J Classif 31, 2–27 (2014). https://doi.org/10.1007/s00357-014-9146-y

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00357-014-9146-y

Keywords

Navigation