Abstract
Multiple-set canonical correlation analysis (Generalized CANO or GCANO for short) is an important technique because it subsumes a number of interesting multivariate data analysis techniques as special cases. More recently, it has also been recognized as an important technique for integrating information from multiple sources. In this paper, we present a simple regularization technique for GCANO and demonstrate its usefulness. Regularization is deemed important as a way of supplementing insufficient data by prior knowledge, and/or of incorporating certain desirable properties in the estimates of parameters in the model. Implications of regularized GCANO for multiple correspondence analysis are also discussed. Examples are given to illustrate the use of the proposed technique.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Abdi, H. (2007). Singular value decomposition (SVD) and generalized singular decomposition (GSVD). In N.J. Salkind (Ed.), Encyclopedia of measurement and statistics (pp. 907–12). Thousand Oak: Sage.
Abdi, H., & Valentin, D. (2007). The STATIS method. In N.J. Salkind (Ed.), Encyclopedia of measurement and statistics (pp. 955–962). Thousand Oaks: Sage.
Adachi, K. (2002). Homogeneity and smoothness analysis for quantifying a longitudinal categorical variable. In S. Nishisato, Y. Baba, H. Bozdogan, & K. Kanefuji (Eds.), Measurement and multivariate analysis (pp. 47–56). Tokyo: Springer.
Beck, A. (1996). BDI-II. San Antonio: Psychological Corporation.
Carroll, J.D. (1968). A generalization of canonical correlation analysis to three or more sets of variables. In Proceedings of the 76th annual convention of the American psychological association (pp. 227–228).
Dahl, T., & Næs, T. (2006). A bridge between Tucker-1 and Carroll’s generalized canonical analysis. Computational Statistics and Data Analysis, 50, 3086–3098.
de Leeuw, J. (1982). Generalized eigenvalue problems with posivite semi-definite matrices. Psychometrika, 47, 87–93.
Devaux, M.-F., Courcoux, P., Vigneau, E., & Novales, B. (1998). Generalized canonical correlation analysis for the interpretation of fluorescence spectral data. Analusis, 26, 310–316.
DiPillo, P.J. (1976). The application of bias to discriminant analysis. Communications in Statistics—Theory and Methods, 5, 843–859.
Efron, B. (1979). Bootstrap methods: another look at the jackknife. Annals of Statistics, 7, 1–26.
Fischer, B., Ross, V., & Buhmann, J.M. (2007). Time-series alignment by non-negative multiple generalized canonical correlation analysis. In F. Massuli & S. Mitra (Eds.), Applications of fuzzy set theory (pp. 505–511). Berlin: Springer.
Friedman, J. (1989). Regularized discriminant analysis. Journal of the American Statistical Association, 84, 165–175.
Gardner, S., Gower, J.C., & le Roux, N.J. (2006). A synthesis of canonical variate analysis, generalized canonical correlation and Procrustes analysis. Computational Statistics and Data Analysis, 50, 107–134.
Gifi, A. (1990). Nonlinear multivariate analysis. Chichester: Wiley.
Greenacre, M.J. (1984). Theory and applications of correspondence analysis. London: Academic Press.
Hoerl, A.F., & Kennard, R.W. (1970). Ridge regression: biased estimation for nonorthgonal problems. Technometrics, 12, 55–67.
Horst, P. (1961). Generalized canonical correlations and their applications to experimental data. Journal of Clinical Psychology, 17, 331–347.
Kroonenberg, P.M. (2008). Applied multiway data analysis. New York: Wiley.
Legendre, P., & Legendre, L. (1998). Numerical ecology. Amsterdam: North Holland.
Maraun, M., Slaney, K., & Jalava, J. (2005). Dual scaling for the analysis of categorical data. Journal of Personality Assessment, 85, 209–217.
Meredith, W. (1964). Rotation to achieve factorial invariance. Psychometrika, 29, 187–206.
Poggio, T., & Girosi, F. (1990). Regularization algorithms for learning that are equivalent to multilayer networks. Science, 247, 978–982.
Ramsay, J.O., & Silverman, B.W. (2005). Functional data analysis, 2nd edn. New York: Springer.
Rao, C.R., & Mitra, S.K. (1971). Generalized inverse of matrices and its applications. New York: Wiley.
Smilde, A., Bro, R., & Geladi, P. (2004). Multi-way analysis: applications in the chemical sciences. New York: Wiley.
Sun, Q.-S., Heng, P.-A., Jin, Z., & Xia, D.-S. (2005). Face recognition based on generalized canonical correlation analysis. In D.S. Huang, X.-P. Zhang, & G.-B. Huang (Eds.), Advances in intelligent computing (pp. 958–967). Berlin: Springer.
Takane, Y. (1980). Analysis of categorizing behavior by a quantification method. Behaviormetrika, 8, 75–86.
Takane, Y., & Hunter, M.A. (2001). Constrained principal component analysis: a comprehensive theory. Applicable Algebra in Engineering, Communication and Computing, 12, 391–419.
Takane, Y., & Hwang, H. (2002). Generalized constrained canonical correlation analysis. Multivariate Behavioral Research, 37, 163–195.
Takane, Y., & Hwang, H. (2006). Regularized multiple correspondence analysis. In M.J. Greenacre & J. Blasius (Eds.), Multiple correspondence analysis and related methods (pp. 259–279). London: Chapman and Hall.
Takane, Y., & Hwang, H. (2007). Regularized linear and kernel redundancy analysis. Computational Statistics and Data Analysis, 52, 392–405.
Takane, Y., & Jung, S. (in press). Regularized partial and/or constrained redundancy analysis. Psychometrika. DOI: 10.1007/s11336-008-9067-y
Takane, Y., & Oshima-Takane, Y. (2002). Nonlinear generalized canonical correlation analysis by neural network models. In S. Nishisato, Y. Baba, H. Bozdogan, & K. Kanefuji (Eds.), Measurement and multivariate analysis (pp. 183–190). Tokyo: Springer.
Takane, Y., & Yanai, H. (2008). On ridge operators. Linear Algebra and Its Applications, 428, 1778–1790.
ten Berge, J.M.F. (1979). On the equivalence of two oblique congruence rotation methods, and orthogonal approximations. Psychometrika, 44, 359–364.
ter Braak, C.J.F. (1990). Update notes: CANOCO Version 3.10. Wageningen: Agricultural Mathematics Group.
Tikhonov, A.N., & Arsenin, V.Y. (1977). Solutions of ill-posed problems. Washington: Winston.
Tillman, B., Dowling, J., & Abdi, H. (2008). Bach, Mozart or Beethoven? Indirect investigations of musical style perception with subjective judgments and sorting tasks. In preparation.
van de Velden, M., & Bijmolt, T.H.A. (2006). Generalized canonical correlation analysis of matrices with missing rows: a simulation study. Psychometrika, 71, 323–331.
van der Burg, E. (1988). Nonlinear canonical correlation and some related techniques. Leiden: DSWO Press.
Vinod, H.D. (1976). Canonical ridge and econometrics of joint production. Journal of Econometrics, 4, 47–166.
Author information
Authors and Affiliations
Corresponding author
Additional information
The work reported in this paper is supported by Grants 10630 and 290439 from the Natural Sciences and Engineering Research Council of Canada to the first and the second authors, respectively. The authors would like to thank the two editors (old and new), the associate editor, and four anonymous reviewers for their insightful comments on earlier versions of this paper. Matlab programs that carried out the computations reported in the paper are available upon request.
Rights and permissions
About this article
Cite this article
Takane, Y., Hwang, H. & Abdi, H. Regularized Multiple-Set Canonical Correlation Analysis. Psychometrika 73, 753–775 (2008). https://doi.org/10.1007/s11336-008-9065-0
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11336-008-9065-0