Kernel Canonical Correlation Analysis and Least Squares Support Vector Machines

  • Tony Van Gestel
  • Johan A. K. Suykens
  • Jos De Brabanter
  • Bart De Moor
  • Joos Vandewalle
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 2130)


A key idea of nonlinear Support Vector Machines (SVMs) is to map the inputs in a nonlinear way to a high dimensional feature space, while Mercer’s condition is applied in order to avoid an explicit expression for the nonlinear mapping. In SVMs for nonlinear classification a large margin classifier is constructed in the feature space. For regression a linear regressor is constructed in the feature space. Other kernel extensions of linear algorithms have been proposed like kernel Principal Component Analysis (PCA) and kernel Fisher Discriminant Analysis. In this paper, we discuss the extension of linear Canonical Correlation Analysis (CCA) to a kernel CCA with application of the Mercer condition. We also discuss links with single output Least Squares SVM (LS-SVM) Regression and Classification.


Support Vector Machine Feature Space Canonical Correlation Analysis Ridge Regression Kernel Principal Component Analysis 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Anderson, T.W.: An introduction to multivariate analysis. Wiley, New York (1966).Google Scholar
  2. 2.
    Baudat, G., Anouar, F.: Generalized Discriminant Analysis Using a Kernel Approach. Neural Computation 12 (2000) 2385–2404CrossRefGoogle Scholar
  3. 3.
    Bishop, C.M.: Neural Networks for Pattern Recognition. Oxford Univ. Press (1995)Google Scholar
  4. 4.
    Cristianini, N., Shawe-Taylor, J.: An Introduction to Support Vector Machines. Cambridge University Press (2000)Google Scholar
  5. 5.
    Evgeniou, T., Pontil, M., Poggio, T.: Regularization Networks and Support Vector Machines. Advances in Computational Mathematics 13 (2001) 1–50CrossRefMathSciNetGoogle Scholar
  6. 6.
    Friedman, J.: Regularized Discriminant Analysis. Journal of the American Statistical Association 84 (1989) 165–175CrossRefMathSciNetGoogle Scholar
  7. 7.
    Lai, P.L., Fyfe, C.: Kernel and nonlinear canonical correlation analysis. International Journal of Neural Systems (2001), accepted for publicationGoogle Scholar
  8. 8.
    Mika, S., Rätsch, G., Weston, J., Schölkopf, B., & Müller, K.-R.: Fisher discriminant analysis with kernels. In: Hu, Y.-H., Larsen, J., Wilson, E., Douglas, S. (Eds.): Proc. Neural Networks for Signal Processing Workshop IX, NNSP-99 (1999)Google Scholar
  9. 9.
    Ripley, B.D.: Neural Networks and Related Methods for Classification. Journal Royal Statistical Society B 56 (1994) 409–456zbMATHMathSciNetGoogle Scholar
  10. 10.
    Schölkopf, B., Smola, A., Müller, K.-M.: Nonlinear component analysis as a kernel eigenvalue problem. Neural Computation 10 (1998) 1299–1319CrossRefGoogle Scholar
  11. 11.
    Suykens, J.A.K., Vandewalle, J.: Least squares support vector machine classifiers. Neural Processing Letters 9 (1999) 293–300CrossRefMathSciNetGoogle Scholar
  12. 12.
    Van Gestel, T., Suykens, J.A.K., Baestaens, D.-E., Lambrechts, A., Lanckriet, G., Vandaele, B., De Moor, B., Vandewalle, J.: Predicting Financial Time Series using Least Squares Support Vector Machines within the Evidence Framework. IEEE Transactions on Neural Networks (2001), to appearGoogle Scholar
  13. 13.
    Vapnik, V.: Statistical Learning Theory. Wiley, New-York (1998)zbMATHGoogle Scholar
  14. 14.
    Williams, C.K.I.: Prediction with Gaussian Processes: from Linear Regression to Linear Prediction and Beyond. In: Jordan, M.I.: Learning and Inference in Graphical Models. Kluwer Academic Press (1998)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2001

Authors and Affiliations

  • Tony Van Gestel
    • 1
  • Johan A. K. Suykens
    • 1
  • Jos De Brabanter
    • 1
  • Bart De Moor
    • 1
  • Joos Vandewalle
    • 1
  1. 1.Dept. of Electrical Engineering, ESAT-SISTAK.U.LeuvenLeuvenBelgium

Personalised recommendations