Abstract
Canonical Correlation Analysis is a technique for finding pairs of basis vectors that maximise the correlation of a set of paired variables, these pairs can be considered as two views of the same object. This paper provides a convergence analysis of Canonical Correlation Analysis by defining a pattern function that captures the degree to which the features from the two views are similar. We analyse the convergence using Rademacher complexity, hence deriving the error bound for new data. The analysis provides further justification for the regularisation of kernel Canonical Correlation Analysis and is corroborated by experiments on real world data.
Article PDF
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
References
Akaho, S. (2001). A kernel method for canonical correlation analysis. In International meeting of psychometric society, Osaka.
Ambroladze, A., & Shawe-Taylor, J. (2004). Complexity of pattern classes and Lipschitz property. In Proceedings of the conference on algorithmic learning theory, ALT’04.
Bach, F., & Jordan, M. (2002). Kernel independent component analysis. Journal of Machine Leaning Research, 3, 1–48.
Bartlett, P. L., & Mendelson, S. (2002). Rademacher and Gaussian complexities: Risk bounds and structural results. Journal of Machine Learning Research, 3, 463–482.
Breiman, L., & Friedman, J. H. (1985). Estimating optimal transformations for multiple regression. Journal of the American Statistical Association, 80, 580–598.
Friman, O., Borga, M., Lundberg, P., & Knutsson, H. (2003). Adaptive analysis of fMRI data. NeuroImage, 19, 837–845.
Fukumizu, K., Bach, F. R., & Gretton, A. (2006). Consistency of kernel canonical correlation analysis. Journal of Machine Learning Research, 8, 361–383.
Fyfe, C., & Lai, P. (2000). ICA using kernel canonical correlation analysis. In Proc. int. workshop on independent component analysis and blind signal separation.
Hardoon, D. R. (2006). Semantic models for machine learning. Ph.D. thesis, University of Southampton.
Hardoon, D. R., & Shawe-Taylor, J. (2003). KCCA for different level precision in content-based image retrieval. In Proceedings of third international workshop on content-based multimedia indexing, IRISA, Rennes, France.
Hardoon, D. R., Szedmak, S., & Shawe-Taylor, J. (2004). Canonical correlation analysis: an overview with application to learning methods. Neural Computation, 16, 2639–2664.
Hardoon, D. R., Saunders, C., Szedmak, S., & Shawe-Taylor, J. (2006). A correlation approach for automatic image annotation. In Springer LNAI (Vol. 4093, pp. 681–692).
Hardoon, D. R., Mourao-Miranda, J., Brammer, M., & Shawe-Taylor, J. (2007). Unsupervised analysis of fmri data using kernel canonical correlation. NeuroImage, 37(4), 1250–1259.
Hotelling, H. (1936). Relations between two sets of variates. Biometrika, 28, 312–377.
Ketterling, J. R. (1971). Canonical analysis of several sets of variables. Biometrika, 58, 433–451.
Kolenda, T., Hansen, L. K., Larsen, J., & Winther, O. (2002). Independent component analysis for understanding multimedia content. In H. Bourlard, T. Adali, S. Bengio, J. Larsen, & S. Douglas (Eds.), Proceedings of IEEE workshop on neural networks for signal processing XII (pp. 757–766). New York: IEEE Press. Martigny, Valais, Switzerland, Sept. 4–6, 2002.
Kuss, M., & Graepel, T. (2002). The geometry of kernel canonical correlation analysis. Technical report, Max Planck Institute for Biological Cybernetics.
Leurgans, S. E., Moyeed, R. A., & Silverman, B. W. (1993). Canonical correlation analysis when the data are curves. Journal at the Royal Statistical Society, 55, 725–740.
Shawe-Taylor, J., & Cristianini, N. (2004). Kernel methods for pattern analysis. Cambridge: Cambridge University Press.
Vinokourov, A., Shawe-Taylor, J., & Cristianini, N. (2002). Inferring a semantic representation of text via cross-language correlation analysis. In Advances of neural information processing systems 15.
Vinokourov, A., Hardoon, D. R., & Shawe-Taylor, J. (2003). Learning the semantics of multimedia content with application to web image retrieval and classification. In Proceedings of fourth international symposium on independent component analysis and blind source separation, Nara, Japan.
Author information
Authors and Affiliations
Corresponding author
Additional information
Editor: Tony Jebara.
Rights and permissions
About this article
Cite this article
Hardoon, D.R., Shawe-Taylor, J. Convergence analysis of kernel Canonical Correlation Analysis: theory and practice. Mach Learn 74, 23–38 (2009). https://doi.org/10.1007/s10994-008-5085-3
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10994-008-5085-3