Skip to main content
SpringerLink
Log in

Connecting the multivariate partial least squares with canonical analysis: a path-following approach

  • Regular Article
  • Published:
Advances in Data Analysis and Classification Aims and scope Submit manuscript

Abstract

Despite the fact that the regularisation of multivariate methods is a well-known and widely used statistical procedure, very few studies have considered it from the perspective of analytic matrix decomposition. Here, we introduce a link between one variant of partial least squares (PLS) and canonical correlation analysis (CCA) for multiple groups, as well as two groups covered as a special case. A continuation algorithm based on the implicit function theorem is selected, with particular attention paid to potential non-generic points based on real economic data inputs. Both degenerated crossings and multiple eigenvalues are identified on the paths. The theory of Chebyshev polynomials is applied in order to generate novel insights into the phenomenon simply generalisable to a variety of other techniques.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Notes

  1. In terms of their significance in practice, the singular value and eigenvalue decompositions of a real symmetric matrix are identical.

  2. Note that because matrix B is specifically block diagonal in this study, joint processing of its square root and inverse are realised separately for each individual block.

  3. As mentioned above, theory of Lagrange multipliers is standardly used to solve the problem.

  4. Note, for the inductive statistical conclusions, there the transformation should be realised because of relatively high positively skewed data.

References

  • Batles Z, Trefethen L (2004) An extension of Matlab to continuous functions and operators. SIAM J Sci Comput 25(5):1743–1770

    Article  MathSciNet  MATH  Google Scholar 

  • Buja A (1985) Theory of bivariate ACE, Technical report no. 74. Department of Statistics, University of Washington

  • Bunse-Gerstner A, Byers R, Mehrmann V, Nichols NK (1991) Numerical computation of an analytic singular value decomposition of a matrix valued function. Numer Math 60:1–39

    Article  MathSciNet  MATH  Google Scholar 

  • Czech Statistical Office (2017) CSO database (downloaded 2017, October 25). Czech Statistical Office, Prague, Czech Republic. Retrieved from: http://www.czso.cz/csu/czso/statistics

  • Dhooge A, Govaerts W, Kuznetsov YuA (2003) Matcont: a Matlab package for numerical bifurcation analysis of ODEs. ACM Trans Math Softw 29(2):141–164

    Article  MathSciNet  MATH  Google Scholar 

  • Dieci L, Pugliese A (2008) Singular values of two-parameter matrices: an algorithm to accurately find their intersection. Math Comput Simul 79(4):1255–1269

    Article  MathSciNet  MATH  Google Scholar 

  • European Statistical Office (2017) Eurostat database (downloaded 2017, November 3). European Statistical Office, Luxembourg, Luxembourg. Retrieved from: http://ec.europa.eu/eurostat/data/database

  • Gifi A (1990) Nonlinear multivariate analysis. Wiley, New York

    MATH  Google Scholar 

  • Golub GH, van Loan CF (1996) Matrix computations, 3rd edn. The Johns Hopkins University Press, Baltimore

    MATH  Google Scholar 

  • Hoerl AE, Kennard RW (1970) Ridge regression: applications to non-orthogonal problems. Technometrics 12:69–82

    Article  MATH  Google Scholar 

  • Janovská D, Janovský V (2010) The analytic SVD: on the non-generic points on the path. Electron Trans Numer Anal 37:70–86

    MathSciNet  MATH  Google Scholar 

  • Janovská D, Janovský V, Tanabe K (2008) An algorithm for computing the analytic singular value decomposition. World Acad Sci Eng Technol 2(11):115–120

    Google Scholar 

  • Kato T (1976) Perturbation theory for linear operators, 2nd edn. Springer, New York

    MATH  Google Scholar 

  • Krantz S, Parks H (2002) A primer of real analytic functions. Birkhauser, New York

    Book  MATH  Google Scholar 

  • Lax PD (2013) Linear algebra and its applications, 2nd edn. Wiley, New York

    Google Scholar 

  • Leissa W (1974) On a curve veering aberration. J Appl Math Phys 25:99–111

    MATH  Google Scholar 

  • Malec L (2014) Studying economics and tourism industry relations by smooth partial least squares method depending on parameter. 17th applications of mathematics and statistics in economics. Poland, Jerzmanowice, pp 173–179

  • Malec L (2016) Some remarks on the functional relation between canonical correlation analysis and partial least squares. J Stat Comput Simul 86(12):2379–2391

    Article  MathSciNet  MATH  Google Scholar 

  • Malec L, Abrhám J (2016) Determinants of tourism industry in selected European countries: a smooth partial least squares approach. Econ Res 29(1):66–84

    Google Scholar 

  • Mehrmann V, Rath W (1993) Numerical methods for the computation of analytic singular value decompositions. Electron Trans Numer Anal 1:72–88

    MathSciNet  MATH  Google Scholar 

  • Nakatsukasa Y, Noferini V, Trefethen N (2016) Computing the analytic SVD (downloaded 2018, March 23). University of Oxford and Chebfun developers. Retrieved from http://www.chebfun.org/examples/linalg/AnalyticSVD.html

  • Oelker M-R, Tutz G (2017) A uniform framework for the combination of penalties in generalized structured models. Adv Data Anal Classif 11(1):97–120

    Article  MathSciNet  MATH  Google Scholar 

  • Parkhomenko E, Tritchler D, Beyene J (2009) Sparse canonical correlation analysis with application to genomic data integration. Stat Appl Genet Mol Biol 8:1–34

    Article  MathSciNet  MATH  Google Scholar 

  • Ramsay JO, Silverman BW (2002) Applied functional data analysis: methods and case studies. Springer, New York

    Book  MATH  Google Scholar 

  • Seber GAF (2004) Multivariate observations. Wiley, New York

    MATH  Google Scholar 

  • SenGupta A (1991) Generalized correlations in the singular case. J Stat Plan Inference 28:241–245

    Article  MathSciNet  MATH  Google Scholar 

  • Takane Y, Hwang H, Abdi H (2008) Regularized multiple-set canonical correlation analysis. Psychometrika 73:753–775

    Article  MathSciNet  MATH  Google Scholar 

  • Tenenhaus A, Tenenhaus M (2011) Regularized generalized canonical correlation analysis. Psychometrika 76:257–284

    Article  MathSciNet  MATH  Google Scholar 

  • Trefethen LN (2013) Approximation theory and approximation practice. SIAM, Philadelphia

    MATH  Google Scholar 

  • Vinod HD (1976) Canonical ridge and econometrics of joint production. J Econom 4(2):147–166

    Article  MATH  Google Scholar 

  • Wegelin JA (2000) A survey of partial least squares (PLS) methods, with emphasis on two-block case, Technical report no. 371. Department of Statistics, University of Washington

  • Witten DM, Tibshirani R, Hastie T (2009) A penalized matrix decomposition, with applications to sparse principal components and canonical correlation analysis. Biostatistics 10(3):515–534

    Article  MATH  Google Scholar 

  • Wright K (1992) Differential equations for the analytic singular value decomposition of a matrix. Numer Math 63:283–295

    Article  MathSciNet  MATH  Google Scholar 

  • Young LJ, Hwang MC (2001) Curve veering phenomena one-dimensional eigenvalue problems. In: Proceedings of the 18th national conference of the Chinese Society of Mechanical Engineers. Taiwan, Taipei, pp 239–246

Download references

Acknowledgements

This work was supported by the Grant Agency of Academic Alliance under GA/13/2018.

Author information

Authors and Affiliations

  1. Department of Information Technologies and Analytical Methods, University College of Business, Prague, Czech Republic

    Lukáš Malec

  2. Department of Numerical Analysis, Faculty of Mathematics and Physics, Charles University, Prague, Czech Republic

    Vladimír Janovský

Authors
  1. Lukáš Malec
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Vladimír Janovský
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Lukáš Malec.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Malec, L., Janovský, V. Connecting the multivariate partial least squares with canonical analysis: a path-following approach. Adv Data Anal Classif 14, 589–609 (2020). https://doi.org/10.1007/s11634-019-00370-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11634-019-00370-x

Keywords

Mathematics Subject Classification

Search

Navigation