Robust canonical correlations: A comparative study

Summary

Several approaches for robust canonical correlation analysis will be presented and discussed. A first method is based on the definition of canonical correlation analysis as looking for linear combinations of two sets of variables having maximal (robust) correlation. A second method is based on alternating robust regressions. These methods are discussed in detail and compared with the more traditional approach to robust canonical correlation via covariance matrix estimates. A simulation study compares the performance of the different estimators under several kinds of sampling schemes. Robustness is studied as well by breakdown plots.

6 Appendix

Least Squares Alternating Regression Scheme (using the notations of Section 3):

• Step 1: $\boldsymbol{X}_{0}=\boldsymbol{X}-\mathbf{1} \overline{\boldsymbol{x}}^{t}, \boldsymbol{Y}_{0}=\boldsymbol{Y}-\mathbf{1} \overline{\boldsymbol{y}}^{t}$

• Step 2: For l = 1, …, k:

  • Step 2.1: Residual spaces (only if l > 1):

    $$\begin{array}{l}{\boldsymbol{X}_{l-1}=\left(\boldsymbol{I}_{n}-\frac{\boldsymbol{u}_{l-1} \boldsymbol{u}_{l-1}^{t}}{\boldsymbol{u}_{l-1}^{t} \boldsymbol{u}_{l-1}}\right) \boldsymbol{X}_{l-2}} \\ {\boldsymbol{Y}_{l-1}=\left(\boldsymbol{I}_{n}-\frac{\boldsymbol{v}_{l-1} \boldsymbol{v}_{l-1}^{t}}{\boldsymbol{v}_{l-1}^{t} \boldsymbol{v}_{l-1}}\right) \boldsymbol{Y}_{l-2}}\end{array}$$

  • Step 2.2: Starting values (using first principal component $\boldsymbol{z}_{1}^{l-1}$ of X_l−1): $\begin{array}{l}{\hat{b}_{l}^{(0)}\ =\left(\boldsymbol{Y}_{l-1}^{t} \boldsymbol{Y}_{l-1}\right)^{-} \boldsymbol{Y}_{l-1}^{t} \boldsymbol{z}_{1}^{l-1}} \\ {\boldsymbol{\beta}_{l}^{(0)}=\frac{\hat{\boldsymbol{b}}_{l}^{(0)}}{\left\|\hat{\boldsymbol{b}}_{l}^{(0)}\right\|}} \\ {\boldsymbol{v}_{l}^{(0)}=\boldsymbol{Y}_{l-1} \boldsymbol{\beta}_{l}^{(0)}}\end{array}$

  • Step 2.3: From iteration s = 1 to convergence:

    $$\begin{aligned} \hat{\boldsymbol{a}}_{l}^{(s)} &=\left(\boldsymbol{X}_{l-1}^{t} \boldsymbol{X}_{l-1}\right)^{-} \boldsymbol{X}_{l-1}^{t} \boldsymbol{v}_{l}^{(s-1)} \\ \boldsymbol{\alpha}_{l}^{(s)} &=\frac{\hat{\boldsymbol{a}}_{l}^{(s)}}{\left\|\hat{\boldsymbol{a}}_{l}^{(s)}\right\|} \\ \boldsymbol{u}_{l}^{(s)} &=\boldsymbol{X}_{l-1} \boldsymbol{\alpha}_{l}^{(s)} \\ \hat{\boldsymbol{b}}_{l}^{(s)} &=\left(\boldsymbol{Y}_{l-1}^{t} \boldsymbol{Y}_{l-1}\right)^{-} \boldsymbol{Y}_{l-1}^{t} \boldsymbol{u}_{l}^{(s)}\\ \boldsymbol{\beta}_{l}^{(s)} &=\frac{\hat{\boldsymbol{b}}_{l}^{(s)}}{\left\|\hat{\boldsymbol{b}}_{l}^{(s)}\right\|} \\ \boldsymbol{v}_{l}^{(s)} &= \boldsymbol{Y}_{l-1}\boldsymbol{\beta}_{l}^{s}\end{aligned}$$

  • Step 2.4: After convergence, resulting in $\boldsymbol{u}_{l}^{*}, \boldsymbol{v}_{l}^{*}, \boldsymbol{\alpha}_{l}^{*}, \boldsymbol{\beta}_{1}^{*}$:

    $$\left|r_{l}=\operatorname{Corr}\left(\boldsymbol{u}_{l}^{*}, \boldsymbol{v}_{l}^{*}\right)\right|$$

    • Step 2.4.1: If l = 1: $\boldsymbol{u}_{1}=\boldsymbol{u}_{1}^{*}, \boldsymbol{v}_{1}=\boldsymbol{v}_{1}^{*}, \hat{\boldsymbol{\alpha}}_{1}=\boldsymbol{\alpha}_{1}^{*}, \boldsymbol{\beta}_{1}=\boldsymbol{\beta}_{1}^{*}$

    • Step 2.4.2: If l > 1: $\begin{array}{l}{\boldsymbol{u}_{l}=\boldsymbol{u}_{l}^{*}} \\ {\hat{\boldsymbol{\alpha}}_{l}=\left(\boldsymbol{X}_{0}^{t} \boldsymbol{X}_{0}\right)^{-1} \boldsymbol{X}_{0}^{t} \boldsymbol{u}_{l}} \\ {\boldsymbol{v}_{l}=\boldsymbol{v}_{l}^{*}} \\ {\hat{\boldsymbol{\beta}}_{l}=\left(\boldsymbol{Y}_{0}^{t} \boldsymbol{Y}_{0}\right)^{-1} \boldsymbol{Y}_{0}^{t} \boldsymbol{v}_{l}}\end{array}$

Robust Alternating Regression Scheme (using the notations of Section 3):

• Step 1: $\boldsymbol{X}_{0}=\boldsymbol{X}-\mathbf{1} \tilde{\boldsymbol{x}}^{t}, \boldsymbol{Y}_{0}=\boldsymbol{Y}-\mathbf{1} \tilde{\boldsymbol{y}}^{t}$$\tilde{\boldsymbol{x}}$ and ӯ are the column-wise medians of X and Y, respectively.

• Step 2: For l = 1, …, k:

  • Step 2.1: Residual spaces (only if l > 1):

    • X_l−1 are the estimated residuals of X_l−2 = u_l−1c^t + ε₁ using weighted L₁ regressions with weights w_i;(u_l−1)

    • Y_l−1 are the estimated residuals of Y_l−2 = v_l−1d^t + ε₂ using weighted L₁ regressions with weights w_i(v_l−1)

  • Step 2.2: Starting values:

    • Compute the first robust principal component $\boldsymbol{z}_{1}^{l-1}$ of X_l−1 using the algorithm of Croux and Ruiz-Gazen (1996)

    • $\hat{\boldsymbol{b}}_{l}^{(0)}$ are the estimated coefficients of $z_{1}^{l-1}=\boldsymbol{Y}_{l-1} \boldsymbol{b}_{l}^{(0)}+\varepsilon_{3}$ using weighted L₁ regression with weights $w_{i}\left(\boldsymbol{Y}_{l-1}^{*}\right)$

    • $$\begin{array}{l}{\boldsymbol{\beta}_{l}^{(0)}=\frac{\hat{\boldsymbol{b}}_{l}^{(0)}}{\left\|\hat{\boldsymbol{b}}_{l}^{(0)}\right\|}} \\ {v_{l}^{(0)}=\boldsymbol{Y}_{l-1} \boldsymbol{\beta}_{l}^{(0)}}\end{array}$$

  • Step 2.3: From iteration s = 1 upto convergence:

    • $\hat{\boldsymbol{a}}_{l}^{(s)}$ are the estimated coefficients of $\boldsymbol{v}_{l}^{s-1}=\boldsymbol{X}_{l-1} \boldsymbol{a}_{l}^{(s)}+\boldsymbol{\varepsilon}_{4}$ using weighted L₁ regression with weights $w_{i}\left(\boldsymbol{X}_{l-1}^{*}\right)$

    • $\begin{aligned} \boldsymbol{\alpha}_{l}^{(s)} &=\frac{\hat{\boldsymbol{a}}_{l}^{(s)}}{\left\|\hat{\boldsymbol{a}}_{l}^{(s)}\right\|} \\ \boldsymbol{u}_{l}^{(s)} &=\boldsymbol{X}_{l-1} \boldsymbol{\alpha}_{l}^{(s)} \end{aligned}$ are the estimated coefficients of $\boldsymbol{u}_{l}^{s-1}=\boldsymbol{Y}_{l-1} \boldsymbol{b}_{l}^{(s)}+\boldsymbol{\varepsilon}_{5}$ using weighted L₁ regression with weights $w_{i}\left(\boldsymbol{Y}_{l-1}^{*}\right)$

    • $$\begin{array}{l}{\boldsymbol{\beta}_{l}^{(s)}=\frac{\hat{\boldsymbol{b}}_{l}^{(s)}}{\left\|\hat{\boldsymbol{b}}_{l}^{(s)}\right\|}} \\ {\boldsymbol{v}_{l}^{(s)}=\boldsymbol{Y}_{l-1} \boldsymbol{\beta}_{l}^{(s)}}\end{array}$$

    • Step 2.4: After convergence, resulting in $\boldsymbol{u}_{l}^{*}, \boldsymbol{v}_{l}^{*}, \boldsymbol{\alpha}_{l}^{*}, \boldsymbol{\beta}_{1}^{*}$: $r_{l}=\operatorname{Corr}\left(\boldsymbol{u}_{l}^{*}, \boldsymbol{v}_{l}^{*}\right)$; Corr is a robust correlation measure like the bivariate MCD correlation discussed in Section 2

      • Step 2.4.1: If l = 1: $\boldsymbol{u}_{1}=\boldsymbol{u}_{1}^{*}, \boldsymbol{v}_{1}=\boldsymbol{v}_{1}^{*}, \hat{\boldsymbol{\alpha}}_{1}=\boldsymbol{\alpha}_{1}^{*}, \hat{\boldsymbol{\beta}}_{1}=\boldsymbol{\beta}_{1}^{*}$

      • Step 2.4.2: If l > 1:

        • U_l−1 = [u₁, …, u_l−1]

        • ũ_l are the estimated residuals of $\boldsymbol{u}_{l}^{*}=\boldsymbol{U}_{l-1}\boldsymbol{e}+\boldsymbol{\varepsilon}_{6}$ using robust LTS regression

        • $\hat{\boldsymbol{\alpha}}_{l}$ are the estimated coefficients of ũ_l = X₀f + ε₇ using robust LTS regression

        • $\boldsymbol{u}_{l}=\boldsymbol{X}_{0} \hat{\boldsymbol{\alpha}}_{l}$

        • V_l−1 = [v₁, …, v_l−1]

        • ṽ_l are the estimated residuals of $\boldsymbol{v}_{l}^{*}=\boldsymbol{V}_{l-1} \boldsymbol{g}+\boldsymbol{\varepsilon}_{8}$ using robust LTS regression

        • $\hat{\boldsymbol{\beta}}_{l}$ are the estimated coefficients of ṽ_l = X₀h + ε₉ using robust LTS regression

        • $\boldsymbol{v}_{l}=\boldsymbol{Y}_{0} \hat{\boldsymbol{\beta}}_{l}$