A note about the corrected VIF

Abstract

This paper discusses some limitations when applying the CVIF of Curto and Pinto in J Appl Stat 38(7):1499–1507 (2011) and proposes some modifications to overcome them. The concept of modified CVIF is also extended to be applied in ridge estimation.

Appendices

Appendices

1.1 Appendix 1: $R^{2}_{0}(k)$ is a continuous function at $k=0$

$R_{0}^{2}(k)=\sum \nolimits _{j=1}^{p}corr\left( \mathbf {y}^{R},\mathbf {X} _{j}^{R}\right) ^{2}$ will be a continuous function at $k=0$ if $R_{0}^{2}(0)=R_{0}^{2}=\sum \nolimits _{j=1}^{p}corr\left( \mathbf {y}, \mathbf {X}_{j}\right) ^{2}$ where , $\mathbf {X}_{j}^{R}$ is the j-column of

and $corr\left( \mathbf {y}^{R},\mathbf {X}_{j}^{R}\right)$ is the coefficient of correlation between $\mathbf {y}^{R}$ and $\mathbf {X}_{j}^{R}$ , that is:

$$\begin{aligned} corr\left( \mathbf {y}^{R},\mathbf {X}_{j}^{R}\right) =\frac{cov\left( \mathbf { y}^{R},\mathbf {X}_{j}^{R}\right) }{\sqrt{var\left( \mathbf {y}^{R}\right) } \sqrt{var\left( \mathbf {X}_{j}^{R}\right) }},\qquad j=1,\ldots ,p. \end{aligned}$$

Taking into account that:

$$\begin{aligned} y_{i}^{R}= & {} \left\{ \begin{array}{c@{\quad }l} y_{i} &{} i=1,\ldots ,n \\ 0 &{} i=n+1,\ldots ,n+p \end{array} \right. ,\\ X_{ij}^{R}= & {} \left\{ \begin{array}{l@{\quad }l} X_{ij} &{} i=1,\ldots ,n \\ 0 &{} \quad i=n+1,\ldots ,n+p,\ i\not =n+j \\ \sqrt{k} &{} i=n+j \end{array} \right. , \end{aligned}$$

we obtain that:

$$\begin{aligned} cov\left( \mathbf {y}^{R},\mathbf {X}_{j}^{R}\right)= & {} \frac{1}{n+p} \sum \limits _{i=1}^{n}y_{i}X_{ij}-\frac{n\overline{\mathbf {y}}(n\overline{ \mathbf {X}}_{j}+\sqrt{k})}{(n+p)^{2}}, \end{aligned}$$

(11)

$$\begin{aligned} var\left( \mathbf {y}^{R}\right)= & {} \frac{1}{n+p}\sum \limits _{i=1}^{n}y_{i}^{2}-\frac{n^{2}\overline{\mathbf {y}}^{2}}{(n+p)^{2}}, \end{aligned}$$

(12)

$$\begin{aligned} var\left( \mathbf {X}_{j}^{R}\right)= & {} \frac{1}{n+p}\sum \limits _{i=1}^{n} \left( X_{ij}^{2}+k\right) -\left( \frac{n\overline{\mathbf {X}}_{j}+\sqrt{k} }{n+p}\right) ^{2}. \end{aligned}$$

(13)

By considering $k=0$, the previous expressions become:

$$\begin{aligned} cov\left( \mathbf {y}^{R},\mathbf {X}_{j}^{R}\right)= & {} \frac{1}{n+p} \sum \limits _{i=1}^{n}y_{i}X_{ij}-\frac{n^{2}\overline{\mathbf {y}}\overline{ \mathbf {X}}_{j}}{(n+p)^{2}},\\ var\left( \mathbf {y}^{R}\right)= & {} \frac{1}{n+p}\sum \limits _{i=1}^{n}y_{i}^{2}-\frac{n^{2}\overline{\mathbf {y}}^{2}}{(n+p)^{2}},\\ var\left( \mathbf {X}_{j}^{R}\right)= & {} \frac{1}{n+p}\sum \limits _{i=1}^{n}X_{ij}^{2}-\left( \frac{n\overline{\mathbf {X}}_{j}}{n+p} \right) ^{2}, \end{aligned}$$

and it is then evident that $R_{0}^{2}(0)\not =R_{0}^{2}$ except in the case when $p=0$ (which is not possible since it means that there are no independent variables in the model).

However, if the data are standardized, that is, $\overline{\mathbf {y}}=0=\overline{\mathbf {X}}_{j}$ , we have that:

$$\begin{aligned} cov\left( \mathbf {y}^{R},\mathbf {X}_{j}^{R}\right)= & {} \frac{1}{n+p} \sum \limits _{i=1}^{n}y_{i}X_{ij}=\frac{n}{n+p}cov\left( \mathbf {y},\mathbf {X} _{j}\right) , \\ var\left( \mathbf {y}^{R}\right)= & {} \frac{1}{n+p}\sum \limits _{i=1}^{n}y_{i}^{2}=\frac{n}{n+p}var\left( \mathbf {y}\right) ,\\ var\left( \mathbf {X}_{j}^{R}\right)= & {} \frac{1}{n+p}\sum \limits _{i=1}^{n}X_{ij}^{2}=\frac{n}{n+p}var\left( \mathbf {X}_{j}^{R}\right) , \end{aligned}$$

and, in that case, it is clear that

$$\begin{aligned} corr\left( \mathbf {y}^{R},\mathbf {X}_{j}^{R}\right) =corr\left( \mathbf {y}, \mathbf {X}_{j}\right) ,\qquad j=1,\dots ,p, \end{aligned}$$

and, as consequence $R_{0}^{2}(0)=R_{0}^{2}$.

1.2 Appendix 2: $R^{2}(k)$ is a continuous function at $k=0$

$R^{2}(k)$ will be a continuous function at $k=0$ if the coefficient of determination of the model (7) for $k=0$ is equal to the one obtained from model (1), that is, if expressions:

$$\begin{aligned} R^{2}(k)=\frac{\widehat{\varvec{\beta }}(k)^{t}\left( \mathbf {X} ^{R}\right) ^{t}\mathbf {y}^{R}-(n+p)\left( \overline{\mathbf {y}}^{R}\right) ^{2}}{\left( \mathbf {y}^{R}\right) ^{t}\mathbf {y}^{R}-(n+p)\left( \overline{ \mathbf {y}}^{R}\right) ^{2}},\quad R^{2}=\frac{\widehat{\varvec{\beta }} \mathbf {X}^{t}\mathbf {y}-n\overline{\mathbf {y}}^{2}}{\mathbf {y}^{t}\mathbf {y} -n\overline{\mathbf {y}}^{2}}, \end{aligned}$$

coincide for $k=0$.

From the original data we obtain that:

$$\begin{aligned} \left( \mathbf {y}^{R}\right) ^{t}\mathbf {y}^{R}=\mathbf {y}^{t}\mathbf {y} ,\quad \left( \mathbf {X}^{R}\right) ^{t}\mathbf {y}^{R}=\mathbf {X}^{t}\mathbf { y},\quad \overline{\mathbf {y}}^{R}=\frac{n}{n+p}\overline{\mathbf {y}}, \end{aligned}$$

and since $\widehat{\varvec{\beta }}(0)=\widehat{\varvec{\beta }}$ it is verified that:

$$\begin{aligned} R^{2}(0)=\frac{\widehat{\varvec{\beta }}\mathbf {X}^{t}\mathbf {y}-\frac{ n^{2}}{n+p}\overline{\mathbf {y}}^{2}}{\mathbf {y}^{t}\mathbf {y}-\frac{n^{2}}{ n+p}\overline{\mathbf {y}}^{2}}. \end{aligned}$$

It is then clear that $R_{0}^{2}\not =R^{2}$ except in the case when $p=0$ (which it is not possible since it means that there are no independent variables in the model).

However, if the data are standardized ($\overline{\mathbf {y}}=0= \overline{\mathbf {X}}_{j}$), we obtain that:

$$\begin{aligned} R^{2}(0)=\widehat{\varvec{\beta }}\mathbf {X}^{t}\mathbf {y}=R^{2}. \end{aligned}$$

1.3 Appendix 3: $R_{0}^{2}(k)$ is a decreasing function in k

Since the continuity is verified with standardized data, we will work directly from standardized data. By considering Eqs. (11) and (13) for $\overline{\mathbf {y}}=0=\overline{\mathbf {X}} _{j}$ we obtain that:

$$\begin{aligned} cov\left( \mathbf {y}^{R},\mathbf {X}_{j}^{R}\right)= & {} \frac{1}{n+p} \sum \limits _{i=1}^{n}y_{i}X_{ij},\\ var\left( \mathbf {y}^{R}\right)= & {} \frac{1}{n+p}\sum \limits _{i=1}^{n}y_{i}^{2},\\ var\left( \mathbf {X}_{j}^{R}\right)= & {} \frac{1}{n+p}\sum \limits _{i=1}^{n} \left( X_{ij}^{2}+k\right) -\frac{k}{(n+p)^{2}}\\= & {} \frac{1}{n+p}\sum \limits _{i=1}^{n}X_{ij}^{2}+\frac{n+p-1}{(n+p)^{2}}k. \end{aligned}$$

Since it is verified that

$$\begin{aligned} corr\left( \mathbf {y}^{R},\mathbf {X}_{j}^{R}\right) ^{2}=\frac{cov\left( \mathbf {y}^{R},\mathbf {X}_{j}^{R}\right) ^{2}}{var\left( \mathbf {y} ^{R}\right) var\left( \mathbf {X}_{j}^{R}\right) }, \end{aligned}$$

it is evident that

$$\begin{aligned} \frac{\partial }{\partial k}corr\left( \mathbf {y}^{R},\mathbf {X} _{j}^{R}\right) ^{2}= & {} \frac{cov\left( \mathbf {y}^{R},\mathbf {X} _{j}^{R}\right) ^{2}}{var\left( \mathbf {y}^{R}\right) }\left[ \frac{\partial }{\partial k}\frac{1}{var\left( \mathbf {X}_{j}^{R}\right) }\right] \\= & {} \frac{cov\left( \mathbf {y}^{R},\mathbf {X}_{j}^{R}\right) ^{2}}{var\left( \mathbf {y}^{R}\right) }\left[ -\frac{\frac{n+p-1}{(n+p)^{2}}}{var\left( \mathbf {X}_{j}^{R}\right) ^{2}}\right] , \end{aligned}$$

which clearly has a negative sign. Hence we conclude that $R_{0}^{2}(k)$ is a decreasing function in k.