Influence measures in ridge regression when the error terms follow an Ar(1) process

Abstract

Influence concepts have an important place in linear regression models and case deletion is a useful method for assessing the influence of single case. The influence measures in the presence of multicollinearity were discussed under the linear regression models when the errors structure is uncorrelated and homoscedastic. In contrast to other article on this subject, we consider the influence measures in ridge regression with autocorrelated errors. Theoretical results are illustrated with a numerical example and a Monte Carlo simulation is conducted to see the effect autocorrelation coefficient, strength of multicollinearity and sample size on leverage points and influential observations.

Appendix

The difference between $\widehat{b}^{(k)}$ and $\widehat{b}_{(i)}^{(k)}$ The ARR estimator without the ith observation is

$$\begin{aligned} \widehat{b}_{(i)}^{(k)}=\left( Z_{(i)}^{*\prime }Z_{(i)}^{*}+kI_{p}^{*}\right) ^{-1}Z_{(i)}^{*\prime }y_{(i)}^{*}=\left( Z^{*\prime }Z^{*}-z_{i}^{*\prime }z_{i}^{*}+kI_{p}^{*}\right) ^{-1}\left( Z^{*\prime }y^{*}-z_{i}^{*\prime }y_{i}^{*}\right) . \end{aligned}$$

By using Sherman Morrison Woodbury (SMV) Theorem (Myers 1990, p. 459) for the matrix $\left( Z^{*\prime }Z^{*}-z_{i}^{*\prime }z_{i}^{*}+kI_{p}^{*}\right) ^{-1}$, we get

$$\begin{aligned} \widehat{b}_{(i)}^{(k)}= & {} \left( (Z^{*\prime }Z^{*}+kI_{p}^{*})^{-1}+\frac{(Z^{*\prime }Z^{*}+kI_{p}^{*})^{-1}z_{i}^{*\prime }z_{i}^{*}(Z^{*\prime }Z^{*}+kI_{p}^{*})^{-1}}{ 1-z_{i}^{*}(Z^{*\prime }Z^{*}+kI_{p}^{*})^{-1}z_{i}^{*\prime }}\right) \nonumber \\&\times \,\left( Z^{*\prime }y^{*}-z_{i}^{*\prime }y_{i}^{*}\right) \\= & {} \widehat{b}^{(k)}-\frac{\left( Z^{*\prime }Z^{*}+kI_{p}^{*}\right) ^{-1}z_{i}^{*\prime }\widehat{u}^{(k)*}}{1-l_{i}^{(k)}}. \end{aligned}$$

Then the difference between $\widehat{b}^{(k)}$ and $\widehat{b}_{(i)}^{(k)}$ is obtained as

$$\begin{aligned} \widehat{b}^{(k)}-\widehat{b}_{(i)}^{(k)}=\frac{\left( Z^{*\prime }Z^{*}+kI_{p}^{*}\right) ^{-1}z_{i}^{*\prime }\widehat{u} _{i}^{(k)*}}{1-l_{i}^{(k)}}. \end{aligned}$$

Since $z_{i}^{*}(\widehat{b}^{(k)}-\widehat{b}_{(i)}^{(k)})=\frac{ l_{i}^{(k)}\widehat{u}_{i}^{(k)*}}{1-l_{i}^{(k)}}$ and

$$\begin{aligned} se(z_{i}^{*}\widehat{b}^{(k)})= & {} \sigma \left( z_{i}^{*}\left( Z^{*\prime }Z^{*}+kI_{p}^{*}\right) ^{-1}Z^{*\prime }Z^{*}\left( Z^{*\prime }Z^{*}+kI_{p}^{*}\right) ^{-1}z_{i}^{*\prime }\right) ^{1/2} \\= & {} s_{(i)}^{*}\left[ \sum _{j=1}^{n}\left( l_{ij}^{(k)}\right) ^{2}\right] ^{1/2}, \end{aligned}$$

we get the result.