Statistics and Computing

, Volume 4, Issue 3, pp 203–212

# A computational framework for variable selection in multivariate regression

• Bruce E. Barrett
• J. Brian Gray
Papers

## Abstract

Stepwise variable selection procedures are computationally inexpensive methods for constructing useful regression models for a single dependent variable. At each step a variable is entered into or deleted from the current model, based on the criterion of minimizing the error sum of squares (SSE). When there is more than one dependent variable, the situation is more complex. In this article we propose variable selection criteria for multivariate regression which generalize the univariate SSE criterion. Specifically, we suggest minimizing some function of the estimated error covariance matrix: the trace, the determinant, or the largest eigenvalue. The computations associated with these criteria may be burdensome. We develop a computational framework based on the use of the SWEEP operator which greatly reduces these calculations for stepwise variable selection in multivariate regression.

## Keywords

Multivariate regression stepwise regression variable selection

## References

1. Beaton, A. E. (1964). The Use of Special Matrix Operators in Statistical Calculus. Ed.D. Thesis, Harvard University. Reprinted as Educational Testing Service Bulletin 64–51, Princeton, NJ.Google Scholar
2. Draper, N. R. and Smith, H. (1981). Applied Regression Analysis, 2nd Edition. Wiley, New York.Google Scholar
3. Golub, H. G. and Van Loan, C. F. (1983). Matrix Computations. The Johns Hopkins University Press, Baltimore.Google Scholar
4. Goodnight, J. H. (1979). A tutorial on the SWEEP operator. American Statistician, 33, 149–158.Google Scholar
5. Graybill, F. A. (1983). Matrices With Applications in Statistics. Wadsworth, Belmont, CA.Google Scholar
6. Horn, R. A. and Johnson, C. R. (1985). Matrix Analysis. Cambridge University Press.Google Scholar
7. McKay, R. J. (1979). The adequacy of variable subsets in multivariate regression. Technometrics, 21, 475–479.Google Scholar
8. Siotani, M., Hayakawa, T. and Fujikoshi, Y. (1985). Modern Multivariate Statistical Analysis: A Graduate Course and Handbook. American Sciences Press, Columbus, OH.Google Scholar
9. Sparks, R. S., Zucchini, W. and Coutsourides, D. (1985). On variable selection in multivariate regression. Communications in Statistics—Theory and Methods, 14, 1569–1587.Google Scholar
10. Weisberg, S. (1985). Applied Linear Regression. Wiley, New York.Google Scholar