Multivariate Linear Regression

  • David J. Olive


This chapter will show that multivariate linear regression with m ≥ 2 response variables is nearly as easy to use, at least if m is small, as multiple linear regression which has m = 1 response variable. Plots for checking the model are given, and prediction regions that are robust to nonnormality are developed. For hypothesis testing, it is shown that the Wilks’ lambda statistic, Hotelling Lawley trace statistic, and Pillai’s trace statistic are robust to nonnormality.


  1. Anderson, T. W. (1984). An introduction to multivariate statistical analysis (2nd ed.). New York, NY: Wiley.zbMATHGoogle Scholar
  2. Berk, K. N. (1978). Comparing subset regression procedures. Technometrics, 20, 1–6.CrossRefzbMATHGoogle Scholar
  3. Berndt, E. R., & Savin, N. E. (1977). Conflict among criteria for testing hypotheses in the multivariate linear regression model. Econometrika, 45, 1263–1277.MathSciNetCrossRefzbMATHGoogle Scholar
  4. Cook, R. D. (1977). Deletion of influential observations in linear regression. Technometrics,19, 15–18.Google Scholar
  5. Cook, R. D., Helland, I. S., & Su, Z. (2013). Envelopes and partial least squares regression. Journal of the Royal Statistical Society, B, 75, 851–877.MathSciNetCrossRefGoogle Scholar
  6. Cook, R. D., & Setodji, C. M. (2003). A model-free test for reduced rank in multivariate regression. Journal of the American Statistical Association, 98, 340–351.MathSciNetCrossRefzbMATHGoogle Scholar
  7. Cook, R. D., & Su, Z. (2013). Scaled envelopes: Scale-invariant and efficient estimation in multivariate linear regression. Biometrika, 100, 929–954.MathSciNetCrossRefzbMATHGoogle Scholar
  8. Cook, R. D., & Weisberg, S. (1999a). Applied regression including computing and graphics. New York, NY: Wiley.Google Scholar
  9. Cook, R. D., & Weisberg, S. (1999b). Graphs in statistical analysis: Is the medium the message? The American Statistician, 53, 29–37.Google Scholar
  10. Copas, J. B. (1983). Regression prediction and shrinkage (with discussion). Journal of the Royal Statistical Society, B, 45, 311–354.MathSciNetzbMATHGoogle Scholar
  11. Fujikoshi, Y., Sakurai, T., & Yanagihara, H. (2014). Consistency of high-dimensional AIC-type and C p-type criteria in multivariate linear regression. Journal of Multivariate Analysis, 123, 184–200.MathSciNetCrossRefzbMATHGoogle Scholar
  12. Hebbler, B. (1847). Statistics of Prussia. Journal of the Royal Statistical Society, A, 10, 154–186.Google Scholar
  13. Henderson, H. V., & Searle, S. R. (1979). Vec and Vech operators for matrices, with some uses in Jacobians and multivariate statistics. The Canadian Journal of Statistics, 7, 65–81.MathSciNetCrossRefzbMATHGoogle Scholar
  14. Hyndman, R. J. (1996). Computing and graphing highest density regions. The American Statistician, 50, 120–126.Google Scholar
  15. Johnson, R. A., & Wichern, D. W. (1988). Applied multivariate statistical analysis (2nd ed.). Englewood Cliffs, NJ: Prentice Hall.zbMATHGoogle Scholar
  16. Kakizawa, Y. (2009). Third-order power comparisons for a class of tests for multivariate linear hypothesis under general distributions. Journal of Multivariate Analysis, 100, 473–496.MathSciNetCrossRefzbMATHGoogle Scholar
  17. Khattree, R., & Naik, D. N. (1999). Applied multivariate statistics with SAS software (2nd ed.). Cary, NC: SAS Institute.Google Scholar
  18. Kshirsagar, A. M. (1972). Multivariate analysis New York, NY: Marcel Dekker.zbMATHGoogle Scholar
  19. Lei, J., Robins, J., & Wasserman, L. (2013). Distribution free prediction sets. Journal of the American Statistical Association, 108, 278–287.MathSciNetCrossRefzbMATHGoogle Scholar
  20. Lei, J., & Wasserman, L. (2014). Distribution free prediction bands. Journal of the Royal Statistical Society, B, 76, 71–96.CrossRefGoogle Scholar
  21. Maronna, R. A., & Zamar, R. H. (2002). Robust estimates of location and dispersion for high-dimensional datasets. Technometrics, 44, 307–317.MathSciNetCrossRefGoogle Scholar
  22. Miller, D. M. (1984). Reducing transformation bias in curve fitting. The American Statistician, 38, 124–126.Google Scholar
  23. Olive, D. J. (2002). Applications of robust distances for regression. Technometrics, 44, 64–71.MathSciNetCrossRefGoogle Scholar
  24. Olive, D. J. (2008), Applied robust statistics, online course notes, see Google Scholar
  25. Olive, D. J. (2013a), Asymptotically optimal regression prediction intervals and prediction regions for multivariate data. International Journal of Statistics and Probability, 2, 90–100.Google Scholar
  26. Olive, D. J. (2016b). Applications of hyperellipsoidal prediction regions. Statistical Papers, to appear.Google Scholar
  27. Olive, D. J. (2016c). Robust multivariate analysis. New York, NY: Springer, to appear.Google Scholar
  28. Olive, D. J., & Hawkins, D. M. (2005). Variable selection for 1D regression models. Technometrics, 47, 43–50.MathSciNetCrossRefGoogle Scholar
  29. Pelawa Watagoda, L. C. R. (2013). Plots and Testing for Multivariate Linear Regression. Master’s Research Paper, Southern Illinois University, at
  30. R Core Team (2016). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing, Google Scholar
  31. Rencher, A., & Pun, F. (1980). Inflation of R 2 in best subset regression. Technometrics, 22, 49–53.CrossRefzbMATHGoogle Scholar
  32. SAS Institute (1985). SAS user’s guide: Statistics. Version 5. Cary, NC: SAS Institute.Google Scholar
  33. Searle, S. R. (1982). Matrix algebra useful for statistics. New York, NY: Wiley.zbMATHGoogle Scholar
  34. Seber, G. A. F., & Lee, A. J. (2003). Linear regression analysis (2nd ed.). New York, NY: Wiley.CrossRefzbMATHGoogle Scholar
  35. Su, Z., & Cook, R. D. (2012). Inner envelopes: Efficient estimation in multivariate linear regression. Biometrika, 99, 687–702.MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • David J. Olive
    • 1
  1. 1.Department of MathematicsSouthern Illinois UniversityCarbondaleUSA

Personalised recommendations