Advertisement

Moving Beyond Linearity

  • Gareth James
  • Daniela Witten
  • Trevor Hastie
  • Robert Tibshirani
Chapter
Part of the Springer Texts in Statistics book series (STS, volume 103)

Abstract

So far in this book, we have mostly focused on linear models. Linear models are relatively simple to describe and implement, and have advantages over other approaches in terms of interpretation and inference. However, standard linear regression can have significant limitations in terms of predictive power. This is because the linearity assumption is almost always an approximation, and sometimes a poor one. In Chapter 6 we see that we can improve upon least squares using ridge regression, the lasso, principal components regression, and other techniques. In that setting, the improvement is obtained by reducing the complexity of the linear model, and hence the variance of the estimates. But we are still using a linear model, which can only be improved so far! In this chapter we relax the linearity assumption while still attempting to maintain as much interpretability as possible. We do this by examining very simple extensions of linear models like polynomial regression and step functions, as well as more sophisticated approaches such as splines, local regression, and generalized additive models.

Keywords

Polynomial Regression Generalize Additive Model Local Regression Regression Spline Piecewise Constant Function 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  • Gareth James
    • 1
  • Daniela Witten
    • 2
  • Trevor Hastie
    • 3
  • Robert Tibshirani
    • 3
  1. 1.Department of Information and Operations ManagementUniversity of Southern CaliforniaLos AngelesUSA
  2. 2.Department of BiostatisticsUniversity of WashingtonSeattleUSA
  3. 3.Department of StatisticsStanford UniversityStanfordUSA

Personalised recommendations