Skip to main content
SpringerLink
Log in

LMs, GLMs and GAMs

  • Chapter
Vector Generalized Linear and Additive Models

Part of the book series: Springer Series in Statistics ((SSS))

  • 6153 Accesses

Abstract

For completeness, this chapter summarizes some relevant aspects of the linear model (LM) and generalized linear model (GLM) for the book. A basic understanding of these is helpful when considering VGLMs later. Some topics covered include link functions, the exponential family, assumptions, estimation (especially IRLS), numerical and computing aspects, how to fit these models in R, and diagnostics. Also, a section on smoothing is provided, which covers splines and local regression at an introductory level. Generalized additive models are briefly mentioned.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
eBook
USD 16.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    The curvature of a curve y = f(x) is \(\left \vert f''(x)\right \vert \,\left \{1 + \left [f'(x)\right ]^{2}\right \}^{-3/2}\). If the { } term is dropped (because the assumption  | f′(x) | ≪ 1 is almost always made in physics and engineering), then  | f″(x) | is left as an approximation to the curvature. In natural cubic spline interpolation, we are finding a curve with minimal (approximate) curvature over an interval, for the quantity \(\int \left [f''(x)\right ]^{2}\,dx\) is being minimized.

  2. 2.

    Explicitly, letting \(h_{i} = x_{i+1} - x_{i}\) for i = 1, …, n − 1, their nonzero elements are: \((\mathbf{T})_{ii} = (h_{i} + h_{i+1})/3\), \((\mathbf{T})_{i,i-1} = (\mathbf{T})_{i,i+1} = h_{i}/6\), \((\mathbf{Q})_{ii} = h_{i}^{-1}\), \((\mathbf{Q})_{i+1,i} = -(h_{i}^{-1} + h_{i+1}^{-1})\) and \((\mathbf{Q})_{i+2,i} = h_{i+1}^{-1}\).

References

  • Agresti, A. 2015. Foundations of Linear and Generalized Linear Models. Hoboken: Wiley.

    MATH  Google Scholar 

  • Aitkin, M., B. Francis, J. Hinde, and R. Darnell 2009. Statistical Modelling in R. Oxford: Oxford University Press.

    MATH  Google Scholar 

  • Albert, A. and J. A. Anderson 1984. On the existence of maximum likelihood estimates in logistic regression models. Biometrika 71(1):1–10.

    Article  MATH  MathSciNet  Google Scholar 

  • Allison, P. 2004. Convergence problems in logistic regression. See Altman et al. (2004), pp. 238–252.

    Google Scholar 

  • Altman, M., J. Gill, and M. P. McDonald 2004. Numerical Issues in Statistical Computing for the Social Scientist. Hoboken: Wiley-Interscience.

    MATH  Google Scholar 

  • Amemiya, T. 1985. Advanced Econometrics. Oxford: Blackwell.

    Google Scholar 

  • Azzalini, A. 1996. Statistical Inference: Based on the Likelihood. London: Chapman & Hall.

    MATH  Google Scholar 

  • Banerjee, S. and A. Roy 2014. Linear Algebra and Matrix Analysis for Statistics. Boca Raton: CRC Press.

    MATH  Google Scholar 

  • Bellman, R. E. 1961. Adaptive Control Processes. Princeton: Princeton University Press.

    MATH  Google Scholar 

  • Belsley, D. A., E. Kuh, and R. E. Welsch 1980. Regression Diagnostics: Identifying Influential Data and Sources of Collinearity. New York: John Wiley & Sons.

    Book  MATH  Google Scholar 

  • Berlinet, A. and C. Thomas-Agnan 2004. Reproducing Kernel Hilbert Spaces in Probability and Statistics. Boston: Kluwer Academic Publishers.

    Book  MATH  Google Scholar 

  • Buja, A., T. Hastie, and R. Tibshirani 1989. Linear smoothers and additive models. The Annals of Statistics 17(2):453–510. With discussion.

    Google Scholar 

  • Christensen, R. 2011. Plane Answers to Complex Questions: The Theory of Linear Models (4th ed.). New York: Springer-Verlag.

    Book  Google Scholar 

  • Cleveland, W. S. 1979. Robust locally weighted regression and smoothing scatterplots. Journal of the American Statistical Association 74(368):829–836.

    Article  MATH  MathSciNet  Google Scholar 

  • Cleveland, W. S. and S. J. Devlin 1988. Locally weighted regression: An approach to regression analysis by local fitting. Journal of the American Statistical Association 83(403):596–610.

    Article  MATH  Google Scholar 

  • Cleveland, W. S., E. Grosse, and W. M. Shyu 1991. Local regression models. See Chambers and Hastie (1991), pp. 309–376.

    Google Scholar 

  • Cook, R. D. and S. Weisberg 1982. Residuals and Influence in Regression. Monographs on Statistics and Applied Probability. London: Chapman & Hall.

    Google Scholar 

  • de Boor, C. 2001. A Practical Guide to Splines (Revised Edition). New York: Springer.

    Google Scholar 

  • Dobson, A. J. and A. Barnett 2008. An Introduction to Generalized Linear Models (Third ed.). Boca Raton: Chapman & Hall/CRC Press.

    MATH  Google Scholar 

  • Eilers, P. H. C. and B. D. Marx 1996. Flexible smoothing with B-splines and penalties. Statistical Science 11(2):89–121.

    Article  MATH  MathSciNet  Google Scholar 

  • Eubank, R. L. 1999. Spline Smoothing and Nonparametric Regression (Second ed.). New York: Marcel-Dekker.

    MATH  Google Scholar 

  • Fahrmeir, L. and G. Tutz 2001. Multivariate Statistical Modelling Based on Generalized Linear Models (Second ed.). New York: Springer-Verlag.

    Book  MATH  Google Scholar 

  • Fan, J. and I. Gijbels 1996. Local Polynomial Modelling and Its Applications. London: Chapman & Hall.

    MATH  Google Scholar 

  • Faraway, J. J. 2006. Extending the Linear Model with R: Generalized Linear, Mixed Effects and Nonparametric Regression Models. Boca Raton: Chapman and Hall/CRC.

    Google Scholar 

  • Faraway, J. J. 2015. Linear Models with R (Second ed.). Boca Raton: Chapman & Hall/CRC.

    MATH  Google Scholar 

  • Fessler, J. A. 1991. Nonparametric fixed-interval smoothing with vector splines. IEEE Transactions on Signal Processing 39(4):852–859.

    Article  Google Scholar 

  • Firth, D. 1991. Generalized linear models. See Hinkley et al. (1991), pp. 55–82.

    Google Scholar 

  • Fox, J. and S. Weisberg 2011. An R Companion to Applied Regression (Second ed.). Thousand Oaks: Sage Publications.

    Google Scholar 

  • Gentle, J. E., W. K. Härdle, and Y. Mori 2012. Handbook of Computational Statistics: Concepts and Methods (Second ed.). Berlin: Springer.

    Book  Google Scholar 

  • Green, P. J. 1984. Iteratively reweighted least squares for maximum likelihood estimation, and some robust and resistant alternatives. Journal of the Royal Statistical Society, Series B 46(2):149–192. With discussion.

    Google Scholar 

  • Green, P. J. and B. W. Silverman 1994. Nonparametric Regression and Generalized Linear Models: A Roughness Penalty Approach. London: Chapman & Hall.

    Book  MATH  Google Scholar 

  • Gu, C. 2013. Smoothing Spline ANOVA Models (Second ed.). New York, USA: Springer.

    Book  MATH  Google Scholar 

  • Härdle, W. 1987. Smoothing Techniques With Implementation in S. New York, USA: Springer-Verlag.

    Google Scholar 

  • Härdle, W. 1990. Applied Nonparametric Regression. Cambridge: Cambridge University Press.

    Book  MATH  Google Scholar 

  • Härdle, W., H. Liang, and J. Gao 2000. Partially Linear Models. New York, USA: Springer.

    Book  MATH  Google Scholar 

  • Härdle, W., M. Müller, S. Sperlich, and A. Werwatz 2004. Nonparametric and Semiparametric Models. Berlin: Springer.

    Book  MATH  Google Scholar 

  • Harezlak, J., D. Ruppert, and M.P. Wand. 2016. Semiparametric regression in R. New York: Springer.

    Google Scholar 

  • Harville, D. A. 1997. Matrix Algebra From a Statistician’s Perspective. New York, USA: Springer-Verlag.

    Book  MATH  Google Scholar 

  • Hastie, T. 1996. Pseudosplines. Journal of the Royal Statistical Society, Series B 58(2):379–396.

    MATH  MathSciNet  Google Scholar 

  • Hastie, T. J. and D. Pregibon 1991. Generalized linear models. See Chambers and Hastie (1991), pp. 195–247.

    Google Scholar 

  • Hastie, T. J. and R. J. Tibshirani 1990. Generalized Additive Models. London: Chapman & Hall.

    MATH  Google Scholar 

  • Hastie, T. J., R. J. Tibshirani, and J. H. Friedman 2009. The Elements of Statistical Learning: Data Mining, Inference and Prediction (Second ed.). New York, USA: Springer-Verlag.

    Book  Google Scholar 

  • Hauck, J. W. W. and A. Donner 1977. Wald’s test as applied to hypotheses in logit analysis. Journal of the American Statistical Association 72(360):851–853.

    Article  MATH  MathSciNet  Google Scholar 

  • Heinze, G. and M. Schemper 2002. A solution to the problem of separation in logistic regression. Statistics in Medicine 21(16):2409–2419.

    Article  Google Scholar 

  • James, G., D. Witten, T. Hastie, and R. Tibshirani 2013. An Introduction to Statistical Learning with Applications in R. New York, USA: Springer.

    Book  MATH  Google Scholar 

  • Kohn, R. and C. F. Ansley 1987. A new algorithm for spline smoothing based on smoothing a stochastic process. SIAM Journal on Scientific and Statistical Computing 8(1):33–48.

    Article  MATH  MathSciNet  Google Scholar 

  • Lindsey, J. K. 1997. Applying Generalized Linear Models. New York, USA: Springer-Verlag.

    MATH  Google Scholar 

  • Loader, C. 1999. Local Regression and Likelihood. New York, USA: Springer.

    MATH  Google Scholar 

  • Marra, G. and R. Radice 2010. Penalised regression splines: theory and application to medical research. Statistical Methods in Medical Research 19(2):107–125.

    Article  MathSciNet  Google Scholar 

  • McCullagh, P. and J. A. Nelder 1989. Generalized Linear Models (Second ed.). London: Chapman & Hall.

    Book  MATH  Google Scholar 

  • Myers, R. H., D. C. Montgomery, G. G. Vining, and T. J. Robinson 2010. Generalized Linear Models With Applications in Engineering and the Sciences (Second ed.). Hoboken, NJ, USA: Wiley.

    Book  MATH  Google Scholar 

  • Nelder, J. A. and R. W. M. Wedderburn 1972. Generalized linear models. Journal of the Royal Statistical Society, Series A 135(3):370–384.

    Article  Google Scholar 

  • Nosedal-Sanchez, A., C. B. Storlie, T. C. M. Lee, and R. Christensen 2012. Reproducing kernel Hilbert spaces for penalized regression: A tutorial. American Statistician 66(1):50–60.

    Article  MathSciNet  Google Scholar 

  • Rao, C. R. 1973. Linear Statistical Inference and its Applications (Second ed.). New York, USA: Wiley.

    Book  MATH  Google Scholar 

  • Reinsch, C. H. 1967. Smoothing by spline functions. Numerische Mathematik 10(3):177–183.

    Article  MATH  MathSciNet  Google Scholar 

  • Rencher, A. C. and G. B. Schaalje 2008. Linear Models in Statistics (second ed.). New York, USA: John Wiley & Sons.

    MATH  Google Scholar 

  • Ruppert, D., M. P. Wand, and R. J. Carroll 2003. Semiparametric Regression. Cambridge: Cambridge University Press.

    Book  MATH  Google Scholar 

  • Ruppert, D., M. P. Wand, and R. J. Carroll 2009. Semiparametric regression during 2003–2007. Electronic Journal of Statistics 3(1):1193–1256.

    Article  MATH  MathSciNet  Google Scholar 

  • Schimek, M. G. (Ed.) 2000. Smoothing and Regression: Approaches, Computation, and Application. New York, USA: Wiley.

    Google Scholar 

  • Schumaker, L. L. 2007. Spline Functions: Basic Theory (Third ed.). Cambridge: Cambridge University Press.

    Book  Google Scholar 

  • Seber, G. A. F. 2008. A Matrix Handbook for Statisticians. Hoboken, NJ, USA: Wiley.

    Google Scholar 

  • Seber, G. A. F. and A. J. Lee 2003. Linear Regression Analysis (Second ed.). New York, USA: Wiley.

    Book  MATH  Google Scholar 

  • Silverman, B. W. 1984. Spline smoothing: The equivalent variable kernel method. The Annals of Statistics 12(3):898–916.

    Article  MATH  MathSciNet  Google Scholar 

  • Silverman, B. W. 1985. Some aspects of the spline smoothing approach to nonparametric regression curve fitting. Journal of the Royal Statistical Society, Series B 47(1):1–21. With discussion.

    Google Scholar 

  • Venables, W. N. and B. D. Ripley 2002. Modern Applied Statistics With S (4th ed.). New York, USA: Springer-Verlag.

    Book  MATH  Google Scholar 

  • Wahba, G. 1990. Spline models for observational data, Volume 59 of CBMS-NSF Regional Conference Series in Applied Mathematics. Philadelphia, PA, USA: Society for Industrial and Applied Mathematics (SIAM).

    Google Scholar 

  • Wand, M. P. and M. C. Jones 1995. Kernel Smoothing. London: Chapman & Hall.

    Book  MATH  Google Scholar 

  • Wand, M. P. and J. T. Ormerod 2008. On semiparametric regression with O’Sullivan penalized splines. Australian & New Zealand Journal of Statistics 50(2):179–198.

    Article  MATH  MathSciNet  Google Scholar 

  • Wang, Y. 2011. Smoothing Splines: Methods and Applications. Boca Raton, FL, USA: Chapman & Hall/CRC.

    Google Scholar 

  • Wecker, W. E. and C. F. Ansley 1983. The signal extraction approach to nonlinear regression and spline smoothing. Journal of the American Statistical Association 78(381):81–89.

    Article  MATH  MathSciNet  Google Scholar 

  • Wedderburn, R. W. M. 1974. Quasi-likelihood functions, generalized linear models, and the Gauss-Newton method. Biometrika 61(3):439–447.

    MATH  MathSciNet  Google Scholar 

  • Wold, S. 1974. Spline functions in data analysis. Technometrics 16(1):1–11.

    Article  MATH  MathSciNet  Google Scholar 

  • Wood, S. N. 2006. Generalized Additive Models: An Introduction with R. London: Chapman and Hall.

    Google Scholar 

  • Yanai, H., K. Takeuchi, and Y. Takane 2011. Projection Matrices, Generalized Inverse Matrices, and Singular Value Decomposition. New York, USA: Springer.

    Book  MATH  Google Scholar 

  • Yee, T. W. and N. D. Mitchell 1991. Generalized additive models in plant ecology. Journal of Vegetation Science 2(5):587–602.

    Article  Google Scholar 

  • Zhang, C. 2003. Calibrating the degrees of freedom for automatic data smoothing and effective curve checking. Journal of the American Statistical Association 98(463):609–628.

    Article  MATH  MathSciNet  Google Scholar 

  • Zuur, A. F. 2012. A Beginner’s Guide to Generalized Additive Models with R. Newburgh, UK: Highland Statistics Ltd.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Statistics, University of Auckland, Auckland, New Zealand

    Thomas W. Yee

Authors
  1. Thomas W. Yee
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Thomas Yee

About this chapter

Cite this chapter

Yee, T.W. (2015). LMs, GLMs and GAMs. In: Vector Generalized Linear and Additive Models. Springer Series in Statistics. Springer, New York, NY. https://doi.org/10.1007/978-1-4939-2818-7_2

Download citation

Publish with us

Policies and ethics

Search

Navigation