Straightforward intermediate rank tensor product smoothing in mixed models

Abstract

Tensor product smooths provide the natural way of representing smooth interaction terms in regression models because they are invariant to the units in which the covariates are measured, hence avoiding the need for arbitrary decisions about relative scaling of variables. They would also be the natural way to represent smooth interactions in mixed regression models, but for the fact that the tensor product constructions proposed to date are difficult or impossible to estimate using most standard mixed modelling software. This paper proposes a new approach to the construction of tensor product smooths, which allows the smooth to be written as the sum of some fixed effects and some sets of i.i.d. Gaussian random effects: no previously published construction achieves this. Because of the simplicity of this random effects structure, our construction is useable with almost any flexible mixed modelling software, allowing smooth interaction terms to be readily incorporated into any Generalized Linear Mixed Model. To achieve the computationally convenient separation of smoothing penalties, the construction differs from previous tensor product approaches in the penalties used to control smoothness, but the penalties have the advantage over several alternative approaches of being explicitly interpretable in terms of function shape. Like all tensor product smoothing methods, our approach builds up smooth functions of several variables from marginal smooths of lower dimension, but unlike much of the previous literature we treat the general case in which the marginal smooths can be any quadratically penalized basis expansion, and there can be any number of them. We also point out that the imposition of identifiability constraints on smoothers requires more care in the mixed model setting than it would in a simple additive model setting, and show how to deal with the issue. An interesting side effect of our construction is that an ANOVA-decomposition of the smooth can be read off from the estimates, although this is not our primary focus. We were motivated to undertake this work by applied problems in the analysis of abundance survey data, and two examples of this are presented.

This is a preview of subscription content, access via your institution.

References

  1. Bates, D., Maechler, M.: lme4: Linear mixed-effects models using S4 classes (2010). http://CRAN.R-project.org/package=lme4

  2. Belitz, C., Lang, S.: Simultaneous selection of variables and smoothing parameters in structured additive regression models. Comput. Stat. Data Anal. 53(1), 61–81 (2008)

    MathSciNet  MATH  Article  Google Scholar 

  3. Borchers, D.L., Buckland, S.T., Priede, I.G., Ahmadi, S.: Improving the precision of the daily egg production method using generalized additive models. Can. J. Fish. Aquat. Sci. 54, 2727–2742 (1997)

    Article  Google Scholar 

  4. Breslow, N.E., Clayton, D.G.: Approximate inference in generalized linear mixed models. J. Am. Stat. Assoc. 88, 9–25 (1993)

    MATH  Google Scholar 

  5. Davison, A.C.: Statistical Models. Cambridge University Press, Cambridge (2003)

    Google Scholar 

  6. Eilers, P.H.C.: Discussion of Verbyla, A.P., B.R. Cullis, M.G. Kenward and S.J. Welham (1999). The analysis of designed experiments and longitudinal data by using smoothing splines. J. R. Stat. Soc. C 48(3), 307–308 (1999)

    Google Scholar 

  7. Eilers, P.H.C., Marx, B.D.: Flexible smoothing with B-splines and penalties. Stat. Sci. 11(2), 89–102 (1996)

    MathSciNet  MATH  Article  Google Scholar 

  8. Eilers, P.H.C., Marx, B.D.: Multivariate calibration with temperature interaction using two-dimensional penalized signal regression. Chemom. Intell. Lab. Syst. 66, 159–174 (2003)

    Article  Google Scholar 

  9. Fahrmeir, L., Kneib, T., Lang, S.: Penalized structured additive regression for space time data: A Bayesian perspective. Stat. Sin. 14, 731–761 (2004)

    MathSciNet  MATH  Google Scholar 

  10. Gu, C.: Smoothing Spline ANOVA Models. Springer, Berlin (2002)

    Google Scholar 

  11. Gu, C., Kim, Y.-J.: Penalized likelihood regression: general formulation and efficient approximation. Can. J. Stat. 30(4), 619–628 (2002)

    MathSciNet  MATH  Article  Google Scholar 

  12. Harville, D.A.: Matrix Algebra from a Statisticians Perspective. Springer, Berlin (1997)

    Google Scholar 

  13. Hastie, T., Tibshirani, R.: Generalized additive models (with discussion). Stat. Sci. 1, 297–318 (1986)

    MathSciNet  Article  Google Scholar 

  14. Kim, Y.J., Gu, C.: Smoothing spline Gaussian regression: More scalable computation via efficient approximation. J. R. Stat. Soc. B 66, 337–356 (2004)

    MathSciNet  MATH  Article  Google Scholar 

  15. Kimeldorf, G., Wahba, G.: A correspondence between Bayesian estimation of stochastic processes and smoothing by splines. Ann. Math. Stat. 41, 495–502 (1970)

    MathSciNet  MATH  Article  Google Scholar 

  16. Lee, D.-J., Durbán, M.: P-spline ANOVA type interaction models for spatio-temporal smoothing. Stat. Model. 11(1), 49–69 (2011)

    MathSciNet  Article  Google Scholar 

  17. Lin, X., Zhang, D.: Inference in generalized additive mixed models using smoothing splines. J. R. Stat. Soc. B 61, 381–400 (1999)

    MathSciNet  MATH  Article  Google Scholar 

  18. Parker, R., Rice, J.: Discussion of Silverman (1985). J. R. Stat. Soc. B 47(1), 41–42 (1985)

    Google Scholar 

  19. Pinheiro, J.C., Bates, D.M.: Mixed-Effects Models in S and S-PLUS. Springer, Berlin (2000)

    Google Scholar 

  20. R Development Core Team: R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna (2010). ISBN 3-900051-07-0, www.R-project.org

    Google Scholar 

  21. Ruppert, D., Wand, M.P., Carroll, R.J.: Semiparametric Regression. Cambridge University Press, Cambridge (2003)

    Google Scholar 

  22. Silverman, B.W.: Some aspects of the spline smoothing approach to non-parametric regression curve fitting (with discussion). J. R. Stat. Soc. B 47, 1–53 (1985)

    MATH  Google Scholar 

  23. Verbyla, A.P., Cullis, B.R., Kenward, M.G., Welham, S.J.: The analysis of designed experiments and longitudinal data by using smoothing splines. J. R. Stat. Soc. C 48(3), 269–311 (1999)

    MATH  Article  Google Scholar 

  24. Wahba, G.: Spline bases, regularization and generalized cross validation for solving approximation problems with large quantities of noisy data. In: Cheney, E. (ed.) Approximation Theory III. Academic Press, London (1980)

    Google Scholar 

  25. Wahba, G.: Spline Models for Observational Data. SIAM, Philadelphia (1990)

    Google Scholar 

  26. Wood, S.N.: Stable and efficient multiple smoothing parameter estimation for generalized additive models. J. Am. Stat. Assoc. 99, 673–686 (2004)

    MATH  Article  Google Scholar 

  27. Wood, S.N.: Low-rank scale-invariant tensor product smooths for generalized additive mixed models. Biometrics 62(4), 1025–1036 (2006a)

    MathSciNet  MATH  Article  Google Scholar 

  28. Wood, S.N.: Generalized Additive Models: An Introduction with R. Taylor & Francis/CRC Press, London (2006b)

    Google Scholar 

  29. Wood, S.N.: Fast stable direct fitting and smoothness selection for generalized additive models. J. R. Stat. Soc. B 70(3), 495–518 (2008)

    MATH  Article  Google Scholar 

  30. Wood, S.N.: Fast stable restricted maximum likelihood and marginal likelihood estimation of semiparametric generalized linear models. J. R. Stat. Soc. B 73(1), 3–36 (2011)

    Article  Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Simon N. Wood.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Wood, S.N., Scheipl, F. & Faraway, J.J. Straightforward intermediate rank tensor product smoothing in mixed models. Stat Comput 23, 341–360 (2013). https://doi.org/10.1007/s11222-012-9314-z

Download citation

Keywords

  • Tensor product
  • Smooth
  • Smoothing spline ANOVA
  • Low rank
  • Space-time
  • Spatio-temporal
  • Identifiability constraint
  • Mixed model