Statistics and Computing

, Volume 23, Issue 3, pp 341–360 | Cite as

Straightforward intermediate rank tensor product smoothing in mixed models

  • Simon N. Wood
  • Fabian Scheipl
  • Julian J. Faraway


Tensor product smooths provide the natural way of representing smooth interaction terms in regression models because they are invariant to the units in which the covariates are measured, hence avoiding the need for arbitrary decisions about relative scaling of variables. They would also be the natural way to represent smooth interactions in mixed regression models, but for the fact that the tensor product constructions proposed to date are difficult or impossible to estimate using most standard mixed modelling software. This paper proposes a new approach to the construction of tensor product smooths, which allows the smooth to be written as the sum of some fixed effects and some sets of i.i.d. Gaussian random effects: no previously published construction achieves this. Because of the simplicity of this random effects structure, our construction is useable with almost any flexible mixed modelling software, allowing smooth interaction terms to be readily incorporated into any Generalized Linear Mixed Model. To achieve the computationally convenient separation of smoothing penalties, the construction differs from previous tensor product approaches in the penalties used to control smoothness, but the penalties have the advantage over several alternative approaches of being explicitly interpretable in terms of function shape. Like all tensor product smoothing methods, our approach builds up smooth functions of several variables from marginal smooths of lower dimension, but unlike much of the previous literature we treat the general case in which the marginal smooths can be any quadratically penalized basis expansion, and there can be any number of them. We also point out that the imposition of identifiability constraints on smoothers requires more care in the mixed model setting than it would in a simple additive model setting, and show how to deal with the issue. An interesting side effect of our construction is that an ANOVA-decomposition of the smooth can be read off from the estimates, although this is not our primary focus. We were motivated to undertake this work by applied problems in the analysis of abundance survey data, and two examples of this are presented.


Tensor product Smooth Smoothing spline ANOVA Low rank Space-time Spatio-temporal Identifiability constraint Mixed model 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Bates, D., Maechler, M.: lme4: Linear mixed-effects models using S4 classes (2010).
  2. Belitz, C., Lang, S.: Simultaneous selection of variables and smoothing parameters in structured additive regression models. Comput. Stat. Data Anal. 53(1), 61–81 (2008) MathSciNetzbMATHCrossRefGoogle Scholar
  3. Borchers, D.L., Buckland, S.T., Priede, I.G., Ahmadi, S.: Improving the precision of the daily egg production method using generalized additive models. Can. J. Fish. Aquat. Sci. 54, 2727–2742 (1997) CrossRefGoogle Scholar
  4. Breslow, N.E., Clayton, D.G.: Approximate inference in generalized linear mixed models. J. Am. Stat. Assoc. 88, 9–25 (1993) zbMATHGoogle Scholar
  5. Davison, A.C.: Statistical Models. Cambridge University Press, Cambridge (2003) zbMATHCrossRefGoogle Scholar
  6. Eilers, P.H.C.: Discussion of Verbyla, A.P., B.R. Cullis, M.G. Kenward and S.J. Welham (1999). The analysis of designed experiments and longitudinal data by using smoothing splines. J. R. Stat. Soc. C 48(3), 307–308 (1999) Google Scholar
  7. Eilers, P.H.C., Marx, B.D.: Flexible smoothing with B-splines and penalties. Stat. Sci. 11(2), 89–102 (1996) MathSciNetzbMATHCrossRefGoogle Scholar
  8. Eilers, P.H.C., Marx, B.D.: Multivariate calibration with temperature interaction using two-dimensional penalized signal regression. Chemom. Intell. Lab. Syst. 66, 159–174 (2003) CrossRefGoogle Scholar
  9. Fahrmeir, L., Kneib, T., Lang, S.: Penalized structured additive regression for space time data: A Bayesian perspective. Stat. Sin. 14, 731–761 (2004) MathSciNetzbMATHGoogle Scholar
  10. Gu, C.: Smoothing Spline ANOVA Models. Springer, Berlin (2002) zbMATHCrossRefGoogle Scholar
  11. Gu, C., Kim, Y.-J.: Penalized likelihood regression: general formulation and efficient approximation. Can. J. Stat. 30(4), 619–628 (2002) MathSciNetzbMATHCrossRefGoogle Scholar
  12. Harville, D.A.: Matrix Algebra from a Statisticians Perspective. Springer, Berlin (1997) CrossRefGoogle Scholar
  13. Hastie, T., Tibshirani, R.: Generalized additive models (with discussion). Stat. Sci. 1, 297–318 (1986) MathSciNetCrossRefGoogle Scholar
  14. Kim, Y.J., Gu, C.: Smoothing spline Gaussian regression: More scalable computation via efficient approximation. J. R. Stat. Soc. B 66, 337–356 (2004) MathSciNetzbMATHCrossRefGoogle Scholar
  15. Kimeldorf, G., Wahba, G.: A correspondence between Bayesian estimation of stochastic processes and smoothing by splines. Ann. Math. Stat. 41, 495–502 (1970) MathSciNetzbMATHCrossRefGoogle Scholar
  16. Lee, D.-J., Durbán, M.: P-spline ANOVA type interaction models for spatio-temporal smoothing. Stat. Model. 11(1), 49–69 (2011) MathSciNetCrossRefGoogle Scholar
  17. Lin, X., Zhang, D.: Inference in generalized additive mixed models using smoothing splines. J. R. Stat. Soc. B 61, 381–400 (1999) MathSciNetzbMATHCrossRefGoogle Scholar
  18. Parker, R., Rice, J.: Discussion of Silverman (1985). J. R. Stat. Soc. B 47(1), 41–42 (1985) Google Scholar
  19. Pinheiro, J.C., Bates, D.M.: Mixed-Effects Models in S and S-PLUS. Springer, Berlin (2000) zbMATHCrossRefGoogle Scholar
  20. R Development Core Team: R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna (2010). ISBN 3-900051-07-0, Google Scholar
  21. Ruppert, D., Wand, M.P., Carroll, R.J.: Semiparametric Regression. Cambridge University Press, Cambridge (2003) zbMATHCrossRefGoogle Scholar
  22. Silverman, B.W.: Some aspects of the spline smoothing approach to non-parametric regression curve fitting (with discussion). J. R. Stat. Soc. B 47, 1–53 (1985) zbMATHGoogle Scholar
  23. Verbyla, A.P., Cullis, B.R., Kenward, M.G., Welham, S.J.: The analysis of designed experiments and longitudinal data by using smoothing splines. J. R. Stat. Soc. C 48(3), 269–311 (1999) zbMATHCrossRefGoogle Scholar
  24. Wahba, G.: Spline bases, regularization and generalized cross validation for solving approximation problems with large quantities of noisy data. In: Cheney, E. (ed.) Approximation Theory III. Academic Press, London (1980) Google Scholar
  25. Wahba, G.: Spline Models for Observational Data. SIAM, Philadelphia (1990) zbMATHCrossRefGoogle Scholar
  26. Wood, S.N.: Stable and efficient multiple smoothing parameter estimation for generalized additive models. J. Am. Stat. Assoc. 99, 673–686 (2004) zbMATHCrossRefGoogle Scholar
  27. Wood, S.N.: Low-rank scale-invariant tensor product smooths for generalized additive mixed models. Biometrics 62(4), 1025–1036 (2006a) MathSciNetzbMATHCrossRefGoogle Scholar
  28. Wood, S.N.: Generalized Additive Models: An Introduction with R. Taylor & Francis/CRC Press, London (2006b) zbMATHGoogle Scholar
  29. Wood, S.N.: Fast stable direct fitting and smoothness selection for generalized additive models. J. R. Stat. Soc. B 70(3), 495–518 (2008) zbMATHCrossRefGoogle Scholar
  30. Wood, S.N.: Fast stable restricted maximum likelihood and marginal likelihood estimation of semiparametric generalized linear models. J. R. Stat. Soc. B 73(1), 3–36 (2011) CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2012

Authors and Affiliations

  • Simon N. Wood
    • 1
  • Fabian Scheipl
    • 2
  • Julian J. Faraway
    • 1
  1. 1.Mathematical SciencesUniversity of BathBathUK
  2. 2.Department of StatisticsLMUMünchenGermany

Personalised recommendations