, Volume 28, Issue 1, pp 1–39 | Cite as

Modular regression - a Lego system for building structured additive distributional regression models with tensor product interactions

  • Thomas KneibEmail author
  • Nadja Klein
  • Stefan Lang
  • Nikolaus Umlauf
Invited Paper


Semiparametric regression models offer considerable flexibility concerning the specification of additive regression predictors including effects as diverse as nonlinear effects of continuous covariates, spatial effects, random effects, or varying coefficients. Recently, such flexible model predictors have been combined with the possibility to go beyond pure mean-based analyses by specifying regression predictors on potentially all parameters of the response distribution in a distributional regression framework. In this paper, we discuss a generic concept for defining interaction effects in such semiparametric distributional regression models based on tensor products of main effects. These interactions can be assigned anisotropic penalties, i.e. different amounts of smoothness will be associated with the interacting covariates. We investigate identifiability and the decomposition of interactions into main effects and pure interaction effects (similar as in a smoothing spline analysis of variance) to facilitate a modular model building process. The decomposition is based on orthogonality in function spaces which allows for considerable flexibility in setting up the effect decomposition. Inference is based on Markov chain Monte Carlo simulations with iteratively weighted least squares proposals under constraints to ensure identifiability and effect decomposition. One important aspect is therefore to maintain sparse matrix structures of the tensor product also in identifiable, decomposed model formulations. The performance of modular regression is verified in a simulation on decomposed interaction surfaces of two continuous covariates and two applications on the construction of spatio-temporal interactions for the analysis of precipitation on the one hand and functional random effects for analysing house prices on the other hand.


Constrained sampling Distributional regression Functional random effects Markov chain Monte Carlo simulations Penalised splines Smoothing spline analysis of variance Space–time models Tensor product interactions 

Mathematics Subject Classification

62G08 62J12 62H11 



We thank the referees and the associate editor for many valuable comments that lead to a significant improvement in our paper upon the original submission. We are grateful to Jim Hodges for pointing us to the alternative representation of the tensor product precision matrix based on eigen decompositions. Financial support by the German Research Foundation (DFG), Grant KN 922/9-1 is gratefully acknowledged.


  1. Adler D, Kneib T, Lang S, Umlauf N, Zeileis A (2012) BayesXsrc: R Package Distribution of the BayesX C++ Sources. R package version 3.0-0. Accessed 29 Jan 2019
  2. Belitz C, Brezger A, Klein N, Kneib T, Lang S, Umlauf N (2015) BayesX—Software for Bayesian inference in structured additive regression models. Version 3.0.2. Accessed 29 Jan 2019
  3. Besag J, Higdon D (1999) Bayesian analysis of agricultural field experiments. J R Stat Soc Ser B (Methodol) 61:691–746MathSciNetCrossRefzbMATHGoogle Scholar
  4. Brezger A, Lang S (2006) Generalized structured additive regression based on Bayesian P-splines. Comput Stat Data Anal 50:967–991MathSciNetCrossRefzbMATHGoogle Scholar
  5. Fahrmeir L, Kneib T (2011) Bayesian smoothing and regression for longitudinal, spatial and event history data. Oxford University Press, New YorkCrossRefzbMATHGoogle Scholar
  6. Fahrmeir L, Kneib T, Lang S (2004) Penalized structured additive regression for space–time data: a Bayesian perspective. Stat Sin 14:731–761MathSciNetzbMATHGoogle Scholar
  7. Fahrmeir L, Kneib T, Lang S, Marx B (2013) Regression—models, methods and applications. Springer, BerlinzbMATHGoogle Scholar
  8. Gamerman D (1997) Sampling from the posterior distribution in generalized linear mixed models. Stat Comput 7:57–68CrossRefGoogle Scholar
  9. Gelfand AE, Sahu SK (1999) Identifiability, improper priors, and Gibbs sampling for generalized linear models. J Am Stat Assoc 94:247–253MathSciNetCrossRefzbMATHGoogle Scholar
  10. Gelman A (2006) Prior distributions for variance parameters in hierarchichal models. Bayesian Anal 1:515–533MathSciNetCrossRefzbMATHGoogle Scholar
  11. Goicoa T, Adin A, Ugarte MD, Hodges JS (2018) In spatio-temporal disease mapping models, identifiability constraints affet PQL and INLA results. Stoch Environ Res Risk Assess 32:749–770CrossRefGoogle Scholar
  12. Gu C (2002) Smoothing spline ANOVA models. Springer, New YorkCrossRefzbMATHGoogle Scholar
  13. Hodges J S (2013) Richly parameterized linear models: additive, time series, and spatial models using random effects. Chapman & Hall/CRC, New York/Boca RatonzbMATHGoogle Scholar
  14. Hughes J, Haran M (2013) Dimension reduction and alleviation of confounding for spatial generalized linear mixed models. J R Stat Soc Ser B (Stat Methodol) 75:139–159MathSciNetCrossRefGoogle Scholar
  15. Klein N (2018) sdPrior: scale-dependent hyperpriors in structured additive distributional regression. R package version 1.0Google Scholar
  16. Klein N, Kneib T (2016a) Scale-dependent priors for variance parameters in structured additive distributional regression. Bayesian Anal 11:1071–1106MathSciNetCrossRefzbMATHGoogle Scholar
  17. Klein N, Kneib T (2016b) Simultaneous inference in structured additive conditional copula regression models: a unifying Bayesian approach. Stat Comput 26:841–860MathSciNetCrossRefzbMATHGoogle Scholar
  18. Klein N, Kneib T, Klasen S, Lang S (2015a) Bayesian structured additive distributional regression for multivariate responses. J R Stat Soc Ser C (Appl Stat) 64:569–591MathSciNetCrossRefGoogle Scholar
  19. Klein N, Kneib T, Lang S (2015b) Bayesian generalized additive models for location, scale and shape for zero-inflated and overdispersed count data. J Am Stat Assoc 110:405–419MathSciNetCrossRefzbMATHGoogle Scholar
  20. Klein N, Kneib T, Lang S, Sohn A (2015c) Bayesian structured additive distributional regression with with an application to regional income inequality in Germany. Ann Appl Stat 9:1024–1052MathSciNetCrossRefzbMATHGoogle Scholar
  21. Knorr-Held L (2000) Bayesian modelling of inseparable space-time variation in disease risk. Stat Med 19:2555–2567CrossRefGoogle Scholar
  22. Lang S, Brezger A (2004) Bayesian P-splines. J Comput Graph Stat 13:183–212MathSciNetCrossRefzbMATHGoogle Scholar
  23. Lang S, Umlauf N, Wechselberger P, Harttgen K, Kneib T (2014) Multilevel structured additive regression. Stat Comput 24:223–238MathSciNetCrossRefzbMATHGoogle Scholar
  24. Lavine M, Hodges JS (2012) On rigorous specification of icar models. Am Stat 66:42–49MathSciNetCrossRefGoogle Scholar
  25. Lee D-J, Durbán M (2011) P-spline ANOVA type interaction models for spatio temporal smoothing. Stat Model 11:46–69MathSciNetCrossRefGoogle Scholar
  26. Marí-Dell’Olmo M, Martinez-Beneito MA, Mercè Gotsens M, Palència L (2014) A smoothed anova model for multivariate ecological regression. Stoch Environ Res Risk Assess 28:695–706CrossRefGoogle Scholar
  27. Marra G, Radice R (2017) Bivariate copula additive models for location, scale and shape. Comput Stat Data Anal 112:99–113MathSciNetCrossRefzbMATHGoogle Scholar
  28. Marra G, Wood SN (2012) Coverage properties of confidence intervals for generalized additive model components. Scand J Stat 39:53–74MathSciNetCrossRefzbMATHGoogle Scholar
  29. Paciorek CJ (2007) Bayesian smoothing with Gaussian processes using Fourier basis functions in the spectralGP package. J Stat Softw 19:1–38CrossRefGoogle Scholar
  30. R Core Team (2017) R: a Language and Environment for Statistical Computing, R Foundation for Statistical Computing, Vienna, Austria. ISBN 3-900051-07-0. Accessed 29 Jan 2019
  31. Reich BJ, Hodges JS, Zadnik V (2006) Effects of residual smoothing on the posterior of the fixed effects in disease-mapping models. Biometrics 62:1197–1206MathSciNetCrossRefzbMATHGoogle Scholar
  32. Rigby RA, Stasinopoulos DM (2005) Generalized additive models for location, scale and shape (with discussion). J R Stat Soc Ser C (Appl Stat) 54:507–554CrossRefzbMATHGoogle Scholar
  33. Rodriguez Alvarez MX, Lee D-J, Kneib T, Durban M, Eilers P (2015) Fast smoothing parameter separation in multidimensional generalized P-splines: the SAP algorithm. Stat Comput 25:941–957MathSciNetCrossRefzbMATHGoogle Scholar
  34. Rue H, Held L (2005) Gaussian Markov random fields. Chapman & Hall/CRC, New York/Boca RatonCrossRefzbMATHGoogle Scholar
  35. Ruppert D, Wand MP, Carroll RJ (2003) Semiparametric regression. Cambridge University Press, CambridgeCrossRefzbMATHGoogle Scholar
  36. Simpson D, Rue H, Martins TG, Riebler A, Sørbye SH (2017) Penalising model component complexity: a principled, practical approach to constructing prior. Stat Sci 32(1):1–28MathSciNetCrossRefzbMATHGoogle Scholar
  37. Stauffer R, Mayr GJ, Messner JW, Umlauf N, Zeileis A (2016) Spatio-temporal precipitation climatology over complex terrain using a censored additive regression model. Int J Climatol 15:3264Google Scholar
  38. Stauffer R, Umlauf N, Messner JW, Mayr GJ, Zeileis A (2017) Ensemble postprocessing of daily precipitation sums over complex terrain using censored high-resolution standardized anomalies. Mon Weather Rev 145(3):955–969CrossRefGoogle Scholar
  39. Ugarte MD, Adin A, Goicoa T (2017) One-dimensional, two-dimensional, and three-dimensional B-splines to specify space-time interations in bayesian disease mapping: model fitting and model identifiability. Spat Stat 22:451–468MathSciNetCrossRefGoogle Scholar
  40. Umlauf N, Klein N, Zeileis A, Köhler M (2018) bamlss : Bayesian additive models for location scale and shape (and Beyond). R package version 1.0-0. Accessed 29 Jan 2019
  41. Wahba G, Wang Y, Gu C, Klein R, Klein B (1995) Smoothing spline anova for exponential families, with application to the Wisconsin epidemiological study of diabetic retinopathy. Ann Stat 23:1865–1895MathSciNetCrossRefzbMATHGoogle Scholar
  42. Wood SN (2006) Low-rank scale-invariant tensor product smooths for generalized additive mixed models. Biometrics 62:1025–1036MathSciNetCrossRefzbMATHGoogle Scholar
  43. Wood SN (2008) Fast stable direct fitting and smoothness selection for generalized additive models. J R Stat Soc Ser B (Stat Methodol) 70:495–518MathSciNetCrossRefzbMATHGoogle Scholar
  44. Wood S (2015) mgcv: Mixed GAM computation vehicle with GCV/AIC/REML smoothness estimations. R package version 1.8-5Google Scholar
  45. Wood SN (2017) Generalized additive models : an introduction with R. Chapman & Hall/CRC, New York/Boca RatonCrossRefzbMATHGoogle Scholar
  46. Wood SN, Scheipl F, Faraway JJ (2013) Straightforward intermediate rank tensor product smoothing in mixed models. Stat Comput 23:341–360MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© Sociedad de Estadística e Investigación Operativa 2019

Authors and Affiliations

  • Thomas Kneib
    • 1
    Email author
  • Nadja Klein
    • 2
  • Stefan Lang
    • 3
  • Nikolaus Umlauf
    • 3
  1. 1.Chair of StatisticsGeorg-August-Universität GöttingenGöttingenGermany
  2. 2.Humboldt Universität zu BerlinBerlinGermany
  3. 3.Department of StatisticsUniversität InnsbruckInnsbruckAustria

Personalised recommendations