Statistics and Computing

, Volume 29, Issue 1, pp 123–138 | Cite as

Generalized additive models with flexible response functions

  • Elmar SpiegelEmail author
  • Thomas Kneib
  • Fabian Otto-Sobotka


Common generalized linear models depend on several assumptions: (i) the specified linear predictor, (ii) the chosen response distribution that determines the likelihood and (iii) the response function that maps the linear predictor to the conditional expectation of the response. Generalized additive models (GAM) provide a convenient way to overcome the restriction to purely linear predictors. Therefore, the covariates may be included as flexible nonlinear or spatial functions to avoid potential bias arising from misspecification. Single index models, on the other hand, utilize flexible specifications of the response function and therefore avoid the deteriorating impact of a misspecified response function. However, such single index models are usually restricted to a linear predictor and aim to compensate for potential nonlinear structures only via the estimated response function. We will show that this is insufficient in many cases and present a solution by combining a flexible approach for response function estimation using monotonic P-splines with additive predictors as in GAMs. Our approach is based on maximum likelihood estimation and also allows us to provide confidence intervals of the estimated effects. To compare our approach with existing ones, we conduct extensive simulation studies and apply our approach on two empirical examples, namely the mortality rate in São Paulo due to respiratory diseases based on the Poisson distribution and credit scoring of a German bank with binary responses.


Flexible response function Generalized additive model Monotonic P-spline Single index model 



We thank Sebastian Petry for providing the code to the paper of Tutz and Petry (2016), such that we could compare our method with the boosting approach. We also want to thank two anonymous referees and an associate editor for their helpful comments improving this paper. Moreover, we acknowledge financial support by the German Research Foundation (DFG), Grant KN 922/4-2.

Supplementary material

11222_2017_9799_MOESM1_ESM.pdf (37.4 mb)
Supplementary material 1 (pdf 38310 KB)
11222_2017_9799_MOESM2_ESM.r (17 kb)
Supplementary material 2 (R 16 KB)
Supplementary material 3 (GZ 21 KB)


  1. Bollaerts, K., Eilers, P.H., Mechelen, I.: Simple and multiple P-splines regression with shape constraints. Br. J. Math. Stat. Psychol. 59(2), 451–469 (2006)MathSciNetCrossRefGoogle Scholar
  2. Bühlmann, P., Hothorn, T.: Boosting algorithms: regularization, prediction and model fitting. Stat. Sci. 22(4), 477–505 (2007)MathSciNetCrossRefzbMATHGoogle Scholar
  3. Cameron, A.C., Trivedi, P.K.: Microeconometrics: Methods and Applications. Cambridge University Press, New York (2005)CrossRefzbMATHGoogle Scholar
  4. Carroll, R.J., Fan, J., Gijbels, I., Wand, M.P.: Generalized partially linear single-index models. J. Am. Stat. Assoc. 92(438), 477–489 (1997)MathSciNetCrossRefzbMATHGoogle Scholar
  5. Czado, C., Santner, T.J.: The effect of link misspecification on binary regression inference. J. Stat. Plan. Inference 33(2), 213–231 (1992)MathSciNetCrossRefzbMATHGoogle Scholar
  6. de Boor, C.: A Practical Guide to Splines. Springer, New York (1978)CrossRefzbMATHGoogle Scholar
  7. Eilers, P.H.C., Marx, B.D.: Flexible smoothing with B-splines and penalties. Stat. Sci. 11(2), 89–121 (1996)MathSciNetCrossRefzbMATHGoogle Scholar
  8. Fahrmeir, L., Hamerle, A., Tutz, G.: Multivariate Statistische Verfahren. Walter de Gruyter GmbH & Co KG, Berlin (1996)zbMATHGoogle Scholar
  9. Fahrmeir, L., Kneib, T., Lang, S., Marx, B.: Regression: Models, Methods and Applications. Springer Science & Business Media, New Yor (2013)CrossRefzbMATHGoogle Scholar
  10. Friedman, J.H., Stuetzle, W.: Projection pursuit regression. J. Am. Stat. Assoc. 76(376), 817–823 (1981)MathSciNetCrossRefGoogle Scholar
  11. Hand, D.J.: Measuring classifier performance: a coherent alternative to the area under the ROC curve. Mach. Learn. 77(1), 103–123 (2009)CrossRefGoogle Scholar
  12. Hastie, T., Tibshirani, R.: Generalized additive models. Stat. Sci. 1(3), 297–310 (1986)MathSciNetCrossRefzbMATHGoogle Scholar
  13. Held, L., Sabanés Bové, D.: Applied Statistical Inference. Springer, Berlin (2014)CrossRefzbMATHGoogle Scholar
  14. Ichimura, H.: Semiparametric least squares (SLS) and weighted SLS estimation of single-index models. J. Econom. 58(1), 71–120 (1993)MathSciNetCrossRefzbMATHGoogle Scholar
  15. Jørgensen, B.: The delta algorithm and GLIM. Int. Stat. Rev./Rev. Int. Stat. 52(3), 283–300 (1984)MathSciNetCrossRefzbMATHGoogle Scholar
  16. Klein, R.W., Spady, R.H.: An efficient semiparametric estimator for binary response models. Econometrica 61(2), 387–421 (1993)MathSciNetCrossRefzbMATHGoogle Scholar
  17. Koenker, R., Yoon, J.: Parametric links for binary choice models: a Fisherian–Bayesian colloquy. J. Econom. 152(2), 120–130 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  18. Leitenstorfer, F., Tutz, G.: Generalized monotonic regression based on B-splines with an application to air pollution data. Biostatistics 8(3), 654–673 (2007)CrossRefzbMATHGoogle Scholar
  19. Leitenstorfer, F., Tutz, G.: Estimation of single-index models based on boosting techniques. Stat. Model. 11(3), 203–217 (2011)MathSciNetCrossRefGoogle Scholar
  20. Li, Q., Racine, J.S.: Nonparametric Econometrics: Theory and Practice. Princeton University Press, Princeton (2007)zbMATHGoogle Scholar
  21. Lobo, J.M., Jiménez-Valverde, A., Real, R.: AUC: a misleading measure of the performance of predictive distribution models. Glob. Ecol. Biogeogr. 17(2), 145–151 (2008)CrossRefGoogle Scholar
  22. Marra, G., Wood, S.N.: Practical variable selection for generalized additive models. Comput. Stat. Data Anal. 55(7), 2372–2387 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
  23. Marra, G., Wood, S.N.: Coverage properties of confidence intervals for generalized additive model components. Scand. J. Stat. 39(1), 53–74 (2012)MathSciNetCrossRefzbMATHGoogle Scholar
  24. Marschner, I.C., et al.: glm2: fitting generalized linear models with convergence problems. R J. 3(2), 12–15 (2011)Google Scholar
  25. Marx, B.D.: Varying-coefficient single-index signal regression. Chemom. Intell. Lab. Syst. 143, 111–121 (2015)CrossRefGoogle Scholar
  26. McCullagh, P., Nelder, J.A.: Generalized Linear Models, 2nd edn. Chapman & Hall, London (1989)CrossRefzbMATHGoogle Scholar
  27. Muggeo, V.M., Ferrara, G.: Fitting generalized linear models with unspecified link function: a P-spline approach. Comput. Stat. Data Anal. 52(5), 2529–2537 (2008)MathSciNetCrossRefzbMATHGoogle Scholar
  28. Pya, N.: scam: Shape Constrained Additive Models. R package version 1.2-2 (2017)Google Scholar
  29. Pya, N., Wood, S.N.: Shape constrained additive models. Stat. Comput. 25(3), 543–559 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
  30. R Core Team: R: a language and environment for statistical computing. R Foundation for Statistical Computing, Vienna. (2017)
  31. Tutz, G., Petry, S.: Nonparametric estimation of the link function including variable selection. Stat. Comput. 22(2), 545–561 (2012)MathSciNetCrossRefzbMATHGoogle Scholar
  32. Tutz, G., Petry, S.: Generalized additive models with unknown link function including variable selection. J. Appl. Stat. 43(15), 2866–2885 (2016)MathSciNetCrossRefGoogle Scholar
  33. Wang, J.L., Xue, L., Zhu, L., Chong, Y.S., et al.: Estimation for a partial-linear single-index model. Ann. Stat. 38(1), 246–274 (2010)MathSciNetzbMATHGoogle Scholar
  34. Weisberg, S., Welsh, A.: Adapting for the missing link. Ann. Stat. 22(4), 1674–1700 (1994)MathSciNetCrossRefzbMATHGoogle Scholar
  35. Wood, S.: Monotonic smoothing splines fitted by cross validation. SIAM J. Sci. Comput. 15(5), 1126–1133 (1994)MathSciNetCrossRefzbMATHGoogle Scholar
  36. Wood, S.N.: Generalized Additive Models: An Introduction with R, 2nd edn. CRC Press, Boca Raton (2017)CrossRefzbMATHGoogle Scholar
  37. Wood, S.N., Fasiolo, M.: A generalized Fellner–Schall method for smoothing parameter optimization with application to tweedie location, scale and shape models. Biometrics (2017).
  38. Yu, Y., Ruppert, D.: Penalized spline estimation for partially linear single-index models. J. Am. Stat. Assoc. 97(460), 1042–1054 (2002)MathSciNetCrossRefzbMATHGoogle Scholar
  39. Yu, Y., Wu, C., Zhang, Y.: Penalised spline estimation for generalised partially linear single-index models. Stat. Comput. 27(2), 571–582 (2017)MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2017

Authors and Affiliations

  1. 1.University of GoettingenGoettingenGermany
  2. 2.Carl von Ossietzky University OldenburgOldenburgGermany

Personalised recommendations