Generalized additive models with flexible response functions

Abstract

Common generalized linear models depend on several assumptions: (i) the specified linear predictor, (ii) the chosen response distribution that determines the likelihood and (iii) the response function that maps the linear predictor to the conditional expectation of the response. Generalized additive models (GAM) provide a convenient way to overcome the restriction to purely linear predictors. Therefore, the covariates may be included as flexible nonlinear or spatial functions to avoid potential bias arising from misspecification. Single index models, on the other hand, utilize flexible specifications of the response function and therefore avoid the deteriorating impact of a misspecified response function. However, such single index models are usually restricted to a linear predictor and aim to compensate for potential nonlinear structures only via the estimated response function. We will show that this is insufficient in many cases and present a solution by combining a flexible approach for response function estimation using monotonic P-splines with additive predictors as in GAMs. Our approach is based on maximum likelihood estimation and also allows us to provide confidence intervals of the estimated effects. To compare our approach with existing ones, we conduct extensive simulation studies and apply our approach on two empirical examples, namely the mortality rate in São Paulo due to respiratory diseases based on the Poisson distribution and credit scoring of a German bank with binary responses.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

References

  1. Bollaerts, K., Eilers, P.H., Mechelen, I.: Simple and multiple P-splines regression with shape constraints. Br. J. Math. Stat. Psychol. 59(2), 451–469 (2006)

    MathSciNet  Article  Google Scholar 

  2. Bühlmann, P., Hothorn, T.: Boosting algorithms: regularization, prediction and model fitting. Stat. Sci. 22(4), 477–505 (2007)

    MathSciNet  Article  MATH  Google Scholar 

  3. Cameron, A.C., Trivedi, P.K.: Microeconometrics: Methods and Applications. Cambridge University Press, New York (2005)

    Google Scholar 

  4. Carroll, R.J., Fan, J., Gijbels, I., Wand, M.P.: Generalized partially linear single-index models. J. Am. Stat. Assoc. 92(438), 477–489 (1997)

    MathSciNet  Article  MATH  Google Scholar 

  5. Czado, C., Santner, T.J.: The effect of link misspecification on binary regression inference. J. Stat. Plan. Inference 33(2), 213–231 (1992)

    MathSciNet  Article  MATH  Google Scholar 

  6. de Boor, C.: A Practical Guide to Splines. Springer, New York (1978)

    Google Scholar 

  7. Eilers, P.H.C., Marx, B.D.: Flexible smoothing with B-splines and penalties. Stat. Sci. 11(2), 89–121 (1996)

    MathSciNet  Article  MATH  Google Scholar 

  8. Fahrmeir, L., Hamerle, A., Tutz, G.: Multivariate Statistische Verfahren. Walter de Gruyter GmbH & Co KG, Berlin (1996)

    Google Scholar 

  9. Fahrmeir, L., Kneib, T., Lang, S., Marx, B.: Regression: Models, Methods and Applications. Springer Science & Business Media, New Yor (2013)

    Google Scholar 

  10. Friedman, J.H., Stuetzle, W.: Projection pursuit regression. J. Am. Stat. Assoc. 76(376), 817–823 (1981)

    MathSciNet  Article  Google Scholar 

  11. Hand, D.J.: Measuring classifier performance: a coherent alternative to the area under the ROC curve. Mach. Learn. 77(1), 103–123 (2009)

    Article  Google Scholar 

  12. Hastie, T., Tibshirani, R.: Generalized additive models. Stat. Sci. 1(3), 297–310 (1986)

    MathSciNet  Article  MATH  Google Scholar 

  13. Held, L., Sabanés Bové, D.: Applied Statistical Inference. Springer, Berlin (2014)

    Google Scholar 

  14. Ichimura, H.: Semiparametric least squares (SLS) and weighted SLS estimation of single-index models. J. Econom. 58(1), 71–120 (1993)

    MathSciNet  Article  MATH  Google Scholar 

  15. Jørgensen, B.: The delta algorithm and GLIM. Int. Stat. Rev./Rev. Int. Stat. 52(3), 283–300 (1984)

    MathSciNet  Article  MATH  Google Scholar 

  16. Klein, R.W., Spady, R.H.: An efficient semiparametric estimator for binary response models. Econometrica 61(2), 387–421 (1993)

    MathSciNet  Article  MATH  Google Scholar 

  17. Koenker, R., Yoon, J.: Parametric links for binary choice models: a Fisherian–Bayesian colloquy. J. Econom. 152(2), 120–130 (2009)

    MathSciNet  Article  MATH  Google Scholar 

  18. Leitenstorfer, F., Tutz, G.: Generalized monotonic regression based on B-splines with an application to air pollution data. Biostatistics 8(3), 654–673 (2007)

    Article  MATH  Google Scholar 

  19. Leitenstorfer, F., Tutz, G.: Estimation of single-index models based on boosting techniques. Stat. Model. 11(3), 203–217 (2011)

    MathSciNet  Article  Google Scholar 

  20. Li, Q., Racine, J.S.: Nonparametric Econometrics: Theory and Practice. Princeton University Press, Princeton (2007)

    Google Scholar 

  21. Lobo, J.M., Jiménez-Valverde, A., Real, R.: AUC: a misleading measure of the performance of predictive distribution models. Glob. Ecol. Biogeogr. 17(2), 145–151 (2008)

    Article  Google Scholar 

  22. Marra, G., Wood, S.N.: Practical variable selection for generalized additive models. Comput. Stat. Data Anal. 55(7), 2372–2387 (2011)

    MathSciNet  Article  MATH  Google Scholar 

  23. Marra, G., Wood, S.N.: Coverage properties of confidence intervals for generalized additive model components. Scand. J. Stat. 39(1), 53–74 (2012)

    MathSciNet  Article  MATH  Google Scholar 

  24. Marschner, I.C., et al.: glm2: fitting generalized linear models with convergence problems. R J. 3(2), 12–15 (2011)

    Google Scholar 

  25. Marx, B.D.: Varying-coefficient single-index signal regression. Chemom. Intell. Lab. Syst. 143, 111–121 (2015)

    Article  Google Scholar 

  26. McCullagh, P., Nelder, J.A.: Generalized Linear Models, 2nd edn. Chapman & Hall, London (1989)

    Google Scholar 

  27. Muggeo, V.M., Ferrara, G.: Fitting generalized linear models with unspecified link function: a P-spline approach. Comput. Stat. Data Anal. 52(5), 2529–2537 (2008)

    MathSciNet  Article  MATH  Google Scholar 

  28. Pya, N.: scam: Shape Constrained Additive Models. R package version 1.2-2 (2017)

  29. Pya, N., Wood, S.N.: Shape constrained additive models. Stat. Comput. 25(3), 543–559 (2015)

    MathSciNet  Article  MATH  Google Scholar 

  30. R Core Team: R: a language and environment for statistical computing. R Foundation for Statistical Computing, Vienna. https://www.R-project.org/ (2017)

  31. Tutz, G., Petry, S.: Nonparametric estimation of the link function including variable selection. Stat. Comput. 22(2), 545–561 (2012)

    MathSciNet  Article  MATH  Google Scholar 

  32. Tutz, G., Petry, S.: Generalized additive models with unknown link function including variable selection. J. Appl. Stat. 43(15), 2866–2885 (2016)

    MathSciNet  Article  Google Scholar 

  33. Wang, J.L., Xue, L., Zhu, L., Chong, Y.S., et al.: Estimation for a partial-linear single-index model. Ann. Stat. 38(1), 246–274 (2010)

    MathSciNet  MATH  Google Scholar 

  34. Weisberg, S., Welsh, A.: Adapting for the missing link. Ann. Stat. 22(4), 1674–1700 (1994)

    MathSciNet  Article  MATH  Google Scholar 

  35. Wood, S.: Monotonic smoothing splines fitted by cross validation. SIAM J. Sci. Comput. 15(5), 1126–1133 (1994)

    MathSciNet  Article  MATH  Google Scholar 

  36. Wood, S.N.: Generalized Additive Models: An Introduction with R, 2nd edn. CRC Press, Boca Raton (2017)

    Google Scholar 

  37. Wood, S.N., Fasiolo, M.: A generalized Fellner–Schall method for smoothing parameter optimization with application to tweedie location, scale and shape models. Biometrics (2017). https://doi.org/10.1111/biom.12666

  38. Yu, Y., Ruppert, D.: Penalized spline estimation for partially linear single-index models. J. Am. Stat. Assoc. 97(460), 1042–1054 (2002)

    MathSciNet  Article  MATH  Google Scholar 

  39. Yu, Y., Wu, C., Zhang, Y.: Penalised spline estimation for generalised partially linear single-index models. Stat. Comput. 27(2), 571–582 (2017)

    MathSciNet  Article  MATH  Google Scholar 

Download references

Acknowledgements

We thank Sebastian Petry for providing the code to the paper of Tutz and Petry (2016), such that we could compare our method with the boosting approach. We also want to thank two anonymous referees and an associate editor for their helpful comments improving this paper. Moreover, we acknowledge financial support by the German Research Foundation (DFG), Grant KN 922/4-2.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Elmar Spiegel.

Electronic supplementary material

Appendix

Appendix

Algorithm 1

(FlexGAM1)

The inner iteration is done until the convergence of \(\varvec{\gamma }^{(k)}\), meaning \(\frac{\left| \left| \varvec{\gamma }^{(k)} - \varvec{\gamma }^{(k-1)} \right| \right| }{\left| \left| \varvec{\gamma }^{(k)} \right| \right| }~<~\varepsilon _1\). Then, the outer iteration is repeated. The inner and outer loops are iterated until the coefficients of \(\varPsi ^{(m)}\) are constant, meaning \(\frac{\left| \left| \varvec{\nu }^{(m)} - \varvec{\nu }^{(m-1)} \right| \right| }{\left| \left| \varvec{\nu }^{(m)} \right| \right| } < \varepsilon _2\).

Algorithm 2

(FlexGAM2)

Again the inner iteration is done until the convergence of \(\varvec{\gamma }^{(k)}\). Then, the outer iteration is repeated. The inner and outer loops are iterated until the coefficients of \(\hat{h}^{(m)}\) are constant, meaning \(\frac{\left| \left| \varvec{\nu }^{(m)} - \varvec{\nu }^{(m-1)} \right| \right| }{\left| \left| \varvec{\nu }^{(m)} \right| \right| }~<~\varepsilon _2\).

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Spiegel, E., Kneib, T. & Otto-Sobotka, F. Generalized additive models with flexible response functions. Stat Comput 29, 123–138 (2019). https://doi.org/10.1007/s11222-017-9799-6

Download citation

Keywords

  • Flexible response function
  • Generalized additive model
  • Monotonic P-spline
  • Single index model