Abstract
Generalized additive models are an effective regression tool, popular in the statistics literature, that provides an automatic extension of traditional linear models to nonlinear systems. We present a distributed algorithm for fitting generalized additive models, based on the alternating direction method of multipliers (ADMM). In our algorithm the component functions of the model are fit independently, in parallel; a simple iteration yields convergence to the optimal generalized additive model. This is in contrast to the traditional approach of backfitting, where the component functions are fit sequentially. We illustrate the method on different classes of problems such as generalized additive, logistic, and piecewise constant models, with various types of regularization, including those that promote smoothness and sparsity.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Becker S, Bobin J, Candès EJ (2011a) NESTA: a fast and accurate first-order method for sparse recovery. SIAM J Imaging Sci 4(1):1–39
Becker S, Candès EJ, Grant M (2011b) Templates for convex cone problems with applications to sparse signal recovery. Math Program Comput 3(3)
Bottomley C, Van Belle V, Pexsters A, Papageorghiou A, Mukri F, Kirk E, Van Huffel S, Timmerman D, Bourne T (2011) A model and scoring system to predict outcome of intrauterine pregnancies of uncertain viability. Ultrasound Obstet Gynecol 37(5):588–595
Boyd S, Vandenberghe L (2004) Convex optimization. Cambridge University Press, Cambridge
Boyd S, Parikh N, Chu E, Peleato B, Eckstein J (2010) Distributed optimization and statistical learning via the alternating direction method of multipliers. Found Trends Mach Learn 3(1):1–122
Douglas J, Rachford HH (1956) On the numerical solution of heat conduction problems in two and three space variables. Trans Am Math Soc 82:421–439
Everett H (1963) Generalized Lagrange multiplier method for solving problems of optimum allocation of resources. Oper Res 11(3):399–417
Friedman JH, Hastie T, Tibshirani R (2010) Regularization paths for generalized linear models via coordinate descent. J Stat Softw 33(1):1–22
Guisan A, Edwards T, Hastie T (2002) Generalized linear and generalized additive models in studies of species distributions: setting the scene. Ecol Model 157:89–100
Hastie T, Tibshirani R (1986) Generalized additive models. Stat Sci 1(3):297–318
Hastie T, Tibshirani R (1995) Generalized additive models for medical research. In: Encyclopedia for biostatistics, vol 4, pp 187–196
Hastie T, Tibshirani R, Friedman J (2009) The elements of statistical learning: data mining, inference and prediction. Springer, Berlin
Hegland M, McIntosh I, Turlach B (1999) A parallel solver for generalised additive models. Comput Stat Data Anal 31:377–396
Kim S-J, Koh K, Lustig M, Boyd S, Gorinevsky D (2007) An interior-point method for large-scale ℓ 1-regularized least squares. IEEE J Sel Top Signal Process 1(4):606–617
Reinsch CH (1967) Smoothing by spline functions. Numer Math 10:177–183
Tibshirani R (1996) Regression shrinkage and selection via the lasso. J R Stat Soc, Ser B, Stat Methodol 58(1):267–288
Yuan M, Lin Y (2006) Model selection and estimation in regression with grouped variables. J R Stat Soc, Ser B, Stat Methodol 68:49–67
Zhao P, Rocha G, Yu B (2009) The composite absolute penalties family for grouped and hierarchical variable selection. Ann Stat 37(6A):3468–3497
Author information
Authors and Affiliations
Corresponding author
Additional information
This material is based upon work supported by AFOSR grant FA9550-09-1-0704, by DARPA XDATA FA8750-12-2-0306, and by NASA grant NNX07AEIIA.
Rights and permissions
About this article
Cite this article
Chu, E., Keshavarz, A. & Boyd, S. A distributed algorithm for fitting generalized additive models. Optim Eng 14, 213–224 (2013). https://doi.org/10.1007/s11081-013-9215-9
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11081-013-9215-9