Statistics and Computing

, Volume 1, Issue 1, pp 47–62 | Cite as

Computational methods for local regression

  • William S. Cleveland
  • E. Grosse


Local regression is a nonparametric method in which the regression surface is estimated by fitting parametric functions locally in the space of the predictors using weighted least squares in a moving fashion similar to the way that a time series is smoothed by moving averages. Three computational methods for local regression are presented. First, fast surface fitting and evaluation is achieved by building ak-d tree in the space of the predictors, evaluating the surface at the corners of the tree, and then interpolating elsewhere by blending functions. Second, surfaces are made conditionally parametric in any proper subset of the predictors by a simple alteration of the weighting scheme. Third degree-of-freedom quantities that would be extremely expensive to compute exactly are approximated, not by numerical methods, but through a statistical model that predicts the quantities from the trace of the hat matrix, which can be computed easily.


Nonparametric regression loess k-d tree blending function semi-parametric model 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Art, D., Gnanadesikan, R. and Kettenring, J. R. (1982) Databased metrics for hierarchical cluster analysis.Utilitas Mathematica,21A, 75–99.Google Scholar
  2. Barnhill, R. E. (1977) Representation and approximation of surfaces.Mathematical Software III, Academic Press, New York.Google Scholar
  3. Bates, D. M., Lindstrom, M. J., Wahba, G. and Yandell, B. S. (1987) GCVPACK-routines for generalized cross-validation.Comm. Stat.-Simula.,16, 263–297.Google Scholar
  4. Birkhoff, G., Cavendish, J. C. and Gordon, W. J. (1974) Multivariate approximation by locally blended univariate interpolants.Proc. National Academy of Sciences USA,71, 3423–3425.Google Scholar
  5. Breiman, L. (to appear) The II method for estimating multivariate functions from noisy data.Technometrics.Google Scholar
  6. Brinkman, N. D. (1981) Ethanol fuel—a single-cylinder engine study of efficiency and exhaust emissions.SAE Transactions,90, no. 810345, 1410–1424.Google Scholar
  7. Buta, R. (1987) The structure and dynamics of ringed galaxies, III: surface photometry and kinematics of the ringed nonbarred spiral NGC7531.The Astrophysical Journal, Supplement Ser.,64, 1–37.Google Scholar
  8. Cavendish, J. C. (1975) Local mesh refinement using rectangular blended finite elements.J. Comp. Physics,19, 211–228.Google Scholar
  9. Chan, T. F.-C. (1982) Algorithm 581: An improved algorithm for computing the singular value decomposition.ACM Transactions on Mathematical Software,8, 84–88.Google Scholar
  10. Cleveland, W. S. (1979) Robust locally-weighted regression and smoothing scatterplots.J. Amer. Statist. Assoc.,74, 829–836.Google Scholar
  11. Cleveland, W. S. and Grosse, E. (in press)Fitting Functions to Data, Wadsworth, Pacific Grove, Calif.Google Scholar
  12. Cleveland, W. S., Devlin, S. J. and Grosse, E. (1988) Regression by local fitting: methods, properties, and computational algorithms.Journal of Econometrics,37, 87–114.Google Scholar
  13. Cleveland, W. S., Grosse, and E. Shyu, W. M. (1991) Local regression models, inStatistical Models in S, Chambers J. M. and Hastie, T. (eds), Wadsworth, Pacific Grove, Calif.Google Scholar
  14. Dongarra, J. J. and Grosse, E. (1987) Distribution of mathematical software via electronic mail.Communications of the ACM,30, 403–407.Google Scholar
  15. Farin, G. (1990)Curves and Surfaces for Computer Aided Design: A Practical Guide (2nd edn), Academic Press, New York.Google Scholar
  16. Floyd, R. W. and Rivest, R. L. (1975) Expected time bounds for selection.Communications of the ACM,18, 165–172.Google Scholar
  17. Franke, R. and Schumaker, L. L. (1987) A bibliography of multivariate approximation, inTopics in Multivariate Approximation, Chui, C. K., Schumaker, L. L. and Utreras, F. (eds), Academic Press, New York.Google Scholar
  18. Friedman, J. H. (1979) A tree-structured approach to, nonparametric multiple regression, inSmoothing, Techniques for Curve Estimation, Gasser, T. and Rosenblatt, M. (eds), Springer Verlag, New York.Google Scholar
  19. Friedman, J. H. (1984) A variable span smoother. Technical Report LCS5. Dept. of Statistics, Stanford.Google Scholar
  20. Friedman, J. H. (1991) Multivariate adaptive regression splines (with discussion).Ann. Statist.,19, 1–141.Google Scholar
  21. Friedman, J. J., Bentley, J. L. and Finkel, R. A. (1977) An algorithm for finding best matches in logarithmic expected time.ACM Transactions on Mathematical Software,3, 209–226.Google Scholar
  22. Friedman, J. H., Grosse, E. H. and Stuetzle, W. (1983) Multidimensional additive spline approximation.SIAM J. Sci. Stat. Comp.,4, 291–301.Google Scholar
  23. Friedman, J. H. and Silverman, B. W. (1989) Flexible parsimonious smoothing and additive modelling (with discussion).Technometrics,31, 3–39.Google Scholar
  24. Friedman, J. H. and Stuetzle, W. (1981) Projection pursuit regression.J. Amer. Statist. Assoc.,76, 817–823.Google Scholar
  25. Gordon, W. J. (1969) Distributive lattices and the approximation of multivariate functions, inProceedings of the Symposium on Approximation with Special Emphasis on Splines Schoenberg, I. J. (ed), Academic Press, New York.Google Scholar
  26. Grosse, E. (1990) A catalogue of algorithms for approximation, inAlgorithms for Approximation II, Mason, J. and Cox, M. (eds), Chapman and Hall, London.Google Scholar
  27. Gu, C. and Wahba, G. (1988) Minimizing GCV/GML scores with multiple smoothing parameters via the Newton method. Technical Report 847. Department of Statistics, University of Wisconsin.Google Scholar
  28. Hastie, T. and Tibshirani, R. (1990)Generalized Additive Models, Chapman and Hall, London.Google Scholar
  29. Lancaster, P. and Šalkauskas, K. (1986)Curve and Surface Fitting: An Introduction, Academic Press, New York.Google Scholar
  30. Macauley, F. R. (1931)The Smoothing of Time Series, National Bureau of Economic Research, New York.Google Scholar
  31. McLain, D. H. (1974) Drawing contours from arbitrary data points.Computer J.,17, 318–324.Google Scholar
  32. Schumaker, L. L. (1976) Fitting surfaces to scattered data, inApproximation Theory II, Lorentz, G. G., Chui, C. K., and Schumaker, L. L. (eds), Academic Press, New York.Google Scholar
  33. Stone, C. J. (1977) Consistent nonparametric regression.Ann. Stat.,5, 595–620.Google Scholar
  34. Wahba, G. (1978) Improper priors, spline smoothing, and the problem of guarding against model errors in regression.J. R. Stat. Soc. B,40, 364–372.Google Scholar
  35. Watson, G. S. (1964) Smooth regression analysis.Sankhya A,26, 359–372.Google Scholar

Copyright information

© Chapman and Hall Ltd 1991

Authors and Affiliations

  • William S. Cleveland
    • 1
  • E. Grosse
    • 1
  1. 1.AT&T Bell LaboratoriesMurray HillUSA

Personalised recommendations