Encyclopedia of Machine Learning and Data Mining

2017 Edition
| Editors: Claude Sammut, Geoffrey I. Webb


  • Novi QuadriantoEmail author
  • Wray L. Buntine
Reference work entry
DOI: https://doi.org/10.1007/978-1-4899-7687-1_716


Regression is a fundamental problem in statistics and machine learning. In regression studies, we are typically interested in inferring a real-valued function (called a regression function) whose values correspond to the mean of a dependent (or response or output) variable conditioned on one or more independent (or input) variables. Many different techniques for estimating this regression function have been developed, including parametric, semi-parametric, and nonparametric methods.

Motivation and Background

Assume that we are given a set of data points sampled from an underlying but unknown distribution, each of which includes input x and output y. An example is given in Fig.  1. The task of regression is to learn a hidden functional relationship between x and y from observed and possibly noisy data points. In Fig.  1, the input–output relationship is a Gaussian-corrupted sinusoidal relationship, that is, \(y =\mathrm{ sin}(2\pi x)+\epsilon\)
This is a preview of subscription content, log in to check access.

Recommended Reading1

  1. Bishop C (2006) Pattern recognition and machine learning. Springer, New YorkzbMATHGoogle Scholar
  2. Gaffney S, Smyth P (1999) Trajectory clustering with mixtures of regression models. In: ACM SIGKDD, vol 62. ACM, New York, pp 63–72Google Scholar
  3. Geman S, Bienenstock E, Doursat R (1992) Neural networks and the bias/variance dilemma. Neural Comput 4:1–58CrossRefGoogle Scholar
  4. Goldberg P, Williams C, Bishop C (1998) Regression with input-dependent noise: a Gaussian process treatment. In: Neural information processing systems, vol 10. MITGoogle Scholar
  5. Hastie T, Tibshirani R, Friedman J (Corrected ed) (2003) The elements of statistical learning: data mining, inference, and prediction. Springer, New YorkGoogle Scholar
  6. Koenker R (2005) Quantile regression. Cambridge University Press, CambridgezbMATHCrossRefGoogle Scholar
  7. Nelder JA, Wedderburn RWM (1972) Generalized linear models. J R Stat Soc: Ser A 135: 370–384Google Scholar
  8. Seber G, Wild C (1989) Nonlinear regression. Wiley, New YorkzbMATHCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2017

Authors and Affiliations

  1. 1.Department of Informatics, SMiLe CLiNiCUniversity of SussexBrightonUK
  2. 2.Statistical Machine Learning ProgramNICTACanberraAustralia
  3. 3.Faculty of Information TechnologyMonash UniversityClaytonAustralia