# Encyclopedia of Machine Learning and Data Mining

2017 Edition
| Editors: Claude Sammut, Geoffrey I. Webb

# Regression

Reference work entry
DOI: https://doi.org/10.1007/978-1-4899-7687-1_716

## Definition

Regression is a fundamental problem in statistics and machine learning. In regression studies, we are typically interested in inferring a real-valued function (called a regression function) whose values correspond to the mean of a dependent (or response or output) variable conditioned on one or more independent (or input) variables. Many different techniques for estimating this regression function have been developed, including parametric, semi-parametric, and nonparametric methods.

## Motivation and Background

Assume that we are given a set of data points sampled from an underlying but unknown distribution, each of which includes input x and output y. An example is given in Fig.  1. The task of regression is to learn a hidden functional relationship between x and y from observed and possibly noisy data points. In Fig.  1, the input–output relationship is a Gaussian-corrupted sinusoidal relationship, that is, $$y =\mathrm{ sin}(2\pi x)+\epsilon$$
This is a preview of subscription content, log in to check access.

## Recommended Reading1

1. Bishop C (2006) Pattern recognition and machine learning. Springer, New York
2. Gaffney S, Smyth P (1999) Trajectory clustering with mixtures of regression models. In: ACM SIGKDD, vol 62. ACM, New York, pp 63–72Google Scholar
3. Geman S, Bienenstock E, Doursat R (1992) Neural networks and the bias/variance dilemma. Neural Comput 4:1–58
4. Goldberg P, Williams C, Bishop C (1998) Regression with input-dependent noise: a Gaussian process treatment. In: Neural information processing systems, vol 10. MITGoogle Scholar
5. Hastie T, Tibshirani R, Friedman J (Corrected ed) (2003) The elements of statistical learning: data mining, inference, and prediction. Springer, New YorkGoogle Scholar
6. Koenker R (2005) Quantile regression. Cambridge University Press, Cambridge
7. Nelder JA, Wedderburn RWM (1972) Generalized linear models. J R Stat Soc: Ser A 135: 370–384Google Scholar
8. Seber G, Wild C (1989) Nonlinear regression. Wiley, New York

## Copyright information

© Springer Science+Business Media New York 2017

## Authors and Affiliations

1. 1.Department of Informatics, SMiLe CLiNiCUniversity of SussexBrightonUK
2. 2.Statistical Machine Learning ProgramNICTACanberraAustralia
3. 3.Faculty of Information TechnologyMonash UniversityClaytonAustralia