Abstract
Regression estimates functional dependencies between features. Linear regression models can be efficiently computed from covariances but are restricted to linear dependencies. Substitution allows us to identify specific nonlinear dependencies by linear regression. Robust regression finds models that are robust against outliers. A popular family of nonlinear regression methods are universal approximators. We present two well-known examples for universal approximators from the field of artificial neural networks: the multilayer perceptron and radial basis function networks. Universal approximators can realize arbitrarily small training errors, but cross-validation is required to find models with low validation errors that generalize well on other data sets. Feature selection allows us to include only relevant features in regression models leading to more accurate models.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
H. Akaike. A new look at the statistical model identification. IEEE Transactions on Automatic Control, AC–19:716–723, 1974.
Y. Bengio. Learning deep architectures for AI. Foundations and Trends in Machine Learning, 2(1):1–127, 2009.
P. Craven and G. Wahba. Smoothing noisy data with spline functions: Estimating the correct degree of smoothing by the method of generalized cross–validation. Numerical Mathematics, 31:377–403, 1979.
R. Hecht-Nielsen. Neurocomputing. Addison-Wesley, 1990.
K. Hornik, M. Stinchcombe, and H. White. Multilayer feedforward networks are universal approximators. Neural Networks, 2(5):359–366, 1989.
P. J. Huber. Robust Statistics. Wiley, New York, 2nd edition, 2009.
A. N. Kolmogorov. On the representation of continuous functions of many variables by superposition of continuous functions of one variable and addition. Doklady Akademii Nauk SSSR, 144:679–681, 1957.
R. Meiria and J. Zahavi. Using simulated annealing to optimize the feature selection problem in marketing applications. European Journal of Operational Research, 171(3):842–858, 2006.
J. Moody and C. Darken. Fast learning in networks of locally-tuned processing units. Neural Computation, 1:281–294, 1989.
F. Rosenblatt. The perceptron: A probabilistic model for information storage and organization in the brain. Psychological Reviews, 65:386–408, 1958.
P. J. Rousseeuw and A. M. Leroy. Robust Regression and Outlier Detection. Wiley, New York, 1987.
D. E. Rumelhart, G. E. Hinton, and R. J. Williams. Learning internal representations by error backpropagation. In D. E. Rumelhart and J. L. McClelland, editors, Parallel Distributed Processing. Explorations in the Microstructure of Cognition, volume 1, pages 318–362. MIT Press, Cambridge, 1986.
W. Siedlecki and J. Sklansky. A note on genetic algorithms for large–scale feature selection. Pattern Recognition Letters, 10(5):335–347, 1989.
S. Vieira, J. M. Sousa, and T. A. Runkler. Multi-criteria ant feature selection using fuzzy classifiers. In C. A. Coello Coello, S. Dehuri, and S. Ghosh, editors, Swarm Intelligence for Multi-objective Problems in Data Mining, pages 19–36. Springer, 2009.
P. J. Werbos. The Roots of Backpropagation: From Ordered Derivatives to Neural Networks and Political Forecasting (Adaptive and Learning Systems for Signal Processing, Communications and Control Series). Wiley–Interscience, 1994.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Copyright information
© 2016 Springer Fachmedien Wiesbaden
About this chapter
Cite this chapter
Runkler, T. (2016). Regression. In: Data Analytics. Springer Vieweg, Wiesbaden. https://doi.org/10.1007/978-3-658-14075-5_6
Download citation
DOI: https://doi.org/10.1007/978-3-658-14075-5_6
Published:
Publisher Name: Springer Vieweg, Wiesbaden
Print ISBN: 978-3-658-14074-8
Online ISBN: 978-3-658-14075-5
eBook Packages: Computer ScienceComputer Science (R0)