Abstract
“Improper linear models” (see Dawes, Am. Psychol. 34:571–582, 1979), such as equal weighting, have garnered interest as alternatives to standard regression models. We analyze the general circumstances under which these models perform well by recasting a class of “improper” linear models as “proper” statistical models with a single predictor. We derive the upper bound on the mean squared error of this estimator and demonstrate that it has less variance than ordinary least squares estimates. We examine common choices of the weighting vector used in the literature, e.g., single variable heuristics and equal weighting, and illustrate their performance in various test cases.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Azen, R., & Budescu, D.V. (2009). Applications of multiple regression in psychological research. In R. Millsap, & A.M. Olivares (Eds.), Handbook of quantitative methods in psychology (pp. 285–310). Thousand Oaks: Sage.
Ben-Haim, Z., & Eldar, Y.C. (2005). Minimax estimators dominating the least-squares estimator. In Acoustics, speech, and signal processing, 2005. Proceedings (ICASSP ’05). IEEE international conference on (Vol. 4, pp. 53–56).
Camerer, C.F. (1981). General conditions for the success of bootstrapping models. Organizational Behavior and Human Performance, 27, 411–422.
Dana, J. (2008). What makes improper linear models tick? In J. Krueger (Ed.), Rationality and social responsibility: essays in honor of Robyn Mason Dawes. Mahwah: Lawrence Erlbaum Associates.
Dana, J., & Dawes, R.M. (2004). The superiority of simple alternatives to regression for social science predictions. Journal of Educational and Behavioral Statistics, 3, 317–331.
Dawes, R.M. (1979). The robust beauty of improper linear models. American Psychologist, 34, 571–582.
Dawes, R.M., & Corrigan, B. (1974). Linear models in decision making. Psychological Bulletin, 81, 95–106.
Dorans, N., & Drasgow, F. (1978). Alternative weighting schemes for linear prediction. Organizational Behavior and Human Performance, 21, 316–345.
Draper, N.R., & van Nostrand, R.C. (1979). Ridge regression and James–Stein estimation: Review and comments. Technometrics, 21, 451–466.
Einhorn, H.J., & Hogarth, R.M. (1975). Unit weighting schemes for decision making. Organizational Behavior and Human Performance, 13, 171–192.
Eldar, Y.C., Ben-Tal, A., & Nemirovski, A. (2005). Robust mean-squared error estimation in the presence of model uncertainties. IEEE Transactions on Signal Processing, 53, 168–181.
Gigerenzer, G., & Goldstein, D.G. (1996). Reasoning the fast and frugal way: Models of bounded rationality. Psychological Review, 103, 650–669.
Gigerenzer, G., Todd, P.M., & the ABC Research Group (1999). Simple heuristics that make us smart. New York: Oxford University Press.
Goldstein, D.G., & Gigerenzer, G. (2009). Fast and frugal forecasting. International Journal of Forecasting, 25, 760–772.
Guttman, L. (1984). A corollary of least-squares: For every unbiased estimator there is generally a better biased estimator. Unpublished manuscript.
Hocking, R.R. (1983). Developments in linear regression methodology: 1959–1982. Technometrics, 25, 219–230.
Hoerl, A.E., & Kennard, R.W. (1970). Ridge regression: Biased estimation for nonorthogonal problems. Technometrics, 12, 55–67.
James, W., & Stein, C. (1961). Estimation with quadratic loss. In Proceedings of the 4th Berkeley symposium on mathematical statistics and probability (Vol. 1, pp. 361–379). Berkeley: Univ. California Press.
Kendall, M.G. (1957). A course in multivariate analysis. London: Griffin.
Keren, G., & Newman, J.R. (1978). Additional considerations with regard to multiple regression and equal weighting. Organizational Behavior and Human Performance, 22, 143–164.
Marquardt, D.W. (1970). Generalized inverses, ridge regression, biased linear estimation, and nonlinear estimation. Technometrics, 12, 591–612.
Pruzek, R.M., & Frederick, B.C. (1978). Weighting predictors in linear models: Alternatives to least squares and limitations of equal weights. Psychological Bulletin, 85, 254–266.
Simon, H.A. (1956). Rational choice and the structure of the environment. Psychological Review, 63, 129–138.
Simon, H.A. (1982). Models of bounded rationality. Cambridge: MIT Press.
Wainer, H. (1976). Estimating coefficients in linear models: It don’t make no nevermind. Psychological Bulletin, 83, 213–217.
Waller, N.G. (2008). Fungible weights in multiple regression. Psychometrika, 73, 691–703.
Wang, M.W., & Stanley, J.C. (1970). Differential weighting: A review of methods and empirical studies. Review of Educational Research, 40, 663–705.
Wilks, S.S. (1938). Weighting systems for linear functions of correlated variables when there is no dependent variable. Psychometrika, 3, 23–40.
Vinod, H.D. (1978). A survey of ridge regression and related techniques for improvements over ordinary least squares. The Review of Economics and Statistics, 60, 121–131.
Author information
Authors and Affiliations
Corresponding author
Additional information
Much of this work was made possible by a predoctoral trainee fellowship awarded to the first author from the National Institutes of Mental Health under Training Grant Award Nr. PHS 2 T32 MH014257 entitled “Quantitative Methods for Behavioral Research” (to M. Regenwetter, PI). The first author was also supported by a Dissertation Completion Fellowship awarded by the University of Illinois. Any opinions, findings, and conclusions or recommendations expressed in this publication are those of the authors and do not necessarily reflect the views of the National Institutes of Mental Health or the University of Illinois. We would like to thank Lawrence Hubert and Konstantinos Katsikopoulos for comments on an earlier draft. We would also like to thank the editor, action editor, and three anonymous referees for their comments and suggestions.
Rights and permissions
About this article
Cite this article
Davis-Stober, C.P., Dana, J. & Budescu, D.V. A Constrained Linear Estimator for Multiple Regression. Psychometrika 75, 521–541 (2010). https://doi.org/10.1007/s11336-010-9162-8
Received:
Revised:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11336-010-9162-8