Fungible Weights in Multiple Regression

Psychometrika volume 73, Article number: 691 (2008) Cite this article

Abstract

Every set of alternate weights (i.e., nonleast squares weights) in a multiple regression analysis with three or more predictors is associated with an infinite class of weights. All members of a given class can be deemed fungiblebecause they yield identical SSE (sum of squared errors) and R 2 values. Equations for generating fungible weights are reviewed and an example is given that illustrates how fungible weights can be profitably used to evaluate parameter sensitivity in multiple regression.

This is a preview of subscription content, log in to check access.

References

  1. Abadir, K.M., & Magnus, J.R. (2002). Notation in econometrics: A proposal for a standard. Econometrics Journal, 5, 76–90.

    Article  Google Scholar 

  2. Achen, C. (1982). Interpreting and using regression. Beverly Hills, CA: Sage Publications.

    Google Scholar 

  3. Bring, J. (1994). How to standardize regression coefficients. American Statistician, 48, 209–213.

    Article  Google Scholar 

  4. Budescu, D.V. (1993). Dominance analysis: A new approach to the problem of relative importance of predictors in multiple regression. Psychological Bulletin, 114, 542–551.

    Article  Google Scholar 

  5. Carroll, J.D., & Green, P.E. (1997). Mathematical tools for applied multivariate analysis. San Diego, CA: Academic Press.

    Google Scholar 

  6. Dana, J., & Dawes, R.M. (2004). The superiority of simple alternatives to regression for social science predictions. Journal of Educational and Behavior Statistics, 29, 317–331.

    Article  Google Scholar 

  7. Darlington, R.B. (1968). Multiple regression in psychological research and practice. Psychological Bulletin, 69, 161–182.

    Article  PubMed  Google Scholar 

  8. Darlington, R.B. (1990). Regression and linear models. New York: McGraw-Hill.

    Google Scholar 

  9. Dawes, R.M., & Corrigan, B. (1974). Linear models in decision making. Psychological Bulletin, 81, 95–106.

    Article  Google Scholar 

  10. Einhorn, H.J., & Hogarth, R.M. (1975). Unit weighting schemes for decision making. Organizational Behavior and Human Performance, 13, 171–192.

    Article  Google Scholar 

  11. Ferguson, C.C. (1979). Intersections of ellipsoids and planes of arbitrary orientation and position. Mathematical Geology, 11, 329–333.

    Article  Google Scholar 

  12. Goldberger, A.S. (1968). Topics in regression analysis. New York: Macmillan.

    Google Scholar 

  13. Green, B.F. (1977). Parameter sensitivity in multivariate methods. Multivariate Behavioral Research, 12, 263–287.

    Article  Google Scholar 

  14. Green, P.E., Carroll, J.D., & DeSarbo, W.S. (1978). A new measure of predictor importance in multiple regression. Journal of Marketing Research, 15, 356–360.

    Article  Google Scholar 

  15. Greenland, S., Schlesselman, J.J., & Criqui, M.H. (1986). The fallacy of employing standardized regression coefficients and correlations as measures of effect. American Journal of Epidemiology, 123, 203–208.

    PubMed  Google Scholar 

  16. Grömping, U. (2007). Estimators of relative importance in linear regression based on variance decomposition. The American Statistician, 61, 139–147.

    Article  Google Scholar 

  17. Grove, W.M. (2001). Correction and extension of Wainer’s “Estimating coefficients in linear models: It don’t make no nevermind.” Unpublished manuscript.

  18. Guttman, L. (1955). The determinancy of factor score matrices with implications for five other basic problems of common factor theory. British Journal of Statistical Psychology, 8, 65–81.

    Google Scholar 

  19. Hoerl, A.E., & Kennard, R.W. (1970). Ridge regression: Biased estimation for nonorthogonal problems. Technometrics, 12, 55–67.

    Article  Google Scholar 

  20. Johnson, J.W. (2000). A heuristic method for estimating the relative weight of predictor variables in multiple regression. Multivariate Behavioral Research, 35, 1–19.

    Article  Google Scholar 

  21. Keren, G., & Newman, J.R. (1978). Additional considerations with regard to multiple regression and equal weighting. Organizational Behavior and Human Decision Processes, 22, 143–164.

    Google Scholar 

  22. Kim, J.O., & Ferree, G. (1981). Standardization in causal analysis. Sociological Methods and Research, 10, 187–210.

    Google Scholar 

  23. King, G. (1986). How not to lie with statistics: Avoiding common mistakes in quantitative political science. American Journal of Political Science, 30, 666–687.

    Article  Google Scholar 

  24. Koopman, R.F. (1988). On the sensitivity of a composite to its weights. Psychometrika, 53, 547–552.

    Article  Google Scholar 

  25. Kuncel, N.R., Hezlett, S.A., & Ones, D.S. (2001). A comprehensive meta-analysis of the predictive validity of the graduate record examinations: Implications for graduate student selection and performance. Psychological Bulletin, 127, 162–181.

    Article  PubMed  Google Scholar 

  26. Laughlin, J.E. (1978). Comment on “Estimating coefficients in linear models: It don’t make no nevermind”. Psychological Bulletin, 85, 247–253.

    Article  Google Scholar 

  27. Mulaik, S.A. (1972). The foundations of factor analysis. New York: McGraw-Hill.

    Google Scholar 

  28. Pruzek, R.M., & Fredrick, B.C. (1978). Weighting procedures in linear models: Alternatives to least squares and limitations of equal weights. Psychological Bulletin, 85, 254–266.

    Article  Google Scholar 

  29. Raju, N.S., Bilgic, R., Edwards, J.E., & Fleer, P.F. (1997). Methodology review: Estimation of population validity and cross-validity, and the use of equal weights in prediction. Applied Psychological Measurement, 21, 291–305.

    Article  Google Scholar 

  30. R Development Core Team (2007). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. ISBN 3-900051-07-0, URL http://www.R-project.org.

    Google Scholar 

  31. Rencher, A.C. (2000). Linear models in statistics. New York: John Wiley & Sons.

    Google Scholar 

  32. Rozeboom, W.W. (1979). Sensitivity of a linear composite of predictor items to differential item weighting. Psychometrika, 44, 289–296.

    Article  Google Scholar 

  33. Schmidt, F.L. (1971). The relative efficiency of regression and simple unit predictor weights in applied differential psychology. Educational and Psychological Measurement, 31, 699–714.

    Article  Google Scholar 

  34. Tate, R.L., & Bryant, J.L. (1986). Parameter sensitivity for discriminant analysis. Multivariate Behavioral Research, 21, 411–427.

    Article  Google Scholar 

  35. Tatsuoka, M.M. (1971). Multivariate analysis in educational and psychological research. New York: Wiley.

    Google Scholar 

  36. Tukey, J.W. (1948). Approximate weights. Annals of Mathematical Statistics, 19, 91–92.

    Article  Google Scholar 

  37. Wainer, H. (1976). Estimating coefficients in linear models: It don’t make no nevermind. Psychological Bulletin, 83, 213–217.

    Article  Google Scholar 

  38. Wainer, H. (1978). On the sensitivity of regression and regressors. Psychological Bulletin, 85, 267–273.

    Article  Google Scholar 

  39. Wilks, S.S. (1938). Weighting schemes for linear functions of correlated variables when there is no dependent variable. Psychometrika, 3, 23–40.

    Article  Google Scholar 

Download references

Author information

Affiliations

  1. Department of Psychology, University of Minnesota, N218 Elliott Hall, 75 East River Road, Minneapolis, MN, 55455, USA

    Niels G. Waller

Authors
  1. Niels G. Waller
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Niels G. Waller.

Additional information

The author wishes to thank Drs. Robyn Dawes, William Grove, Markus Keel, Leslie Yonce, Joe Rausch, the editor, and three anonymous reviewers for helpful comments on earlier versions of this article.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Waller, N.G. Fungible Weights in Multiple Regression. Psychometrika 73, 691 (2008). https://doi.org/10.1007/s11336-008-9066-z

Download citation

Keywords

  • multiple regression
  • alternate weights
  • fungible weights
  • parameter sensitivity