Research in Higher Education

, Volume 38, Issue 2, pp 215–227 | Cite as

Working with Low Survey Response Rates: The Efficacy of Weighting Adjustments

  • Eric L. Dey

Abstract

National data show a continuing decline in the willingness of people to respond to surveys. This trend is troubling given the central role that survey research plays in collecting data for institutional research purposes. This paper examines the effectiveness of a weighting procedure described by Astin and Molm for adjusting survey results to correct for nonresponse bias. Using data from a Cooperative Institutional Research Program (CIRP) follow-up survey, the results indicate that the weighting procedure is highly effective at reducing nonresponse bias in univariate distributions. The effectiveness of the weighting procedure in adjusting correlation and regression analyses is less clear. This may be due in part to the observation that even when individual variables are noticeably biased, their relationships with each other tend not to be.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

REFERENCES

  1. Astin, A. W. (1968). The College Environment. Washington, DC: American Council on Education.Google Scholar
  2. Astin, A. W. (1977). Four Critical Years. San Francisco: Jossey-Bass.Google Scholar
  3. Astin, A. W. (1982). Minorities in American Higher Education. San Francisco: Jossey-Bass.Google Scholar
  4. Astin, A. W. (1993). What Matters in College. San Francisco: Jossey-Bass.Google Scholar
  5. Astin, A. W., and Molm, L. D. (1972). Correcting for Nonresponse Bias in Follow-up Surveys. Unpublished paper, Office of Research, American Council on Education, Washington, DC.Google Scholar
  6. Astin, A. W., and Panos, R. J. (1969). The Educational and Vocational Development of College Students. Washington, DC: American Council on Education.Google Scholar
  7. Astin, A. W., Panos, R. J., and Creager, J. A. (1966). A program of longitudinal research on higher education. ACE Research Reports, vol. 1(1). Washington, DC: American Council on Education.Google Scholar
  8. Cabrera, A. F., Castañeda, M. B., Nora, A., and Hengstler, D. (1992). The convergence between two theories of college persistence. Journal of Higher Education 63(2): 143–164.Google Scholar
  9. Cote, L. S., Grinnell, R. E., and Tompkins, L. D. (1986). Increasing response rates to mail surveys: The impact of adherence to Dillman-like procedures and techniques. Review of Higher Education 9(2): 229–242.Google Scholar
  10. Davis, J. A., and Smith, T. W. (1992). The NORC General Social Survey: A User's Guide. Newbury Park, CA: Sage.Google Scholar
  11. Dey, E. L., and Astin, A. W. (1993). Statistical alternatives for studying college student retention: A comparative analysis of logit, probit, and linear regression. Research in Higher Education 34(4): 569–581.Google Scholar
  12. Dey, E. L., Astin, A. W., and Korn, W. S. (1991). The American Freshman: Twenty-Five Year Trends. Los Angeles: Higher Education Research Institute, UCLA.Google Scholar
  13. Dillman; D. A. (1978). Mail and Telephone Surveys: The Total Design Method. New York: Wiley.Google Scholar
  14. Dillman, D. A. (1983). Mail and self-administered surveys. In P. H. Rossi, J. D. Wright, and A. B. Anderson (eds.), Handbook of Survey Research. New York: Academic Press, pp. 359–377.Google Scholar
  15. Dillman, D. A. (1991). The design and administration of mail surveys. Annual Review of Sociology 17: 225–249.Google Scholar
  16. Goldberger, A. S. (1981). Linear regression after selection. Journal of Econometrics 15(2): 357–366.Google Scholar
  17. Grosset, J. M. (1995). The biasing effects of bad addresses on information gathered by mail surveys. Journal of Applied Research in the Community College 2(2): 179–191.Google Scholar
  18. Groves, R. M. (1989). Survey Errors and Survey Costs. New York: Wiley.Google Scholar
  19. Higher Education Research Institute (1989). The American College Student, 1987: National Norms for 1983 and 1985 College Students. Los Angeles: Higher Education Research Institute, UCLA.Google Scholar
  20. Higher Education Research Institute (1991). The American College Student, 1989: National Norms for 1985 and 1987 College Students. Los Angeles: Higher Education Research Institute, UCLA.Google Scholar
  21. Higher Education Research Institute (1992). The American College Student, 1991: National Norms for 1987 and 1989 College Students. Los Angeles: Higher Education Research Institute, UCLA.Google Scholar
  22. Kalton, G. (1983). Compensating for Missing Survey Data. Ann Arbor: Survey Research Center, University of Michigan.Google Scholar
  23. Madow, W. G., Olkin, I., and Rubin, D. B. (eds.). (1983). Incomplete Data in Sample Surveys: Theory and Bibliographies, vol. 2. New York: Academic Press.Google Scholar
  24. Oh, H. L., and Scheuren, F. J. (1983). Weighting adjustments for unit nonresponse. In W. G. Madlow, I. Olkin, and D. B. Rubin (eds.), Incomplete Data in Sample Surveys: Theory and Bibliographies, vol. 2. New York: Academic Press, pp. 143–184.Google Scholar
  25. Pascarella, E. T., Brier, E. M., Smart, J. C., and Herzog, L. (1987). Becoming a physician: The influence of the undergraduate experience. Research in Higher Education 26(2): 180–201.Google Scholar
  26. Pascarella, E. T., Ethington, C. A., and Smart, J. C. (1988). The influence of college on humanitarian/civic involvement values. Journal of Higher Education 59(4): 412–437.Google Scholar
  27. Rossi, P. H., Wright, J. D., and Anderson, A. B. (eds.) (1983). Handbook of Survey Research. New York: Academic Press.Google Scholar
  28. Rubin, D. B. (1987). Multiple Imputation for Nonresponse in Surveys. New York: Wiley.Google Scholar
  29. Schiltz, M. E. (1988). Professional standards for survey research. Research in Higher Education 28(1): 67–75.Google Scholar
  30. Skinner, C. J, Holt, D., and Smith, T. M. F. (eds.) (1989). Analysis of Complex Surveys. New York: Wiley.Google Scholar
  31. Smith, K., and Bers, T. (1987). Improving alumni survey response rates: An experiment and cost benefit analysis. Research in Higher Education 27: 218–225.Google Scholar
  32. Steeh, C. J. (1981). Trends in nonresponse rates. Public Opinion Quarterly 45: 40–57.Google Scholar

Copyright information

© Human Sciences Press, Inc. 1997

Authors and Affiliations

  • Eric L. Dey
    • 1
  1. 1.Center for the Study of Higher and Postsecondary EducationUniversity of MichiganAnn Arbor

Personalised recommendations