Quality and Quantity

, Volume 26, Issue 3, pp 233–244 | Cite as

The effect of questionnaire length on survey response

  • Brendan Burchell
  • Catherine Marsh
Article

Abstract

Survey textbooks suggest that long questionnaires should be avoided, and a careful reading of the available empirical evidence confirms the negative effects of substantial length on both response rates and the quality of those responses which are obtained. Data is presented from a lengthy survey in Britain in 1987. Analysis of reasons for nonresponse to this survey suggest that length may indeed have been a significant disincentive to respond for many. However, no effect of length was found on item quality as measured by the number of responses given to open-ended questions. Unexpectedly, the variance in number of responses was greater when the questions were asked later in the questionnaire. The results are interpreted as resulting from the greater power that respondents gain as the survey proceeds.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Anderson, A. B., Basilevsky, A. and Hum, D. (1983). Missing data: a review of the literature. In Rossi et al. (eds.), Handbook of Survey Research. New York: Academic Press.Google Scholar
  2. Andrews, F. M. (1984). Construct validity and error components of survey measures: a structural modelling approach. Public Opinion Quarterly 48: 409–442.CrossRefPubMedGoogle Scholar
  3. Backstrom, C. H. and Hursh-Cesar, G. (1981). Survey Research, 2nd edition. New York: Wiley.Google Scholar
  4. Berdie, D. R. (1973). Questionnaire length and response rate. Journal of Applied Psychology 58: 278–280.Google Scholar
  5. Bernstein, B. (1971). On the classification and framing of educational knowledge. In M. F. D. Young (ed.), Knowledge and Control: New Directions for the Sociology of Education. London: Collier-Macmillan.Google Scholar
  6. Billiet, J. and Loosveldt, G. (1988). Interviewer training and quality of responses. Public Opinion Quarterly 52: 190–211.Google Scholar
  7. Blumberg, H. H., Fuller, C. and Hare, A. P. (1974). Response rates in postal surveys. Public Opinion Quarterly 38: 113–123.Google Scholar
  8. Bradburn, N. (1977). Respondent burden. US Department of Health, Education and Welfare, National Center for Health Services Research, Health Survey Research Methods, Research Proceedings from the 2nd Biennial Conference, Publication No (PHS) 79-3207: 49–53.Google Scholar
  9. Brook, L. (1978). Postal survey procedures. In E. G. Hoinville and R. Jowell (eds.), Handbook of Survey Research, New York: Academic Press.Google Scholar
  10. Cannell, C. F. and Kahn, R. L. (1968). Interviewing. In G. Lindzey and E. Aronson (eds.), The Handbook of Social Psychology. Vol II (2nd ed). Reading, MA: Addison-Wesley.Google Scholar
  11. Collins, M. (1978). Interviewer variability: the North Yorkshire experiment. Journal of the Market Research Society 20: 59–72.Google Scholar
  12. Courtenay, G. (1978). Questionnaire construction. In G. Hoinville and R. Jowell (eds.), Survey Research Practice. London: Heinemann.Google Scholar
  13. Dillman, D. A. (1978). Mail and Telephone Surveys: The Total Design Method. New York: Wiley.Google Scholar
  14. Dillman, D., Christenson, J. A., Carpenter, E. H. and Brooks, R. M. (1974). Increasing mail questionnaire response: a four state comparison. American Sociological Review 39: 44–56.Google Scholar
  15. Ferber, R. (1966). Item nonresponse in a consumer survey. Public Opinion Quarterly 30: 399–415.Google Scholar
  16. Festinger, L. and Katz, D. (1965). Research Methods in the Behavioral Sciences. London: Holt, Rinehart and Winston.Google Scholar
  17. Furnham, A. (1982). Explanations for unemployment in Britain. European Journal of Social Psychology 12: 335–352.Google Scholar
  18. Gibson, D. M. and Aitkenhead, W. (1983). The elderly respondent: experiences from a largescale survey of the aged. Research on Ageing 5: 283–296.Google Scholar
  19. Goode, W. J. and Hatt, P. K. (1952). Methods in Social Research. London: McGraw-Hill.Google Scholar
  20. Goyder, J. C. (1982). Further evidence on factors affecting response rates to mailed questionnaires. American Sociological Review 47: 550–553.Google Scholar
  21. Hanson, R. H. and Marks, E. S. (1958). Influence of the interviewer on the accuracy of survey results. Journal of the American Statistical Association 53: 635–55.Google Scholar
  22. Heberlein, T. A. and Baumgartner, R. (1978). Factors affecting response rates to mailed questionnaires: a quantitative analysis of the published literature. American Sociological Review 43: 447–462.Google Scholar
  23. Herzog, A. R. and Bachman, J. G. (1981). Effects of questionnaire length on response quality. Public Opinion Quarterly 45: 49–59.Google Scholar
  24. Hoinville, G. and Jowell, R. (1978). Survey Research Practice. London: Heinemann.Google Scholar
  25. Johnson, W. R., Seiveking, N. A. and Clanton, E. S. (1974). Effects of alternative positioning of open-ended questions in multiple-choice questionnaires. Journal of Applied Psychology 59: 776–778.Google Scholar
  26. Jones, H. J. M., Lawson, H. B. and Newman, D. (1973). Population census: some recent British developments in methodology. Journal of the Royal Statistical Society (A) 136: 505–538.Google Scholar
  27. Kraut, A. I., Wolfson, A. D. and Rothenberg, A. (1975). Some effects of position on opinion survey items. Journal of Applied Psychology 60: 774–776.Google Scholar
  28. Layne, B. H. and Thompson, D. N. (1981). Questionnaire page length and return rate. Journal of Social Psychology 113: 291–292.Google Scholar
  29. Mason, W. S., Dressel, R. J. and Bain, R. K. (1961). An experimental study of factors affecting response to a mail survey of beginning teachers. Public Opinion Quarterly 25: 296–299.Google Scholar
  30. Morton-Williams, J. and Young, P. (1986). Interviewer Strategies on the Doorstep. Paper presented to the British Market Research Society Conference, Brighton, March.Google Scholar
  31. Morton-Williams, J. and Young, P. (1987). Obtaining the survey interview. Journal of the Market Research Society 29: 35–54.Google Scholar
  32. Roper, B. (1986). Evaluating polls with poll data. Public Opinion Quarterly 50: 10–16.Google Scholar
  33. Rossi, P. H., Wright, J. D. and Anderson, A. B. (1983). Handbook of Survey Research. New York: Academic Press.Google Scholar
  34. Schleifer, S. (1986). Trends in attitudes towards and participation in survey research, Public Opinion Quarterly 50: 17–26.Google Scholar
  35. Scott, C. (1961). Research on mail surveys. Journal of the Royal Statistical Society 124: 143–206.Google Scholar
  36. Shapiro, M. J. (1970). Discovering interviewer bias in open-ended responses. Public Opinion Quarterly 34: 412–415.Google Scholar
  37. Sharp, L. M. and Frankel, J. (1983). Respondent burden: a test of some common assumptions. Public Opinion Quarterly 47: 36–53.Google Scholar
  38. Sheatsley, P. B. (1983). Questionnaire construction and item writing. In P. Rossi et al. (eds.), Handbook of Survey Research. New York: Academic Press.Google Scholar
  39. Sheth, J. N., Roscoe, A. M. and Talbert, T. L. (1973). Mail surveys: can segmentation and experimentation improve them? Public Opinion Quarterly 37: 482–483.Google Scholar
  40. Sirken, M. G., Pifer, J. W. and Brown, M. L. (1960). Survey procedures for supplementing mortality statistics. American Journal of Public Health 50: 1753–1764.Google Scholar
  41. Sletto, R. F. (1940). Pretesting of questionnaires. American Sociological Review 5: 193–200.Google Scholar
  42. Sudman, S. and Bradburn, N. M. (1974). Response Effects in Surveys: a Review and Synthesis. Chicago: Aldine.Google Scholar
  43. Yu, J. and Cooper, H. (1983). A quantitative review of research design effects on response rates to questionnaires. Journal of Marketing Research XX: 36–44.Google Scholar

Copyright information

© Kluwer Academic Publishers 1992

Authors and Affiliations

  • Brendan Burchell
    • 1
  • Catherine Marsh
    • 2
  1. 1.Social and Political Sciences DepartmentUniversity of CambridgeCambridge
  2. 2.Faculty of Economic and Social StudiesUniversity of ManchesterManchester

Personalised recommendations