Quality and Quantity

, Volume 36, Issue 4, pp 325–345 | Cite as

Nonresponse Bias and Stimulus Effects in the Dutch National Election Study

  • Robert J. J. Voogt
  • Hetty Van Kempen


Bias is a much-debated issue in survey research. Answer effects (respondents claim to have behaved differently than they did in reality), nonresponse bias (nonrespondents differ on important variables from the respondents) and stimulus effects (by participating in a previous wave of a study, respondents change their behavior or attitude) can seriously distort the results of survey research. By using data from the 1998 Dutch National Election Study the authors show that the results of election research can indeed be affected by bias. Not only are significant effects found in the distribution of political attitude and voting behavior variables as a result of both nonresponse bias and stimulus effects, it is also shown that relations between variables change as a result of bias.

election research Hawthorne-effect nonresponse bias stimulus effects 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Aarts, K, & Van der Kolk, H. (1999). Tegen de stroom in? Non-respons en Hawthorne effecten in het Nationaal Kiezersonderzoek 1998 (Against The Stream? Nonresponse and Hawthorne effects in the 1998 Dutch National Election Study). Paper presented at the NVMC Spring conference 'De Waan van de Dag', Utrecht, The Netherlands, April 23.Google Scholar
  2. Aarts, K., Van der Kolk, H. & Kamp, M. (1999). Dutch Parliamentary Election Study 1998-Codebook. Enschede: Dutch Electoral Research Foundation (SKON).Google Scholar
  3. Andeweg, R. B. (1997). Institutional reform in Dutch politics: elected prime minister, personalized PR, and popular veto in comparative perspective. Acta Politica 32: 227-257.Google Scholar
  4. Andeweg, R. B. & Van Holsteyn, J. J. M. (1996). A hidden confidence gap? The question of nonresponse bias in measuring political interest. Netherlands Journal of Social Science 32: 127-142.Google Scholar
  5. Armstrong, J. S. & Overton, T. S. (1977). Estimating nonresponse bias in mail surveys. Journal of Marketing Research 14: 396-402.CrossRefGoogle Scholar
  6. Bebbington, A. C. (1970). The effect of nonresponse in the sample survey with an example. Human Relations 23: 169-180.CrossRefGoogle Scholar
  7. Betlehem, J. G. & Kersten, H. M. P. (1986). Werken Met Nonresponse (Working With Nonresponse). Voorburg: Centraal Bureau voor de Statistiek.Google Scholar
  8. Bishop, G. F., Oldendick, R. W. & Tuchfarber, A. J. (1984). What must my interest in politics be if I just told you 'I don't know'?. Public Opinion Quarterly 48: 510-519.CrossRefGoogle Scholar
  9. Blalock Jr., H. M. (1960). Social Statistics. New York: McGraw-Hill.Google Scholar
  10. Bradburn, N. M. (1992). Presidential address; a response to the nonresponse problem. Public Opinion Quarterly 56: 391-397.CrossRefGoogle Scholar
  11. Brehm, J. (1993). The Phantom Respondents; Opinion Surveys and Political Representation. Ann Arbor: University Of Michigan Press.Google Scholar
  12. Burchell, B. & Marsh, C. (1992). The effect of questionnaire length on survey response. Quality & Quantity 26: 233-244.CrossRefGoogle Scholar
  13. Collins, M. & Sykes, W. (1987). The problems of non-coverage and unlisted numbers in telephone surveys in Britain. Journal of The Royal Statistical Society, series A 150: 241-253.CrossRefGoogle Scholar
  14. Couper, M. P. (1997). Survey introductions and data quality. Public Opinion Quarterly 61: 317-338.CrossRefGoogle Scholar
  15. Couper, M. P. & Groves, R. M. (1996). Social environmental impacts on survey cooperation. Quality & Quantity 30: 173-188.CrossRefGoogle Scholar
  16. De Heer, W. (1999). International response trends: results of an international survey. Journal of Official Statistics 15: 129-142.Google Scholar
  17. De Leeuw, E. D. (1992). Data Quality in Mail, Telephone and Face to Face Surveys. Amsterdam: T.T.-Publikaties.Google Scholar
  18. DeMaio, T. J. (1980). Refusals: who, where and why. Public Opinion Quarterly 44: 223-233.CrossRefGoogle Scholar
  19. Donovan, R. J., Holman, C. D. J., Corti, B. & Jalleh, G. (1997). Face-to-face household interviews versus telephone interviews for health surveys. Australian and New Zealand Journal of Public Health 21: 134-140.CrossRefGoogle Scholar
  20. Ellis, R. A., Endo C. M. & Armer, J. M. (1970). The use of potential nonrespondents for studying nonresponse bias. Pacific Sociological Review 13: 103-109.Google Scholar
  21. Filion, F. L. (1975). Estimating bias due to nonresponse in mail surveys. Public Opinion Quarterly 39: 482-492.CrossRefGoogle Scholar
  22. Filion, F. L. (1976). Exploring and correcting for nonresponse bias using follow-ups of nonrespondents. Pacific Sociological Review 19: 401-408.Google Scholar
  23. Fitzgerald, R. & Fuller, L. (1982). I hear you knocking but you can't come in. Sociological Methods & Research 11: 3-32.CrossRefGoogle Scholar
  24. Goudy, W. J. (1976). Nonresponse effects on relationships between variables. Public Opinion Quarterly 40: 360-369.CrossRefGoogle Scholar
  25. Goyder, J. (1987). The Silent Minority: Nonrespondents on Sample Surveys. Boulder: Westview Press.Google Scholar
  26. Granberg, D. & Homberg, S. (1991). Self-reported turnout and voter validation. American Journal of Political Science 35: 448-459.CrossRefGoogle Scholar
  27. Greenwald, A., Carnot, C., Beach, R. & Young, B. (1987). Increasing voting behavior by asking people if they expect to vote. Journal of Applied Psychology 72: 315-318.CrossRefGoogle Scholar
  28. Groves, R. M. (1989). Survey Errors and Survey Costs. New York: Wiley.CrossRefGoogle Scholar
  29. Groves, R. M. & Couper, M. P. (1998). Nonresponse in Household Interview Surveys. New York: Wiley.Google Scholar
  30. Groves, R. M. & Kahn, R. L. (1979). Surveys by Telephone, A National Comparison with Personal Interviews. New York: Academic Press.Google Scholar
  31. Groves, R. M. & Lyberg, L. E. (1988). An overview of nonresponse issues in telephone surveys. In: R. M. Groves, P. P. Biemer, L. E. Lyberg, J. T. Massey, W. L. Nicholls II & J. Waksberg (eds), Telephone Survey Methodology. New York: Wiley, pp. 191-212.Google Scholar
  32. Hawkins, D. F. (1975). Estimation of nonresponse bias. Sociological Methods & Research 3: 461-485.CrossRefGoogle Scholar
  33. Herzog, A. R. & Rodgers, W. L. (1988). Age and response rates to interview sample surveys. Journal of Gerontology 43: 200-205.Google Scholar
  34. Hochstim, J. R. (1967). A critical comparison of three strategies of collecting data from households. Journal of The American Statistical Association 62: 976-989.CrossRefGoogle Scholar
  35. Hox, J. H. & De Leeuw, E. D. (1994). A comparison of nonresponse in mail, telephone, and face-to-face surveys. Quality & Quantity 28: 329-344.CrossRefGoogle Scholar
  36. Lagay, B. W. (1969). Assessing bias: a comparison of two methods. Public Opinion Quarterly 33: 615-618.CrossRefGoogle Scholar
  37. Lin, I. F. & Schaeffer, N. C. (1995). Using survey participants to estimate the impact of nonparticipation. Public Opinion Quarterly 59: 236-258.CrossRefGoogle Scholar
  38. Locander, W., Sudman, S. & Bradburn, N. (1976). An investigation of interview method, threat and response distortion. Journal of The American Statistical Association 71: 269-275.CrossRefGoogle Scholar
  39. Lund, E. & Gram, I. T. (1998). Response rate according to title and length of questionnaire. Scandinavian Journal of Social Medicine 26: 154-160.Google Scholar
  40. O'Neil, M. J. (1979). Estimating the nonresponse bias due to refusals in telephone surveys. Public Opinion Quarterly 43: 218-232.CrossRefGoogle Scholar
  41. Pavalko, R. & Lutterman, K. G. (1973). Characteristics of willing and reluctant respondents. Pacific Sociological Review 16: 463-476.Google Scholar
  42. Perry, J. B., Jr. (1968). A note on the use of telephone directories as a sample source. Public Opinion Quarterly 32: 691-695.CrossRefGoogle Scholar
  43. Rich, C. L. (1977). Is random digit dialing really necessary?. Journal of Marketing Research 14: 300-305.CrossRefGoogle Scholar
  44. Saris, W. E. & Van Den Putte, B. (1988). True score or factor models-a secondary analysis of the ALLBUS-Test-Retest data. Sociological Methods and Research 17: 123-157.CrossRefGoogle Scholar
  45. Saris, W. E. & Hagenaars, J. A. (1997). Mode effects in the standard Eurobarometer questions. In: W. E. Saris & M. Kaase (eds), Eurobarometer-Measurement Instruments for Opinions in Europe. Mannheim: ZUMA, pp. 87-100.Google Scholar
  46. Smeets, I. (1995). Facing another gap: an exploration of the discrepancies between voting turnout in survey research and official statistics. Acta Politica 30: 307-334.Google Scholar
  47. Smith, T. W. (1983). The hidden 25 percent: an analysis of nonresponse on the 1980 General Social Survey. Public Opinion Quarterly 47: 386-404.CrossRefGoogle Scholar
  48. Smith, T. W. (1984). Estimating nonresponse bias with temporary refusals. Sociological Perspectives 27: 473-489.Google Scholar
  49. Smith, T. W. (1990). Phone home? An analysis of household telephone ownership. International Journal of Public Opinion Research 2: 369-390.CrossRefGoogle Scholar
  50. Steeh, C. G. (1981). Trends in nonresponse rates, 1952-1979. Public Opinion Quarterly 45: 40-57.CrossRefGoogle Scholar
  51. Stinchcombe, A. L., Jones, C. & Sheatsley, P. (1981). Nonresponse bias for attitude questions. Public Opinion Quarterly 45: 359-375.CrossRefGoogle Scholar
  52. Swaddle, K. & Heath, A. (1989). Official and reported turnout in the British General Election of 1987. British Journal of Political Science 19: 527-541.CrossRefGoogle Scholar
  53. Sykes, W. & Collins, M. (1988). Effects of mode of interview: experiments in the UK. In: R. M. Groves, P. P. Biemer, L. E. Lyberg, J. T. Massey, W. L. Nicholls II & J. Waksberg (eds), Telephone Survey Methodology. New York: Wiley, pp. 301-320.Google Scholar
  54. Traugott, M. W. & Katosh, J. P. (1979). Response validity in surveys of voting behavior. Public Opinion Quarterly 42: 359-377.CrossRefGoogle Scholar
  55. Van Goor, B. (1996). Het lijk uit de kast: non-respons in het Nationaal Kiezersonderzoek (The corpse in the cupboard: nonresponse in the National Election Study). Sociologische Gids 18: 166-170.Google Scholar
  56. Visscher, G. (1995). Kiezersonderzoek op een Dwaalspoor (Election Studies on the Wrong Track) Den Haag: SDU.Google Scholar
  57. Voogt, R. J. J., Saris, W. E. & Niemöller, B. (1998). Non-response, and the gulf between the public and the politicians. Acta Politica 33: 250-280.Google Scholar
  58. Wilcox, J. B. (1977). The interaction of refusal and not-at-home sources of nonresponse bias. Journal of Marketing Research 14: 592-597.CrossRefGoogle Scholar
  59. Wolfle, L. M. (1979). Characteristics of persons with and without home telephones. Journal of Marketing Research 16: 421-425.CrossRefGoogle Scholar
  60. Yalch, R. F. (1976). Pre-election interview effects on voter turnout. Public Opinion Quarterly 40: 331-336.CrossRefGoogle Scholar

Copyright information

© Kluwer Academic Publishers 2002

Authors and Affiliations

  1. 1.ASCOR/University Of AmsterdamAmsterdamThe Netherlands
  2. 2.ASCOR/University Of AmsterdamAmsterdamThe Netherlands

Personalised recommendations