Advertisement

The effects of survey mode and sampling in Belgian election studies: a comparison of a national probability face-to-face survey and a nonprobability Internet survey

  • Ruth DassonnevilleEmail author
  • André Blais
  • Marc Hooghe
  • Kris Deschouwer
Original Article

Abstract

National probability election surveys are more and more abandoned. Decreasing response rates and the escalating costs of face-to-face and telephone interviews have strengthened election scholars’ reliance on nonprobability internet samples to conduct election surveys online. In a number of countries, experiments with alternative ways of recruiting respondents and different interview modes have been well documented. For other countries, however, substantially less is known about the consequences of relying on nonprobability internet panels. In this paper, we investigate the effects of survey mode and sampling method in the Belgian context. This is a particularly important and relevant case study because election researchers in Belgium can draw a sample of voters directly from the National Register. In line with previous studies, we find important differences in the marginal distributions of variables measured in the two surveys. When considering vote choice models and the inferences that scholars would draw, in contrast, we find minor differences.

Keywords

Election study Belgium Survey mode effects Representativeness Nonprobability sample 

Notes

Acknowledgements

A previous version of this paper was presented during the Making Electoral Democracy Work mini-conference at the 113th Annual Meeting of the American Political Science Meeting, San Francisco, August 31-September 3, 2017. We thank Filip Kostelka for providing technical information on the MEDW-survey and Fernando Feitosa for research assistance. We are grateful to Shane Singh and Dieter Stiers for commenting on previous drafts of the papers and the anonymous reviewers of this journal for excellent suggestions.

Supplementary material

41269_2018_110_MOESM1_ESM.pdf (182 kb)
Supplementary material 1 (pdf 181 KB)

References

  1. Alvarez, R. Michael, Robert P. Sherman, and Carla VanBeselaere. 2003. Subject Acquisition for Web-Based Surveys. Political Analysis 11 (1): 23–43.CrossRefGoogle Scholar
  2. André, Audrey, and Sam Depauw. 2015. A Divided Nation? The 2014 Belgian Federal Elections. West European Politics 38 (1): 228–237.CrossRefGoogle Scholar
  3. Ansolabehere, Stephen, and Brian F. Schaffner. 2014. Does Survey Mode Still Matter? Findings from a 2010 Multi-Mode Comparison. Political Analysis 22 (3): 285–303.CrossRefGoogle Scholar
  4. Baker, Reg, Stephen J. Blumberg, Michael J. Brick, Mick P. Couper, J. Melanie Courtright, Michael Dennis, Don Dillman, et al. 2010. AAPOR Report on Online Panels. Public Opinion Quarterly 74 (4): 711–781.CrossRefGoogle Scholar
  5. Berrens, Robert P., Alok K. Bohara, Hank Jenkins-Smith, Carol Silva, and David L. Weimer. 2003. The Advent of Internet Surveys for Political Research: A Comparison of Telephone and Internet Samples. Political Analysis 11 (1): 1–22.CrossRefGoogle Scholar
  6. Breton, Charles, Fred Cutler, Lachance Sarah, and Alex Mierke-Zatwarnicki. 2017. Telephone versus Online Survey Modes for Election Studies: Comparing Canadian Public Opinion and Vote Choice in the 2015 Federal Election. Canadian Journal of Political Science 50 (4): 1005–1036.CrossRefGoogle Scholar
  7. Bytzek, Evelyn, and Ina E. Bieber. 2016. Does Survey Mode Matter for Studying Electoral Behaviour? Evidence from the 2009 German Longitudinal Election Study. Electoral Studies 43: 41–51.CrossRefGoogle Scholar
  8. Cameron, Sarah M., and Ian McAllister. 2016. Trends in Australian Political Opinion: Results from the Australian Election Study 1987–2016. Canberra: Australian National University.Google Scholar
  9. Chang, Linchiat, and Jon A. Krosnick. 2009. National Surveys via RDD Telephone Interviewing versus the Internet. Comparing Sample Representativeness and Response Quality. Public Opinion Quarterly 73 (4): 641–678.CrossRefGoogle Scholar
  10. Converse, Philip, John Meisel, Maurice Pinard, Peter Regenstreif, and Mildred Schwartz. 2002. Canadian National Election Study, 1965. ICPSR 7225 [Data file]. Ann Arbor (MI): Inter-University Consortium for Political and Social Research [distributor].Google Scholar
  11. Couper, Mick P. 2000. Web Surveys: A Review of Issues and Approaches. Public Opinion Quarterly 64 (4): 464–494.CrossRefGoogle Scholar
  12. Crewe, Ivor, Bo Sarlvik, and James Alt. 1977. Partisan Dealignment in Britain 1964–1974. British Journal of Political Science 7 (2): 129–190.CrossRefGoogle Scholar
  13. Curtin, Richard, Stanley Presser, and Eleanor Singer. 2005. Changes in Telephone Survey Nonresponse over the Past Quarter Century. Public Opinion Quarterly 69 (1): 87–98.CrossRefGoogle Scholar
  14. Dassonneville, Ruth, and Dieter Stiers. 2018. Electoral Volatility in Belgium (2009–2014). Is There a Difference between Stable and Volatile Voters. Acta Politica 53 (1): 68–97.CrossRefGoogle Scholar
  15. Deschouwer, Kris (ed.). 2018. Mind the Gap. Political Participation and Representation in Belgium. Lanham, MD: Rowman & Littlefield.Google Scholar
  16. Dillman, Don A. 2000. Mail and Internet Surveys: The Tailored Design Method. New York: Harper & Row.Google Scholar
  17. Durand, Claire, André Blais, and Mylène Larochelle. 2004. The Polls-Review. The Polls in the, 2002. French Presidential Election: An Autopsy. Public Opinion Quarterly 68 (4): 602–622.CrossRefGoogle Scholar
  18. Fieldhouse, Ed, Jane Green, Geoffrey Evans, Herman Schmitt, Cees van der Eijk, Jonathan Mellon, and Chris Prosser. 2016. British Election Study, 2015: Face-to-Face Post-Election Survey [data collection]. UK Data Service. SN: 7972.  https://doi.org/10.5255/UKDA-SN-7972-1.
  19. Fieldhouse, Ed, Jane Green, Geoffrey Evans, Herman Schmitt, Cees van der Eijk, Jonathan Mellon, and Chris Prosser. 2017. British Election Study Internet Panel Waves 1-13 [datafile].  https://doi.org/10.15127/1.293723.
  20. Foucault, Martial. 2017. L’enquête électorale française (www.enef.fr). Paris: CEVIPOF SciencesPo.Google Scholar
  21. Gelman, Andrew, Jennifer Hill, and Masanao Yajima. 2012. Why We (Usually) Don’t Have to Worry About Multiple Comparisons. Journal of Research on Educational Effectiveness 5 (2): 189–211.CrossRefGoogle Scholar
  22. Groves, Robert M. 2006. Nonresponse Rates and Nonresponse Bias in Household Surveys. Public Opinion Quarterly 70 (5): 646–675.CrossRefGoogle Scholar
  23. Hooghe, Marc, Sofie Marien, and Teun Pauwels. 2011. Where Do Distrusting Voters Turn if There is No Viable Exit or Voice Option? The Impact of Political Trust on Electoral Behaviour in the Belgian Regional Elections of June 2009. Government and Opposition 46 (2): 245–273.CrossRefGoogle Scholar
  24. Hooghe, Marc, Sara Vissers, Dietlind Stolle, and Valérie-Anne Mahéo. 2010. The Potential of Internet Mobilization: An Experimental Study on the Effect of Internet and Face-to-Face Mobilization Efforts. Political Communication 27 (4): 406–431.CrossRefGoogle Scholar
  25. Keeter, Scott, Courtney Kennedy, April Clark, Trevor Tompson, and Mike Mokrzycki. 2007. What’s Missing from National Landline RDD Surveys? The Impact of the Growing Cell-Only Population. Public Opinion Quarterly 71 (5): 772–792.CrossRefGoogle Scholar
  26. Lynn, Peter. 2015. Alternative Sequential Mixed-Mode Designs: Effects on Attrition Rates, Attrition Bias, and Costs. Journal of Survey Statistics and Methodology 1 (2): 183–205.CrossRefGoogle Scholar
  27. Malhotra, Neil, and Jon A. Krosnick. 2007. The Effect of Survey Mode and Sampling on In-ferences about Political Attitudes and Behavior: Comparing the 2000 and 2004 ANES to Internet Surveys with Nonprobability Samples. Political Analysis 15 (3): 286–323.CrossRefGoogle Scholar
  28. Manfreda, Katja Lozar, Jernej Berzelak, Vasja Vehovar, Michael Bosnjak, and Iris Haas. 2008. Web Surveys Versus Other Survey Modes: A Meta-Analysis Comparing Response Rates. International Journal of Market Research 50 (2): 269–291.Google Scholar
  29. MEDW, Making Electoral Democracy Work. 2014. Making Electoral Democracy Work. Belgium Regional, National and European Election-Brussels, Flanders and Wallonia. Technical Report. Montréal.Google Scholar
  30. Mellon, Jonathan, and Christopher Prosser. 2017. Missing Nonvoters and Misweighted Samples. Explaining the 2015 Great British Polling Miss. Public Opinion Quarterly 81 (3): 661–667.CrossRefGoogle Scholar
  31. Mood, Carina. 2010. Logistic Regression: Why We Cannot Do What We Think We Can Do, and What We Can Do About It. European Sociological Review 26 (1): 67–82.CrossRefGoogle Scholar
  32. Morton, Susan M.B., Dinusha K. Bandara, Elizabeth M. Robinson, and Polly E. Atatoa Carr. 2012. In the 21st Century, What is an Acceptable Response Rate? Australian and New Zealand Journal of Public Health 36 (2): 106–108.CrossRefGoogle Scholar
  33. Motta, Matthew P., Timothy H. Callaghan, and Brianna Smith. 2016. Looking for Answers: Identifying Search Behavior and Improving Knowledge-Based Data Quality in Online Surveys. International Journal of Public Opinion Research 29 (4): 575–603.Google Scholar
  34. Northrup, David. 2016. The 2015 Canadian Election Study. Toronto: Institute for Social Research, York University.Google Scholar
  35. PartiRep. 2014. PartiRep Voter Panel Survey 2014. Technical Report. Brussels.Google Scholar
  36. Pasek, Josh. 2016. When will Nonprobability Surveys Mirror Probability Surveys? Considering Types of Inference and Weighting Strategies as Criteria for Correspondence. International Journal of Public Opinion Research 28 (2): 269–291.CrossRefGoogle Scholar
  37. Sanders, David, Harold D. Clarke, Marianne C. Stewart, and Paul Whiteley. 2007. Does Mode Matter for Modeling Political Choice? Evidence From the 2005 British Election Study. Political Analysis 15 (3): 257–285.CrossRefGoogle Scholar
  38. Schoen, Harald, and Thorsten Faas. 2005. When Methodology Interferes with Substance: The Difference of Attitudes towards E-Campaigning in Online and Offline Surveys. Social Science Computer Review 23 (3): 326–333.CrossRefGoogle Scholar
  39. Selb, Peter, and Simon Munzert. 2013. Voter Overrepresentation, Vote Misreporting, and Turnout Bias in Postelection Surveys. Electoral Studies 32 (1): 186–196.CrossRefGoogle Scholar
  40. Simmons, Alicia D., and Lawrence D. Bobo. 2015. Can Non-Full-Probability Internet Surveys Yield Useful Data? A Comparison with Full-Probability Face-to-Face Surveys in the Domain of Race and Social Inequality Attitudes. Sociological Methodology 45 (1): 357–387.CrossRefGoogle Scholar
  41. Stephenson, Laura B., and Jean Crête. 2010. Studying Political Behavior: A Comparison of Internet and Telephone Surveys. International Journal of Public Opinion Research 23 (1): 24–55.CrossRefGoogle Scholar
  42. Stern, Michael J., Ipek Bilgen, and Don A. Dillman. 2014. The State of Survey Methodology: Challenges, Dilemmas, and New Frontiers in the Era of the Tailored Design. Field Methods 26 (3): 284–301.CrossRefGoogle Scholar
  43. Vavreck, Lynn, and Douglas Rivers. 2008. The 2006 Cooperative Congressional Election Study. Journal of Elections, Public Opinion and Parties 18 (4): 355–366.CrossRefGoogle Scholar
  44. Yeager, David S., Jon A. Krosnick, Chang LinChiat, Harold S. Javitz, Matthew S. Levendusky, Alberto Simpser, and Rui Wang. 2011. Comparing the Accuracy of RDD Telephone Surveys and Internet Surveys Conducted with Probability and Non-Probability Samples. Public Opinion Quarterly 75 (4): 709–747.CrossRefGoogle Scholar

Copyright information

© Springer Nature Limited 2018

Authors and Affiliations

  • Ruth Dassonneville
    • 1
    Email author
  • André Blais
    • 1
  • Marc Hooghe
    • 2
  • Kris Deschouwer
    • 3
  1. 1.Département de science politiqueUniversité de MontréalMontréalCanada
  2. 2.Centre for Citizenship and DemocracyKU LeuvenLeuvenBelgium
  3. 3.Department of Political ScienceVrije Universiteit BrusselBrusselBelgium

Personalised recommendations