Abstract
Measuring citizens’ political knowledge is important for understanding public opinion formation. In view of the increasing popularity of Web surveys, this paper examines the limitations of this interviewing facility when measuring factual political knowledge. We show that Web surveys contain a source of measurement error as respondents can “Google” the correct answers. This cheating activity is our principal concern. Past efforts are extended by: (1) offering a self-reported estimate of the share of Googling cheaters, (2) showing that the positive effect of education on factual political knowledge is most probably underestimated when cheating occurs, and (3) showing that self-reported cheating activity is inversely related to actual response time. In the concluding section, we discuss the implications of these results and the extent to which cheating can be reduced. The empirical analyses are based on a Danish Web sample from 2012 (N \(=\) 1,509).
Similar content being viewed by others
Notes
The empirical analyses were also performed without weighting, and the results differed only marginally from the results presented. The weight has no influence on any of the substantial conclusions we draw in this paper.
All main findings presented below can be reproduced with an index in which “don’t know” responses are treated as missing. Yet, because a very large share of the respondents (68.8 %) had at least one “don’t know” answer, the number of observations are reduced to less than a third when using this index.
In a study of knowledge about science, Fricker et al. (2005, p. 389) also found that “Web users who completed the survey online seemed to know more about science than those who completed the survey on the telephone”. They ascribed this difference to the pace of telephone interviews compared to the Web survey—without considering the “Googling” effect.
An approximately similar result occurred when response time was treated as a ranking variable, and thus transformed into a metric variable (ranging from 1 to 1509) with higher values indicating longer response time. The bivariate correlation \(=\) \(-0.05\). The key relationship between cheating (vs non-cheating) and response time is still negative.
References
Alvarez, R.M., Sherman, R.P., VanBeselaere, C.: Subject acquisition for web-based surveys. Political Anal. 11, 23–43 (2003)
Ansolabehere, S., Schaffner, B.F.: Does survey mode still matter? Findings from a 2010 multi-mode comparison. Unpublished manuscript: downloaded 5 January 2013. (2011)
Berrens, R.P., Bohara, A.K., Jenkins-Smith, H., Silva, C., Weimer, D.L.: The advent of internet surveys for political research: a comparison of telephone and internet samples. Political Anal. 11, 1–22 (2003)
Biancotti, C.: A polarization of inequality? The distribution of national Gini coefficients 1970–1996. J. Econ. Inequal. 4, 1–32 (2006)
Chang, L., Krosnick, J.A.: Comparing oral interviewing with self-administered computerized questionnaires. Public Opin. Q. 74, 154–167 (2010)
Clarke, H.D., Sanders, D., Stewart, M.C., Whiteley, P.: Leader heuristics, political knowledge and voting in Britain’s AV referendum. Elect. Stud. XXX, 1–12 (2013)
Cook, C., Heath, F., Thompson, R.L.: A meta-analysis of response rates in web- or internet-based surveys. Educ. Psychol. Meas. 60, 821–836 (2000)
Couper, M.P.: Web surveys. A review of issues and approaches. Public Opin. Q. 64(4), 464–494 (2000)
Delli Carpini, M.X., Keeter, S.: Measuring political knowledge: putting first things first. Am J Pol Sci 37(4), 1179–1206 (1993)
Delli Carpini, M.X., Keeter, S.: What Americans know about politics and why it matters. Yale University Press, New Haven (1996)
Elo, K.: Asking factual knowledge questions: reliability in web-based, passive sampling surveys. Soc. Sci. Comput. Rev. 27, 1–16 (2009)
Fricker, S., Galesic, M., Tourangeau, R., Yan, T.: An experimental comparison of web and telephone surveys. Public Opin. Q. 69, 370–392 (2005)
Gilens, M.: Political ignorance and collective policy preferences. Am. Polit. Sci. Rev. 95, 379–396 (2001)
Grönlund, K., Milner, H.: The determinants of political knowledge in comparative perspective. Scand. Polit. Stud. 29(4), 386–406 (2006)
Heerweg, D.: Mode differences between face-to-face and web surveys: an experimental investigation of data quality and social desirability effects. Int. J. Public Opin. Res. 21, 111–121 (2009)
Heerweg, D., Loosveldt, G.: Face-to-face versus web surveying in a high-internet-coverage population. Public Opin. Q. 72, 836–846 (2008)
Hooghe, M., Reeskens, T., Stolle, D., Trappers, A.: Ethnic diversity and generalized trust in Europe: a cross-national multilevel study. Comp. Polit. Stud. 42, 198–223 (2009)
Jennings, M.K.: Political knowledge over time and across generations. Public Opin. Q. 60, 228–252 (1996)
Kreuter, F., Presser, S., Tourangeau, R.: Social desirability bias in CATI, IVR, and web surveys. Public Opin. Q. 72, 847–865 (2008)
Kunovich, R.M.: Political knowledge in Poland. Communist Post-Communist Stud. |, 1–14 (2013)
Lambert, R.D., Curtis, J.E., Kay, B.J., Brown, S.D.: The social sources of political knowledge. Can. J. Polit. Sci. 21(2), 359–374 (1988)
Luskin, R.C.: Measuring political sophistication. Am. J. Polit. Sci. 31, 856–899 (1987)
Mondak, J.J.: Reconsidering the measurement of political knowledge. Polit. Anal. 8, 57–82 (1999)
Prior, M., Lupia, A.: Money, time, and political knowledge: distinguishing quick recall and political learning skills. Am. J. Polit. Sci. 52(1), 169–183 (2008)
Stephenson, L.B., Crête, J.: Studying political behavior: a comparison of internet and telephone surveys. Int. J. Public Opin. Res. 23, 24–55 (2010)
Strabac, Z., Aalberg, T.: Measuring political knowledge in telephone and web surveys. A cross-national comparison. Soc. Sci. Comput. Rev. 29(2), 175–192 (2011)
Sturgis, P., Allum, N., Smith, P.: An experiment on the measurement of political knowledge in surveys. Public Opin. Q. 85, 90–102 (2008)
Viklund, M.J.: Trust and risk perception in Western Europe: a cross-national study. Risk Anal. 23, 727–738 (2003)
Zaller, J.R.: The nature and origins of mass opinion. Cambridge University Press, Cambridge (1992)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Jensen, C., Thomsen, J.P.F. Self-reported cheating in web surveys on political knowledge. Qual Quant 48, 3343–3354 (2014). https://doi.org/10.1007/s11135-013-9960-z
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11135-013-9960-z