Skip to main content
Log in

Self-reported cheating in web surveys on political knowledge

  • Published:
Quality & Quantity Aims and scope Submit manuscript

Abstract

Measuring citizens’ political knowledge is important for understanding public opinion formation. In view of the increasing popularity of Web surveys, this paper examines the limitations of this interviewing facility when measuring factual political knowledge. We show that Web surveys contain a source of measurement error as respondents can “Google” the correct answers. This cheating activity is our principal concern. Past efforts are extended by: (1) offering a self-reported estimate of the share of Googling cheaters, (2) showing that the positive effect of education on factual political knowledge is most probably underestimated when cheating occurs, and (3) showing that self-reported cheating activity is inversely related to actual response time. In the concluding section, we discuss the implications of these results and the extent to which cheating can be reduced. The empirical analyses are based on a Danish Web sample from 2012 (N \(=\) 1,509).

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Notes

  1. The empirical analyses were also performed without weighting, and the results differed only marginally from the results presented. The weight has no influence on any of the substantial conclusions we draw in this paper.

  2. All main findings presented below can be reproduced with an index in which “don’t know” responses are treated as missing. Yet, because a very large share of the respondents (68.8 %) had at least one “don’t know” answer, the number of observations are reduced to less than a third when using this index.

  3. In a study of knowledge about science, Fricker et al. (2005, p. 389) also found that “Web users who completed the survey online seemed to know more about science than those who completed the survey on the telephone”. They ascribed this difference to the pace of telephone interviews compared to the Web survey—without considering the “Googling” effect.

  4. An approximately similar result occurred when response time was treated as a ranking variable, and thus transformed into a metric variable (ranging from 1 to 1509) with higher values indicating longer response time. The bivariate correlation \(=\) \(-0.05\). The key relationship between cheating (vs non-cheating) and response time is still negative.

References

  • Alvarez, R.M., Sherman, R.P., VanBeselaere, C.: Subject acquisition for web-based surveys. Political Anal. 11, 23–43 (2003)

    Article  Google Scholar 

  • Ansolabehere, S., Schaffner, B.F.: Does survey mode still matter? Findings from a 2010 multi-mode comparison. Unpublished manuscript: downloaded 5 January 2013. (2011)

  • Berrens, R.P., Bohara, A.K., Jenkins-Smith, H., Silva, C., Weimer, D.L.: The advent of internet surveys for political research: a comparison of telephone and internet samples. Political Anal. 11, 1–22 (2003)

    Article  Google Scholar 

  • Biancotti, C.: A polarization of inequality? The distribution of national Gini coefficients 1970–1996. J. Econ. Inequal. 4, 1–32 (2006)

    Article  Google Scholar 

  • Chang, L., Krosnick, J.A.: Comparing oral interviewing with self-administered computerized questionnaires. Public Opin. Q. 74, 154–167 (2010)

    Article  Google Scholar 

  • Clarke, H.D., Sanders, D., Stewart, M.C., Whiteley, P.: Leader heuristics, political knowledge and voting in Britain’s AV referendum. Elect. Stud. XXX, 1–12 (2013)

    Google Scholar 

  • Cook, C., Heath, F., Thompson, R.L.: A meta-analysis of response rates in web- or internet-based surveys. Educ. Psychol. Meas. 60, 821–836 (2000)

    Article  Google Scholar 

  • Couper, M.P.: Web surveys. A review of issues and approaches. Public Opin. Q. 64(4), 464–494 (2000)

    Article  Google Scholar 

  • Delli Carpini, M.X., Keeter, S.: Measuring political knowledge: putting first things first. Am J Pol Sci 37(4), 1179–1206 (1993)

    Article  Google Scholar 

  • Delli Carpini, M.X., Keeter, S.: What Americans know about politics and why it matters. Yale University Press, New Haven (1996)

    Google Scholar 

  • Elo, K.: Asking factual knowledge questions: reliability in web-based, passive sampling surveys. Soc. Sci. Comput. Rev. 27, 1–16 (2009)

    Google Scholar 

  • Fricker, S., Galesic, M., Tourangeau, R., Yan, T.: An experimental comparison of web and telephone surveys. Public Opin. Q. 69, 370–392 (2005)

    Article  Google Scholar 

  • Gilens, M.: Political ignorance and collective policy preferences. Am. Polit. Sci. Rev. 95, 379–396 (2001)

    Article  Google Scholar 

  • Grönlund, K., Milner, H.: The determinants of political knowledge in comparative perspective. Scand. Polit. Stud. 29(4), 386–406 (2006)

    Article  Google Scholar 

  • Heerweg, D.: Mode differences between face-to-face and web surveys: an experimental investigation of data quality and social desirability effects. Int. J. Public Opin. Res. 21, 111–121 (2009)

    Article  Google Scholar 

  • Heerweg, D., Loosveldt, G.: Face-to-face versus web surveying in a high-internet-coverage population. Public Opin. Q. 72, 836–846 (2008)

    Article  Google Scholar 

  • Hooghe, M., Reeskens, T., Stolle, D., Trappers, A.: Ethnic diversity and generalized trust in Europe: a cross-national multilevel study. Comp. Polit. Stud. 42, 198–223 (2009)

    Article  Google Scholar 

  • Jennings, M.K.: Political knowledge over time and across generations. Public Opin. Q. 60, 228–252 (1996)

    Article  Google Scholar 

  • Kreuter, F., Presser, S., Tourangeau, R.: Social desirability bias in CATI, IVR, and web surveys. Public Opin. Q. 72, 847–865 (2008)

    Article  Google Scholar 

  • Kunovich, R.M.: Political knowledge in Poland. Communist Post-Communist Stud. |, 1–14 (2013)

    Google Scholar 

  • Lambert, R.D., Curtis, J.E., Kay, B.J., Brown, S.D.: The social sources of political knowledge. Can. J. Polit. Sci. 21(2), 359–374 (1988)

    Article  Google Scholar 

  • Luskin, R.C.: Measuring political sophistication. Am. J. Polit. Sci. 31, 856–899 (1987)

    Article  Google Scholar 

  • Mondak, J.J.: Reconsidering the measurement of political knowledge. Polit. Anal. 8, 57–82 (1999)

    Article  Google Scholar 

  • Prior, M., Lupia, A.: Money, time, and political knowledge: distinguishing quick recall and political learning skills. Am. J. Polit. Sci. 52(1), 169–183 (2008)

    Article  Google Scholar 

  • Stephenson, L.B., Crête, J.: Studying political behavior: a comparison of internet and telephone surveys. Int. J. Public Opin. Res. 23, 24–55 (2010)

    Article  Google Scholar 

  • Strabac, Z., Aalberg, T.: Measuring political knowledge in telephone and web surveys. A cross-national comparison. Soc. Sci. Comput. Rev. 29(2), 175–192 (2011)

    Article  Google Scholar 

  • Sturgis, P., Allum, N., Smith, P.: An experiment on the measurement of political knowledge in surveys. Public Opin. Q. 85, 90–102 (2008)

    Article  Google Scholar 

  • Viklund, M.J.: Trust and risk perception in Western Europe: a cross-national study. Risk Anal. 23, 727–738 (2003)

    Article  Google Scholar 

  • Zaller, J.R.: The nature and origins of mass opinion. Cambridge University Press, Cambridge (1992)

    Book  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Jens Peter Frølund Thomsen.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Jensen, C., Thomsen, J.P.F. Self-reported cheating in web surveys on political knowledge. Qual Quant 48, 3343–3354 (2014). https://doi.org/10.1007/s11135-013-9960-z

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11135-013-9960-z

Keywords

Navigation