Behavior Research Methods

, Volume 40, Issue 4, pp 1144–1149 | Cite as

Individual payments as a longer-term incentive in online panels

  • Anja S. Göritz
  • Hans-Georg Wolff
  • Daniel G. Goldstein


Does it pay to pay online panel members? A three-wave longitudinal experiment was conducted with an online panel to examine whether per person payments, paid through an online intermediary, influence response and retention rates. In the payment condition, participants were promised payment for participation at each wave, whereas control participants were not offered any payment. The promise of a payment had a negative effect on response in Wave 1, but a positive effect on response in Wave 2. Payment had no significant effect on retention. Completing a given wave was an indicator for responding to a subsequent invitation.


  1. Bailar, B. A. (1984). The quality of survey data. Proceedings of the Section on Survey Research Methods of the American Statistical Association, 43–52.Google Scholar
  2. Batinic, B., & Moser, K. (2005). Determinanten der Rücklaufquote in Online-Panels [Determinants of response in online panels]. Zeitschrift für Medienpsychologie, 17, 64–74.CrossRefGoogle Scholar
  3. Biswas, D., & Biswas, A. (2004). The diagnostic role of signals in the context of perceived risks in online shopping: Do signals matter more on the Web? Journal of Interactive Marketing, 18, 30–45.CrossRefGoogle Scholar
  4. Bošnjak, M., & Tuten, T. L. (2003). Prepaid and promised incentives in Web surveys: An experiment. Social Science Computer Review, 21, 208–217.CrossRefGoogle Scholar
  5. Brennan, M., & Hoek, J. (1992). The behavior of respondents, nonrespondents, and refusers across mail surveys. Public Opinion Quarterly, 56, 530–535.CrossRefGoogle Scholar
  6. Church, A. H. (1993). Estimating the effect of incentives on mail survey response rates: A meta-analysis. Public Opinion Quarterly, 57, 62–79.CrossRefGoogle Scholar
  7. Cohen, J. (1987). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum.Google Scholar
  8. Couper, M. P. (2000). Web surveys: A review of issues and approaches. Public Opinion Quarterly, 64, 464–494.PubMedCrossRefGoogle Scholar
  9. Deci, E. L. (1971). Effects of externally mediated rewards on intrinsic motivation. Journal of Personality & Social Psychology, 18, 105–115.CrossRefGoogle Scholar
  10. de Kampen, J. K. (2006). The impact of survey methodology and context on central tendency, nonresponse and associations of subjective indicators of government performance. Quality & Quantity, 41, 793–813.CrossRefGoogle Scholar
  11. de Leeuw, E. D. (1992). Data quality in mail, telephone, and face to face surveys. Amsterdam: TT-Publikaties.Google Scholar
  12. Dillman, D. A. (1978). Mail and telephone surveys: The total design method. New York: Wiley.Google Scholar
  13. Dillman, D. A. (2000). Mail and Internet surveys: The tailored design method (2nd ed.). New York: Wiley.Google Scholar
  14. Dillman, D. A., & Bowker, D. K. (2001). The Web questionnaire challenge to survey methodologists. In U.-D. Reips & M. Bošnjak (Eds.), Dimensions of Internet science (pp. 159–178). Lengerich, Germany: Pabst.Google Scholar
  15. Edwards, P., Roberts, I., Clarke, M., DiGuiseppi, C., Pratap, S., Wentz, R., & Kwan, I. (2002). Increasing response rates to postal questionnaires: Systematic review. British Medical Journal, 324, 1183–1185.PubMedCrossRefGoogle Scholar
  16. Fay, R. E. (1986). Causal models for patterns of nonresponse. Journal of the American Statistical Association, 81, 354–365.CrossRefGoogle Scholar
  17. Göritz, A. S. (2005). Contingent versus unconditional incentives in WWW-studies. Metodološki Zvezki—Advances in Methodology & Statistics, 2, 1–14.Google Scholar
  18. Göritz, A. S. (2006a). Cash lotteries as incentives in online panels. Social Science Computer Review, 24, 445–459.CrossRefGoogle Scholar
  19. Göritz, A. S. (2006b). Incentives in Web studies: Methodological issues and a review. International Journal of Internet Science, 1, 58–70.Google Scholar
  20. Göritz, A. S. (2007). Using online panels in psychological research. In A. N. Joinson, K. Y. A. McKenna, T. Postmes, & U.-D. Reips (Eds.), The Oxford handbook of Internet psychology (pp. 473–485). Oxford: Oxford University Press.Google Scholar
  21. Göritz, A. S. (2008). The long-term effect of material incentives on participation in online panels. Field Methods, 20, 211–225.CrossRefGoogle Scholar
  22. Göritz, A. S. (in press). Incentives in Web-based research. In S. Gosling & J. A. Johnson (Eds.), Advanced methods for behavioral research on the Internet}. Washington, DC: American Psychological Association. Göritz, A. S., & Wolff, H.-G. (2007). Lotteries as incentives in longitudinal Web studies. Social Science Computer Review, 25, 99-110.Google Scholar
  23. Göritz, A. S., Wolff, H.-G., & Goldstein, D. G. (2008). Payments via Paypal as an incentive in longitudinal Web-based studies. Manuscript submitted for publication.Google Scholar
  24. Groves, R. M. (1989). Survey errors and survey costs. New York: Wiley.CrossRefGoogle Scholar
  25. Groves, R. M., & Peytcheva, E. (2008). The impact of nonresponse rates on nonresponse bias: A meta-analysis. Public Opinion Quarterly, 72, 167–189.CrossRefGoogle Scholar
  26. Hagenaars, J. A. (1990). Categorical longitudinal data: Log-linear panel, trend, and cohort analysis. Newbury Park, NY: Sage.Google Scholar
  27. Hill, D. H., & Willis, R. J. (2001). Reducing panel attrition: A search for effective policy instruments. Journal of Human Resources, 36, 416–438.CrossRefGoogle Scholar
  28. Hubbard, R., & Little, E. L. (1988). Promised contributions to charity and mail survey responses: Replication with extension. Public Opinion Quarterly, 52, 223–230.CrossRefGoogle Scholar
  29. Raftery, A. E. (1995). Bayesian model selection in social research. Sociological Methodology, 25, 111–163.CrossRefGoogle Scholar
  30. Stoop, I. A. L. (2005). The hunt for the last respondent: Nonresponse in sample surveys. The Hague: Transaction Publishers.Google Scholar
  31. Vermunt, J. K. (1997a). LEM 1.0: A general program for the analysis of categorical data. Tilburg, The Netherlands: Tilburg University. Accessed September 2008 at http: // Scholar
  32. Vermunt, J. K. (1997b). Log-linear models for event histories. Thousand Oaks, CA: Sage.Google Scholar
  33. Yu, J., & Cooper, H. (1983). A quantitative review of research design effects on response rates to questionnaires. Journal of Marketing Research, 20, 36–44.CrossRefGoogle Scholar

Copyright information

© Psychonomic Society, Inc. 2008

Authors and Affiliations

  • Anja S. Göritz
    • 1
  • Hans-Georg Wolff
    • 2
  • Daniel G. Goldstein
    • 3
  1. 1.University of Erlangen-NurembergNurembergGermany
  2. 2.London Business SchoolLondonEngland
  3. 3.Work and Organizational PsychologyWürzburg UniversityWürzburgGermany

Personalised recommendations