Abstract
Does it pay to pay online panel members? A three-wave longitudinal experiment was conducted with an online panel to examine whether per person payments, paid through an online intermediary, influence response and retention rates. In the payment condition, participants were promised payment for participation at each wave, whereas control participants were not offered any payment. The promise of a payment had a negative effect on response in Wave 1, but a positive effect on response in Wave 2. Payment had no significant effect on retention. Completing a given wave was an indicator for responding to a subsequent invitation.
Article PDF
Similar content being viewed by others
References
Bailar, B. A. (1984). The quality of survey data. Proceedings of the Section on Survey Research Methods of the American Statistical Association, 43–52.
Batinic, B., & Moser, K. (2005). Determinanten der Rücklaufquote in Online-Panels [Determinants of response in online panels]. Zeitschrift für Medienpsychologie, 17, 64–74.
Biswas, D., & Biswas, A. (2004). The diagnostic role of signals in the context of perceived risks in online shopping: Do signals matter more on the Web? Journal of Interactive Marketing, 18, 30–45.
Bošnjak, M., & Tuten, T. L. (2003). Prepaid and promised incentives in Web surveys: An experiment. Social Science Computer Review, 21, 208–217.
Brennan, M., & Hoek, J. (1992). The behavior of respondents, nonrespondents, and refusers across mail surveys. Public Opinion Quarterly, 56, 530–535.
Church, A. H. (1993). Estimating the effect of incentives on mail survey response rates: A meta-analysis. Public Opinion Quarterly, 57, 62–79.
Cohen, J. (1987). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum.
Couper, M. P. (2000). Web surveys: A review of issues and approaches. Public Opinion Quarterly, 64, 464–494.
Deci, E. L. (1971). Effects of externally mediated rewards on intrinsic motivation. Journal of Personality & Social Psychology, 18, 105–115.
de Kampen, J. K. (2006). The impact of survey methodology and context on central tendency, nonresponse and associations of subjective indicators of government performance. Quality & Quantity, 41, 793–813.
de Leeuw, E. D. (1992). Data quality in mail, telephone, and face to face surveys. Amsterdam: TT-Publikaties.
Dillman, D. A. (1978). Mail and telephone surveys: The total design method. New York: Wiley.
Dillman, D. A. (2000). Mail and Internet surveys: The tailored design method (2nd ed.). New York: Wiley.
Dillman, D. A., & Bowker, D. K. (2001). The Web questionnaire challenge to survey methodologists. In U.-D. Reips & M. Bošnjak (Eds.), Dimensions of Internet science (pp. 159–178). Lengerich, Germany: Pabst.
Edwards, P., Roberts, I., Clarke, M., DiGuiseppi, C., Pratap, S., Wentz, R., & Kwan, I. (2002). Increasing response rates to postal questionnaires: Systematic review. British Medical Journal, 324, 1183–1185.
Fay, R. E. (1986). Causal models for patterns of nonresponse. Journal of the American Statistical Association, 81, 354–365.
Göritz, A. S. (2005). Contingent versus unconditional incentives in WWW-studies. Metodološki Zvezki—Advances in Methodology & Statistics, 2, 1–14.
Göritz, A. S. (2006a). Cash lotteries as incentives in online panels. Social Science Computer Review, 24, 445–459.
Göritz, A. S. (2006b). Incentives in Web studies: Methodological issues and a review. International Journal of Internet Science, 1, 58–70.
Göritz, A. S. (2007). Using online panels in psychological research. In A. N. Joinson, K. Y. A. McKenna, T. Postmes, & U.-D. Reips (Eds.), The Oxford handbook of Internet psychology (pp. 473–485). Oxford: Oxford University Press.
Göritz, A. S. (2008). The long-term effect of material incentives on participation in online panels. Field Methods, 20, 211–225.
Göritz, A. S. (in press). Incentives in Web-based research. In S. Gosling & J. A. Johnson (Eds.), Advanced methods for behavioral research on the Internet}. Washington, DC: American Psychological Association. Göritz, A. S., & Wolff, H.-G. (2007). Lotteries as incentives in longitudinal Web studies. Social Science Computer Review, 25, 99-110.
Göritz, A. S., Wolff, H.-G., & Goldstein, D. G. (2008). Payments via Paypal as an incentive in longitudinal Web-based studies. Manuscript submitted for publication.
Groves, R. M. (1989). Survey errors and survey costs. New York: Wiley.
Groves, R. M., & Peytcheva, E. (2008). The impact of nonresponse rates on nonresponse bias: A meta-analysis. Public Opinion Quarterly, 72, 167–189.
Hagenaars, J. A. (1990). Categorical longitudinal data: Log-linear panel, trend, and cohort analysis. Newbury Park, NY: Sage.
Hill, D. H., & Willis, R. J. (2001). Reducing panel attrition: A search for effective policy instruments. Journal of Human Resources, 36, 416–438.
Hubbard, R., & Little, E. L. (1988). Promised contributions to charity and mail survey responses: Replication with extension. Public Opinion Quarterly, 52, 223–230.
Raftery, A. E. (1995). Bayesian model selection in social research. Sociological Methodology, 25, 111–163.
Stoop, I. A. L. (2005). The hunt for the last respondent: Nonresponse in sample surveys. The Hague: Transaction Publishers.
Vermunt, J. K. (1997a). LEM 1.0: A general program for the analysis of categorical data. Tilburg, The Netherlands: Tilburg University. Accessed September 2008 at http: //spitswww.uvt.nl/~vermunt/.
Vermunt, J. K. (1997b). Log-linear models for event histories. Thousand Oaks, CA: Sage.
Yu, J., & Cooper, H. (1983). A quantitative review of research design effects on response rates to questionnaires. Journal of Marketing Research, 20, 36–44.
Author information
Authors and Affiliations
Corresponding author
Additional information
This work was supported by a University of Erlangen-Nuremberg postdoctoral scholarship (HWP) to the first author.
Rights and permissions
About this article
Cite this article
Göritz, A.S., Wolff, HG. & Goldstein, D.G. Individual payments as a longer-term incentive in online panels. Behavior Research Methods 40, 1144–1149 (2008). https://doi.org/10.3758/BRM.40.4.1144
Received:
Accepted:
Issue Date:
DOI: https://doi.org/10.3758/BRM.40.4.1144