Skip to main content
SpringerLink
Log in

The cost-effectiveness of cash versus lottery incentives for a web-based, stated-preference community survey

  • Original Paper
  • Published:
The European Journal of Health Economics Aims and scope Submit manuscript

Abstract

We present the results of a randomized experiment to test the effectiveness and cost-effectiveness of response incentives for a stated-preference survey of a general community population. The survey was administered using a mixed-mode approach, in which community members were invited to participate using a traditional mailed letter using contact information for a representative sample of the community; but individuals completed the survey via the web, which exploited the advantages of electronic capture. Individuals were randomized to four incentive groups: (a) no incentive, (b) prepaid cash incentive ($2), (c) a low lottery (10 prizes of $25) and (d) a high lottery (2 prizes of $250). Letters of invitation were mailed to 3,000 individuals. In total, 405 individuals (14.4%) contacted the website and 277 (9.8%) provided complete responses. The prepaid cash incentive generated the highest contact and response rates (23.3 and 17.3%, respectively), and no incentive generated the lowest (9.1 and 5.7%, respectively). The high lottery, however, was the most cost-effective incentive for obtaining completed surveys: compared with no incentive, the incremental cost-effectiveness ratio (ICER) per completed survey for high lottery was $13.89; for prepaid cash, the ICER was $18.29. This finding suggests that the preferred response incentive for community-based, stated-preference surveys is a lottery with a small number of large prizes.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Notes

  1. Response rates are less well documented for health economic surveys, but the experience of those who regularly conduct stated preference surveys indicates that response rates have been falling in recent years [10].

  2. This includes individuals who may not have access to the Internet. Because we have no way of identifying these individuals, we must assume all those who received the letter were eligible to respond.

References

  1. Ryan, M., Gerard, K., Amaya–Amaya, M.: Using Discrete Choice Experiments to Value Health and Health Care. Springer, Dordrecht (2008)

    Book  Google Scholar 

  2. Hurley, J., Buckley, N., Cuff, K., Giacomini, M., Cameron, D.: Judgments regarding the fair division of goods: the impact of verbal versus quantitative descriptions of alternative principles. Social Choice and Welfare Forthcoming (2010)

  3. Nord, E., Richardson, J., Street, A., Kuhse, H., Singer, P.: Maximizing health benefits vs. Egalitarianism: an Australian survey of health issues. Soc. Sci. Med. 41(10), 1429–1437 (1995)

    Article  PubMed  CAS  Google Scholar 

  4. Dolan, P., Tsuchiya, A.: Health priorities and public preferences: the relative importance of past health experience and future health prospects. J. Health Econ. 24, 703–714 (2005)

    Article  PubMed  Google Scholar 

  5. Sheehan, K.: E-mail survey response rates: A review. J. Comput. Mediat. Commun. 6(2). (2001)

  6. Warriner, K., Goyder, J., Gjersten, H., honhner, P., McSpurren, K.: Charities, no; lotteries, no; cash, yes. Main effects and interactions in a Canadian incentives experiment. Public Opin Q 60, 542–562 (1996)

    Article  Google Scholar 

  7. Bickart, B., Schmittlein, D.: The distribution of survey contact and participation in the United States: constructing a survey-based estimate. J. Mark. Res. 36(2), 286–294 (1999)

    Article  Google Scholar 

  8. Kulka, R., Eyerman, J., McNeeley, M.: The use of monetary incentives in federal surveys on substance use and abuse. J. Econ. Soc. Meas. 30(2–3), 233–249 (2005)

    Google Scholar 

  9. Dixon, J., Tucker, C.: Survey nonresponse. In: Marsden, P., Wright, J. (eds.) Handbook of Survey Research. Emerald Publishing Group, Bingley, UK (2010)

    Google Scholar 

  10. Ryan, M.: Personal E-mail Communication, August 17. (2010)

  11. Bosnjak, M., Tuten, T.: Prepaid and promised incentives in web surveys. Soc. Sci. Comp. Rev. 21(2), 208–217 (2003)

    Article  Google Scholar 

  12. Umbach, P.D.: Web-based surveys: best practices. New Dir. Inst. Res. 121, 23–38 (2004)

    Google Scholar 

  13. Heerwegh, D.: An investigation of the effect of lotteries on web survey response rates. Field Methods 18(2), 205–220 (2006)

    Article  Google Scholar 

  14. Edwards, P., Roberts, I., Clarke, M., DiGuiseppi, C., Pratap, S., Wentz, R., Kwan, I.: Increasing response rates to postal questionnaires: a systematic review. BMJ 324, 1–9 (2002)

    Google Scholar 

  15. Armstrong, J.S.: Monetary incentives in mailed surveys. Public Opin. Q. 39, 111–116 (1975)

    Article  Google Scholar 

  16. Linsky, A.S.: Stimulating responses to mailed questionnaires: a review. Public Opin. Q. 39, 82–101 (1975)

    Article  Google Scholar 

  17. Church, A.H.: Estimating the effect of incentives on mail survey response rates: a meta-analysis. Public Opin. Q. 57, 62–79 (1993)

    Article  Google Scholar 

  18. Leung, G.M., Ho, L.M., Chan, M.F., Johnston, J.M., Wong, F.K.: The effects of cash and lottery incentives on mailed surveys to physicians. A randomized trial. J. Clin. Epidemiol. 55, 801–807 (2002)

    Article  PubMed  Google Scholar 

  19. Porter, S.R., Whitcomb, M.E.: The impact of lottery incentives on student survey response rates. Res. High. Educ. 44(4), 389–407 (2003)

    Article  Google Scholar 

  20. Göritz, A.S.: Incentives in web-based studies: methodological issues and review. Int. J. Internet Sci. 1, 58–70 (2006)

    Google Scholar 

  21. Frickl, A., Bachtiger, M.T., Reips, U.-D.: Financial incentives, personal information and drop-out rate in online studies. In: Reips, U.-D., Bosnjak, M. (eds.) Dimensions of Internet Science, pp. 209–219. Pabst Science Publishers, Berlin (2001)

    Google Scholar 

  22. Duetskens, E., Ruyter, K.D., Wetzels, M., Oosterveld, P.: Response rate and response quality of internet-based surveys: an experimental study. Mark. Lett. 15(1), 21–36 (2004)

    Article  Google Scholar 

  23. Göritz, A.S.: The impact of material incentives on response quality, sample composition, survey income, and cost in online access panels. Int. J. Mark. Res. 46, 327–345 (2004)

    Google Scholar 

  24. Su, J., Shao, S., Fang, J.: Effects of incentives on web-based surveys. Tsinghua Sci. Technol. 13(3), 344–347 (2008)

    Article  Google Scholar 

  25. Birnholtz, J.P., Horn, D.B., Finholt, T.A., Bae, S.J.: The effects of cash, electronic and paper gift certificates for a web-based survey of technologically sophisticated respondents. Soc. Sci. Comp. Rev. 22, 355–362 (2004)

    Article  Google Scholar 

  26. Smith, R.: The discrete-choice willingness-to-pay question format in health economics: should we adopt environmental guidelines. Med. Decis. Mak. 20(2), 194–204 (2000)

    Article  CAS  Google Scholar 

  27. Messick, D.M., McClintock, C.G.: Motivational bases of choice n experimental games. J. Exp. Soc. Psychol. 4, 1–25 (1968)

    Article  Google Scholar 

  28. Hurley, J., Mentzakis, E.: The Existence and Magnitude of Health Care Externalities: Evidence from a Choice Experiment. Hamilton, ON: McMaster University Centre for Health Economics and Policy Analysis Working Paper 11–01. (2011)

  29. Dillman, D.A., Phelps, G., Tortora, R., Swift, K., Kohrell, J., Berck, J., Messer, B.: Mail and Internet Survey: The Tailored Design Method. Wiley, New York (2007)

    Google Scholar 

  30. Offerman, T., Sonnemans, J., Schram, A.: Value orientations, expectations and voluntary contributions in public goods. Econ. J. 106(437), 817–845 (1996)

    Article  Google Scholar 

  31. Buckley, N., Chan, K.S., Chowhan, J., Mestelman, S., Shehata, M.: Value orientations, income and displacement effects, and voluntary contributions. Exp. Econ. 4(2), 183–195 (2001)

    Google Scholar 

  32. Dillman, D.A.: Response rate and measurement differences in mixed mode surveys using mail, telephone, interactive voice response and the internet. Soc. Sci. Res. 38(1), 1–18 (2009)

    Article  Google Scholar 

  33. Manfreda, L., Bosnjak, M., Haas, J., Vehovar, V.: Web surveys versus other survey modes: a meta-analysis comparing response rates. Int. J. Mark. Res. 50, 79–104 (2008)

    Google Scholar 

  34. Statistics Canada: Canadian Internet Use Survey. Statistics Canada, retrieved from http://www.statcan.gc.ca/daily-quotidien/100510a/dq100510a-eng.htm, July 2010, Ottawa (2009)

Download references

Acknowledgments

We acknowledge helpful comments from two anonymous referees, Emmanouil Mentzakis, Neil Buckley, Stuart Mestelman, Andrew Muller, Katherine Cuff, Jingjing Zhang, and David Karp of the Department of Economics and the Experimental Economics Laboratory, McMaster University. Funding: This research was funded by the Canadian Institutes of Health Research (Grant # 76670). We also acknowledge funding from the Ontario Ministry of Health and Long-Term Care to the Centre for Health Economics and Policy Analysis, McMaster University, and the use of resources associated with the McMaster Experimental Economics Laboratory. This study was reviewed and approved by the McMaster University Research Ethics Board. The views expressed are those of the authors alone.

Conflicts of interest

No conflicts to declare.

Author information

Authors and Affiliations

  1. Department of Economics, McMaster University, KTH 430, 1280 Main Street West, Hamilton, ON, L8S 4M4, Canada

    Aleksandra Gajic, David Cameron & Jeremiah Hurley

  2. Centre for Health Economics and Policy Analysis, McMaster University, Hamilton, Canada

    Aleksandra Gajic, David Cameron & Jeremiah Hurley

  3. Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, Canada

    Jeremiah Hurley

Authors
  1. Aleksandra Gajic
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. David Cameron
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jeremiah Hurley
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jeremiah Hurley.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Gajic, A., Cameron, D. & Hurley, J. The cost-effectiveness of cash versus lottery incentives for a web-based, stated-preference community survey. Eur J Health Econ 13, 789–799 (2012). https://doi.org/10.1007/s10198-011-0332-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10198-011-0332-0

Keywords

JEL Codes

Search

Navigation