Advertisement

The European Journal of Health Economics

, Volume 13, Issue 6, pp 789–799 | Cite as

The cost-effectiveness of cash versus lottery incentives for a web-based, stated-preference community survey

  • Aleksandra Gajic
  • David Cameron
  • Jeremiah Hurley
Original Paper

Abstract

We present the results of a randomized experiment to test the effectiveness and cost-effectiveness of response incentives for a stated-preference survey of a general community population. The survey was administered using a mixed-mode approach, in which community members were invited to participate using a traditional mailed letter using contact information for a representative sample of the community; but individuals completed the survey via the web, which exploited the advantages of electronic capture. Individuals were randomized to four incentive groups: (a) no incentive, (b) prepaid cash incentive ($2), (c) a low lottery (10 prizes of $25) and (d) a high lottery (2 prizes of $250). Letters of invitation were mailed to 3,000 individuals. In total, 405 individuals (14.4%) contacted the website and 277 (9.8%) provided complete responses. The prepaid cash incentive generated the highest contact and response rates (23.3 and 17.3%, respectively), and no incentive generated the lowest (9.1 and 5.7%, respectively). The high lottery, however, was the most cost-effective incentive for obtaining completed surveys: compared with no incentive, the incremental cost-effectiveness ratio (ICER) per completed survey for high lottery was $13.89; for prepaid cash, the ICER was $18.29. This finding suggests that the preferred response incentive for community-based, stated-preference surveys is a lottery with a small number of large prizes.

Keywords

Stated-preference survey Discrete-choice survey Response incentives 

JEL Codes

I10 C83 C90 

Notes

Acknowledgments

We acknowledge helpful comments from two anonymous referees, Emmanouil Mentzakis, Neil Buckley, Stuart Mestelman, Andrew Muller, Katherine Cuff, Jingjing Zhang, and David Karp of the Department of Economics and the Experimental Economics Laboratory, McMaster University. Funding: This research was funded by the Canadian Institutes of Health Research (Grant # 76670). We also acknowledge funding from the Ontario Ministry of Health and Long-Term Care to the Centre for Health Economics and Policy Analysis, McMaster University, and the use of resources associated with the McMaster Experimental Economics Laboratory. This study was reviewed and approved by the McMaster University Research Ethics Board. The views expressed are those of the authors alone.

Conflicts of interest

No conflicts to declare.

References

  1. 1.
    Ryan, M., Gerard, K., Amaya–Amaya, M.: Using Discrete Choice Experiments to Value Health and Health Care. Springer, Dordrecht (2008)CrossRefGoogle Scholar
  2. 2.
    Hurley, J., Buckley, N., Cuff, K., Giacomini, M., Cameron, D.: Judgments regarding the fair division of goods: the impact of verbal versus quantitative descriptions of alternative principles. Social Choice and Welfare Forthcoming (2010)Google Scholar
  3. 3.
    Nord, E., Richardson, J., Street, A., Kuhse, H., Singer, P.: Maximizing health benefits vs. Egalitarianism: an Australian survey of health issues. Soc. Sci. Med. 41(10), 1429–1437 (1995)PubMedCrossRefGoogle Scholar
  4. 4.
    Dolan, P., Tsuchiya, A.: Health priorities and public preferences: the relative importance of past health experience and future health prospects. J. Health Econ. 24, 703–714 (2005)PubMedCrossRefGoogle Scholar
  5. 5.
    Sheehan, K.: E-mail survey response rates: A review. J. Comput. Mediat. Commun. 6(2). (2001)Google Scholar
  6. 6.
    Warriner, K., Goyder, J., Gjersten, H., honhner, P., McSpurren, K.: Charities, no; lotteries, no; cash, yes. Main effects and interactions in a Canadian incentives experiment. Public Opin Q 60, 542–562 (1996)CrossRefGoogle Scholar
  7. 7.
    Bickart, B., Schmittlein, D.: The distribution of survey contact and participation in the United States: constructing a survey-based estimate. J. Mark. Res. 36(2), 286–294 (1999)CrossRefGoogle Scholar
  8. 8.
    Kulka, R., Eyerman, J., McNeeley, M.: The use of monetary incentives in federal surveys on substance use and abuse. J. Econ. Soc. Meas. 30(2–3), 233–249 (2005)Google Scholar
  9. 9.
    Dixon, J., Tucker, C.: Survey nonresponse. In: Marsden, P., Wright, J. (eds.) Handbook of Survey Research. Emerald Publishing Group, Bingley, UK (2010)Google Scholar
  10. 10.
    Ryan, M.: Personal E-mail Communication, August 17. (2010)Google Scholar
  11. 11.
    Bosnjak, M., Tuten, T.: Prepaid and promised incentives in web surveys. Soc. Sci. Comp. Rev. 21(2), 208–217 (2003)CrossRefGoogle Scholar
  12. 12.
    Umbach, P.D.: Web-based surveys: best practices. New Dir. Inst. Res. 121, 23–38 (2004)Google Scholar
  13. 13.
    Heerwegh, D.: An investigation of the effect of lotteries on web survey response rates. Field Methods 18(2), 205–220 (2006)CrossRefGoogle Scholar
  14. 14.
    Edwards, P., Roberts, I., Clarke, M., DiGuiseppi, C., Pratap, S., Wentz, R., Kwan, I.: Increasing response rates to postal questionnaires: a systematic review. BMJ 324, 1–9 (2002)Google Scholar
  15. 15.
    Armstrong, J.S.: Monetary incentives in mailed surveys. Public Opin. Q. 39, 111–116 (1975)CrossRefGoogle Scholar
  16. 16.
    Linsky, A.S.: Stimulating responses to mailed questionnaires: a review. Public Opin. Q. 39, 82–101 (1975)CrossRefGoogle Scholar
  17. 17.
    Church, A.H.: Estimating the effect of incentives on mail survey response rates: a meta-analysis. Public Opin. Q. 57, 62–79 (1993)CrossRefGoogle Scholar
  18. 18.
    Leung, G.M., Ho, L.M., Chan, M.F., Johnston, J.M., Wong, F.K.: The effects of cash and lottery incentives on mailed surveys to physicians. A randomized trial. J. Clin. Epidemiol. 55, 801–807 (2002)PubMedCrossRefGoogle Scholar
  19. 19.
    Porter, S.R., Whitcomb, M.E.: The impact of lottery incentives on student survey response rates. Res. High. Educ. 44(4), 389–407 (2003)CrossRefGoogle Scholar
  20. 20.
    Göritz, A.S.: Incentives in web-based studies: methodological issues and review. Int. J. Internet Sci. 1, 58–70 (2006)Google Scholar
  21. 21.
    Frickl, A., Bachtiger, M.T., Reips, U.-D.: Financial incentives, personal information and drop-out rate in online studies. In: Reips, U.-D., Bosnjak, M. (eds.) Dimensions of Internet Science, pp. 209–219. Pabst Science Publishers, Berlin (2001)Google Scholar
  22. 22.
    Duetskens, E., Ruyter, K.D., Wetzels, M., Oosterveld, P.: Response rate and response quality of internet-based surveys: an experimental study. Mark. Lett. 15(1), 21–36 (2004)CrossRefGoogle Scholar
  23. 23.
    Göritz, A.S.: The impact of material incentives on response quality, sample composition, survey income, and cost in online access panels. Int. J. Mark. Res. 46, 327–345 (2004)Google Scholar
  24. 24.
    Su, J., Shao, S., Fang, J.: Effects of incentives on web-based surveys. Tsinghua Sci. Technol. 13(3), 344–347 (2008)CrossRefGoogle Scholar
  25. 25.
    Birnholtz, J.P., Horn, D.B., Finholt, T.A., Bae, S.J.: The effects of cash, electronic and paper gift certificates for a web-based survey of technologically sophisticated respondents. Soc. Sci. Comp. Rev. 22, 355–362 (2004)CrossRefGoogle Scholar
  26. 26.
    Smith, R.: The discrete-choice willingness-to-pay question format in health economics: should we adopt environmental guidelines. Med. Decis. Mak. 20(2), 194–204 (2000)CrossRefGoogle Scholar
  27. 27.
    Messick, D.M., McClintock, C.G.: Motivational bases of choice n experimental games. J. Exp. Soc. Psychol. 4, 1–25 (1968)CrossRefGoogle Scholar
  28. 28.
    Hurley, J., Mentzakis, E.: The Existence and Magnitude of Health Care Externalities: Evidence from a Choice Experiment. Hamilton, ON: McMaster University Centre for Health Economics and Policy Analysis Working Paper 11–01. (2011)Google Scholar
  29. 29.
    Dillman, D.A., Phelps, G., Tortora, R., Swift, K., Kohrell, J., Berck, J., Messer, B.: Mail and Internet Survey: The Tailored Design Method. Wiley, New York (2007)Google Scholar
  30. 30.
    Offerman, T., Sonnemans, J., Schram, A.: Value orientations, expectations and voluntary contributions in public goods. Econ. J. 106(437), 817–845 (1996)CrossRefGoogle Scholar
  31. 31.
    Buckley, N., Chan, K.S., Chowhan, J., Mestelman, S., Shehata, M.: Value orientations, income and displacement effects, and voluntary contributions. Exp. Econ. 4(2), 183–195 (2001)Google Scholar
  32. 32.
    Dillman, D.A.: Response rate and measurement differences in mixed mode surveys using mail, telephone, interactive voice response and the internet. Soc. Sci. Res. 38(1), 1–18 (2009)CrossRefGoogle Scholar
  33. 33.
    Manfreda, L., Bosnjak, M., Haas, J., Vehovar, V.: Web surveys versus other survey modes: a meta-analysis comparing response rates. Int. J. Mark. Res. 50, 79–104 (2008)Google Scholar
  34. 34.
    Statistics Canada: Canadian Internet Use Survey. Statistics Canada, retrieved from http://www.statcan.gc.ca/daily-quotidien/100510a/dq100510a-eng.htm, July 2010, Ottawa (2009)

Copyright information

© Springer-Verlag 2011

Authors and Affiliations

  • Aleksandra Gajic
    • 1
    • 2
  • David Cameron
    • 1
    • 2
  • Jeremiah Hurley
    • 1
    • 2
    • 3
  1. 1.Department of EconomicsMcMaster UniversityHamiltonCanada
  2. 2.Centre for Health Economics and Policy AnalysisMcMaster UniversityHamiltonCanada
  3. 3.Department of Clinical Epidemiology and BiostatisticsMcMaster UniversityHamiltonCanada

Personalised recommendations