The cost-effectiveness of cash versus lottery incentives for a web-based, stated-preference community survey
- 653 Downloads
We present the results of a randomized experiment to test the effectiveness and cost-effectiveness of response incentives for a stated-preference survey of a general community population. The survey was administered using a mixed-mode approach, in which community members were invited to participate using a traditional mailed letter using contact information for a representative sample of the community; but individuals completed the survey via the web, which exploited the advantages of electronic capture. Individuals were randomized to four incentive groups: (a) no incentive, (b) prepaid cash incentive ($2), (c) a low lottery (10 prizes of $25) and (d) a high lottery (2 prizes of $250). Letters of invitation were mailed to 3,000 individuals. In total, 405 individuals (14.4%) contacted the website and 277 (9.8%) provided complete responses. The prepaid cash incentive generated the highest contact and response rates (23.3 and 17.3%, respectively), and no incentive generated the lowest (9.1 and 5.7%, respectively). The high lottery, however, was the most cost-effective incentive for obtaining completed surveys: compared with no incentive, the incremental cost-effectiveness ratio (ICER) per completed survey for high lottery was $13.89; for prepaid cash, the ICER was $18.29. This finding suggests that the preferred response incentive for community-based, stated-preference surveys is a lottery with a small number of large prizes.
KeywordsStated-preference survey Discrete-choice survey Response incentives
JEL CodesI10 C83 C90
We acknowledge helpful comments from two anonymous referees, Emmanouil Mentzakis, Neil Buckley, Stuart Mestelman, Andrew Muller, Katherine Cuff, Jingjing Zhang, and David Karp of the Department of Economics and the Experimental Economics Laboratory, McMaster University. Funding: This research was funded by the Canadian Institutes of Health Research (Grant # 76670). We also acknowledge funding from the Ontario Ministry of Health and Long-Term Care to the Centre for Health Economics and Policy Analysis, McMaster University, and the use of resources associated with the McMaster Experimental Economics Laboratory. This study was reviewed and approved by the McMaster University Research Ethics Board. The views expressed are those of the authors alone.
Conflicts of interest
No conflicts to declare.
- 2.Hurley, J., Buckley, N., Cuff, K., Giacomini, M., Cameron, D.: Judgments regarding the fair division of goods: the impact of verbal versus quantitative descriptions of alternative principles. Social Choice and Welfare Forthcoming (2010)Google Scholar
- 5.Sheehan, K.: E-mail survey response rates: A review. J. Comput. Mediat. Commun. 6(2). (2001)Google Scholar
- 8.Kulka, R., Eyerman, J., McNeeley, M.: The use of monetary incentives in federal surveys on substance use and abuse. J. Econ. Soc. Meas. 30(2–3), 233–249 (2005)Google Scholar
- 9.Dixon, J., Tucker, C.: Survey nonresponse. In: Marsden, P., Wright, J. (eds.) Handbook of Survey Research. Emerald Publishing Group, Bingley, UK (2010)Google Scholar
- 10.Ryan, M.: Personal E-mail Communication, August 17. (2010)Google Scholar
- 12.Umbach, P.D.: Web-based surveys: best practices. New Dir. Inst. Res. 121, 23–38 (2004)Google Scholar
- 14.Edwards, P., Roberts, I., Clarke, M., DiGuiseppi, C., Pratap, S., Wentz, R., Kwan, I.: Increasing response rates to postal questionnaires: a systematic review. BMJ 324, 1–9 (2002)Google Scholar
- 20.Göritz, A.S.: Incentives in web-based studies: methodological issues and review. Int. J. Internet Sci. 1, 58–70 (2006)Google Scholar
- 21.Frickl, A., Bachtiger, M.T., Reips, U.-D.: Financial incentives, personal information and drop-out rate in online studies. In: Reips, U.-D., Bosnjak, M. (eds.) Dimensions of Internet Science, pp. 209–219. Pabst Science Publishers, Berlin (2001)Google Scholar
- 23.Göritz, A.S.: The impact of material incentives on response quality, sample composition, survey income, and cost in online access panels. Int. J. Mark. Res. 46, 327–345 (2004)Google Scholar
- 28.Hurley, J., Mentzakis, E.: The Existence and Magnitude of Health Care Externalities: Evidence from a Choice Experiment. Hamilton, ON: McMaster University Centre for Health Economics and Policy Analysis Working Paper 11–01. (2011)Google Scholar
- 29.Dillman, D.A., Phelps, G., Tortora, R., Swift, K., Kohrell, J., Berck, J., Messer, B.: Mail and Internet Survey: The Tailored Design Method. Wiley, New York (2007)Google Scholar
- 31.Buckley, N., Chan, K.S., Chowhan, J., Mestelman, S., Shehata, M.: Value orientations, income and displacement effects, and voluntary contributions. Exp. Econ. 4(2), 183–195 (2001)Google Scholar
- 33.Manfreda, L., Bosnjak, M., Haas, J., Vehovar, V.: Web surveys versus other survey modes: a meta-analysis comparing response rates. Int. J. Mark. Res. 50, 79–104 (2008)Google Scholar
- 34.Statistics Canada: Canadian Internet Use Survey. Statistics Canada, retrieved from http://www.statcan.gc.ca/daily-quotidien/100510a/dq100510a-eng.htm, July 2010, Ottawa (2009)