Abstract
We present the results of a randomized experiment to test the effectiveness and cost-effectiveness of response incentives for a stated-preference survey of a general community population. The survey was administered using a mixed-mode approach, in which community members were invited to participate using a traditional mailed letter using contact information for a representative sample of the community; but individuals completed the survey via the web, which exploited the advantages of electronic capture. Individuals were randomized to four incentive groups: (a) no incentive, (b) prepaid cash incentive ($2), (c) a low lottery (10 prizes of $25) and (d) a high lottery (2 prizes of $250). Letters of invitation were mailed to 3,000 individuals. In total, 405 individuals (14.4%) contacted the website and 277 (9.8%) provided complete responses. The prepaid cash incentive generated the highest contact and response rates (23.3 and 17.3%, respectively), and no incentive generated the lowest (9.1 and 5.7%, respectively). The high lottery, however, was the most cost-effective incentive for obtaining completed surveys: compared with no incentive, the incremental cost-effectiveness ratio (ICER) per completed survey for high lottery was $13.89; for prepaid cash, the ICER was $18.29. This finding suggests that the preferred response incentive for community-based, stated-preference surveys is a lottery with a small number of large prizes.
Similar content being viewed by others
Notes
Response rates are less well documented for health economic surveys, but the experience of those who regularly conduct stated preference surveys indicates that response rates have been falling in recent years [10].
This includes individuals who may not have access to the Internet. Because we have no way of identifying these individuals, we must assume all those who received the letter were eligible to respond.
References
Ryan, M., Gerard, K., Amaya–Amaya, M.: Using Discrete Choice Experiments to Value Health and Health Care. Springer, Dordrecht (2008)
Hurley, J., Buckley, N., Cuff, K., Giacomini, M., Cameron, D.: Judgments regarding the fair division of goods: the impact of verbal versus quantitative descriptions of alternative principles. Social Choice and Welfare Forthcoming (2010)
Nord, E., Richardson, J., Street, A., Kuhse, H., Singer, P.: Maximizing health benefits vs. Egalitarianism: an Australian survey of health issues. Soc. Sci. Med. 41(10), 1429–1437 (1995)
Dolan, P., Tsuchiya, A.: Health priorities and public preferences: the relative importance of past health experience and future health prospects. J. Health Econ. 24, 703–714 (2005)
Sheehan, K.: E-mail survey response rates: A review. J. Comput. Mediat. Commun. 6(2). (2001)
Warriner, K., Goyder, J., Gjersten, H., honhner, P., McSpurren, K.: Charities, no; lotteries, no; cash, yes. Main effects and interactions in a Canadian incentives experiment. Public Opin Q 60, 542–562 (1996)
Bickart, B., Schmittlein, D.: The distribution of survey contact and participation in the United States: constructing a survey-based estimate. J. Mark. Res. 36(2), 286–294 (1999)
Kulka, R., Eyerman, J., McNeeley, M.: The use of monetary incentives in federal surveys on substance use and abuse. J. Econ. Soc. Meas. 30(2–3), 233–249 (2005)
Dixon, J., Tucker, C.: Survey nonresponse. In: Marsden, P., Wright, J. (eds.) Handbook of Survey Research. Emerald Publishing Group, Bingley, UK (2010)
Ryan, M.: Personal E-mail Communication, August 17. (2010)
Bosnjak, M., Tuten, T.: Prepaid and promised incentives in web surveys. Soc. Sci. Comp. Rev. 21(2), 208–217 (2003)
Umbach, P.D.: Web-based surveys: best practices. New Dir. Inst. Res. 121, 23–38 (2004)
Heerwegh, D.: An investigation of the effect of lotteries on web survey response rates. Field Methods 18(2), 205–220 (2006)
Edwards, P., Roberts, I., Clarke, M., DiGuiseppi, C., Pratap, S., Wentz, R., Kwan, I.: Increasing response rates to postal questionnaires: a systematic review. BMJ 324, 1–9 (2002)
Armstrong, J.S.: Monetary incentives in mailed surveys. Public Opin. Q. 39, 111–116 (1975)
Linsky, A.S.: Stimulating responses to mailed questionnaires: a review. Public Opin. Q. 39, 82–101 (1975)
Church, A.H.: Estimating the effect of incentives on mail survey response rates: a meta-analysis. Public Opin. Q. 57, 62–79 (1993)
Leung, G.M., Ho, L.M., Chan, M.F., Johnston, J.M., Wong, F.K.: The effects of cash and lottery incentives on mailed surveys to physicians. A randomized trial. J. Clin. Epidemiol. 55, 801–807 (2002)
Porter, S.R., Whitcomb, M.E.: The impact of lottery incentives on student survey response rates. Res. High. Educ. 44(4), 389–407 (2003)
Göritz, A.S.: Incentives in web-based studies: methodological issues and review. Int. J. Internet Sci. 1, 58–70 (2006)
Frickl, A., Bachtiger, M.T., Reips, U.-D.: Financial incentives, personal information and drop-out rate in online studies. In: Reips, U.-D., Bosnjak, M. (eds.) Dimensions of Internet Science, pp. 209–219. Pabst Science Publishers, Berlin (2001)
Duetskens, E., Ruyter, K.D., Wetzels, M., Oosterveld, P.: Response rate and response quality of internet-based surveys: an experimental study. Mark. Lett. 15(1), 21–36 (2004)
Göritz, A.S.: The impact of material incentives on response quality, sample composition, survey income, and cost in online access panels. Int. J. Mark. Res. 46, 327–345 (2004)
Su, J., Shao, S., Fang, J.: Effects of incentives on web-based surveys. Tsinghua Sci. Technol. 13(3), 344–347 (2008)
Birnholtz, J.P., Horn, D.B., Finholt, T.A., Bae, S.J.: The effects of cash, electronic and paper gift certificates for a web-based survey of technologically sophisticated respondents. Soc. Sci. Comp. Rev. 22, 355–362 (2004)
Smith, R.: The discrete-choice willingness-to-pay question format in health economics: should we adopt environmental guidelines. Med. Decis. Mak. 20(2), 194–204 (2000)
Messick, D.M., McClintock, C.G.: Motivational bases of choice n experimental games. J. Exp. Soc. Psychol. 4, 1–25 (1968)
Hurley, J., Mentzakis, E.: The Existence and Magnitude of Health Care Externalities: Evidence from a Choice Experiment. Hamilton, ON: McMaster University Centre for Health Economics and Policy Analysis Working Paper 11–01. (2011)
Dillman, D.A., Phelps, G., Tortora, R., Swift, K., Kohrell, J., Berck, J., Messer, B.: Mail and Internet Survey: The Tailored Design Method. Wiley, New York (2007)
Offerman, T., Sonnemans, J., Schram, A.: Value orientations, expectations and voluntary contributions in public goods. Econ. J. 106(437), 817–845 (1996)
Buckley, N., Chan, K.S., Chowhan, J., Mestelman, S., Shehata, M.: Value orientations, income and displacement effects, and voluntary contributions. Exp. Econ. 4(2), 183–195 (2001)
Dillman, D.A.: Response rate and measurement differences in mixed mode surveys using mail, telephone, interactive voice response and the internet. Soc. Sci. Res. 38(1), 1–18 (2009)
Manfreda, L., Bosnjak, M., Haas, J., Vehovar, V.: Web surveys versus other survey modes: a meta-analysis comparing response rates. Int. J. Mark. Res. 50, 79–104 (2008)
Statistics Canada: Canadian Internet Use Survey. Statistics Canada, retrieved from http://www.statcan.gc.ca/daily-quotidien/100510a/dq100510a-eng.htm, July 2010, Ottawa (2009)
Acknowledgments
We acknowledge helpful comments from two anonymous referees, Emmanouil Mentzakis, Neil Buckley, Stuart Mestelman, Andrew Muller, Katherine Cuff, Jingjing Zhang, and David Karp of the Department of Economics and the Experimental Economics Laboratory, McMaster University. Funding: This research was funded by the Canadian Institutes of Health Research (Grant # 76670). We also acknowledge funding from the Ontario Ministry of Health and Long-Term Care to the Centre for Health Economics and Policy Analysis, McMaster University, and the use of resources associated with the McMaster Experimental Economics Laboratory. This study was reviewed and approved by the McMaster University Research Ethics Board. The views expressed are those of the authors alone.
Conflicts of interest
No conflicts to declare.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Gajic, A., Cameron, D. & Hurley, J. The cost-effectiveness of cash versus lottery incentives for a web-based, stated-preference community survey. Eur J Health Econ 13, 789–799 (2012). https://doi.org/10.1007/s10198-011-0332-0
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10198-011-0332-0