Abstract
We report the results of a randomized experiment conducted in Japan as part of a mail survey of voters. In this experiment, we examined whether or not inserting a pen in the questionnaire would improve the response rate. We employed a two-by-two treatment in which we varied the price of the pen (high-priced or low-priced) and whether or not the pen was inserted in a paper-made box. We found that providing a pen increased the response rate by more than nine percentage points. We also found that a low-priced pen inserted in a box was more effective than other condisions. However, the effect of the high-priced pen was not statistically distinguishable from that of the low-priced one.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Ando, N. (2009). Chōsa Gaisha no Kakaeru Kadai. [Challenges Research Firms Face] Shakai to Chōsa, 3, 65–71.
Brennan, M., & Charbonneau J. (2009). Improving Mail Survey Response Rates Using Chocolate and Replacement Questionnaires. Public Opinion Quarterly, 73(2), 368–78.
Church, A. H. (1993). Estimating the Effect of Incentives on Mail Survey Response Rates: A Meta-Analysis. Public Opinion Quarterly, 57(1), 62–79.
Cialdini, R. B. (2007). Influence: The Psychology of Persuasion. Revised Edition. New York: HarperCollins.
Clark, T. J., Khan, K. S., & Gupta, J. K. (2001). Provision of Pen Along with Questionnaire Does Not Increase the Response Rate to a Postal Survey: A Randomised Controlled Trial. Journal of Epidemiology and Community Health, 55(8), 595–596.
Dillman, D. A. (1991). The Design and Administration of Mail Surveys. Annual Review of Sociology, 17, 225–249.
Edwards, P., Roberts, I., Clarke M., DiGuiseppi, C., Pratap, S., Wentz, R., & Kwan, I. (2002). Increasing Response Rates to Postal Questionnaires: Systematic Review. BMJ, 324(7347), 1183.
Gerber, A. S., & Green, D. P. (2012). Field Experiments: Design, Analysis, and Interpretation. New York: W. W. Norton.
Gonzalez-Ocantos, E., Kiewiet, C., Jonge, D., Meléndez, C., Osorio, J., & Nickerson, D. W. (2012). Vote Buying and Social Desirability Bias: Experimental Evidence from Nicaragua. American Journal of Political Science, 56(1), 202–17.
Hagihara, G., Ota, H., & Fujii, S. (2006). Ankēto Chōsa Kaishūritsu ni Kansuru Jikken Kenkyu (An Experimental Study about Survey Response Rate: Basic Study of Efficient Strategies to Raise the Participation Rate of Mobility Management). Doboku Keikaku Kenkyu Ronbunshu, 23(1), 117-23 (in Japanese).
Hayashi, H. (2006). Yūsō Chōsaho (Mail Survey Methodology). Osaka: Kansai Daigaku Shup-panbu (in Japanese).
Hayashi, H. (2010). Yūsō Chōsaho — Aratana Jidai no Shuryoku Shuhō to Narieruka (Reappraising Mail Surveys and Current Topics). Kōdō Keiryōgaku, 37(2), 127-45 (in Japanese).
Jobber, D., Saunders, J., & Mitchell, V.-W. (2004). Prepaid Monetary Incentive Effects on Mail Survey Response. Journal of Business Research, 57(1), 21–25.
Kanuk, L., & Berenson, C. (1975). Mail Surveys and Response Rates: A Literature Review. Journal of Marketing Research, 12(4), 440–453.
Kaplowitz, M. D., Hadlock, T. D., & Levine, R. (2004). A Comparison of Web and Mail Survey Response Rates. Public Opinion Quarterly, 68(1), 94–101.
Larson, P. D., & Poist, R. F. (2004). Improving Response Rates to Mail Surveys: A Research Note. Transportation Journal, 43(4), 67–74.
Matsuda, E. (2006). Chōsa o Meguru Konnichi teki Kadai to Tenbō ≈ Denwa (Rdd) ho no genkai, Mensetsuhō no Sanjō, Yūsōho no Saihyōka, Sosite Internet ha... ≈ (Problems and Prospects of Survey ≈ Limitation of Telephone Survey (RDD), Disastrous Situation of Face-to-Face Interview, Reevaluation of Mail Survey, and Internet). Shin Joho, 94, 8–17 (in Japanese).
Matsuda, E. (2008). Yūsō Chōsa no Kōyō to Kanōsei (Usefulness and Capabilities of Mail Surveys). Kōdō Keiryōgaku, 35(1), 17–45 (in Japanese).
Moore, D. L., & Tarnai, J. (2002). Evaluating Nonresponse Error in Mail Surveys. in Survey Nonresponse, eds. Groves, R. M., Dillman, D. A., Eltinge, J. L. & Little, R. J. A., New York: Wiley.
Regan, D. T. (1971). Effects of a Favor and Liking on Compliance. Journal of Experimental Social Psychology, 7(6), 627—639.
Sharp, L., Cochran, C., Cotton, S. C., Gray, N. M., & Gallagher, M. E. (2006). Enclosing a Pen with a Postal Questionnaire Can Significantly Increase the Response Rate. Journal of Clinical Epidemiology, 59(7), 747–54.
Singer, E. (2002). The Use of Incentives to Reduce Nonresponse in Household Surveys. in Survey Nonresponse, eds. Groves, R. M., Dillman, D. A., Eltinge, J. L. & Little, R. J. A., New York: Wiley.
Tomz, M., Wittenberg, J., & King, G. (2003). CLARIFY: Software for Interpreting and Presenting Statistical Results. Version 2.1. Cambridge, MA: Harvard University. http://gking.harvard.edu
Tourangeau, R., & Yan, T. (2007). Sensitive Questions in Surveys. Psychological Bulletin, 133(5), 859–883.
Willimack, D. K., Schuman, H., Pennell, B.-E., & Lepkowski, J. M. (1995). Effects of a Prepaid Nonmonetary Incentive on Response Rates and Response Quality in a Face-to-Face Survey. Public Opinion Quarterly, 59(1), 78–92.
Yammarino, F. J., Skinner, S. J., & Childers, T. L. (1991). Understanding Mail Survey Response Behavior a Meta-Analysis. Public Opinion Quarterly, 55(4), 613–39.
Yu, J., & Cooper, H. (1983). A Quantitative Review of Research Design Effects on Response Rates to Questionnaires. Journal of Marketing Research, 20(1), 36–44.
Author information
Authors and Affiliations
Corresponding author
About this article
Cite this article
Arai, K., Yamada, K. Examining the Effect of Providing a Pen on the Response Rate to a Mail Survey. Behaviormetrika 43, 83–101 (2016). https://doi.org/10.2333/bhmk.43.83
Received:
Revised:
Published:
Issue Date:
DOI: https://doi.org/10.2333/bhmk.43.83