A Brief Comment About Sample Size and Response Rates for Clinical Research: A Multi-site Case Example
- 15 Downloads
During the past few decades, response rates have declined across social science research. Although some researchers are willing to accept this state of affairs, in this comment, we argue that low response rates weaken the value of social science research, thereby jeopardizing science research and associated knowledge. We describe the case of a Hong Kong-based multi-site study of addiction treatment seekers to examine approaches to stimulating clinical research and response rates. For this case of addiction treatment seekers, the overall response rate was acceptable (69.7%), despite variation among the three study sites. We discuss factors that likely contributed to this response rate variation to illustrate practical steps that investigators can undertake to improve response rates instead of capitulating to the status quo.
KeywordsResponse rate Research methods Sample size
Compliance with Ethical Standards
Funding and Conflict of Interest
During approximately the past 5 years, the Division on Addiction received funding from a variety of sources. We do not believe any of the following funding sources constitute a conflict of interest for this project; however, we want to fully disclose our funding sources. The Division on Addiction received funding from the Foundation for Advancing Alcohol Responsibility (FAAR), The Healing Lodge of the Seven Nations via the National Institutes of Health (NIH) and Indian Health Services (IHS), the Integrated Centre on Addiction Prevention and Treatment of the Tung Wah Group of Hospitals—which receives funding from The Hong Kong Jockey Club Charities Trust, DraftKings, the Massachusetts Gaming Commission, GVC Holdings PLC, and the Massachusetts Department of Public Health. In addition, during approximately the past 5 years, Shaffer or the Division on Addiction received funding from National Center for Responsible Gambling, National Institutes of Health, the Alcohol Beverage Management Research Fund, the Danish Council for Independent Research, Heineken USA, Inc., bwin.party, St. Francis House, the State of Florida (i.e., as a subcontract to Spectrum Gaming Group), the Massachusetts Residential Substance Abuse Treatment for State Prisoners Grant Program (i.e., as a subcontracted evaluator for Worcester House of Corrections), and the Massachusetts Juvenile Accountability Block Grant Program—as a subcontracted evaluator for Cambridge Police Department). Howard Shaffer also has received speaker honoraria, or royalties, and compensation for consultation from the American Psychological Association, Las Vegas Sands Corp., Davies Ward Phillips and Vineberg, LLP, Freshfields Bruckhaus Deringer, LLP, and from the Dunes of Easthampton, a residential addiction treatment program, for serving as a consultant. Regarding his work with the international group on responsible gambling, he received reimbursement from Laval University for travel expenses, but no honorarium.
- Babbie, E. R. (2007). The practice of social research (11th ed.). Belmont: Thomson Wadsworth.Google Scholar
- Basson, D. (2008). Completion rate. In P. Lavrakas (Ed.), Encyclopedia of Survey Research Methods (p. 113). Thousand Oaks: Sage Publications, Inc..Google Scholar
- de Leeuw, E., & de Heer, W. (2002). Trends in household survey nonresponse: a longitudinal and international comparison. In R. M. Groves, D. A. Dillman, J. L. Eltinge, & R. J. A. Little (Eds.), Survey Nonresponse (pp. 41–54). New York: Wiley.Google Scholar
- Ekholm, O., Gundgaard, J., Rasmussen, N. K. R., & Hansen, E. H. (2010). The effect of health, socio-economic position, and mode of data collection on non-response in health interview surveys. Scandinavian Journal of Public Health, 38(7), 699–706. https://doi.org/10.1177/1403494810382474.CrossRefPubMedGoogle Scholar
- Frankel, J. R. (1982). On the definition of response rates: a special report of the CASRO task force on completion rates. Retrieved from Port Jefferson, NY.Google Scholar
- Hulley, S. B., Gove, S., Browner, W. S., & Cummings, S. R. (1988). Choosing the study subjects: specification and sampling. In S. B. Hulley & S. R. Cummings (Eds.), Designing clinical research: an epidemiologic approach (p. 247). Baltimore: Williams & Wilkins.Google Scholar
- Johnson, T., & Owens, L. (2003). Survey response rate reporting in the professional literature. American Association for Public Opinion Research, 127–133.Google Scholar
- Kazdin, A. E. (2003). Research design in clinical psychology (4th ed.). Boston: Allyn and Bacon.Google Scholar
- Kennedy, J. F. (1962). Yale University Commencement Address. American Rhetoric. Retrieved from American Rhetoric website: https://www.americanrhetoric.com/speeches/jfkyalecommencement.htm. Accessed 24 Oct 2019.
- Kessler, R. C., Hwang, I., Labrie, R. A., Petukhova, M., Sampson, N. A., Winters, K. C., & Shaffer, H. J. (2008). DSM-IV pathological gambling in the National Comorbidity Survey Replication. Psychological Medicine, 38, 1351–1360. https://doi.org/10.1017/S0033291708002900.CrossRefPubMedPubMedCentralGoogle Scholar
- Shaffer, H. J., Tom, M., Wong, M. F. Y., Chan, E. M. L., Cheng, G. L. F., Lo, C. K. M., . . . Wong, R. H. Y. (2017). Treating addiction as a syndrome: bridging research and clinical practice. Trends in Medicine, (April). Retrieved from Trends in Medicine website: https://trends.hms.harvard.edu. Accessed 24 Oct 2019.
- Shaffer, H. J., Tom, M. A., Wiley, R. C., Wong, M. F. Y., Chan, E. M. L., Cheng, G. L. F., et al. (2018). Using the syndrome model of addiction: a preliminary consideration of psychological states and traits. International Journal of Mental Health and Addiction, 16(6), 11373–11393. https://doi.org/10.1007/s11469-018-9952-2.CrossRefGoogle Scholar
- Singleton, R., & Straits, B. C. (2005). Approaches to social research (4th ed.). New York: Oxford University Press.Google Scholar
- Sivo, S., Saunders, C., Chang, Q., & Jiang, J. (2006). How low should you go? Low response rates and the validity of inference in IS questionnaire research. Journal of the Association for Information Systems, 7(6), 351–356,359-360,362-372,374-414. https://doi.org/10.17705/1jais.00093.CrossRefGoogle Scholar
- Studer, J., Baggio, S., Mohler-Kuo, M., Dermota, P., Gaume, J., Bertholet, N., et al. (2013). Examining non-response bias in substance use research—are late respondents proxies for non-respondents? Drug and Alcohol Dependence, 132, 316–323. https://doi.org/10.1016/j.drugalcdep.2013.02.029.CrossRefPubMedGoogle Scholar
- The Insight Association. (2011). Code of standards & ethics for market, opinion, and social research. Port Jefferson: The Insight Association.Google Scholar
- Vander Bilt, J., Hall, M. N., Shaffer, H. J., Storti, S., & Church, O. (1997). An assessment of substance abuse treatment training needs among nurses (part I): evaluating skill, knowledge, and training characteristics. Journal of Substance Misuse for Nursing, Health and Social Care, 2(3), 150–157.CrossRefGoogle Scholar
- Volberg, R. A., & Bernhard, B. (2006). The 2006 study of gambling and problem gambling in New Mexico: report to the Responsible Gaming Association of New Mexico. Retrieved from University of Calgary Libraries and Cultural Resources website: https://prism.ucalgary.ca/handle/1880/44211. Accessed 24 Oct 2019.