Mode of administration does not cause bias in patient-reported outcome results: a meta-analysis
- 446 Downloads
Technological advances in recent decades have led to the availability of new modes to administer patient-reported outcomes (PROs). To aid selecting optimal modes of administration (MOA), we undertook a systematic review to determine whether differences in bias (both size and direction) exist among modes.
We searched five electronic databases from 2004 (date of last comprehensive review on this topic) to April 2014, cross-referenced and searched reference lists. Studies that compared two or more MOA for a health-related PRO measure in adult samples were included. Two reviewers independently applied inclusion and quality criteria and extracted findings. Meta-analyses and meta-regressions were conducted using random-effects models.
Of 5100 papers screened, 222 were considered potentially relevant and 56 met eligibility criteria. No evidence of bias was found for: (1) paper versus electronic self-complete; and (2) self-complete versus assisted MOA. Heterogeneity for paper versus electronic comparison was explained by type of construct (i.e. physical vs. psychological). Heterogeneity for self-completion versus assisted modes was in part explained by setting (clinic vs. home); the largest bias was introduced when assisted completion occurred in the clinic and follow-up was by self-completion (either electronic or paper) in the home.
Self-complete paper and electronic MOA can be used interchangeably for research in clinic and home settings. Self-completion and assisted completion produce equivalent scores overall, although heterogeneity may be induced by setting. These results support the use of mixed MOAs within a research study, which may be a useful strategy for reducing missing PRO data.
KeywordsSystematic review Patient-reported outcome Mode of administration Bias
We thank HR and LG for their contribution to this project as volunteers. We also acknowledge the support from our faculty librarian, Matthew Davis, in developing the search strategy.
C.R., D.C., R.M.B. and M.K. are supported by the Australian Government through Cancer Australia. No additional funding was sought for this review.
Compliance with ethical standards
Conflict of interest
All authors declare that they have no conflict of interest.
This article is a secondary analysis of published literature. It does not contain any studies with human participants or animals performed by any of the authors.
- 1.Food and Drug Administration. (2009). Patient reported outcome measures: Use in medical product development to support labelling claims. MD: US Department of Health & Human Support Food & Drug Administration.Google Scholar
- 4.Basch, E., Abernethy, A. P., Mullins, C. D., Reeve, B. B., Smith, M. L., Coons, S. J., et al. (2012). Recommendations for incorporating patient-reported outcomes into clinical comparative effectiveness research in adult oncology. Journal of Clinical Oncology, 30(34), 4249–4255.CrossRefPubMedGoogle Scholar
- 13.Kmet, L., Lee, R., & Cook, L. (2004). Standard quality assessment criteria for evaluating primary research papers from a variety of fields. Health Technology Assessment, 13, 1–294.Google Scholar
- 14.Lipsey, M., & Wilson, D. (2001). Practical meta-analysis. Thousand Oaks: Sage.Google Scholar
- 17.Reichmann, W. M., Losina, E., Seage, G. R., Arbelaez, C., Safren, S. A., Katz, J. N., et al. (2010). Does modality of survey administration impact data quality: Audio computer assisted self interview (ACASI) versus self-administered pen and paper? PLoS One, 5(1), e8728.PubMedCentralCrossRefPubMedGoogle Scholar
- 20.Rodriguez, H. P., von Glahn, T., Rogers, W. H., Chang, H., Fanjiang, G., & Safran, D. G. (2006). Evaluating patients’ experiences with individual physicians: A randomized trial of mail, internet, and interactive voice response telephone administration of surveys. Medical Care, 44(2), 167–174.CrossRefPubMedGoogle Scholar
- 22.Rutherford, C., Nixon, J., Brown, J. M., Lamping, D. L., & Cano, S. J. (2014). Using mixed methods to select optimal mode of administration for a patient-reported outcome instrument for people with pressure ulcers. BMC Medical Research Methodology, 14(22), 1471–2288.Google Scholar
- 24.Shih, T., & Fan, X. (2007). Response rates and mode preferences in web-mail mixed-mode surveys: A meta-analysis. International Journal of Internet Science, 2, 59–82.Google Scholar
- 29.Cerrada, C. J., Weinberg, J., Sherman, K. J., & Saper, R. B. (2014). Inter-method reliability of paper surveys and computer assisted telephone interviews in a randomized controlled trial of yoga for low back pain. BMC Research Notes, 7, 227. doi:10.1186/1756-0500-7-227.PubMedCentralCrossRefPubMedGoogle Scholar
- 30.Chang, Y. J., Chang, C. H., Peng, C. L., Wu, H. C., Lin, H. C., Wang, J. Y., et al. (2014). Measurement equivalence and feasibility of the EORTC QLQ-PR25: Paper-and-pencil versus touch-screen administration. Health and Quality of Life Outcomes, 12, 23. doi:10.1186/1477-7525-12-23.CrossRefGoogle Scholar
- 40.Hauer, K., Yardley, L., Beyer, N., Kempen, G., Dias, N., Campbell, M., et al. (2010). Validation of the falls efficacy scale and falls efficacy scale international in geriatric patients with and without cognitive impairment: results of self-report and interview-based questionnaires. Gerontology, 56(2), 190–199.CrossRefPubMedGoogle Scholar
- 43.Hedman, E., Ljotsson, B., Blom, K., Alaoui, S. E., Kraepelien, M., Ruck, C., et al. (2013). Telephone versus internet administration of self-report measures of social anxiety, depressive symptoms, and insomnia: Psychometric evaluation of a method to reduce the impact of missing data. Journal of Medical Internet Research, 15(10), 131–138.CrossRefGoogle Scholar
- 44.Hollandare, F., Andersson, G., & Engstrom, I. (2010). A comparison of psychometric properties between internet and paper versions of two depression instruments (BDI-II and MADRS-S) administered to clinic patients. Journal of Medical Internet Research, 12(5), e49.Google Scholar
- 49.Lungenhausen, M., Lange, S., Maier, C., Schaub, C., Trampisch, H. J., & Endres. H. G. (2007). Randomised controlled comparison of the Health Survey Short Form (SF-12) and the Graded Chronic Pain Scale (GCPS) in telephone interviews versus self-administered questionnaires. Are the results equivalent? BMC Medical Research Methodology, 7(50). doi:10.1186/1471-2288-7-50.PubMedCentralPubMedGoogle Scholar
- 51.Matthew, A. G., Currie, K. L., Irvine, J., Ritvo, P., Santa Mina, D., Jamnicky, L., et al. (2007). Serial personal digital assistant data capture of health-related quality of life: A randomized controlled trial in a prostate cancer clinic. Health Qual Life Outcomes, 5, 38.PubMedCentralCrossRefPubMedGoogle Scholar
- 59.Salaffi, F., Gasparini, S., Ciapetti, A., Gutierrez, M., & Grassi, W. (2013). Usability of an innovative and interactive electronic system for collection of patient-reported data in axial spondyloarthritis: Comparison with the traditional paper-administered format. Rheumatology, 52(11), 2062–2070.CrossRefPubMedGoogle Scholar
- 61.Sousa, P. C., Mendes, F. M., Imparato, J. C., & Ardenghi, T. M. (2009). Differences in responses to the Oral Health Impact Profile (OHIP14) used as a questionnaire or in an interview. Pesquisa Odontologica Brasileira—Brazilian Oral Research, 23(4), 358–364.Google Scholar
- 63.Swartz, R. J., de Moor, C., Cook, K. F., Fouladi, R. T., Basen-Engquist, K., Eng, C., & Taylor, C. L. C. (2007). Mode effects in the center for epidemiologic studies depression (CES-D) scale: Personal digital assistant vs. paper and pencil administration. Quality of Life Research, 16(5), 803–813.CrossRefPubMedGoogle Scholar
- 64.Tiplady, B., Goodman, K., Cummings, G., Lyle, D., Carrington, R., Battersby, C., & Ralston, S. H. (2010). Patient-reported outcomes in rheumatoid arthritis: Assessing the equivalence of electronic and paper data collection. The Patient: Patient Centered Outcomes Research, 3(3), 133–143.CrossRefGoogle Scholar
- 65.Weiler, K., Christ, A. M., Woodworth, G. G., Weiler, R. L., & Weiler, J. M. (2004). Quality of patient-reported outcome data captured using paper and interactive voice response diaries in an allergic rhinitis study: Is electronic data capture really better? Annals of Allergy, Asthma & Immunology, 92(3), 335–339.CrossRefGoogle Scholar
- 66.Whitehead, L. (2011). Methodological issues in Internet-mediated research: A randomized comparison of internet versus mailed questionnaires. Journal of Medical Internet Research, 13(4), e109.Google Scholar
- 67.Wu, L. T., Pan, J. J., Blazer, D. G., Tai, B., Brooner, R. K., Stitzer, M. L., et al. (2009). The construct and measurement equivalence of cocaine and opioid dependences: A National Drug Abuse Treatment Clinical Trials Network (CTN) study. Drug and Alcohol Dependence, 103(3), 114–123.PubMedCentralCrossRefPubMedGoogle Scholar