Abstract
Purpose
Patient-reported outcome measures (PROMs) are used to collect data on disease symptoms in support of clinical trial endpoints. Clinical studies can last a year or more, and the patients’ adherence and response time to daily at-home questionnaires may vary significantly over time. The aim of this study was to understand patterns and changes in patients’ completion of daily PROMs during longitudinal clinical studies.
Methods
Data were collected from 1342 patients randomized into three respiratory clinical trials (NCT03401229, NCT03347279, and NCT03406078). PROMs were completed by patients using electronic handheld devices that collected the starting and completion times. A Bayesian generalized linear mixed-effects model was used to identify unbiased coefficients associated with PROM adherence and response time using patient, site, and calendar features as covariates.
Results
Adherence decreased over time after randomization, and the rate of decrease was higher in younger patients. The 14-day pre-randomization adherence was correlated with adherence throughout the study. Patients were also more adherent during working days compared to non-working days. Oldest patients took twice as long to complete PROMs throughout the study; however, the response time for all patients decreased during the first month of the study regardless of age. Response time increased 7 days before and after the date of a scheduled clinic visit and when a patient-reported higher symptom burden.
Conclusion
Detailed analyses of adherence and response time for daily PROMs in clinical trials can provide significant insights about trends of patient behavior in longitudinal clinical studies with high baseline adherence.
Similar content being viewed by others
Data availability
Data underlying the findings described in this manuscript may be obtained in accordance with AstraZeneca’s data sharing policy described at https://astrazenecagrouptrials.pharmacm.com/ST/Submission/Disclosure. Data for studies directly listed on Vivli can be requested through Vivli at www.vivli.org. Data for studies not listed on Vivli could be requested through Vivli at https://vivli.org/members/enquiries-about-studies-not-listed-on-the-vivli-platform/. AstraZeneca Vivli member page is also available outlining further details: https://vivli.org/ourmember/astrazeneca/.
References
US Food and Drug Administration, ‘Guidance for industry: Patient-Reported Outcome Measures: Use in Medical Product Development to Support Labeling Claims’. (2009 Dec). Accessed: 31 Oct 2022. [Online]. Available: https://www.fda.gov/regulatory-information/search-fda-guidance-documents/patient-reported-outcome-measures-use-medical-product-development-support-labeling-claims
Weldring, T., & Smith, S. M. S. (2013). Patient-reported outcomes (PROs) and patient-reported outcome measures (PROMs). Health Services Insights, 6, 61–68. https://doi.org/10.4137/HSI.S11093
Mercieca-Bebber, R., King, M. T., Calvert, M. J., Stockler, M. R., & Friedlander, M. (2018). The importance of patient-reported outcomes in clinical trials and strategies for future optimization. Patient Related Outcome Measures, 9, 353–367. https://doi.org/10.2147/PROM.S156279
Au, H.-J., Ringash, J., Brundage, M., Palmer, M., Richardson, H., & Meyer, R. M. (2010). Added value of health-related quality of life measurement in cancer clinical trials: The experience of the NCIC CTG. Expert Review of Pharmacoeconomics & Outcomes Research, 10(2), 119–128. https://doi.org/10.1586/erp.10.15
Basch, E., Deal, A. M., Kris, M. G., Scher, H. I., Hudis, C. A., Sabbatini, P., Rogak, L., Bennett, A. V., Dueck, A. C., Atkinson, T. M., Chou, J. F., Dulko, D., Sit, L., Barz, A., Novotny, P., Fruscione, M., Sloan, J. A., & Schrag, D. (2016). Symptom monitoring with patient-reported outcomes during routine cancer treatment: A randomized controlled trial. Journal of Clinical Oncology, 34(6), 557–565. https://doi.org/10.1200/JCO.2015.63.0830
Etkind, S. N., Daveson, B. A., Kwok, W., Witt, J., Bausewein, C., Higginson, I. J., & Murtagh, F. E. M. (2015). Capture, transfer, and feedback of patient-centered outcomes data in palliative care populations: Does it make a difference? A systematic review. Journal of Pain and Symptom Management, 49(3), 611–624. https://doi.org/10.1016/j.jpainsymman.2014.07.010
Coons, S. J., Eremenco, S., Lundy, J. J., O’Donohoe, P., O’Gorman, H., & Malizia, W. (2015). Capturing patient-reported outcome (PRO) data electronically: The past, present, and promise of ePRO measurement in clinical trials. The Patient, 8(4), 301–309. https://doi.org/10.1007/s40271-014-0090-z
Rosa, C., Campbell, A. N. C., Miele, G. M., Brunner, M., & Winstanley, E. L. (2015). Using e-technologies in clinical trials. Contemporary Clinical Trials, 45(Pt A), 41–54. https://doi.org/10.1016/j.cct.2015.07.007
Stone, A. A., Shiffman, S., Schwartz, J. E., Broderick, J. E., & Hufford, M. R. (2003). Patient compliance with paper and electronic diaries. Controlled Clinical Trials, 24(2), 182–199. https://doi.org/10.1016/s0197-2456(02)00320-3
Lane, S. J., Heddle, N. M., Arnold, E., & Walker, I. (2006). A review of randomized controlled trials comparing the effectiveness of hand held computers with paper methods for data collection. BMC Medical Informatics and Decision Making, 6, 23. https://doi.org/10.1186/1472-6947-6-23
Ganser, A. L., Raymond, S. A., & Pearson, J. D. (2010). Data quality and power in clinical trials: A comparison of ePRO and paper in a randomized trial. ePro: Electronic solutions for patient-reported data by tiplady B and byrom B (pp. 49–78). Routledge.
Rintala, A., Wampers, M., Myin-Germeys, I., & Viechtbauer, W. (2019). Response compliance and predictors thereof in studies using the experience sampling method. Psychological Assessment, 31(2), 226–235. https://doi.org/10.1037/pas0000662
Di Fraia, M., Tripodi, S., Arasi, S., Dramburg, S., Castelli, S., Villalta, D., Buzzulini, F., Sfika, I., Villella, V., Potapova, E., Perna, S., Brighetti, M. A., Travaglini, A., Verardo, P., Pelosi, S., Zicari, A. M., & Matricardi, P. M. (2020). Adherence to prescribed E-diary recording by patients with seasonal allergic rhinitis: Observational study. Journal of Medical Internet Research, 22(3), e16642. https://doi.org/10.2196/16642
Jones, S. E., Moore, R. C., Pinkham, A. E., Depp, C. A., Granholm, E., & Harvey, P. D. (2021). A cross-diagnostic study of adherence to ecological momentary assessment: Comparisons across study length and daily survey frequency find that early adherence is a potent predictor of study-long adherence. Personalized Medicine in Psychiatry, 29–30, 100085. https://doi.org/10.1016/j.pmip.2021.100085
Morren, M., van Dulmen, S., Ouwerkerk, J., & Bensing, J. (2009). Compliance with momentary pain measurement using electronic diaries: A systematic review. European Journal of Pain, 13(4), 354–365. https://doi.org/10.1016/j.ejpain.2008.05.010
Grüne, B., Menold, H., Lenhart, M., Mühlbauer, J., Walach, M. T., Waldbillig, F., Neuberger, M., Nuhn, P., Michel, M. S., Koenig, J., Kriegmair, M. C., & Wessels, F. (2023). Patient compliance in assessing electronic patient-reported outcome measures after urologic surgery. Urologia Internationalis, 107(3), 280–287. https://doi.org/10.1159/000520755
Glimmerveen, A., Holewijn, S., & Vermeer, S. (2023). Association between clinician reported outcome and patient reported outcome measures one year after stroke. Journal of Stroke and Cerebrovascular Diseases, 32(8), 107156. https://doi.org/10.1016/j.jstrokecerebrovasdis.2023.107156
Aiyegbusi, O. L., Roydhouse, J., Rivera, S. C., Kamudoni, P., Schache, P., Wilson, R., Stephens, R., & Calvert, M. (2022). Key considerations to reduce or address respondent burden in patient-reported outcome (PRO) data collection. Nature Communications, 13, 6026. https://doi.org/10.1038/s41467-022-33826-4
Bodart, S., Byrom, B., Crescioni, M., Eremenco, S., & Flood, E. (2019). Perceived burden of completion of patient-reported outcome measures in clinical trials: Results of a preliminary study. Therapeutic Innovation & Regulatory Science, 53(3), 318–323. https://doi.org/10.1177/2168479018788053
Atkinson, T. M., Schwartz, C. E., Goldstein, L., Garcia, I., Storfer, D. F., Li, Y., Zhang, J., Bochner, B. H., & Rapkin, B. D. (2019). Perceptions of response burden associated with completion of patient-reported outcome assessments in oncology. Value in Health, 22(2), 225–230. https://doi.org/10.1016/j.jval.2018.07.875
Nowojewski, A., Bark, E., Shih, V. H., O’Quinn, S., & Dearden, R. (2023). Leveraging ePRO paradata for patient-centered trial designs. Value & Outcomes Spotlight, 9(6), 29–31.
Bachert, C., Han, J. K., Desrosiers, M. Y., Gevaert, P., Heffler, E., Hopkins, C., Tversky, J. R., Barker, P., Cohen, D., Emson, C., Martin, U. J., Shih, V. H., Necander, S., Kreindler, J. L., Jison, M., & Werkström, V. (2022). Efficacy and safety of benralizumab in chronic rhinosinusitis with nasal polyps: A randomized, placebo-controlled trial. The Journal of Allergy and Clinical Immunology, 149(4), 1309-1317.e12. https://doi.org/10.1016/j.jaci.2021.08.030
Menzies-Gow, A., Corren, J., Bourdin, A., Chupp, G., Israel, E., Wechsler, M. E., Brightling, C. E., Griffiths, J. M., Hellqvist, Å., Bowen, K., Kaur, P., Almqvist, G., Ponnarambil, S., & Colice, G. (2021). Tezepelumab in adults and adolescents with severe, uncontrolled asthma. The New England Journal of Medicine, 384(19), 1800–1809. https://doi.org/10.1056/NEJMoa2034975
Wechsler, M. E., Menzies-Gow, A., Brightling, C. E., Kuna, P., Korn, S., Welte, T., Griffiths, J. M., Sałapa, K., Hellqvist, Å., Almqvist, G., Lal, H., Kaur, P., Skärby, T., Colice, G., & SOURCE study group. (2022). Evaluation of the oral corticosteroid-sparing effect of tezepelumab in adults with oral corticosteroid-dependent asthma (SOURCE): A randomised, placebo-controlled, phase 3 study. The Lancet. Respiratory Medicine, 10(7), 650–660. https://doi.org/10.1016/S2213-2600(21)00537-3
O’Quinn, S., Shih, V. H., Martin, U. J., Meyers, O., Crooks, P., Bailey, J., & Slagle, A. F. (2022). Measuring the patient experience of chronic rhinosinusitis with nasal polyposis: Qualitative development of a novel symptom diary. International Forum of Allergy & Rhinology, 12(8), 996–1005. https://doi.org/10.1002/alr.22952
Globe, G., Martin, M., Schatz, M., Wiklund, I., Lin, J., von Maltzahn, R., & Mattera, M. S. (2015). Symptoms and markers of symptom severity in asthma—Content validity of the asthma symptom diary. Health and Quality of Life Outcomes, 13, 21. https://doi.org/10.1186/s12955-015-0217-5
Globe, G., Wiklund, I., Lin, J., Chen, W.-H., Martin, M., Mattera, M. S., von Maltzahn, R., Feng, J. Y., Chon, Y., Viswanathan, H. N., & Schatz, M. (2016). Psychometric properties of the asthma symptom diary (ASD), a diary for use in clinical trials of persistent asthma. The Journal of Allergy and Clinical Immunology in Practice, 4(1), 60-66.e4. https://doi.org/10.1016/j.jaip.2015.07.008
Globe, G., Wiklund, I., Mattera, M., Zhang, H., & Revicki, D. A. (2019). Evaluating minimal important differences and responder definitions for the asthma symptom diary in patients with moderate to severe asthma. Journal of Patient-Reported Outcomes, 3(1), 22. https://doi.org/10.1186/s41687-019-0109-2
Moineddin, R., Matheson, F. I., & Glazier, R. H. (2007). A simulation study of sample size for multilevel logistic regression models. BMC Medical Research Methodology, 7, 34. https://doi.org/10.1186/1471-2288-7-34
Gomes, D. G. E. (2022). Should I use fixed effects or random effects when I have fewer than five levels of a grouping factor in a mixed-effects model? PeerJ, 10, e12794. https://doi.org/10.7717/peerj.12794
Hale, T., Angrist, N., Goldszmidt, R., Kira, B., Petherick, A., Phillips, T., Webster, S., Cameron-Blake, E., Hallas, L., Majumdar, S., & Tatlow, H. (2021). A global panel database of pandemic policies (Oxford COVID-19 government response tracker). Nature Human Behaviour, 5(4), 529–538. https://doi.org/10.1038/s41562-021-01079-8
Edouard Mathieu, Hannah Ritchie, Lucas Rodés-Guirao, Cameron Appel, Charlie Giattino, Joe Hasell, Bobbie Macdonald, Saloni Dattani, Diana Beltekian, Esteban Ortiz-Ospina, & Max Roser. (2020). Coronavirus Pandemic (COVID-19) https://ourworldindata.org/coronavirus
Stan Development Team. (2023). Stan Modeling Language Users Guide and Reference Manual, 2.32. Available: https://mc-stan.org
Kruschke, J. K. (2021). Bayesian analysis reporting guidelines. Nature Human Behaviour, 5(10), 1282–1291. https://doi.org/10.1038/s41562-021-01177-7
Jenkinson, A. F. (1955). The frequency distribution of the annual maximum (or minimum) values of meteorological elements. Quarterly Journal Royal Meteorological Society, 81(348), 158–171. https://doi.org/10.1002/qj.49708134804
Rigby, R. A., & Stasinopoulos, D. M. (2005). Generalized additive models for location, scale and shape. Journal of the Royal Statistical Society Series C, 54(3), 507–554.
Benedetti, A., Platt, R., & Atherton, J. (2014). Generalized linear mixed models for binary data: Are matching results from penalized quasi-likelihood and numerical integration less biased? PLoS ONE, 9(1), e84601. https://doi.org/10.1371/journal.pone.0084601
Wedderburn, R. W. M. (1974). Quasi-likelihood functions, generalized linear models, and the Gauss—Newton method. Biometrika, 61(3), 439–447. https://doi.org/10.1093/biomet/61.3.439
Van Rossum, G., & Drake, F. L. (2009). Python 3 reference manual. Scotts Valley, CreateSpace.
R Core Team. (2023). R: A language and environment for statistical computing. R Foundation for Statistical Computing. Available: https://www.R-project.org/
Bürkner, P.-C. (2018). Advanced Bayesian multilevel modeling with the R package brms. The R Journal, 10(1), 395–411.
Pratap, A., Neto, E. C., Snyder, P., Stepnowsky, C., Elhadad, N., Grant, D., Mohebbi, M. H., Mooney, S., Suver, C., Wilbanks, J., Mangravite, L., Heagerty, P. J., Areán, P., & Omberg, L. (2020). Indicators of retention in remote digital health studies: A cross-study evaluation of 100,000 participants. NPJ Digital Medicine, 3, 21. https://doi.org/10.1038/s41746-020-0224-8
Kim, S. J., Kwon, O. D., Han, E. B., Lee, C. M., Oh, S.-W., Joh, H.-K., Oh, B., Kwon, H., Cho, B., & Choi, H. C. (2019). Impact of number of medications and age on adherence to antihypertensive medications. Medicine, 98(49), e17825. https://doi.org/10.1097/MD.0000000000017825
Brysbaert, M. (2019). How many words do we read per minute? A review and meta-analysis of reading rate. Journal of Memory and Language, 109, 104047.
Therrien, W. J. (2004). Fluency and comprehension gains as a result of repeated reading: A meta-analysis. Remedial and Special Education, 25(4), 252–261.
Chang, A.-S., & Millett, S. (2013). Improving reading rates and comprehension through timed repeated reading. Reading in Foreign Language, 25(2), 126–148.
Trauzettel-Klosinski, S., Dietz, K., & IReST Study Group. (2012). Standardized assessment of reading performance: The new international reading speed texts IReST. Investigative Opthalmology & Visual Science, 53(9), 5452–5461. https://doi.org/10.1167/iovs.11-8284
Chmykhova, E., Davydov, D., & Lavrova, T. (2014). The factors of the reading speed: An experimental study. Пcиxoлoгия Oбyчeния, 9, 26–36.
Kieffer, M. J. (2010). Socioeconomic status, English proficiency, and late-emerging reading difficulties. Educational Researcher, 36(6), 484–486.
Deutscher, D., Weil, C., Chodick, G., Tsukinovsky, S., Bodger, K., Waterman, M., & Kariv, R. (2023). Implementing electronic patient reported outcomes in inflammatory bowel disease: Patient participation, score reliability and validity. Health and Quality of Life Outcomes, 21(1), 2. https://doi.org/10.1186/s12955-023-02087-0
Acknowledgements
We are grateful to Romina Dennehy and Davorin Miletic of Clario (formerly ERT) for providing us the timestamped ePROM data on an ad hoc basis. We also appreciate Sean O’Quinn, William Dott, and Olga Elizarova of AstraZeneca as well as Priyanka Sen for proofreading and commenting on the manuscript.
Funding
AstraZeneca sponsored the clinical studies and also funded this work.
Author information
Authors and Affiliations
Contributions
AN, EB, VHS, and RD designed and directed the project. EB and VHS obtained the datasets. AN designed and executed the analysis. EB and VHS validated the results. AN wrote the manuscript with critical review and input from all authors. The final manuscript was reviewed and approved for submission by all authors.
Corresponding author
Ethics declarations
Competing interest
All authors are or were employees and shareholders of AstraZeneca at the time of study conduct.
Ethical approval
The details of the design and results of the OSTRO, NAVIGATOR, and SOURCE trials have been previously reported (ClinicalTrials.gov Identifiers: NCT03401229 [22], NCT03347279 [23], and NCT03406078 [24]). The protocols were approved by the independent ethics committee at each participating site, and all patients provided written informed consent. A review process was followed that ensured that the proposed analysis was within the scope of original approvals and signed informed consent forms (INT-20211208–893, 8 December 2021). This study used existing fully de-identified data and the investigators or patients could not be identified, directly or through identifiers linked to patients.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
Below is the link to the electronic supplementary material.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Nowojewski, A., Bark, E., Shih, V.H. et al. Patient adherence and response time in electronic patient-reported outcomes: insights from three longitudinal clinical trials. Qual Life Res (2024). https://doi.org/10.1007/s11136-024-03644-w
Accepted:
Published:
DOI: https://doi.org/10.1007/s11136-024-03644-w