Abstract
Purpose
To assess whether, in the retrospective assessment of the pragmatic/explanatory features of pragmatic randomized controlled trials (pRCTs), the nine PRECIS-2 domain scores using the information provided in articles were modified after using the information reported in other publicly available sources.
Methods
This is a cross-sectional study of participant-level pRCTs published in July 2018 to December 2019 in the four highest-impact general medicine journals. The articles described the main results of pRCTs assessing medicines in one or more arms that were not in the pre-licensing phases. The information reported in trial full protocols, published protocols, and other publications, registries, and trial websites were assessed and scored, and compared with that previously obtained after reviewing the information reported in the articles.
Results
Out of 76 articles on pRCTs, 13 (17%) were included in the analysis. All were two-arm trials, assessing medicines only (n = 7), medicine vs device (n = 2), medicine vs surgery (n = 1), or medicine vs placebo (n = 3). Seven were open-label trials, and six had any type of masking. All except one had the full protocol available and/or published protocol; seven had other types of publication available. The assessment of the nine PRECIS-2 domains with the information reported in the 13 articles was changed in all trials after using the information included in other additional available sources. Between one (n = 1 article) and six (n = 2) domains were modified in each pRCT. The domains that most commonly changed were “organization” (n = 12), “recruitment” (n = 11), and “follow-up” (n = 8). “Primary outcome” and “primary analysis” were not modified in any trial. Eight percent of all domains could not be assessed due to inadequate or lack of information in seven articles; those were “recruitment” (n = 3), “organization” (n = 3), “setting” (n = 2), and “flexibility:adherence” (n = 1).
Conclusion
Articles describing the trial main results are usually insufficient for the appropriate retrospective assessment of the pragmatic/explanatory features of a pRCT by authors not involved in the conduct of the trial. To address this issue, editors should require the submission of the original full protocol and final full protocol with the history of amendments to be published as supplementary material to the article.
Similar content being viewed by others
References
Schwartz D, Lellouch J (1967) Explanatory and pragmatic attitudes in therapeutical trials. J Chron Dis 20:637–645
Loudon K, Treweek S, Sullivan F, Donnan P, Thorpe KE, Zwarenstein M (2015) The PRECIS-2 tool: designing trials that are fit for purpose. BMJ 350:h2147
Dal-Ré R, Janiaud P, Ioannidis JPA (2018) Real-world evidence: how pragmatic are randomized controlled trials labeled as pragmatic? BMC Med. 16:49
PRECIS─2. https://www.precis-2.org/Trials
Krebs EE, Jensen AC, Nugent S, DeRonne B, Rutks I, Leverty D et al (2017) Design, recruitment outcomes, and sample characteristics of the Strategies for Prescribing Analgesics Comparative Effectiveness (SPACE) trial. Contemp Clin Trials. 62:130–139
Tobe SW, Yeates K, Campbell NRC, Maar MA, Perkins N, Liu PP et al (2019) Diagnosing hypertension in Indigenous Canadians (DREAM-GLOBAL): a randomized controlled trial to compare the effectiveness of short message service messaging for management of hypertension: main results. J Clin Hypertens (Greenwich). 21:29–36
Jordan AE, Perlman DC, Smith DJ, Reed JR, Hagan H (2018) Use of the PRECIS-II instrument to categorize reports along the efficacy-effectiveness spectrum in an hepatitis C virus care continuum systematic review and meta-analysis. J Clin Epidemiol 93:66–75
Dal-Re R (2018) Could phase 3 medicine trials be tagged as pragmatic? A case study. The Salford COPD trial. J Eval Clin Pract 24:258–261
Sepehrvand N, Alemayehu W, Das D, Gupta AK, Gouda P, Ghimire A et al (2019) Trends in the explanatory or pragmatic nature of cardiovascular clinical trials over 2 decades. JAMA Cardiol. 4:1122–1128
Choi MY, Barnabe C, Barber CE, Bykerk V, Pope JE, Hazlewood GS (2019) Pragmaticism of randomized controlled trials of biologic treatment with methotrexate in rheumatoid arthritis: a systematic review. Arthritis Care Res. 71:620–628
Devos F, Foissac F, Bouazza N, Ancel PY, Tréluyer JM, Chappuy H (2019) Study characteristics impacted the pragmatism of randomized controlled trial published in nursing: a meta-epidemiological study. J Clin Epidemiol. 116:18–25
Braend AM, Straand J, Klovning A (2017) Clinical drug trials in GP & external validity. BMC Fam Pract 18:113
Malmivaara A (2019) Generalizability of findings from randomized controlled trials is limited in the leading general medical journals. J Clin Epidemiol. 107:36–41
Knight M, Chiocchia V, Partlett C, Rivero-Arias O, Hua X, Hinshaw K et al (2019) Prophylactic antibiotics in the prevention of infection after operative vaginal delivery (ANODE): a multicentre randomised controlled trial. Lancet. 393:2395–2403
Hofmeyr GJ, Betrán AP, Singata-Madliki M, Cormick G, Munjanja SP, Fawcus S et al (2019) Prepregnancy and early pregnancy calcium supplementation among women at high risk of pre-eclampsia: a multicentre, double-blind, randomised, placebo-controlled trial. Lancet. 393:330–339
Lyttle MD, Rainford NEA, Gamble C, Messahel S, Humphreys A, Hickey H et al (2019) Levetiracetam versus phenytoin for second-line treatment of paediatric convulsive status epilepticus (EcLiPSE): a multicentre, open-label, randomised trial. Lancet. 393:2125–2134
Ross JDC, Brittain C, Cole M, Dewsnap C, Harding J, Hepburn T et al (2019) Gentamicin compared with ceftriaxone for the treatment of gonorrhoea (G-ToG): a randomised non-inferiority trial. Lancet 393:2511–2520 Erratum in: Lancet. 2019;393:2590. Lancet. 2019;394:1230
Chesterton LS, Blagojevic-Bucknall M, Burton C, Dziedzic KS, Davenport G, Jowett SM et al (2018) The clinical and cost-effectiveness of corticosteroid injection versus night splints for carpal tunnel syndrome (INSTINCTS trial): an open-label, parallel group, randomised controlled trial. Lancet. 392:1423–1433
Adnan N, Conlan-Trant R, McCormick C, Boland F, Murphy DJ (2018) Intramuscular versus intravenous oxytocin to prevent postpartum haemorrhage at vaginal delivery: randomised controlled trial. BMJ. 362:k3546
Gazzard G, Konstantakopoulou E, Garway-Heath D, Garg A, Vickerstaff V, Hunter R et al (2019) Selective laser trabeculoplasty versus eye drops for first-line treatment of ocular hypertension and glaucoma (LiGHT): a multicentre randomised controlled trial. Lancet. 393:1505–1516
Harris PNA, Tambyah PA, Lye DC, Mo Y, Lee TH, Yilmaz M et al (2018) Effect of piperacillin-tazobactam vs meropenem on 30-day mortality for patients With E. coli or Klebsiella pneumoniae bloodstream infection and ceftriaxone resistance: a randomized clinical trial. JAMA. 320:984–994
Landoni G, Lomivorotov VV, Nigro Neto C, Monaco F, Pasyuga VV, Bradic N et al (2019) Volatile anesthetics versus total intravenous anesthesia for cardiac surgery. N Engl J Med. 380:1214–1225
Li HK, Rombach I, Zambellas R, Walker AS, McNally MA, Atkins BL et al (2019) Oral versus intravenous antibiotics for bone and joint infection. N Engl J Med 380:425–436
Wilson MJA, MacArthur C, Hewitt CA, Handley K, Gao F, Beeson L et al (2018) Intravenous remifentanil patient-controlled analgesia versus intramuscular pethidine for pain relief in labour (RESPITE): an open-label, multicentre, randomised controlled trial. Lancet. 392:662–672
Krag M, Marker S, Perner A, Wetterslev J, Wise MP, Schefold JC et al (2018) Pantoprazole in patients at risk for gastrointestinal bleeding in the ICU. N Engl J Med. 379:2199–2208
Hajek P, Phillips-Waller A, Przulj D, Pesola F, Myers Smith K, Bisal N et al (2019) A randomized trial of E-cigarettes versus nicotine-replacement therapy. N Engl J Med. 380:629–637
Chan AW, Hrobjartsson A (2018) Promoting public access to clinical trials protocols: challenges and recommendations. Trials 19:116
van Rosmalen BV, Alldinger I, Cieslak KP, Wennink R, Clarke M, Ali UA et al (2017) Worldwide trends in volume and quality of published protocols of randomized controlled trials. PLoS ONE 12:e0173042
Spence O, Hong K, Uba RO, Doshi P (2020) Availability of study protocols for randomized trials published in highest-impact medical journals: a cross-sectional analysis. Clin Trials. 17:99–105
Zarin DA, Fain KM, Dobbins HD, Tse T, Williams RJ (2019) 10-Year update on study results submitted to ClinicalTrials.gov. N Engl J Med 381:1966–1974
Banno M, Tsujimoto Y, Kataoka Y (2019) Studies registered in non-ClinicalTrials.gov accounted for an increasing proportion of protocol registrations in medical research. J Clin Epidemiol. 116:106–113
Doussau A, Vinarov E, Barsanti-Innes B, Kimmelman J (2020) Comparison between protocols and publications for prognostic and predictive cancer biomarker studies. Clin Trials. 17:61–68
Loudon K, Zwarenstein M, Sullivan FM, Donnan PT, Gágyor I, Hobbelen HJSM et al (2017) The PRECIS─2 tool has good interrater reliability and modest discriminant valididy. J Clin Epidemiol 88:113–121
Forbes G, Loudon K, Treweek S, Taylor SJC, Eldridge S (2017) Understanding the applicability of results from primary care trials: lessons learned from applying PRECIS-2. J Clin Epidemiol. 90:119–126
Janiaud P, Dal-Ré R, Ioannidis JPA (2018) Assessment of pragmatism in recently published randomized clinical trials. JAMA Intern Med. 178:1278–1280
Dal-Ré R, de Boer A, James SK (2020) The design can limit PRECIS-2 retrospective assessment of the clinical trial explanatory/pragmatic features. J Clin Epidemiol. https://doi.org/10.1016/j.jclinepi.2020.03.027
Goldacre B, Drysdale H, Dale A, Milosevic I, Slade E, Hartley P et al (2019) COMPare: prospective cohort study correcting and monitoring 58 misreported trials in real time. Trials. 20:118
Chen T, Li C, Qin R, Wang Y, Yu D, Dodd J, Wang D, Cornelius V (2019) Comparison of clinicaltrial changes in primary outcome and reported intervention effect size between trial registration and publication. JAMA Netw Open. 2:e197242
Dal-Ré R, Ross JS, Marušić A (2016) Compliance with prospective trial registration guidance remained low in highest-impact journals and has implications for primary end point reporting. J Clin Epidemiol. 75:100–107
Gopal AD, Wallach JD, Aminawung JA, Gonsalves G, Dal-Ré R, Miller JE et al (2018) Adherence to the International Committee of Medical Journal Editors' (ICMJE) prospective registration policy and implications for outcome integrity: a cross-sectional analysis of trials published in highest-impact specialty society journals. Trials. 19:448
Trinquart L, Dunn AG, Bourgeois FT (2018) Registration of published randomized trials: a systematic review and meta-analysis. BMC Medicine 16:173
International Committee of Medical Journal Editors. (2019) Recommendations for the conduct, reporting, editing, and publication of scholarly work in medical journals. December. http://www.icmje.org/icmje-recommendations.pdf
Funding
This research required no funding
Author information
Authors and Affiliations
Contributions
RDR conceived the study, conducted the search and data extraction, analyzed the data, and wrote the manuscript.
Corresponding author
Ethics declarations
Conflicts of interest
The author has received honorarium from Palex Medical for giving a lecture on low-risk pragmatic trials, outside from the submitted work.
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Electronic supplementary material
ESM 1
All pragmatic randomized controlled trials published in the four highest-impact general/internal medicine journals between July 2018 and December 2019 and changes in the PRECIS-2 tool scores in the assessed trials (PDF 879 kb)
Rights and permissions
About this article
Cite this article
Dal-Ré, R. Articles provided insufficient information to conduct an appropriate retrospective assessment of the pragmatic/explanatory features of medicine trials with the PRECIS-2 tool. Eur J Clin Pharmacol 76, 1093–1102 (2020). https://doi.org/10.1007/s00228-020-02901-4
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00228-020-02901-4