Pragmatic Trials as an Additional Tool in the Evidence Building Toolbox

  • Rebekah J. WalkerEmail author
  • Leonard E. Egede

Though randomized controlled trials (RCTs) are the gold standard in medicine, extensive evidence shows their influence on clinical practice is often delayed, with the very aspects that increase internal validity of the studies limiting its applicability in real-world settings.1, 2, 3, 4 In this issue of JGIM, Moin et al. examined the effectiveness of shared decision-making for diabetes prevention using a pragmatic trial design.5 Using a cluster-randomized design of 20 primary care clinics within a regional health system, Moin et al. found that participants receiving the shared decision-making intervention led by a pharmacist had higher uptake of efforts to increase prevention of diabetes, specifically use of metformin or participation in a local Diabetes Prevention Program (DPP).5 In addition, within the target population of overweight or obese adults with prediabetes, weight loss was greater at 12 month for intervention participants compared with a propensity score–matched control group.5 This article highlights a number of decisions that are necessary, and the real-world evidence that can result from a well-designed pragmatic trial.

In 1967, Schwartz and Lellouch outlined pragmatic designs as a way to test effectiveness of an intervention in routine practice using broad patient groups as opposed to testing efficacy of an intervention in a well-controlled setting like RCTs.6 Rather than a strict categorization, pragmatism has been conceptualized as a continuum, initially with a set of 7 domains, and more recently with 9 dimensions to consider.7 As such, pragmatic features are to be adopted where feasible without compromising the quality of a trial in its ability to answer the research question.1 The 9 dimensions for assessing the level of pragmatism can be categorized into characteristics specific to recruitment of individuals, characteristics of the intervention and its delivery, and characteristics of follow-up and outcomes.1, 7 Studies can use pragmatic aspects within their trial design while maintaining the rigor necessary for valid results. For example, Moin et al. did not consent patients in the control arm of the study, did not provide incentives for participation, and delivered the trial within the primary care clinic to mirror usual care as much as possible.5 However, as a result of the design, they were limited in their outcomes and conducted a number of sensitivity analyses to examine how reliable their results were given those limitations.5

Use of a pragmatic design does require significant forethought to ensure results can be used by providers in a real-world setting, rather than simply having designed an intervention that can be deployed similar to usual care, but lacking evidence of effectiveness. Moin et al. used two primary outcomes of interest: uptake of either DPP or metformin at 4 months follow-up and weight change at 12 months follow-up.5 Participants were monitored through attendance data from DPP suppliers, EHR medication reconciliation notes, and natural language queries of EHR progress notes to capture participation in another structured weight loss program.5 Weight change was calculated based on weight measures available in the EHR.5 While these are all pragmatic approaches, and other than natural language processing could be accomplished with minimal increase in resources for usual care, the internal validity of adherence without asking participants may be limited. Though concerns with lengthy questionnaires are valid, pragmatic trials have been conducted using brief, sensitive measures that are useful for clinicians and policy makers as well as researchers.8

A combination of measures is recommended by Glasgow et al. in designing high-quality pragmatic trials.7 Additional factors such as quality of life, behavior change, and economic outcomes are recommended as important outcomes for informing both clinical decision-making and policy recommendations and should be considered in future studies.8 The lack of consent by control participants is a pragmatic consideration that will impact decisions on measures; however, given the rise in use of brief patient-reported outcome scales in regular practice screening procedures, collection through the clinic staff as part of both intervention and usual care groups could be considered.9, 10

High-quality pragmatic trials should be seen as one way to increase the likelihood of translating trial data into clinical practice and an equally valid option for large-scale randomized studies designed to inform care. While significant focus has been placed on using evidence from well-powered RCTs, the need for answers developed and tested in a more pragmatic fashion can be seen in the delay in using evidence and the continually increasing cost of care. The importance of high-quality evidence to inform clinical care and policy changes cannot rely on only one kind of clinical trial.2 As shown by Moin et al. in this issue of JGIM, a pragmatic approach to a trial design can provide important information when planned and conducted with attention to high internal and external validity.5 Decisions regarding recruitment, intervention delivery, and outcome measurement all require use of a continuum of options from explanatory to pragmatic, rather than focus on one type fitting all questions.


Funding Information

This study was partially supported by the National Institute of Diabetes and Digestive Kidney Disease (K24DK093699, R01DK118038, R01DK120861, PI: Egede), the National Institute for Minority Health and Health Disparities (R01MD013826, PI: Egede/Walker), and the American Diabetes Association (1-19-JDF-075, PI: Walker).

Compliance with Ethical Standards

Conflict of Interest

The authors declare that they do not have a conflict of interest.


  1. 1.
    Ford I, Noorie J. Pragmatic trials. NEJM. 2016; 375: 454–463.CrossRefGoogle Scholar
  2. 2.
    Tunis SR, Stryer DB, Clancy CM. Practical clinical trials: increasing the value of clinical research for decision making in clinical and health policy. JAMA. 2003;290:1624–1632.CrossRefGoogle Scholar
  3. 3.
    Zwarenstein M, Oxman A. Why are so few randomized trials useful, and what can we do about it? J Clin Epidemiol. 2006; 59:1125–1126.CrossRefGoogle Scholar
  4. 4.
    Balas EA, Boren SA. Managing clinical knowledge for health care improvement. In: Bemmel J, McCray AT, editors. Yearbook of Medical Informatics 2000: Patient-Centered Systems. Stuttgart: Schattauer Verlagsgesellschaft mbH; 2000:65–70.Google Scholar
  5. 5.
    Moin T, Duru OK, Turk N, Chon JS, Frosch DL, Martin J, Jeffers KS, Castellon-Lopez Y, Tseng C, Norris K, Mangione CM. Effectiveness of shared decision-making for diabetes prevention: 12-month results from the Prediabetes Informed Decision and Education (PRIDE) Trial. JGIM. 2019.
  6. 6.
    Schwartz D, Lellouch J. Explanatory and pragmatic attitudes in therapeutic trials. J Chronic Dis. 1967; 20: 637–648.CrossRefGoogle Scholar
  7. 7.
    Loudon K, Treweek S, Sullivan F, Donnan P, Thorpe KE, Zwarenstein M. The PREICIS-2 tool: designing trials that are fit for purpose. BMJ. 2015; 350:h2147.CrossRefGoogle Scholar
  8. 8.
    Glasgow RE, Magid DJ, Beck A, Ritzwoller D, Estabrooks PA. Practical clinical trials for translating research to practice: design and measurement recommendations. Med Care., 2005;43(6):551–557.CrossRefGoogle Scholar
  9. 9.
    Wu AW, Kharrazi H, Boulware LE, Snyder CF. Measure one, cut twice – adding patient-reported outcome measures to the electronic health record for comparative effectiveness research. J Clin Epidemiol. 2013; 66(8): S12-S20.CrossRefGoogle Scholar
  10. 10.
    Foster A, Croot L, Brazier J, Harris J, O’Cathain A. The facilitators and barriers to implementing patient reported outcome measures in organizations delivering health related services: a systematic review of reviews. J Patient Rep Outcomes. 2018; 2: 46.CrossRefGoogle Scholar

Copyright information

© Society of General Internal Medicine 2019

Authors and Affiliations

  1. 1.Division of General Internal Medicine, Department of MedicineMedical College of WisconsinMilwaukeeUSA
  2. 2.Center for Advancing Population ScienceMedical College of WisconsinMilwaukeeUSA

Personalised recommendations