Skip to main content
SpringerLink
Log in

Similarities and Differences Between Pragmatic Trials and Hybrid Effectiveness-Implementation Trials

  • Research and Reporting Methods
  • Published:
Journal of General Internal Medicine Aims and scope Submit manuscript

Abstract

Pragmatism in clinical trials is focused on increasing the generalizability of research findings for routine clinical care settings. Hybridism in clinical trials (i.e., assessing both clinical effectiveness and implementation success) is focused on speeding up the process by which evidence-based practices are developed and adopted into routine clinical care. Even though pragmatic trial methodologies and implementation science evolved from very different disciplines, Pragmatic Trials and Hybrid Effectiveness-Implementation Trials share many similar design features. In fact, these types of trials can easily be conflated, creating the potential for investigators to mislabel their trial type or mistakenly use the wrong trial type to answer their research question. Blurred boundaries between trial types can hamper the evaluation of grant applications, the scientific interpretation of findings, and policy-making. Acknowledging that most trials are not pure Pragmatic Trials nor pure Hybrid Effectiveness-Implementation Trials, there are key differences in these trial types and they answer very different research questions. The purpose of this paper is to clarify the similarities and differences of these trial types for funders, researchers, and policy-makers. In addition, recommendations are offered to help investigators choose, label, and operationalize the most appropriate trial type to answer their research question. These recommendations complement existing reporting guidelines for clinical effectiveness trials (TIDieR) and implementation trials (StaRI).

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Figure 1
Figure 2

Similar content being viewed by others

Data Availability:

There are no data associated with this manuscript.

References

  1. March J, Kraemer HC, Trivedi M, et al. What have we learned about trial design from NIMH-funded pragmatic trials? Neuropsychopharmacology. 2010;35(13):2491-2501.

    Article  PubMed  PubMed Central  Google Scholar 

  2. Thorpe KE, Zwarenstein M, Oxman AD, et al. A pragmatic-explanatory continuum indicator summary (PRECIS): a tool to help trial designers. J Clin Epidemiol. 2009;62(5):464-475.

    Article  PubMed  Google Scholar 

  3. Loudon K, Treweek S, Sullivan F, Donnan P, Thorpe KE, Zwarenstein M. The PRECIS-2 tool: designing trials that are fit for purpose. BMJ (Clinical Research Ed). 2015;350:h2147.

    PubMed  Google Scholar 

  4. Powell BJ, Waltz TJ, Chinman MJ, et al. A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project. Implement Sci. 2015;10:21.

    Article  PubMed  PubMed Central  Google Scholar 

  5. Bauer MS, Damschroder L, Hagedorn H, Smith J, Kilbourne AM. An introduction to implementation science for the non-specialist. BMC Psychol. 2015;3(1):32.

    Article  PubMed  PubMed Central  Google Scholar 

  6. Glasgow RE, McKay HG, Piette JD, Reynolds KD. The RE-AIM framework for evaluating interventions: what can it tell us about approaches to chronic illness management? Patient Educ Couns. 2001;44(2):119-127.

    Article  CAS  PubMed  Google Scholar 

  7. Norton WE, Loudon K, Chambers DA, Zwarenstein M. Designing provider-focused implementation trials with purpose and intent: introducing the PRECIS-2-PS tool. Implement Sci. 2021;16(1):7.

    Article  PubMed  PubMed Central  Google Scholar 

  8. Zatzick D, Palinkas L, Chambers DA, et al. Integrating pragmatic and implementation science randomized clinical trial approaches: a PRagmatic Explanatory Continuum Indicator Summary-2 (PRECIS-2) analysis. Trials. 2023;24(1):288.

    Article  PubMed  PubMed Central  Google Scholar 

  9. Curran GM, Bauer M, Mittman B, Pyne JM, Stetler C. Effectiveness-implementation hybrid designs: combining elements of clinical effectiveness and implementation research to enhance public health impact. Med Care. 2012;50(3):217-226.

    Article  PubMed  PubMed Central  Google Scholar 

  10. Crocker AM, Kessler R, van Eeghen C, et al. Integrating Behavioral Health and Primary Care (IBH-PC) to improve patient-centered outcomes in adults with multiple chronic medical and behavioral health conditions: study protocol for a pragmatic cluster-randomized control trial. Trials. 2021;22(1):200.

    Article  PubMed  PubMed Central  Google Scholar 

  11. Littenberg B, Clifton J, Crocker AM, et al. A cluster randomized trial of primary care practice redesign to integrate behavioral health for those who need it most: Patients with multiple chronic conditions. Ann Fam Med. 2023;21(6):483-495.

  12. Kessler RS, Auxier A, Hitt JR, et al. Development and validation of a measure of primary care behavioral health integration. Fam Syst Health. 2016;34(4):342-356.

    Article  PubMed  Google Scholar 

  13. Dal-Ré R, Janiaud P, Ioannidis JPA. Real-world evidence: how pragmatic are randomized controlled trials labeled as pragmatic? BMC Med. 2018;16(1):49.

    Article  PubMed  PubMed Central  Google Scholar 

  14. Landes SJ, McBain SA, Curran GM. An introduction to effectiveness-implementation hybrid designs. Psychiatry Res. 2019;280:112513.

    Article  PubMed  PubMed Central  Google Scholar 

  15. Powell BJ, Beidas RS, Lewis CC, et al. Methods to improve the selection and tailoring of implementation strategies. J Behav Health Serv Res. 2017;44(2):177-194.

    Article  PubMed  PubMed Central  Google Scholar 

  16. Swindle T, Rutledge JM, Selig JP, et al. Obesity prevention practices in early care and education settings: an adaptive implementation trial. Implement Sci. 2022;17(1):25.

    Article  PubMed  PubMed Central  Google Scholar 

  17. Fortney JC, Rajan S, Reisinger HS, et al. Deploying a telemedicine collaborative care intervention for posttraumatic stress disorder in the U.S. Department of Veterans Affairs: a stepped wedge evaluation of an adaptive implementation strategy. Gen Hosp Psychiatry. 2022;77:109-117.

    Article  PubMed  Google Scholar 

  18. Kilbourne AM, Almirall D, Goodrich DE, et al. Enhancing outreach for persons with serious mental illness: 12-month results from a cluster randomized trial of an adaptive implementation strategy. Implement Sci. 2014;9:163.

    Article  PubMed  PubMed Central  Google Scholar 

  19. Lewis CC, Klasnja P, Powell BJ, et al. From classification to causality: advancing understanding of mechanisms of change in implementation science. Front Public Health. 2018;6:136.

    Article  PubMed  PubMed Central  Google Scholar 

  20. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implementat Sci IS. 2009;4:50.

    Article  PubMed  Google Scholar 

  21. Williams NJ, Glisson C, Hemmelgarn A, Green P. Mechanisms of change in the ARC organizational strategy: increasing mental health clinicians’ EBP adoption through improved organizational culture and capacity. Adm Policy Ment Health. 2017;44(2):269-283.

    Article  PubMed  PubMed Central  Google Scholar 

  22. French C, Pinnock H, Forbes G, Skene I, Taylor SJC. Process evaluation within pragmatic randomised controlled trials: what is it, why is it done, and can we find it?-a systematic review. Trials. 2020;21(1):916.

    Article  PubMed  PubMed Central  Google Scholar 

  23. Proctor E, Silmere H, Raghavan R, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health. 2011;38(2):65-76.

    Article  PubMed  Google Scholar 

  24. Chambers DA, Glasgow RE, Stange KC. The dynamic sustainability framework: addressing the paradox of sustainment amid ongoing change. Implement Sci. 2013;8:117.

    Article  PubMed  PubMed Central  Google Scholar 

  25. Wiltsey Stirman S, Baumann AA, Miller CJ. The FRAME: an expanded framework for reporting adaptations and modifications to evidence-based interventions. Implement Sci. 2019;14(1):58.

    Article  PubMed  PubMed Central  Google Scholar 

  26. Audrey S, Holliday J, Parry-Langdon N, Campbell R. Meeting the challenges of implementing process evaluation within randomized controlled trials: the example of ASSIST (A Stop Smoking in Schools Trial). Health Educ Res. 2006;21(3):366-377.

    Article  PubMed  Google Scholar 

  27. Moore GF, Audrey S, Barker M, et al. Process evaluation of complex interventions: Medical Research Council guidance. BMJ (Clinical Research Ed). 2015;350:h1258.

    PubMed  PubMed Central  Google Scholar 

  28. Esmail LC, Barasky R, Mittman BS, Hickam DH. Improving comparative effectiveness research of complex health interventions: standards from the Patient-Centered Outcomes Research Institute (PCORI). J Gen Intern Med. 2020;35(Suppl 2):875-881.

    Article  PubMed  PubMed Central  Google Scholar 

  29. Oakley A, Strange V, Bonell C, Allen E, Stephenson J. Process evaluation in randomised controlled trials of complex interventions. BMJ (Clinical Research Ed). 2006;332(7538):413-416.

    Article  PubMed  PubMed Central  Google Scholar 

  30. Hawe P, Shiell A, Riley T. Complex interventions: how “out of control” can a randomised controlled trial be? BMJ (Clinical Research Ed). 2004;328(7455):1561-1563.

    Article  PubMed  PubMed Central  Google Scholar 

  31. Ford I, Norrie J. Pragmatic Trials. N Engl J Med. 2016;375(5):454-463.

    Article  PubMed  Google Scholar 

  32. Grol R, Grimshaw J. Evidence-based implementation of evidence-based medicine. Jt Comm J Qual Improv. 1999;25(10):503-513.

    CAS  PubMed  Google Scholar 

  33. Flum DR, Davidson GH, Monsell SE, et al. A randomized trial comparing antibiotics with appendectomy for appendicitis. N Engl J Med. 2020;383(20):1907-1919.

    Article  PubMed  Google Scholar 

  34. Adrian M, Lyon AR, Nicodimos S, Pullmann MD, McCauley E. Enhanced “train and hope” for scalable, cost-effective professional development in youth suicide prevention. Crisis. 2018;39(4):235-246.

    Article  PubMed  Google Scholar 

  35. Cucciare MA, Marchant K, Abraham T, et al. A randomized controlled trial comparing a manual and computer version of CALM in VA community-based outpatient clinics. J Affect Disord Rep. 2021;6:100202.

    Article  PubMed  PubMed Central  Google Scholar 

  36. Kolko DJ, McGuier EA, Turchi R, et al. Care team and practice-level implementation strategies to optimize pediatric collaborative care: study protocol for a cluster-randomized hybrid type III trial. Implement Sci. 2022;17(1):20.

    Article  PubMed  PubMed Central  Google Scholar 

  37. Kolko DJ, Campo J, Kilbourne AM, Hart J, Sakolsky D, Wisniewski S. Collaborative care outcomes for pediatric behavioral health problems: a cluster randomized trial. Pediatrics. 2014;133(4):e981-992.

    Article  PubMed  PubMed Central  Google Scholar 

  38. Hogue A, Ozechowski TJ, Robbins MS, Waldron HB. Making fidelity an intramural game: localizing quality assurance procedures to promote sustainability of evidence‐based practices in usual care. Clin Psychol Sci Pract. 2013;20(1):60.

    Article  Google Scholar 

  39. Hartzler B, Lyon AR, Walker DD, Matthews L, King KM, McCollister KE. Implementing the teen marijuana check-up in schools-a study protocol. Implement Sci. 2017;12(1):103.

    Article  PubMed  PubMed Central  Google Scholar 

  40. Hoffmann TC, Glasziou PP, Boutron I, et al. Better reporting of interventions: template for intervention description and replication (TIDieR) checklist and guide. BMJ (Clinical Research Ed). 2014;348:g1687.

    PubMed  Google Scholar 

  41. Pinnock H, Barwick M, Carpenter CR, et al. Standards for Reporting Implementation Studies (StaRI) statement. BMJ (Clinical Research Ed). 2017;356:i6795.

    Article  PubMed  PubMed Central  Google Scholar 

  42. Pawson R. Pragmatic trials and implementation science: grounds for divorce? BMC Med Res Methodol. 2019;19(1):176.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Funding

This work was supported by grants from the Patient-Centered Outcomes Research Institute (PTSD-2019C1-15636), National Institute of Mental Health (UF1 MH121942), and the Department of Veterans Affairs (QUE 20–007, RCS 17–153) to Dr. Fortney. Drs. Fortney and Lyon are supported by the National Institute of Mental Health (P50MH115837). Dr. Curran is supported by the Translational Research Institute (UL1 TR003107), through the National Center for Advancing Translational Sciences of the National Institutes of Health. Dr. Check is supported by the National Institutes of Health (NIH) Pragmatic Trials Collaboratory funded by the NIH Common Fund through cooperative agreement (U24AT009676) from the Office of Strategic Coordination within the Office of the NIH Director, and by the NIH HEAL Initiative (U24AT010961).

Author information

Authors and Affiliations

  1. Department of Psychiatry and Behavioral Sciences, School of Medicine, University of Washington, Seattle, WA, USA

    John C. Fortney PhD & Aaron R. Lyon PhD

  2. Department of Veterans Affairs, Health Systems Research, Center of Innovation for Veteran-Centered and Value-Driven Care, Seattle, WA, USA

    John C. Fortney PhD

  3. College of Pharmacy, University of Arkansas for Medical Sciences, Little Rock, AR, USA

    Geoffrey M. Curran PhD

  4. Department of Nursing and Midwifery, University of Limerick, Limerick, Ireland

    Geoffrey M. Curran PhD

  5. Department of Population Health Sciences, School of Medicine, Duke University, Durham, NC, USA

    Devon K. Check PhD

  6. Department of Surgery, School of Medicine, University of Washington, Seattle, WA, USA

    David R. Flum MD

Authors
  1. John C. Fortney PhD
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Geoffrey M. Curran PhD
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Aaron R. Lyon PhD
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Devon K. Check PhD
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. David R. Flum MD
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to John C. Fortney PhD.

Ethics declarations

Contributors:

None.

Conflict of Interest:

The authors declare that they do not have a conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Prior presentations:

None.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Fortney, J.C., Curran, G.M., Lyon, A.R. et al. Similarities and Differences Between Pragmatic Trials and Hybrid Effectiveness-Implementation Trials. J GEN INTERN MED (2024). https://doi.org/10.1007/s11606-024-08747-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11606-024-08747-1

KEY WORDS

Search

Navigation