Abstract
Evidence-based interventions (EBIs) are practices or programs that have peer-reviewed, documented empirical evidence of effectiveness on a clinical outcome. Clinicians rely on the scientific evidence supporting EBIs to ensure they are delivering effective treatments for patients and their families. However, we often do not know and do not ask how the evidence was generated for a given intervention, which could cause problems transferring EBIs from a research context to real-world practice. EBIs are often adapted from their original form when implemented in a new setting or with a new patient group to improve fit. However, changing the core components of an EBI could reduce its effectiveness. This chapter will review how the evidence is generated for EBIs, discuss challenges that can arise when EBIs do not fit when transferred to routine practice settings, and present principles for guiding adaptations to EBIs while maintaining the core functions that make them effective. Finally, it will provide a discussion of the overall merits and shortcomings of current approaches to generating EBIs and how we can encourage practice innovations and discourage the use of interventions with limited support for their effectiveness.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Aarons, G. A., Sklar, M., Mustanski, B., Benbow, N., & Brown, C. H. (2017). “Scaling-out” evidence-based interventions to new populations or new health care delivery systems. Implementation Science, 12(1), 111. https://doi.org/10.1186/s13012-017-0640-6
Aarons Gregory, A., Miller Elizabeth, A., Green Amy, E., Perrott Jennifer, A., & Bradway, R. (2012). Adaptation happens: a qualitative case study of implementation of The Incredible Years evidence-based parent training programme in a residential substance abuse treatment programme. Journal of Children’s Services, 7(4), 233–245. https://doi.org/10.1108/17466661211286463
Aschbrenner, K. A., Bond, G. R., Pratt, S. I., Jue, K., Williams, G., Banerjee, S., & Bartels, S. J. (2020). Evaluating agency-led adaptions to an evidence-based lifestyle intervention for adults with serious mental illness. Implementation Research and Practice, 1, 2633489520943200. https://doi.org/10.1177/2633489520943200
Bauchner, H., Golub, R. M., & Fontanarosa, P. B. (2019). Reporting and Interpretation of Randomized Clinical Trials. JAMA, 322(8), 732–735. https://doi.org/10.1001/jama.2019.12056
Baumann, A. A., Cabassa, L. J., & Stirman, S. W. (2017). Adaptation in dissemination and implementation science. In Dissemination and Implementation Research in Health: Translating Science to Practice, Second Edition (pp. 285–300). Oxford University Press. https://doi.org/10.1093/oso/9780190683214.003.0017
Bond, G. R., & Drake, R. E. (2020). Assessing the Fidelity of Evidence-Based Practices: History and Current Status of a Standardized Measurement Methodology. Administration and Policy in Mental Health and Mental Health Services Research, 47(6), 874–884. https://doi.org/10.1007/s10488-019-00991-6
Bond, G. R., Drake, R. E., & Becker, D. R. (2012). Generalizability of the Individual Placement and Support (IPS) model of supported employment outside the US. World Psychiatry, 11(1), 32–39. https://doi.org/10.1016/j.wpsyc.2012.01.005
Bond, G. R., Evans, L., Salyers, M. P., Williams, J., & Kim, H. W. (2000). Measurement of fidelity in psychiatric rehabilitation. Ment Health Serv Res, 2(2), 75–87. https://doi.org/10.1023/a:1010153020697
Bowen, D. J., Kreuter, M., Spring, B., Cofta-Woerpel, L., Linnan, L., Weiner, D., Bakken, S., Kaplan, C. P., Squiers, L., Fabrizio, C., & Fernandez, M. (2009). How we design feasibility studies. American Journal of Preventive Medicine, 36(5), 452–457. https://doi.org/10.1016/j.amepre.2009.02.002
Breitenstein, S. M., Gross, D., Garvey, C. A., Hill, C., Fogg, L., & Resnick, B. (2010). Implementation fidelity in community-based interventions. Research in nursing & health, 33(2), 164–173. https://doi.org/10.1002/nur.20373
Carroll, C., Patterson, M., Wood, S., Booth, A., Rick, J., & Balain, S. (2007). A conceptual framework for implementation fidelity. Implementation Science, 2(1), 40. https://doi.org/10.1186/1748-5908-2-40
Carroll, K. M., & Rounsaville, B. J. (2003). Bridging the Gap: A Hybrid Model to Link Efficacy and Effectiveness Research in Substance Abuse Treatment. Psychiatric Services, 54(3), 333–339. https://doi.org/10.1176/appi.ps.54.3.333
Chambers, D. A., Glasgow, R. E., & Stange, K. C. (2013). The dynamic sustainability framework: addressing the paradox of sustainment amid ongoing change. Implementation Science, 8(1), 117. https://doi.org/10.1186/1748-5908-8-117
Czajkowski, S. M., Powell, L. H., Adler, N., Naar-King, S., Reynolds, K. D., Hunter, C. M., Laraia, B., Olster, D. H., Perna, F. M., Peterson, J. C., Epel, E., Boyington, J. E., & Charlson, M. E. (2015). From ideas to efficacy: The ORBIT model for developing behavioral treatments for chronic diseases. Health Psychol, 34(10), 971–982. https://doi.org/10.1037/hea0000161
Drake, R. E., Wilkniss, S. M., Frounfelker, R. L., Whitley, R., Zipple, A. M., McHugo, G. J., & Bond, G. R. (2009). Public-Academic Partnerships: The Thresholds-Dartmouth Partnership and Research on Shared Decision Making. Psychiatric Services, 60(2), 142–144. https://doi.org/10.1176/ps.2009.60.2.142
Eccles, M. P., & Mittman, B. S. (2006). Welcome to Implementation Science. Implementation Science, 1(1), 1. https://doi.org/10.1186/1748-5908-1-1
El-Kotob, R., & Giangregorio, L. M. (2018). Pilot and feasibility studies in exercise, physical activity, or rehabilitation research. Pilot and Feasibility Studies, 4(1), 137. https://doi.org/10.1186/s40814-018-0326-0
Escoffery, C., Lebow-Skelley, E., Haardoerfer, R., Boing, E., Udelson, H., Wood, R., Hartman, M., Fernandez, M. E., & Mullen, P. D. (2018). A systematic review of adaptations of evidence-based public health interventions globally. Implementation Science, 13(1), 125. https://doi.org/10.1186/s13012-018-0815-9
Gitlin, L. N. (2013). Introducing a new intervention: an overview of research phases and common challenges. The American journal of occupational therapy: official publication of the American Occupational Therapy Association, 67(2), 177–184. https://doi.org/10.5014/ajot.2013.006742
Glasgow, R. E., & Riley, W. T. (2013). Pragmatic Measures: What They Are and Why We Need Them. American Journal of Preventive Medicine, 45(2), 237–243. https://doi.org/10.1016/j.amepre.2013.03.010
Green, L. W. (2014). Closing the chasm between research and practice: evidence of and for change. Health Promotion Journal of Australia, 25(1), 25–29. https://doi.org/10.1071/he13101
Kilbourne, A. M., Neumann, M. S., Waxmonsky, J., Bauer, M. S., Kim, H. M., Pincus, H. A., Hepburn, B., & Thomas, Marshall. (2012). Public-Academic Partnerships: Evidence-Based Implementation: The Role of Sustained Community-Based Practice and Research Partnerships. Psychiatric Services, 63(3), 205–207. https://doi.org/10.1176/appi.ps.201200032
Kirk, M., Haines, E. R., Rokoske, F. S., Powell, B., Weinberger, M., Hanson, L., & Birken, S. (2019). A case study of a theory-based method for identifying and reporting core functions and forms of evidence-based interventions. Transl Behav Med.
Kistin, C., & Silverstein, M. (2015). Pilot Studies: A Critical but Potentially Misused Component of Interventional Research. JAMA, 314(15), 1561–1562. https://doi.org/10.1001/jama.2015.10962
Kraemer, H. C., Mintz, J., Noda, A., Tinklenberg, J., & Yesavage, J. A. (2006). Caution regarding the use of pilot studies to guide power calculations for study proposals. Arch Gen Psychiatry, 63(5), 484–489. https://doi.org/10.1001/archpsyc.63.5.484
Leon, A. C., Davis, L. L., & Kraemer, H. C. (2011). The role and interpretation of pilot studies in clinical research. Journal of psychiatric research, 45(5), 626–629. https://doi.org/10.1016/j.jpsychires.2010.10.008
Marchand, E., Stice, E., Rohde, P., & Becker, C. B. (2011). Moving from efficacy to effectiveness trials in prevention research. Behav Res Ther, 49(1), 32–41. https://doi.org/10.1016/j.brat.2010.10.008
McGurk, S. R., Mueser, K. T., Xie, H., Welsh, J., Kaiser, S., Drake, R. E., Becker, D., Bailey, E., Fraser, G., Wolfe, R., & McHugo, G. J. (2015). Cognitive enhancement treatment for people with mental illness who do not respond to supported employment: A randomized controlled trial. American Journal of Psychiatry, 172(9), 852–861.
Möller, H.-J. (2011). Effectiveness studies: advantages and disadvantages. Dialogues in clinical neuroscience, 13(2), 199–207. https://pubmed.ncbi.nlm.nih.gov/21842617 https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3181999/
Moncher, F. J., & Prinz, R. J. (1991). Treatment fidelity in outcome studies. Clin Psychol Rev, 11(3), 247–266. https://doi.org/10.1016/0272-7358(91)90103-2
Morris, Z. S., Wooding, S., & Grant, J. (2011). The answer is 17 years, what is the question: understanding time lags in translational research. Journal of the Royal Society of Medicine, 104(12), 510–520. https://doi.org/10.1258/jrsm.2011.110180
Onken, L. S. (2019). History and Evaluation of the NIH Stage Model: Overcoming hurdles to create behavioral interventions to improve public health. In S. Dimidjian (Ed.), Evidence-based practice in action: bridging clinical science and intervention (pp. 28–42). The Guilford Press.
Perez Jolles, M., Lengnick-Hall, R., & Mittman, B. S. (2019). Core Functions and Forms of Complex Health Interventions: a Patient-Centered Medical Home Illustration. J Gen Intern Med, 34(6), 1032–1038. https://doi.org/10.1007/s11606-018-4818-7
Powell, B. J., Garcia, K., & Fernandez, M. (2019). Implementation Strategies. In D. Chambers, C. Vinson, & W. Norton (Eds.), Optimizing the Cancer Control Continuum: Advancing Implementation Research (pp. 98–120). Oxford University Press.
Powell, B. J., Waltz, T. J., Chinman, M. J., Damschroder, L. J., Smith, J. L., Matthieu, M. M., Proctor, E. K., & Kirchner, J. E. (2015). A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project. Implementation Science, 10(1), 21. https://doi.org/10.1186/s13012-015-0209-1
Proctor, E., Silmere, H., Raghavan, R., Hovmand, P., Aarons, G., Bunger, A., Griffey, R., & Hensley, M. (2011). Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Administration and policy in mental health, 38(2), 65–76. https://doi.org/10.1007/s10488-010-0319-7
Proctor, E. K., Landsverk, J., Aarons, G., Chambers, D., Glisson, C., & Mittman, B. (2009). Implementation research in mental health services: an emerging science with conceptual, methodological, and training challenges. Administration and policy in mental health, 36(1), 24–34. https://doi.org/10.1007/s10488-008-0197-4
Proctor, E. K., Powell, B. J., & McMillen, J. C. (2013). Implementation strategies: recommendations for specifying and reporting. Implementation science: IS, 8, 139–139. https://doi.org/10.1186/1748-5908-8-139
Rothwell, P. M. (2005). External validity of randomised controlled trials: "to whom do the results of this trial apply?". Lancet, 365(9453), 82–93. https://doi.org/10.1016/s0140-6736(04)17670-8
Roy-Byrne, P. P., Sherbourne, C. D., Craske, M. G., Stein, M. B., Katon, W., Sullivan, G., Means-Christensen, A., & Bystritsky, A. (2003). Moving Treatment Research From Clinical Trials to the Real World. Psychiatric Services, 54(3), 327–332. https://doi.org/10.1176/appi.ps.54.3.327
Shean, G. D. (2012). Some Limitations on the External Validity of Psychotherapy Efficacy Studies and Suggestions for Future Research. American Journal of Psychotherapy, 66(3), 227–242. https://doi.org/10.1176/appi.psychotherapy.2012.66.3.227
Singal, A. G., Higgins, P. D. R., & Waljee, A. K. (2014). A primer on effectiveness and efficacy trials. Clinical and translational gastroenterology, 5(1), e45–e45. https://doi.org/10.1038/ctg.2013.13
Teague, G. B., Mueser, K. T., & Rapp, C. A. (2012). Advances in fidelity measurement for mental health services research: four measures. Psychiatr Serv, 63(8), 765–771. https://doi.org/10.1176/appi.ps.201100430
Torpy, J. M., Lynm, C., & Glass, R. M. (2010). Randomized Controlled Trials. JAMA, 303(12), 1216–1216. https://doi.org/10.1001/jama.303.12.1216
Waltz, J., Addis, M. E., Koerner, K., & Jacobson, N. S. (1993). Testing the integrity of a psychotherapy protocol: assessment of adherence and competence. J Consult Clin Psychol, 61(4), 620–630. https://doi.org/10.1037//0022-006x.61.4.620
Wiltsey Stirman, S., Baumann, A. A., & Miller, C. J. (2019). The FRAME: an expanded framework for reporting adaptations and modifications to evidence-based interventions. Implementation Science, 14(1), 58. https://doi.org/10.1186/s13012-019-0898-y
Wiltsey Stirman, S., Gamarra, J. M., Bartlett, B. A., Calloway, A., & Gutner, C. A. (2017). Empirical Examinations of Modifications and Adaptations to Evidence-Based Psychotherapies: Methodologies, Impact, and Future Directions. Clinical Psychology: Science and Practice, 24(4), 396–420. https://doi.org/10.1111/cpsp.12218
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this chapter
Cite this chapter
Aschbrenner, K.A., Torrey, W.C. (2022). Developing, Evaluating and Implementing Evidence-Based Interventions in Real World Practice.. In: Sowers, W.E., McQuistion, H.L., Ranz, J.M., Feldman, J.M., Runnels, P.S. (eds) Textbook of Community Psychiatry. Springer, Cham. https://doi.org/10.1007/978-3-031-10239-4_21
Download citation
DOI: https://doi.org/10.1007/978-3-031-10239-4_21
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-10238-7
Online ISBN: 978-3-031-10239-4
eBook Packages: Behavioral Science and PsychologyBehavioral Science and Psychology (R0)