Multilevel Mechanisms of Implementation Strategies in Mental Health: Integrating Theory, Research, and Practice

  • Nathaniel J. WilliamsEmail author
Original Article


A step toward the development of optimally effective, efficient, and feasible implementation strategies that increase evidence-based treatment integration in mental health services involves identification of the multilevel mechanisms through which these strategies influence implementation outcomes. This article (a) provides an orientation to, and rationale for, consideration of multilevel mediating mechanisms in implementation trials, and (b) systematically reviews randomized controlled trials that examined mediators of implementation strategies in mental health. Nine trials were located. Mediation-related methodological deficiencies were prevalent and no trials supported a hypothesized mediator. The most common reason was failure to engage the mediation target. Discussion focuses on directions to accelerate implementation strategy development in mental health.


Mechanism Mediation Multilevel Implementation strategy Mental health Systematic review 



This work was supported by funding from the National Institute of Mental Health under Award Number F31MH099846 to Nathaniel J. Williams. The content is solely the responsibility of the author and does not necessarily represent the official views of the National Institutes of Health.


  1. Aarons, G. A., Ehrhart, M. G., Farahnak, L. R., & Sklar, M. (2014). Aligning leadership across systems and organizations to develop a strategic climate for evidence-based practice implementation. Annual Review of Public Health, 35, 255–274. doi: 10.1146/annurev-publhealth-032013-182447.CrossRefPubMedPubMedCentralGoogle Scholar
  2. Aarons, G. A., Hurlburt, M., & Horwitz, S. M. (2011). Advancing a conceptual model of evidence-based practice implementation in public sectors. Administration and Policy in Mental Health, 38, 4–23. doi: 10.1007/s10488-010-0327-7.CrossRefPubMedGoogle Scholar
  3. Atkins, M. S., Frazier, S. L., Leathers, S. J., Graczyk, P. A., Talbott, E., Jakobsons, L., et al. (2008). Teacher key opinion leaders and mental health consultation in low-income urban schools. Journal of Consulting and Clinical Psychology, 76, 905–908. doi: 10.1037/a0013036.CrossRefPubMedGoogle Scholar
  4. Baer, J. S., Wells, E. A., Rosengren, D. B., Hartzler, B., Beadnell, B., & Dunn, C. (2009). Agency context and tailored training in technology transfer: A pilot evaluation of motivational interviewing training for community counselors. Journal of Substance Abuse Treatment, 37, 191–202. doi: 10.1016/j.jsat.2009.01.003.CrossRefPubMedPubMedCentralGoogle Scholar
  5. Baron, R. M., & Kenny, D. A. (1986). The moderator-mediator variable distinction in social psychological research: Conceptual, strategic, and statistical considerations. Journal of Personality and Social Psychology, 51, 1173–1182. doi: 10.1037/0022-3514.51.6.1173.CrossRefPubMedGoogle Scholar
  6. Barwick, M. A., Schachter, H. M., Bennett, L. M., McGowan, J., Ly, M., Wilson, A., et al. (2012). Knowledge translation efforts in child and youth mental health: A systematic review. Journal of Evidence Based Social Work, 9, 369–395. doi: 10.1080/15433714.2012.663667.CrossRefPubMedPubMedCentralGoogle Scholar
  7. Bauer, D. J., Preacher, K. J., & Gil, K. M. (2006). Conceptualizing and testing random indirect effects and moderated mediation in multilevel models: New procedures and recommendations. Psychological Methods, 11, 142–163. doi: 10.1037/1082-989x.11.2.142.supp.CrossRefPubMedGoogle Scholar
  8. Berwick, D. M. (2008). The science of improvement. Journal of the American Medical Association, 299, 1182–1184. doi: 10.1001/jama.299.10.1182.CrossRefPubMedGoogle Scholar
  9. Biesanz, J. C., Falk, C. F., & Savalei, V. (2010). Assessing mediational models: Testing and interval estimation for indirect effects. Multivariate Behavioral Research, 45, 661–701. doi: 10.1080/00273171.2010.498292.CrossRefPubMedGoogle Scholar
  10. Chan, D. (1998). Functional relations among constructs in the same content domain at different levels of analysis: A typology of composition models. Journal of Applied Psychology, 83, 234–246. doi: 10.1037/0021-9010.83.2.234.CrossRefGoogle Scholar
  11. Chaudoir, S. R., Dugan, A. G., & Barr, C. H. (2013). Measuring factors affecting implementation of health innovations: a systematic review of structural, organizational, provider, patient, and innovation level measures. Implementation Science, 8, 22. doi: 10.1186/1748-5908-8-22.CrossRefPubMedPubMedCentralGoogle Scholar
  12. Chen, H. (1990). Theory-driven evaluations. Newbury Park, CA: Sage.Google Scholar
  13. Collins, P. Y., Patel, V., Joestl, S. S., March, D., Insel, T. R., & Daar, A. S. (2011). Grand challenges in global mental health. Nature, 475, 27–30. doi: 10.1038/475027a.CrossRefPubMedPubMedCentralGoogle Scholar
  14. Damschroder, L. J., Aron, D. C., Keith, R. E., Kirsh, S. R., Alexander, J. A., & Lowery, J. C. (2009). Fostering implementation of health services research findings into practice: A consolidated framework for advancing implementation science. Implementation Science, 4, 50. doi: 10.1186/1748-5908-4-50.CrossRefPubMedPubMedCentralGoogle Scholar
  15. Davies, P., Walker, A. E., & Grimshaw, J. M. (2010). A systematic review of the use of theory in the design of guideline dissemination and implementation strategies and interpretation of the results of rigorous evaluations. Implementation Science, 5(14), 5908-5. doi: 10.1186/1748-5908-5-14.Google Scholar
  16. Doss, B. D. (2004). Changing the way we study change in psychotherapy. Clinical Psychology: Science and Practice, 11, 368–386. doi: 10.1186/1748-5908-5-14.Google Scholar
  17. Edwards, J. R., & Lambert, L. S. (2007). Methods for integrating moderation and mediation: A general analytical framework using moderated path analysis. Psychological Methods, 12, 1–22. doi: 10.1037/1082-989x.12.1.1.supp.CrossRefPubMedGoogle Scholar
  18. Enders, C. (2013). Centering predictors and contextual effects. In M. A. Scott, J. S. Simonoff, & B. D. Marx (Eds.), The SAGE handbook of multilevel modeling (pp. 89–108). Thousand Oaks, CA: Sage Publications Inc.CrossRefGoogle Scholar
  19. Garland, A. F., Haine-Schlagel, R., Brookman-Frazee, L., Baker-Ericzen, M., Trask, E., & Fawley-King, K. (2013). Improving community-based mental health care for children: Translating knowledge into action. Administration and Policy in Mental Health, 40, 6–22. doi: 10.1371/journal.pmed.1001434.CrossRefPubMedPubMedCentralGoogle Scholar
  20. Garner, B. R., Godley, S. H., & Bair, C. M. L. (2011). The impact of pay-for-performance on therapists’ intentions to deliver high quality treatment. Journal of Substance Abuse Treatment, 41, 97–103. doi: 10.1016/j.jsat.2011.01.012.CrossRefPubMedPubMedCentralGoogle Scholar
  21. Glisson, C., Schoenwald, S. K., Hemmelgarn, A., Green, P., Dukes, D., Armstrong, K. S., & Chapman, J. E. (2010). Randomized trial of MST and ARC in a two-level evidence-based treatment implementation strategy. Journal of Consulting and Clinical Psychology, 78, 537–550. doi: 10.1037/a0019160.CrossRefPubMedPubMedCentralGoogle Scholar
  22. Glisson, C., & Williams, N. J. (2015). Assessing and changing organizational social contexts for effective mental health services. Annual Review of Public Health, 36, 507–523. doi: 10.1146/annurev-publhealth-031914-122435.CrossRefPubMedGoogle Scholar
  23. Godin, G., Belanger-Gravel, A., Eccles, M., & Grimshaw, J. (2008). Healthcare professionals’ intentions and behaviors: A systematic review of studies based on social cognitive theories. Implementation Science, 3(36), 1–12.Google Scholar
  24. Greenhalgh, T., Robert, G., MacFarlane, F., Bate, P., & Kyriakidou, O. (2004). Diffusion of innovations in service organizations: A systematic review and recommendations. Milbank Quarterly, 82, 581–629. doi: 10.1111/j.0887-378x.2004.00325.x.CrossRefPubMedPubMedCentralGoogle Scholar
  25. Grol, R. P., Bosch, M. C., Hulscher, M. E., Eccles, M. P., & Wensing, M. (2007). Planning and studying improvement in patient care: The use of theoretical perspectives. Milbank Quarterly, 85, 93–138. doi: 10.1111/j.1468-0009.2007.00478.x.CrossRefPubMedPubMedCentralGoogle Scholar
  26. Hayes, A. F., & Scharkow, M. (2013). The relative trustworthiness of inferential tests of the indirect effect in statistical mediation analysis: Does method really matter? Psychological Science, 24, 1918–1927. doi: 10.1177/0956797613480187.CrossRefPubMedGoogle Scholar
  27. Holth, P., Torsheim, T., Sheidow, A. J., Ogden, T., & Henggeler, S. W. (2011). Intensive quality assurance of therapist adherence to behavioral interventions for adolescent substance use problems. Journal of Child and Adolescent Substance Abuse, 20, 289–313. doi: 10.1080/1067828x.2011.581974.CrossRefPubMedPubMedCentralGoogle Scholar
  28. Insel, T. R. (2009). Translating scientific opportunity into public health impact: A strategic plan for research on mental illness. Archives of General Psychiatry, 66, 128–133. doi: 10.1001/archgenpsychiatry.2008.540.CrossRefPubMedGoogle Scholar
  29. Institute of Medicine. (2001). Crossing the quality chasm: A new health system for the 21st century. Washington, DC: Author.Google Scholar
  30. Judd, C. M., & Kenny, D. A. (1981). Process analysis: Estimating mediation in treatment evaluations. Evaluation Review, 5, 602–619. doi: 10.1080/1067828x.2011.581974.CrossRefGoogle Scholar
  31. Kauth, M. R., Sullivan, G., Blevins, D., Cully, J. A., Landes, R. D., Said, Q., & Teasdale, T. A. (2010). Employing external facilitation to implement cognitive behavioral therapy in VA clinics: A pilot study. Implementation Science, 5, 75. doi: 10.1186/1748-5908-5-75.CrossRefPubMedPubMedCentralGoogle Scholar
  32. Kazdin, A. E. (2007). Mediators and mechanisms of change in psychotherapy research. Annual Review of Clinical Psychology, 3, 1–27. doi: 10.1146/annurev.clinpsy.3.022806.091432.CrossRefPubMedGoogle Scholar
  33. Kenny, D. A., Korchmaros, J. D., & Bolger, N. (2003). Lower level mediation in multilevel models. Psychological Methods, 8, 115–128. doi: 10.1037/1082-989x.8.2.115.CrossRefPubMedGoogle Scholar
  34. Kessler, R. C., Aguilar-Gaxiola, S., Alonso, J., Chatterji, S., Lee, S., Ormel, J., et al. (2009). The global burden of mental disorders: An update from the WHO World Mental Health (WMH) Surveys. Epidemiologia e Psichiatria Sociale, 18(1), 23–33. doi: 10.1017/s1121189x00001421.CrossRefPubMedPubMedCentralGoogle Scholar
  35. Klein, K. J., Dansereau, F., & Hall, R. J. (1994). Levels issues in theory development, data collection, and analysis. Academy of Management Review, 19, 195–229. doi: 10.5465/AMR.1994.9410210745.Google Scholar
  36. Kozlowski, S. W. J., & Klein, K. J. (2000). A multilevel approach to theory and research in organizations: Contextual, temporal, and emergent properties. In K. J. Klein & S. W. J. Kozlowski (Eds.), Multilevel theory, research, and methods in organizations: Foundations, extensions, and new directions (pp. 3–90). San Francisco: Jossey-Bass.Google Scholar
  37. Kraemer, H. C., Kazdin, A. E., Offord, D. R., Kessler, R. C., Jensen, P. S., & Kupfer, D. J. (1997). Coming to terms with the terms of risk. Archives of General Psychiatry, 54, 337–343. doi: 10.1001/archpsyc.1997.01830160065009.CrossRefPubMedGoogle Scholar
  38. Kreft, I. G. G., de Leeuw, J., & Aiken, L. S. (1995). The effects of different forms of centering in hierarchical linear models. Multivariate Behavioral Research, 30, 1–21. doi: 10.1207/s15327906mbr3001_1.CrossRefPubMedGoogle Scholar
  39. Krull, J. L., & MacKinnon, D. P. (2001). Multilevel modeling of individual and group level mediated effects. Multivariate Behavioral Research, 36, 249–277. doi: 10.1207/S15327906MBR3602_06.CrossRefPubMedGoogle Scholar
  40. Landsverk, J., Brown, C. H., Reutz, J. R., Palinkas, L., & Horwitz, S. M. (2011). Design elements in implementation research: A structured review of child welfare and mental health studies. Administration and Policy in Mental Health, 38, 54–63. doi: 10.1007/s10488-010-0315-y.CrossRefPubMedPubMedCentralGoogle Scholar
  41. LeBreton, J. M., & Senter, J. L. (2008). Answers to 20 questions about interrater reliability and interrater agreement. Organizational Research Methods, 11, 815–852. doi: 10.1177/1094428106296642.CrossRefGoogle Scholar
  42. Lochman, J. E., Boxmeyer, C., Powell, N., Qu, L., Wells, K., & Windle, M. (2009). Dissemination of the coping power program: Importance of intensity of counselor training. Journal of Consulting and Clinical Psychology, 77, 397–409. doi: 10.1037/a0014514.CrossRefPubMedGoogle Scholar
  43. MacKinnon, D. P. (2008). Introduction to statistical mediation analysis. New York: Lawrence Erlbaum Associates.Google Scholar
  44. MacKinnon, D. P., Fairchild, A. J., & Fritz, M. S. (2007). Mediation analysis. Annual Review of Psychology, 58, 593–614. doi: 10.1146/annurev.psych.58.110405.085542.CrossRefPubMedPubMedCentralGoogle Scholar
  45. MacKinnon, D. P., Lockwood, C. M., Hoffman, J. M., West, S. G., & Sheets, V. (2002). A comparison of methods to test mediation and other intervening variable effects. Psychological Methods, 7, 83–104. doi: 10.1037/1082-989X.7.1.83.CrossRefPubMedPubMedCentralGoogle Scholar
  46. MacKinnon, D. P., Lockwood, C. M., & Williams, J. (2004). Confidence limits for the indirect effect: Distribution of the product and resampling methods. Multivariate Behavioral Research, 39, 99–128. doi: 10.1207/s15327906mbr3901_4.CrossRefPubMedPubMedCentralGoogle Scholar
  47. Mathieu, J. E., DeShon, R. P., & Bergh, D. D. (2008). Mediational inferences in organizational research: Then, now, and beyond. Organizational Research Methods, 11, 203–223. doi: 10.1177/1094428107310089.CrossRefGoogle Scholar
  48. Mathieu, J. E., & Taylor, S. R. (2007). A framework for testing meso-mediational relationships in organizational behavior. Journal of Organizational Behavior, 28, 141–172. doi: 10.1002/job.436.CrossRefGoogle Scholar
  49. McHugh, R. K., & Barlow, D. H. (2012). The reach of evidence-based psychological interventions. In R. K. McHugh & D. H. Barlow (Eds.), Dissemination and implementation of evidence-based psychological interventions (pp. 3–15). New York: Oxford University Press.Google Scholar
  50. Michie, S., van Stralen, M. M., & West, R. (2011). The behavior change wheel: A new method for characterizing and designing behavior change interventions. Implementation Science, 6, 42. doi: 10.1186/1748-5908-6-42.CrossRefPubMedPubMedCentralGoogle Scholar
  51. Neuhaus, J. M., & Kalbfleisch, J. D. (1998). Between- and within-cluster covariate effects in the analysis of clustered data. Biometrics, 54, 638–645. doi: 10.2307/3109770.CrossRefPubMedGoogle Scholar
  52. Novins, D. K., Green, A. E., Legha, R. K., & Aarons, G. A. (2013). Dissemination and implementation of evidence-based practices for child and adolescent mental health: A systematic review. Journal of the American Academy of Child and Adolescent Psychiatry, 52, 1009–1025. doi: 10.1016/j.jaac.2013.07.012.CrossRefPubMedPubMedCentralGoogle Scholar
  53. Pituch, K. A., Murphy, D. L., & Tate, R. L. (2010). Three-level models for indirect effects in school- and class-randomized experiments in education. Journal of Experimental Education, 78, 60–95. doi: 10.1080/00220970903224685.CrossRefGoogle Scholar
  54. Powell, B. J., Proctor, E. K., & Glass, J. E. (2014). A systematic review of strategies for implementing empirically supported mental health interventions. Research on Social Work Practice, 24, 192–212. doi: 10.1177/1049731513505778.CrossRefPubMedGoogle Scholar
  55. Preacher, K. J., & Hayes, A. F. (2004). Asymptotic and resampling strategies for assessing and comparing indirect effects in multiple mediator models. Behavior Research Methods, 40, 879–891. doi: 10.3758/brm.40.3.879.CrossRefGoogle Scholar
  56. Preacher, K. J., Rucker, D. D., & Hayes, A. F. (2007). Addressing moderated mediation hypotheses: Theory, methods, and prescriptions. Multivariate Behavioral Research, 42, 185–227. doi: 10.1080/00273170701341316.CrossRefPubMedGoogle Scholar
  57. Preacher, K. J., & Selig, J. P. (2012). Advantages of Monte Carlo confidence intervals for indirect effects. Communication Methods and Measures, 6, 77–98. doi: 10.1080/19312458.2012.679848.CrossRefGoogle Scholar
  58. Preacher, K. J., Zyphur, M. J., & Zhang, Z. (2010). A general multilevel SEM framework for assessing multilevel mediation. Psychological Methods, 15, 209–233. doi: 10.1037/a0020141.CrossRefPubMedGoogle Scholar
  59. Proctor, E., Silmere, H., Raghavan, R., Hovmand, P., Aarons, G., Bunger, A., et al. (2011). Outcomes for implementation research: Conceptual distinctions, measurement challenges, and research agenda. Administration and Policy in Mental Health, 38, 65–76. doi: 10.1007/s10488-010-0319-7.CrossRefPubMedGoogle Scholar
  60. Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models: Applications and data analysis (2nd ed.). Thousand Oaks, CA: Sage publications.Google Scholar
  61. Rohrbach, L. A., Graham, J. W., & Hansen, W. B. (1993). Diffusion of a school-based substance abuse prevention program: Predictors of program implementation. Preventive Medicine, 22, 237–260. doi: 10.1006/pmed.1993.1020.CrossRefPubMedGoogle Scholar
  62. Rousseau, D. M. (1985). Issues of level in organizational research: Multi-level and cross-level perspectives. In L. L. Cummings & B. M. Straw (Eds.), Research in organizational behavior (Vol. 7, pp. 1–37). Greenwich, CT: JAI Press.Google Scholar
  63. Saloner, B., Carson, N., & Lê Cook, B. (2014). Episodes of mental health treatment among a nationally representative sample of children and adolescents. Medical Care Research and Review, 71, 261–279. doi: 10.1177/1077558713518347.CrossRefPubMedGoogle Scholar
  64. Sobel, M. E. (1982). Asymptotic confidence intervals for indirect effects in structural equation models. Sociological Methodology, 13, 290–312. doi: 10.2307/270723.CrossRefGoogle Scholar
  65. Steel, Z., Marnane, C., Iranpour, C., Chey, T., Jackson, J. W., Patel, V., & Silove, D. (2014). The global prevalence of common mental disorders: A systematic review and meta-analysis 1980–2013. International Journal of Epidemiology, 43, 476–493. doi: 10.1093/ije/dyu038.CrossRefPubMedPubMedCentralGoogle Scholar
  66. Tabak, R. G., Khoong, E. C., Chambers, D. A., & Brownson, R. C. (2012). Bridging research and practice: Models for dissemination and implementation research. American Journal of Preventive Medicine, 43, 337–350. doi: 10.1016/j.amepre.2012.05.024.CrossRefPubMedPubMedCentralGoogle Scholar
  67. Taylor, A. B., MacKinnon, D. P., & Tein, J. Y. (2008). Tests of the three-path mediated effect. Organizational Research Methods, 11, 241–269. doi: 10.1177/1094428107300344.CrossRefGoogle Scholar
  68. Weisz, J. R., Kuppens, S., Eckshtain, D., Ugueto, A. M., Hawley, K. M., & Jensen-Doss, A. (2013). Performance of evidence-based youth psychotherapies compared with usual clinical care: A multilevel meta-analysis. JAMA Psychiatry, 70, 750–761. doi: 10.1001/jamapsychiatry.2013.1176.CrossRefPubMedGoogle Scholar
  69. Williams, N. J. (2015). Mechanisms of change in an organizational culture and climate intervention for increasing clinicians’ evidence-based practice adoption in mental health (Unpublished doctoral dissertation). University of Tennessee, Knoxville.
  70. Williams, N. J., & Glisson, C. (2014). The role of organizational culture and climate in the dissemination and implementation of empirically-supported treatments for youth. In R. S. Beidas & P. C. Kendall (Eds.), Dissemination and implementation of evidence based practices in child and adolescent mental health (pp. 61–81). New York: Oxford University Press.Google Scholar
  71. Williams, J. R., Williams, W. O., Dusablon, T., Blais, M. P., Tregear, S. J., Banks, D., & Hennessy, K. D. (2014). Evaluation of a randomized intervention to increase adoption of comparative effectiveness research by community health organizations. Journal of Behavioral Health Services and Research, 41, 308–323. doi: 10.1007/s11414-013-9369-4.CrossRefPubMedGoogle Scholar
  72. Zhang, Z., Zyphur, M. J., & Preacher, K. J. (2009). Testing multilevel mediation using hierarchical linear models: Problems and solutions. Organizational Research Methods, 12, 695–719. doi: 10.1177/1094428108327450.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2015

Authors and Affiliations

  1. 1.School of Social WorkBoise State UniversityBoiseUSA

Personalised recommendations