American Journal of Community Psychology

, Volume 50, Issue 3–4, pp 481–496 | Cite as

Practical Implementation Science: Developing and Piloting the Quality Implementation Tool

  • Duncan C. Meyers
  • Jason Katz
  • Victoria Chien
  • Abraham Wandersman
  • Jonathan P. Scaccia
  • Annie Wright
Original Paper

Abstract

According to the Interactive Systems Framework for Dissemination and Implementation, implementation is a major mechanism and concern in bridging research and practice. The growing number of implementation frameworks need to be synthesized and translated so that the science and practice of quality implementation can be furthered. In this article, we: (1) use the synthesis of frameworks developed by Meyers et al. (Am J Commun Psychol, 2012) and translate the results into a practical implementation science tool to use for improving quality of implementation (i.e., the Quality Implementation Tool; QIT), and (2) present some of the benefits and limitations of the tool by describing how the QIT was implemented in two different pilot projects. We discuss how the QIT can be used to guide collaborative planning, monitoring, and evaluation of how an innovation is implemented.

Keywords

Implementation Knowledge utilization Practical implementation science Implementation tool Interactive Systems Framework 

References

  1. Bramley, P., & Kitson, B. (1994). Evaluating training against business criteria. Journal of European Industrial Training, 18(1), 10–14.CrossRefGoogle Scholar
  2. Carnevale, A. P., Gainer, L. E., & Meltzer, A. S. (1990). Work-place basics: The essential skills employers want. San Francisco, CA: Jossey-Bass Publishers.Google Scholar
  3. Castro, F. G., Barrera, M., Jr, & Holleran Steiker, L. K. (2010). Issues and challenges in the design of culturally-adapted evidence-based interventions. Annual Review of Clinical Psychology, 6, 213–239.PubMedCrossRefGoogle Scholar
  4. Cherniss, C., (April 15, 2000). Emotional intelligence: What it is and why it matters. Paper presented at the annual meeting of the Society for Industrial and Organizational Psychology, New Orleans, LA.Google Scholar
  5. Chien, V. C. (2010). Training for outcomes: Towards an evidence-informed approach for training effectiveness (in preparation).Google Scholar
  6. Chinman, M., Imm, P., & Wandersman, A. (2004). Getting to outcomes 2004: Promoting accountability through methods and tools for planning, implementation, and evaluation. (No. TR-TR101). Santa Monica, CA: RAND.Google Scholar
  7. Damschroder, L. J., Aron, D. C., Keith, R. E., Kirsh, S. R., Alexander, J. A., & Lowery, J. C. (2009). Fostering implementation of health services research findings into practice: A consolidated framework for advancing implementation science. Implementation Science, 4, 50.PubMedCrossRefGoogle Scholar
  8. Dane, A. V., & Schneider, B. H. (1998). Program integrity in primary and early secondary prevention: Are implementation effects out of control. Clinical Psychology Review, 18, 23–45.PubMedCrossRefGoogle Scholar
  9. Domitrovich, C. E., Gest, S. D., Jones, D., Gill, S., & DeRousie, R. M. S. (2010). Implementation quality: Lessons learned in the context of the Head Start REDI trial. Early Childhood Research Quarterly, 25, 284–298.PubMedCrossRefGoogle Scholar
  10. DuBois, D. L., Holloway, B. E., Valentine, J. C., & Cooper, H. (2002). Effectiveness of mentoring programs for youth: A meta-analytic review. American Journal of Community Psychology, 30, 157–198.PubMedCrossRefGoogle Scholar
  11. Duffy, J. L., Prince, M. S., Johnson, E. E., Alton, F. L., Flynn, S., Mattison Faye, A., et al. (2012). Enhancing teen pregnancy prevention in local communities: Capacity building using the Interactive Systems Framework. American Journal of Community Psychology (this issue).Google Scholar
  12. Durlak, J. A. (1998). Why program implementation is important. Journal of Prevention and Intervention in the Community, 17, 5–18.CrossRefGoogle Scholar
  13. Durlak, J. A. (2010). The importance of doing well in whatever you do: A commentary on the special section, “implementation research in early childhood education”. Early Childhood Research Quarterly, 25, 348–357.CrossRefGoogle Scholar
  14. Durlak, J. A., & DuPre, E. P. (2008). Implementation matters: A review of research on the influence of implementation on program outcomes and the factors affecting implementation. American Journal of Community Psychology, 41, 327–350.PubMedCrossRefGoogle Scholar
  15. Dusenbury, L., Brannigan, R., Falco, M., & Hansen, W. (2003). A review of research on fidelity of implementation: Implications for drug abuse prevention in school settings. Health Education Research, 18, 237–256.PubMedCrossRefGoogle Scholar
  16. Fagan, A. A., Hanson, K., Hawkins, J. D., & Arthur, M. W. (2008). Bridging science to practice: Achieving prevention program implementation fidelity in the Community Youth Development Study. American Journal of Community Psychology, 41, 235–249.PubMedCrossRefGoogle Scholar
  17. Feldstein, A. C., & Glasgow, R. E. (2008). A practical, robust implementation and sustainability model (PRISM) for integrating research findings into practice. Joint Commission Journal on Quality and Patient Safety/Joint Commission Resources, 34, 228–243.PubMedGoogle Scholar
  18. Fixsen, D. L., Blase, K. A., Timbers, G. D., & Wolf, M. M. (2001). In search of program implementation: 792 replications of the Teaching-Family Model. In G. A. Bernfeld, D. P. Farrington, & A. W. Leschied (Eds.), Offender rehabilitation in practice: Implementing and evaluating effective programs (pp. 149–166). London: Wiley.Google Scholar
  19. Fixsen, D. L., Naoom, S. F., Blase, K. A., Friedman, R. M., & Wallace, F. (2005). Implementation research: A synthesis of the literature. Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network. (FMHI Publication #231).Google Scholar
  20. Flaspohler, P., Duffy, J. L., Wandersman, A., Stillman, L., & Maras, M. A. (2008). Unpacking prevention capacity: An intersection of research-to-practice and community centered models. American Journal of Community Psychology, 41, 182–196.PubMedCrossRefGoogle Scholar
  21. Forsner, T., Hansson, J., Brommels, M., Wistedt, A. A., & Forsell (2010). Implementing clinical guidelines in psychiatry: a qualitative study of perceived facilitators and barriers. BMC Psychiatry, 10(8). doi:10.1186/1471-244x-10-8.
  22. Glisson, C., & Schoenwald, S. K. (2005). The ARC organizational and community intervention strategy for implementing evidence-based children’s mental health treatments. Mental Health Services Research, 7, 243–259.PubMedCrossRefGoogle Scholar
  23. Goodman, R. M., Speers, M. A., McLeroy, K., Fawcett, S., Kegler, M., Parker, E., et al. (1998). Identifying and defining the dimensions of community capacity to provide a basis for measurement. Health Education & Behavior, 25(3), 258–278.Google Scholar
  24. Gottfredson, D. C., & Gottfredson, G. D. (2002). Quality of school based prevention programs: Results from a national survey. Journal of Research on Crime and Delinquency, 39, 3–35.CrossRefGoogle Scholar
  25. Greenberg, M. T., Domitrovich, C. E., Graczyk, P. A., & Zins, J. E. (2005). The study of implementation in school-based preventive interventions: Theory, research, and practice (Vol. 3). DHHS Pub. No. (SMA) __________. Rockville, MD: Center for Mental Health Services, Substance Abuse and Mental Health Services Administration. Retrieved from http://casel.org/wp-content/uploads/2011/04/Implementation-VOL-31.pdf.
  26. Greenhalgh, T., Robert, G., MacFarlane, F., Bate, P., & Kyriakidou, O. (2004). Diffusion of innovations in service organizations: Systematic review and recommendations. The Milbank Quarterly, 82, 581–629.PubMedCrossRefGoogle Scholar
  27. Hall, G. E., & Hord, S. M. (2010). Implementing change: Patterns, principles and potholes (3rd ed.). Boston, MA: Allyn and Bacon.Google Scholar
  28. Hansen, W. B., & McNeal, R. B. (1999). Drug education practice: Results of an observational study. Health Education Research, 14, 85–97.PubMedCrossRefGoogle Scholar
  29. Harshbarger, C., Simmons, G., Coelho, H., Sloop, K., & Collins, C. (2006). An empirical assessment of implementation, adaptation, and tailoring: The evaluation of CDC’s national diffusion of VOICES/VOCES. AIDS Education and Prevention, 18, 184–197.PubMedCrossRefGoogle Scholar
  30. Hernandez, M., Gomez, A., Lipien, L., Greenbaum, P. E., Armstrong, K. H., & Gonzalez, P. (2001). Use of the system-of-care practice review in the national evaluation: Evaluating the fidelity of practice to system-of-care principles. Journal of Emotional and Behavioral Disorders, 9, 43–52.CrossRefGoogle Scholar
  31. Hogue, A., Liddle, H. A., Singer, A., & Leckrone, J. (2005). Intervention fidelity in family-based prevention counseling for adolescent problem behaviors. Journal of Community Psychology, 33, 191–211.CrossRefGoogle Scholar
  32. International Organization for Standardization. (1998). ISO/IEC International Standard 13236: Information technologyQuality of service: Framework (1st ed.).Google Scholar
  33. Klein, K. J., & Sorra, J. S. (1996). The challenge of innovation implementation. Academy of Management Review, 21, 1055–1080.Google Scholar
  34. Kumpfer, K. L., Turner, C., Hopkins, R., & Librett, J. (1993). Leadership and team effectiveness in community coalitions for the prevention of alcohol and other drug abuse. Health Education Research, 8, 359–374.CrossRefGoogle Scholar
  35. Lehman, W. E. K., Greener, J. M., & Simpson, D. D. (2002). Assessing organizational readiness for change. Journal of Substance Abuse Treatment, 22, 197–209.PubMedCrossRefGoogle Scholar
  36. McGehee, W., & Thayer, P. W. (1961). Training in business and industry. New York: Wiley.Google Scholar
  37. Meyers, D. C., Durlak, J., & Wandersman, A. (2012). The Quality Implementation Framework: A synthesis of critical steps in the implementation process. American Journal of Community Psychology. doi:10.1007/s10464-012-9522-x.
  38. Mihalic, S., Fagan, A. A., Irwin, K., Ballard, D., & Elliott, D. (2004). Blueprints for violence prevention. Washington, DC: Office of Juvenile Justice and Delinquency Prevention.Google Scholar
  39. Miller, W. R., & Rollnick, S. (2002). Motivational interviewing: Preparing people for change. New York: Guilford Press.Google Scholar
  40. Miller, R. L., & Shinn, M. (2005). Learning from communities: Overcoming difficulties in dissemination of prevention and promotion efforts. American Journal of Community Psychology, 35, 169–183.PubMedCrossRefGoogle Scholar
  41. Mills, S. C., & Ragan, T. J. (2000). A tool for analyzing implementation fidelity of an integrated learning system. Educational Technology Research and Development, 48, 21–41.CrossRefGoogle Scholar
  42. Mowbray, C. T., Holter, M. C., Teague, G. B., & Bybee, D. (2003). Fidelity criteria: Development, measurement, and validation. American Journal of Evaluation, 24, 315–340.Google Scholar
  43. Noe, R. A. (2010). Employee training and development (5th ed.). Boston, MA. Irwin/McGraw-Hill.Google Scholar
  44. O’Donnell, C. L. (2008). Defining, conceptualizing, and measuring fidelity of implementation and its relationship to outcomes in K-12 curriculum intervention research. Review of Educational Research, 78, 33–84.CrossRefGoogle Scholar
  45. Raghavan, R., Bright, C. L., & Shadoin, A. L. (2008). Toward a policy ecology of implementation of evidence-based practices in public mental health settings. Implementation Science, 3, 26.PubMedCrossRefGoogle Scholar
  46. Rapp, C., Goscha, R., & Carlson, L. (2010). Evidence-based practice implementation in Kansas. Community Mental Health Journal, 46, 461–465.PubMedCrossRefGoogle Scholar
  47. Rogers, E. M. (2003). Diffusion of innovations (5th ed.). New York: Free Press.Google Scholar
  48. Saunders, R. P., Ward, D., Felton, G. M., Dowda, M., & Pate, R. R. (2006). Examining the link between program implementation and behavior outcomes in the lifestyle education for activity program (LEAP). Evaluation and Program Planning, 29, 352–364.Google Scholar
  49. Schorr, L. B. (2003). Determining ‘‘what works’’ in social programs and social policies: Toward a more inclusive knowledge base. Washington, DC: Brookings Institution.Google Scholar
  50. Semel, M. E., Resch, S., Haynes, H. B., Funk, L. M., Bader, A., Berry, W. R., et al. (2010). Adopting a surgical safety checklist could save money and improve the quality of care in U.S. hospitals. Health Affairs, 29, 1593–1599.PubMedCrossRefGoogle Scholar
  51. Smith, J. D., Schneider, B. H., Smith, P. K., & Ananiadou, K. (2004). The effectiveness of whole-school antibullying programs: A synthesis of evaluation research. School Psychology Review, 33, 547–560.Google Scholar
  52. Spoth, R., Claire, C., Greenberg, M., Redmont, C., & Shin, C. (2007). Toward dissemination of evidence-based family interventions: Maintenance of community-based recruitment results and associated factors. Journal of Family Psychology, 21, 137–146.PubMedCrossRefGoogle Scholar
  53. Spoth, R., Guyll, M., Redmond, C., Greenberg, M., & Feinberg, M. (2011). Six-year sustainability of evidence-based intervention implementation quality by community–university partnerships: The PROSPER study. American Journal of Community Psychology, 48, 412–425.PubMedCrossRefGoogle Scholar
  54. Stith, S., Pruitt, I., Dees, J., Fronce, M., Green, N., Som, A., et al. (2006). Implementing community-based prevention programming: A review of the literature. The Journal of Primary Prevention, 27, 599–617.PubMedCrossRefGoogle Scholar
  55. Thomas, K. G. F., Gatz, M., & Luczak, S. E. (1997). A tale of two school districts: Lessons to be learned about the impact of relationship building and ecology on consultation. Journal of Educational and Psychological Consultation, 8, 297–320.CrossRefGoogle Scholar
  56. Voss, C. (1992). Successful innovation and implementation of new processes. Business Strategy Review, 20, 29–44.CrossRefGoogle Scholar
  57. Walker, J. S., & Koroloff, N. (2007). Grounded theory and backward mapping: Exploring the implementation context for Wraparound. Journal of Behavioral Health Services & Research, 34, 443–458.CrossRefGoogle Scholar
  58. Wandersman, A. (2003). Community science: Bridging the gap between science and practice with community-centered models. American Journal of Community Psychology, 31, 227–242.PubMedCrossRefGoogle Scholar
  59. Wandersman, A., Chien, V., & Katz. J. (2012). Toward an evidence-based system for innovation support for implementing innovations with quality: Tools, training, technical assistance, and quality assurance/quality improvement. American Journal of Community Psychology, 1–15. doi:10.1007/s10464-012-9509-7.
  60. Wandersman, A., Duffy, J., Flaspohler, P., Noonan, R., Lubell, K., Stillman, L., et al. (2008). Bridging the gap between prevention research and practice: The Interactive Systems Framework for Dissemination and Implementation. American Journal of Community Psychology, 41, 171–181. Google Scholar
  61. Wandersman, A., Imm, P., Chinman, M., & Kaftarian, S. (2000). Getting to outcomes: A results-based approach to accountability. Evaluation and program planning, 23, 389–395.Google Scholar
  62. White, W. (2008). Recovery management and recovery-oriented systems of care: Scientific rationale and promising practices. Northeast Addiction Technology Transfer Center, Great Lakes Addiction Technology Transfer Center, Philadelphia Department of Behavioral Health/Mental Retardation Services.Google Scholar

Copyright information

© Society for Community Research and Action 2012

Authors and Affiliations

  • Duncan C. Meyers
    • 1
  • Jason Katz
    • 1
  • Victoria Chien
    • 1
  • Abraham Wandersman
    • 1
  • Jonathan P. Scaccia
    • 1
  • Annie Wright
    • 1
  1. 1.University of South CarolinaColumbiaUSA

Personalised recommendations