Abstract
An individual or organization that sets out to implement an innovation (e.g., a new technology, program, or policy) generally requires support. In the Interactive Systems Framework for Dissemination and Implementation, a Support System should work with Delivery Systems (national, state and/or local entities such as health and human service organizations, community-based organizations, schools) to enhance their capacity for quality implementation of innovations. The literature on the Support ystem has been under-researched and under-developed. This article begins to conceptualize theory, research, and action for an evidence-based system for innovation support (EBSIS). EBSIS describes key priorities for strengthening the science and practice of support. The major goal of EBSIS is to enhance the research and practice of support in order to build capacity in the Delivery System for implementing innovations with quality, and thereby, help the Delivery System achieve outcomes. EBSIS is guided by a logic model that includes four key support components: tools, training, technical assistance, and quality assurance/quality improvement. EBSIS uses the Getting To Outcomes approach to accountability to aid the identification and synthesis of concepts, tools, and evidence for support. We conclude with some discussion of the current status of EBSIS and possible next steps, including the development of collaborative researcher-practitioner-funder-consumer partnerships to accelerate accumulation of knowledge on the Support System.
Similar content being viewed by others
Notes
Division of Violence Prevention; Division of Reproductive Health.
Getting To Outcomes and GTO are trademarks registered by the University of South Carolina and RAND Corp.
References
Alliger, G. M., Tannenbaum, S. I., Bennet, W., Traver, H., & Shotland, A. (1997). A meta-analysis of the relations among training criteria. Personnel Psychology, 50, 341–358.
Ammerman, R. T., Putnam, F. W., Margolis, P. A., & Van Ginkel, J. B. (2009). Quality improvement in child abuse prevention programs. In K. A. Dodge & D. L. Coleman (Eds.), Preventing child maltreatment: Community approaches (pp. 63–86). New York: The Guilford Press.
Anjard, R. P. (1995). SPC chart selection process. Microelectronics and Reliability, 35(11), 1445–1447.
Ayusawa, M., Sonobe, T., Uemura, S., Ogawa, S., Nakamura, Y., Kiyosawa, N., et al. (2005). Revision of diagnostic guidelines for Kawasaki disease. Pediatrics International, 47(2), 232–234.
Bartholomew, N. G., Joe, G. W., Rowan-Szal, G. A., & Simpson, D. D. (2007). Counselor assessments of training and adoption barriers. Journal of Substance Abuse Treatment, 33, 193–199.
Beard, G. (2008). Improving clinical interventions through successful outreach using Six Sigma quality improvement. Journal of Healthcare Quality, 30(1), 38–43.
Becker, D. R., Lynde, D., & Swanson, S. J. (2008). Strategies for state-wide implementation of supported employment: The Johnson & Johnson—Dartmouth Community Mental Health Program. Psychiatric Rehabilitation Journal, 31(4), 296–299.
Bickman, L. (2008). A measurement feedback system (MFS) is necessary to improve mental health outcomes. Journal of the American Academy of Child and Adolescent Psychiatry, 47(10), 1114–1119.
Blumenthal, B. (2003). Investing in capacity building: A guide to high-impact approaches. New York: The foundation Center.
Bramley, P. (1991). Evaluating training effectiveness. New York, NY, and Maidenhead: McGraw-Hill Book.
Bramley, P., & Kitson, B. (1994). Evaluating training against business criteria. Journal of European Industrial Training, 18(1), 10–14.
Butterfoss, F. (2004). The coalition technical assistance and training framework: Helping community coalitions help themselves. Health Promotion Practice, 5(2), 118–126.
Butterfoss, F. D. (2007). Coalitions and partnerships in community health. San Francisco, CA: Jossey-Bass.
Cannon-Bowers, J. A., Salas, E., Tannenbaum, S. I., & Matthieu, J. E. (1995). Toward theoretically-based principles of trainee effectiveness: A model and initial empirical investigation. Military Psychology, 7, 141–164.
Carnevale, A. P., Gainer, L. E., & Meltzer, A. S. (1990). Work- place basics: The essential skills employers want. San Fran- cisco, CA: Jossey-Bass Publishers.
Chinman, M., Young, A. S., Rowe, M., Forquer, S., Knight, E., & Miller, A. (2003). An instrument to assess competencies of providers treating severe mental illness. Mental Health Services Research, 5, 97–108.
Chinman, M., Hannah, G., Wandersman, A., Ebener, P., Hunter, S. M., Imm, P., et al. (2005). Developing a community science research agenda for building community capacity for effective prevention interventions. American Journal of Community Psychology, 35(3–4), 143–157.
Chinman, M., Hunter, S. B., Ebener, P., Paddock, S. M., Stillman, L., Imm, P., et al. (2008). The getting to outcomes demonstration and evaluation: An illustration of the prevention Support System. American Journal of Community Psychology, 41(3–4), 206–224.
Clement, W. A., Wilson, S., & Bingham, B. J. (2002). A guide to creating your own patient-oriented website. Journal of the Royal Society of Medicine, 95(2), 64–67.
Collins, C., Harshbarger, C., Sawyer, R., & Hamdallah, M. (2006). The diffusion of effective behavioral interventions project: Development, implementation, and lessons learned. AIDS Education and Prevention, 18 (Supplement A), 5–20.
Cowley, K., & Good, K. (2010). Using evaluation to refine and focus technical assistance. In S. Harsh, K. Bradley, K. Good, & J. Ross (Eds.), Capacity building technical assistance. Nashville, TN: Edvantia.
DeMatteo, J. S., Dobbins, G. H., & Lundby, K. M. (1997). The effects of accountability on training effectiveness. Training Research Journal, 3, 39–57.
Donabedian, A. (1996). The effectiveness of quality assurance. International Journal for Quality in Health Care, 8, 401–407.
Drucker, P. (2008). Classic Drucker: Essential wisdom of Peter Drucker. Boston, MA: Harvard Business Press.
Duffy, J. L., Prince, M., Johnson, E. J., Alton, F. L., Flynn, S., Faye, M., et al. (2012). Enhancing teen pregnancy prevention in local communities. Capacity building using the interactive systems framework. American Journal of Community Psychology.
Durlak, J. A., & DuPre, E. P. (2008). Implementation matters: A review of research on the influence of implementation on program outcomes and the factors affecting implementation. American Journal of Community Psychology, 41, 327–350.
Eysenbach, G., & Jadad, A. R. (2001). Evidence-based patient choice and consumer health informatics in the internet age. Journal of Medical Internet Research, 3(2), e19.
Fagan, A. A., Hanson, K., Hawkins, J. D., & Arthur, M. W. (2008). Bridging science to practice: Achieving prevention program implementation fidelity in the community youth development study. American Journal of Community Psychology, 41, 235–249.
Fawcett, S. B., Paine Andrews, A., Francisco, V. T., Schultz, J. A., Richter, K. P., Lewis, R. K., et al. (1995). Using empowerment theory in collaborative partnerships for community health and development. American Journal of Community Psychology, 23(5), 677–697.
Feinberg, M. E., Greenberg, M. T., & Osgood, D. W. (2004). Readiness, functioning, and perceived effectiveness in community prevention coalitions: A study of Communities That Care. American Journal of Community Psychology, 33, 163–176.
Feinberg, M. E., Ridenour, T. A., & Greenberg, M. T. (2008). The longitudinal effect of technical assistance dosage on the functioning of Communities That Care prevention boards in Pennsylvania. The Journal of Primary Prevention, 29(2), 145–165.
Fixsen, D. L., Blase, K. A., Naoom, S. F., & Wallace, F. (2009). Core implementation components. Research on Social Work Practice, 19(5), 531–540.
Flaspohler, P., Duffy, J., Wandersman, A., Stillman, L., & Maras, M. A. (2008). Unpacking prevention capacity: An intersection of research-to-practice models and community-centered models. American Journal of Community Psychology, 41(3–4), 182–196.
Florin, P., Mitchell, R., & Stevenson, J. (1993). Identifying training and technical assistance needs in community coalitions: A developmental approach. Health Education Research: Theory and Practice, 8(30), 417–432.
Furman, C., & Caplan, R. (2007). Applying the Toyota production system: Using a patient safety alert system to reduce error. Joint Commission on Quality & Patient Safety, 33(7), 376–386.
Gawande, A. (2009). The checklist manifesto: How to get things right. New York: Metropolitan books.
Gibbs, D., Napp, D., Jolly, D., Westover, B., & Uhl, G. (2002). Increasing evaluation capacity within community-based HIV prevention programs. Evaluation and Program Planning, 25, 261–269.
Grimshaw, J. M., & Russell, I. T. (1993). Effects of clinical guidelines on medical practice: A systematic review of rigorous evaluations. The Lancet, 342(8883), 1317–1322.
Groi, R., Daihuijsen, J., Thomas, S., In’tVeld, C., Rutten, G., & Mokkink, H. (1998). Attributes of clinical guidelines that influence use of guidelines in general practice: Observational study. British Medical Journal, 317, 858–861.
Grol, R., Baker, R., Roberts, R., & Booth, B. (1997). Systems for quality improvement in general practice. European Journal of General Practice, 3, 65–68.
Haag, S., Cummings, M., McCubbrey, D., Pinsonneault, A., & Donovan, R. (2006). Management information systems for the information age (3rd Canadian ed.). Canada: McGraw Hill Ryerson.
Hall, G. E., & Hord, S. M. (2006). Implementing change: Patterns, principles and potholes (2nd ed.). Boston, MA: Allyn and Bacon.
Harvey, G., & Kitson, A. (1996). Achieving improvement through quality: An evaluation of key factors in the implementation process. Journal of Advanced Nursing, 24(1), 185–195.
Hiruma, F., & Kaiho, H. (1991). Optimal design for a presentation of action-sequences in a user manual. Japanese Journal of Educational Psychology, 39(4), 461–466.
Ho, T., & Antunes, P. (1999). Developing a tool to assist electronic facilitation of decision-making groups. String Processing and Information Retrieval Symposium, 10, 243–252.
Joyce, B., & Showers, B. (2002). Student achievement through staff development. Alexandria, VA: Association for Supervision and Curriculum Development.
Kahn, B. K., Strong, D. M., & Wang, R. Y. (2002). Information quality benchmarks: Product and service performance. Communications of the ACM, 45, 184–192.
Keener, D. C. (2007). Toward a science of capacity building: An examination of technical assistance following a training program for prevention professionals. Unpublished doctoral dissertation, University of South Carolina, Columbia, SC, USA.
Kegeles, S. M., Rebchook, G. M., & Tebbetts, S. (2005). Challenges and facilitators to building program evaluation capacity among community-based organizations. AIDS Education and Prevention, 17(4), 284–299.
Kelly, J. A., Somlai, A. M., DiFranceisco, W. J., Otto-Salaj, L. L., McAuliffe, T. L., Hackl, K. L., et al. (2000). Bridging the gap between the science and service of HIV prevention: Transferring effective research-based HIV prevention interventions to community AIDS service providers. American Journal of Public Health, 90(7), 1082–1088.
King, D. L., Ben-Tovim, D. I., & Bassham, J. (2006). Redesigning emergency department patient flows: Application of lean thinking to health care. Emergency Medicine Australasia, 18(4), 391–397.
Kirkpatrick, D. L. (1959). Techniques for evaluating training programs. Journal of American Society of Training Directors’, 13(3), 21–26.
Kitchenham, B., Pickard, L., & Pfleeger, S. L. (1995). Case studies for method and tool evaluation. IEE Software, 12(4), 52–62.
Knatterud, G. L., Rockhold, F. W., George, S. L., Barton, F. B., Davis, C. E., Fairweather, W. R., et al. (1998). Guidelines for quality assurance in multicenter trials: A position paper. Controlled Clinical Trials, 19(5), 477–493.
Knowles, M. S. (1970). The modern practice of adult education: Andrology versus pedagogy. New York: New York Association Press.
Kraemer, H. C., Measelle, J. R., Ablow, J. C., Essex, M. J., Boyce, T., & Kupfer, D. J. (2003). A new approach to integrating data from multiple informants in psychiatric assessment and research: Mixing and matching contexts and perspectives. Journal of Psychiatry, 160, 1566–1577.
Lammers, J. C., Cretin, S., Gilman, S., & Calingo, E. (1996). Total quality management in hospitals: The contributions of commitment, quality councils, teams, budgets, and training to perceived improvement at Veterans Health Administration Hospitals. Medical Care, 34(5), 463–478.
Lee, Y. W., Strong, D. M., Kahn, B., & Wang, R. Y. (2002). AIMQ: A methodology for information quality assessment. Information & Management, 40, 133–146.
Letts, C. W., Ryan, W. P., & Grossman, A. (1999). High performance nonprofit organizations: Managing upstream for greater impact. New York, NY: Wiley.
Lewis, K., Lesesne C. A., Zahniser, S. C., Wilson, M. M., Desiderio, G., Wandersman, A. & Green, D. C. (2012). Developing a prevention synthesis and translation system to promote science based approaches to teen pregnancy, HIV and STI Prevention. American Journal of Community Psychology. doi:10.1007/s10464-012-9510-1.
Lovelle, J. (2001). Mapping the value stream. IIE Solutions, 33(2), 26–33.
Mahapatra, R., & Lai, V. (2005). Evaluating end-user training programs. Communications of the Association for Computing Machinery, 48(1), 67–70.
Marguerite Casey Foundation. (2007). The Marguerite Casey Foundation organizational capacity assessment tool. From http://www.caseygrants.org. Retrieved December 21, 2010.
McConnon, A., Kirk, S. L., Cockroft, J. E., Harvey, E. L., Greenwood, D. C., Thomas, J. D., et al. (2007). The internet for weight control in an obese sample: Results of a randomized controlled trial. BMC Health Services Research, 7, 206–215.
McGehee, W., & Thayer, P. W. (1961). Training in business and industry. New York: Wiley.
McLaughlin, T. J., Soumerai, S. B., Wilson, D. J., et al. (1996). Adherence to national guideline for drug treatment of suspected acute myocardial infarction: Evidence for under treatment in women and the elderly. Archives of Internal Medicine, 156, 799–805.
Meyers, D. C., Katz, J., Chien, V., Wandersman, A., Scaccia, J., & Wright, A. (2012). Practical implementation science: Developing and piloting the quality implementation tool. American Journal of Community Psychology.
Mihalic, S. F., & Irwin, K. (2003). Blueprints for violence prevention: From research to real world settings—factors influencing the successful replication of model programs. Youth Violence and Juvenile Justice, 1, 307–329.
Mitchell, R. E., Florin, P., & Stevenson, J. F. (2002). Supporting community-based prevention and health promotion initiatives: Developing effective technical assistance systems. Health Education & Behavior, 29(5), 620–639.
Mitchell, R., Stone-Wiggins, B., Stevenson, J. F., & Florin, P. (2004). Cultivating capacity: Outcomes of a statewide Support System for prevention coalitions. Journal of Prevention and Intervention in the Community, 27, 67–87.
Mizzaro, S. (2003). Quality control in scholarly publishing: A new proposal. Journal of American Society for Information Science & Technology, 54(11), 989–1005.
Mumtaz, S. (2000). Factors affecting teachers’ use of information and communication technology: A review of the literature. Journal of Information Technology for Teacher Education, 9(3), 319–342.
Nemec, P. B., Forbess, R., Cohen, M. R., Farkas, M. D., Rogers, E. S., & Anthony, W. (1991). Effectiveness of technical assistance in the development of psychiatric rehabilitation programs. Journal of Mental Health Administration, 18(1), 1–11.
Noe, R. A. (2010). Employee training and development (5th ed.). Boston, MA: Irwin/McGraw-Hill.
O’Donnell, L., Scattergood, P., Alder, M., San Doval, A., & Al, E. (2000). The role of technical assistance in the replication of effective HIV interventions. AIDS Education and Prevention, 12 (Suppl. A), 99–111.
Parnas, D. L., & Lawford, M. (2003). The role of inspection in software quality assurance. IKEE Transaction on Software Engineering, 29(8), 674–676.
Powell, A. E., Rushmer, R. K., & Davies, H. T. O. (2009). A systematic narrative review of quality improvement models in health care. Scotland: NHS Quality Improvement.
Quinby, R. K., Hanson, K., Brooke-Weiss, B., Arthur, M. W., Hawkins, J. D., & Fagan, A. (2008). Installing the communities that care prevention system: Implementation progress and fidelity in a randomized controlled trial. Journal of Community Psychology, 36(3), 313–332.
Ray, M. L., Wilson, M. M., Wandersman, A., Meyers, D., & Katz, J. (2012). Using a training-of-trainers approach and proactive technical assistance to bring evidence-based programs to scale: An operationalization of the Interactive Systems Framework’s Support System. American Journal of Community Psychology.
Rowold, J. (2007). Individual influences on knowledge acquisition in a call center training context in Germany. International Journal of Training and Development, 11(1), 21–34.
Salyers, M. P., McKasson, M., Bond, G. R., & McGrew, J. H. (2007). The role of technical assistance centers in implementing evidence-based practices: Lessons learned. American Journal of Psychiatric Rehabilitation, 10, 85–101.
Salzer, M. S., & Bickman, L. (1997). Delivering effective children’s services in the community: Reconsidering the benefits of system interventions. Applied & Preventive Psychology, 6, 1–13.
Scaduto, A., Lindsay, D., & Chiaburu, D. S. (2008). Leader influences on training effectiveness: Motivation and outcome expectation process. International Journal of Training & Development, 12(3), 158–170.
Shojania, K. G., & Grimshaw, J. M. (2005). Evidence-based quality improvement: The state of the science. Health Affairs, 24, 138–150.
Sieber, J. E. (2008). When academicians collaborate with community agencies in effectiveness research. Clinical Psychology: Science and Practice, 15(2), 137–143.
Silimperi, D. R., Franco, L. M., Van Zanten, T. V., & Macaulay, C. (2002). A framework for institutionalizing quality assurance. International Journal for Quality in Health Care, 14(Suppl 1), 67–73.
Sobeck, J., & Agius, E. (2007). Organizational capacity building: Addressing a research and practice gap. Evaluation and Program Planning, 30, 237–246.
Speroff, T., & O’Connnor, G. T. (2004). Study designs for PDSA quality improvement research. Quality Management in Health care, 13(1), 17–32.
Spoth, R., Claire, C., Greenberg, M., Redmont, C., & Shin, C. (2007). Toward dissemination of evidence-based family interventions: Maintenance of community-based recruitment results and associated factors. Journal of Family Psychology, 21(2), 137–146.
Squires, D., & Preece, J. (1996). Usability and learning: Evaluating the potential of educational software. Computers Education, 27(1), 15–22.
Stevenson, J. F., Florin, P., Mills, D. S., & Andrade, M. (2002). Building evaluation capacity in human service organizations: A case study. Evaluation and Program Planning, 25(3), 233–243.
Stvilia, B., Twidale, M. B., Smith, L. C., Gasser, L. (2005). Assessing information quality of a community-based encyclopedia.
Tai, W. (2006). Effects of training framing, general self-efficacy and training motivation on trainees’ training effectiveness. Personnel Review, 35(1), 51–65.
Thomas, L., Cullum, N., McColl, E., Rousseau, N., Soutter, J., & Steen, N. (2000). Guidelines in professions allied to medicine. Cochrane Database Systematic Review, 2, CD000349.
Trivedi, M. H., Kern, J. K., Marcee, A., Grannemann, B., Kleiber, B., Bettinger, T., et al. (2002). Development and implementation of computerized clinical guidelines: Barriers and solutions. Methods of Information in Medicine, 41, 435–442.
Varkey, P., Reller, M. K., & Resar, R. K. (2007). Basics of quality improvement in health care. Mayo Clinic Proceedings, 82(6), 735–739.
Wandersman, A., Duffy, J., Flaspohler, P., Noonan, R., Lubell, K., Stillman, S., et al. (2008b). Bridging the gap between prevention research and practice: The interactive systems framework for dissemination and implementation. American Journal of Community Psychology, 41(3–4), 171–181.
Wandersman, A., Flaspohler, P., & Saul, J. (2008a). Illuminating the interactive systems framework for dissemination and implementation. Special issue of the American Journal of Community Psychology, 82, 6.
Wandersman, A., Imm, P., Chinman, M., & Kaftarian, S. (2000). Getting to outcomes: A results-based approach to accountability. Evaluation and Program Planning, 23, 389–395.
Wesley, P. W., & Buysse, V. (1996). Supporting early childhood inclusion: Lessons learned through a statewide technical assistance project. Topics in Early Childhood Special Education, 16(4), 476–500.
Whiting, P., Rutjes, A. W., Reitsma, J. B., Bossuyt, P. M., & Kleijnen, J. (2003). The development of QUADAS: A tool for the quality assessment of studies of diagnostic accuracy included in systematic reviews. BMC Medical Research Methodology, 3(25), 1–13.
Yasin, M. M., & Zimmer, T. W. (1996). The role of benchmarking in achieving continuous service quality. International Journal of Contemporary Hospitality Management, 7(4), 27–32.
Young, W. F., Montgomery, D., Nycum, C., Burns-Martin, L., & Buller, D. B. (2006). Web-based technical assistance and training to promote community tobacco control policy change. Health Promotion Practice, 7(1), 78–85.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Wandersman, A., Chien, V.H. & Katz, J. Toward an Evidence-Based System for Innovation Support for Implementing Innovations with Quality: Tools, Training, Technical Assistance, and Quality Assurance/Quality Improvement. Am J Community Psychol 50, 445–459 (2012). https://doi.org/10.1007/s10464-012-9509-7
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10464-012-9509-7