Abstract
Bringing evidence based programs to scale was a major initial impetus for the development of the Interactive Systems Framework for Dissemination and Implementation (ISF). The ISF demonstrates the importance of the Support System in facilitating the uptake of innovations in the community (the Delivery System). Two strategies that members of the Support System commonly use are training-of-trainers (TOT) models and technical assistance (TA). In this article, we focus on the role of the Support System in bringing evidence-based programs (EBPs) to scale in the Delivery System using a case example, with special attention on two strategies employed by Support Systems—training-of-trainers (TOT) and proactive technical assistance. We then report on findings from a case example from the Promoting Science Based Approaches to Teen Pregnancy Prevention project that furthers our conceptualization of these strategies and the evidence base for them. We also report on the limitations in the literature regarding research on TOTs and proactive TA and provide suggestions for future research on TOTs and proactive TA that will enhance the science and practice of support in the ISF.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
Although the term Training-of-Educators implies that all participants in these trainings were educators, it is important to note that not all participants were school teachers.
The five scaled items for Making Proud Choices were: (1) the day’s training was engaging; (2) presentations were clear; (3) presentation styles were a good match for how I learn; (4) I gained new knowledge and/or understanding of Making Proud Choices; and(5) I gained new knowledge and/or capacities to train others on Making Proud Choices. The four point scale was 1 = Not good use at all and 4 = excellent.
In an attempt to gather more detailed information, for Safer Choices we used the following 13 scaled items and the same four point scale as for the other TOTs: (1) the day’s training was engaging; (2) material was presented in a clear and well-organized fashion; (3) trainers used and interactive style that kept all participants involved in the discussions and activities; (4) directions were clear; (5) trainers took time to be sure everyone understood new concepts; (6) activities were presented in easy to understand steps; (7) the presentation styles were a good match for how I learn; (8) trainers made sure everyone felt comfortable and safe stating their opinions; (9) trainers made sure everyone felt comfortable and safe stating their opinions; (10) trainers made sure everyone had a chance to join discussions and that discussions were not dominated by only a few participants; (11) trainers were respectful of different opinions; (12) I gained new knowledge and/or understanding of Safer Choices; and (13) I gained new knowledge and/or capacities to train others on Safer Choices.
The items to assess participants’ feelings of competency to train were different for each EBP. Here is the list of items HTN used to assess TOT ¡Cuídate!: (a) Describe the learning objectives for ¡Cuídate!; (b) (b) Describe the core content of ¡Cuídate!; (c) Describe ¡Cuídate’s! intended audience; (d) Describe the unique features of ¡Cuidate’s!; (e) Describe ¡Cuidate’s! evaluation findings; (f) Describe the six modules that comprise ¡Cuidate!; (g) Describe ¡Cuidate’s! unique features for working with Latino youth; (h) Describe how Latino culture supports safer sex decisions, including abstinence and condom use; (i) Explain the importance of implementing ¡Cuídate! with fidelity; (j) Describe a plan for conducting a ¡Cuídate! training-of-educators (TOE) in the next 6 mo.; (k) Describe how to monitor and evaluate ¡Cuídate!; (l) I have sufficient knowledge about Latino values to train others on ¡Cuídate!;; (m) I really understand how Latino culture supports safer sex.
The following items were used to assess TOT participants’ feelings of general training skills capacity: (a) Demonstrate knowledge of adolescent pregnancy and HIV/STD prevention while training on a specific science-based program; (b) Provide a welcoming, enjoyable, and safe environment for a training; (c) Demonstrate sensitivity and respect for diverse backgrounds and cultures; (d) Demonstrate skills in performing correct condom use, negotiation of abstinence and condom use through the use of role-plays; (e) Facilitate and manage small group work; (f) Answer questions about sensitive matters; (g) Present a short lecture; (h) Facilitate large group discussions; (i) Facilitate brainstorming; (j) Model effective strategies to create and maintain a positive and constructive learning environment for all participants (e.g. control inappropriate talking or commentary, manages disruptive behavior); (k) Maintain control of the agenda, including managing “in flight changes”; (l) Provide clear instructions for tasks and activities; (m) Manage time efficiently according to training design; (n) Present skills in easy to understand steps (e.g. how to negotiate condom use); (o) Demonstrate the skills to be mastered by TOT participants; (p) Use tools and materials provided to ensure the training design, content, and delivery is effective; (q) Guide and support participants through practice activities; (r) Train on a science-based program with fidelity to the training design/curriculum.
The following items were used to assess TOT participants’ feelings of competence to provide TA: (a) Assessing a participant’s ¡Cuídaté! content knowledge; (b) Assessing a participant’s training competencies; (c) Setting TA goals based on participant needs; (d) Defining TA activities to reach goals; (e) Developing a proactive TA plan and timeline with participant; (f) Providing TA for planning and a timeline for all tasks required to implement a TOE; (g) Evaluating the overall effectiveness of the TA in addressing participant needs.
The ratings for quality were: (1) activity not observed; (2) needs improvement; and (3) well done. There was space on the forms for comments on each activity.
Two of the PSBA grantees were separate units in one organization. Therefore there was only one executive director for these two grantees, and the total number of interviews was 12.
References
Assemi, M., Mutha, S., & Hudmon, K. S. (2007). Evaluation of a train-the-trainer program for cultural competence. American Journal of Pharmaceutical Education, 71.
Carruth, A. K., Pryor, S., Cormier, C., Bateman, A., Matzke, B., & Gilmore, K. (2010). Evaluation of a school-based train-the-trainer intervention program to teach first aid and risk reduction among high school students. Journal of School Health, 80, 453–460.
Chinman, M., Imm, P., & Wandersman, A. (2004). Getting to outcomes 2004: Promoting accountability through methods and tools for planning, implementation, and evaluation. (No. TR- TR101). Santa Monica, CA: RAND.
Collins, C., Harshbarger, C., Sawyer, R., & Hamdallah, M. (2006). The Diffusion of Effective Behavioral Interventions Project: Development, implementation, and lessons learned. AIDS Education and Prevention, 18(Suppl A), 5–20.
Corelli, R. L., Fenlon, C. M., Kroon, L. A., Prokhorov, A. V., & Hudmon, K. S. (2007). Evaluation of a train-the-trainer program for tobacco cessation. [Article]. American Journal of Pharmaceutical Education, 71.
Coyle, K. K., Kirby, D., Parcel, G., Basen-Engquist, K., Banspach, S., Rugg, D. & Weil, M. (1996). Safer choices. Scotts Valley, CA: ETR Associates. Retrieved from: http://www.etr.org/recapp/index.cfm?fuseaction=pages.ebpDetail&PageID=131.
Davis, D., O’Brien, M. A., Freemantle, N., Wolf, F. M., Mazmanian, P., & Taylor-Vaisey, A. (1999). Impact of formal continuing medical education: Do conferences, workshops, rounds, and other traditional continuing education activities change physician behavior or health care outcomes? Journal of American Medical Association, 282, 867–874.
Dynes, M., Rahman, A., Beck, D., Moran, A., Pervin, J., Yunus, M., et al. (2011). Home-based life saving skills in Matlab, Bangladesh: A process evaluation of a community based maternal child health programs. Midwifery, 27, 15–22.
Education, Training and Research Associates and CDC. (in press). Adaptation guidance for science-based pregnancy, STD and HIV prevention education programs for adolescents. Scotts Valley, CA: ETR Associates.
Elliott, D., & Mihalic, S. (2004). Issues in disseminating and replicating effective prevention programs. Prevention Science, 5, 47–53.
Feinberg, M., Greenberg, M., Olson, J., & Osgood, W. (2005). Preliminary report: CTC impact in Pennsylvania. Findings from the 2001 and 2003 PA Youth Survey. University Park, PA: College of Health and Human Development, the Pennsylvania State University.
Fixsen, D. L., Blasé, K. A., Naoom, S. F., & Wallace, F. (2009a). Core implementation components. Research on Social Work Practice, 19, 531–540.
Fixsen, D.L., Blasée’, K., Horner, R., & Sugai, G. (2009b). Readiness for change. Scaling-up Brief: State Implementation and Scaling-up of Evidence-based Practices. FPG Child Development Institute. Chapel Hill, NC: The University of North Carolina at Chapel Hill.
Fixsen, D. L., Naoom, K. A., Blasé, S. L., Friedman, R. M., & Wallace. F. (2005). Implementation research: A synthesis of the literature [monograph]. Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health Institute, National Implementation Research Network; Florida Mental Health Institute publication 231.
Granger, R. (2011). The big why: A learning agenda for the scale-up movement (pp. 28–32). Winter.
Hahn, E., Noland, M., Rayens, M., & Christie, D. (2002). Efficacy of training and fidelity of implementation of the life skills training program. Journal of School Health, 72, 282–287.
Hendricks, M., Plantz, M. C., & Pritchard, K. J. (2008). Measuring outcomes of united way-funded programs: Expectations and reality. New Directions for Evaluation, 119, 13–35.
Hiner, C. A., Mandel, B. G., Weaver, M. R., Bruce, D., McLaughlin, R., & Anderson, J. (2009). Effectiveness of a training-of-trainers model in a HIV counseling and testing program in the Caribbean Region. Human Resources for Health. doi:10.1186/1478-4491-7-11.
International Labour Organization. (2002). Project to eliminate trafficking of children and women. Bangkok, Thailand: United Nations Services. Retrieven from: http://www.ilo.org/public/english/region/asro/bangkok/child/trafficking/downloads/tia-f.pdf.
Jemmott, L. S., Jemmott J., III, McCaffree, K. (1998). Making Proud Choices. NYC, NY: Select Media: http://www.selectmedia.org/customer-service/evidence-based-curricula/making-proud-choices/.
Joyce, B., & Showers, B. (2002). Student achievement through staff development. Association for staff and curriculum development (3rd ed.). Alexandria, VA: Association for Supervision and Curriculum Development.
Kegeles, S. M., Rebchook, G. M., & Tebbetts, S. (2005). Challenges and facilitators to building program evaluation capacity among community-based organizations. AIDS Education and Prevention, 17, 284–299.
Kelly, J. A., Somlai, A. M., DiFranceisco, W. J., Otto-Salaj, L. L., McAuliffe, T. L., Hackl, K. L., et al. (2000). Bridging the gap between the science and service of HIV prevention: Transferring effective research-based HIV prevention interventions to community AIDS service providers. American Journal of Public Health, 90, 1082–1088.
Lesesne, C. A., Lewis, K. M., White, C. P., Green, D., Duffy, J., & Wandersman, A. (2008). Promoting Science-Based Approaches to Teen Pregnancy Prevention: Proactively engaging the three systems of the interactive systems framework. American Journal of Community Psychology, 41, 379–392.
Lewis, K., Lesesne, C., Zahniser, S., Wilson, M., Desiderio, G., Wandersman, A., & Green, D. (2012). Developing a prevention synthesis and translation system to promote science-based approaches to teen pregnancy, HIV and STI prevention. American Journal of Community Psychology (This issue). doi:10.1007/s10464-012-9510-1.
Lloyd, B., Rychetnik, L., Maxwell, M., & Nove, T. (2009). Evaluation of a train-the-trainer workshop on sickle cell disease for ED providers. Health Promotion Journal of Australia, 20(2), 151–154.
Martino, S., Gallon, S., Ball, S. A., & Carroll, K. M. (2007). A step forward in teaching addiction counselors how to supervise motivational interviewing using a clinical trials training approach. Journal of Teaching in the Addictions, 6, 37–65.
Muramoto, M. L., Connolly, T., Strayer, L. J., Ranger-Moore, J., Blatt, W., Leischow, R., et al. (2011). Tobacco cessation skills certification in Arizona: Application of a state wide, community based model for diffusion of evidence based practice guidelines. Tobacco Control, 9, 408–414.
Naccarella, J., Pirkis, J., Kohn, F., Morley, B., Burgess, P., & Blashki, G. (2007). Building evaluation capacity: Definitional and practical implications from an Australian case study. Evaluation and Program Planning, 30, 231–236.
Olweus, D. (2005). A useful evaluation design, and effects of the Olweus Bullying Prevention Program. Psychology, Crime and Law, 11, 389–402.
Orfaly, R. A., Frances, J. C., Campbell, P., Whittemore, B. J., Joly, B., & Koh, H. (2005). Train-the-trainer as an educational model in public health preparedness. Journal of Public Health Management Practice, 11, 123–127.
Pratt, C. C. (2000). Measuring program outcomes: Using retrospective pretest methodology. American Journal of Evaluation, 21, 341–349.
Quinby, R. K., Hanson, K., Brooke-Weiss, B., Arthur, M. W., Hawkins, J. D., & Fagan, A. (2008). Installing the communities that care prevention system: Implementation progress and fidelity in a randomized controlled trial. American Journal of Community Psychology, 36, 313–332.
Rubak, S., Mortensen, L., Ringsted, C., & Malling, B. (2008). A controlled study of short- and long-term effects of a train the trainers’ course. Medical Education, 42, 693–702.
Salyers, M. P., McKasson, M., Bond, G. R., & McGrew, J. H. (2007). The role of technical assistance centers in implementing evidence-based practices: Lessons learned. American Journal of Psychiatric Rehabilitation, 10, 85–101.
Sandler, J. (2007). Community-based practices: Integrating dissemination theory with critical theories of power and justice. American Journal of Community Psychology, 40, 272–289.
St. Pierre, T. L., & Kaltreider, D. L. (2004). Tales of refusal, adoption, and maintenance: Evidence-based substance abuse prevention via school-extension collaborations. American Journal of Evaluation, 25, 479–491.
Steinert, Y., Mann, K., Centeno, A., Dolmans, D., Spencer, J., Gelula, M., et al. (2006). A systematic review of faculty development initiatives designed to improve teaching effectiveness in medical education: Best Evidence in Medical Education Guide Medical Teaching, 28, 497–526.
Stevenson, J. F., Florin, P., Mills, D. S., & Andrade, M. (2002). Building evaluation capacity in human service organizations: A case study. Evaluation and Program Planning, 25, 233–243.
U.S. Department of Education, Office of Planning, Evaluation and Policy Development, Policy and Program Studies Service. (2011). Prevalence and implementation fidelity of research-based prevention programs in public schools. Final Report, Washington, DC.
US Dept of Health and Human Services, Office of Adolescent Health. (2010). Programs for replication—Intervention implementation reports. Retrieved from: http://www.hhs.gov/ash/oah/oah-initiatives/tpp/programs.html.
USAID. (2008). Rwanda. Training of trainers manual. Retrieved from: http://pdf.usaid.gov/pdf_docs/PNADM806.pdf.
Valentine, J. C., Biglan, A., Boruch, R. F., Castro, F. G., Collins, L. M., Flay, B. R., Kellam, S., Moscicki, E. K., & Schinke, S. P. (2011). Replication in prevention science. Prevention Science, 12, 103–117. doi:10.1007/s11121-011-0217-6.
Villarruel, A., Jemmott, L. S., III, Jemmott, J. B. (2005). ¡Cuídate! NYC, NY: Select Media. Retrieved from: http://www.selectmedia.org/customer-service/evidence-based-curricula/cuidate/.
Wandersman, A., Chien, V., & Katz, J. (2012). Toward an evidence-based system for innovation support for implementing innovations with quality: Tools, training, technical assistance, and quality assurance/quality improvement. American Journal of Community Psychology. doi:10.1007/s10464-012-9509-7.
Wandersman, A., Duffy, J., Flaspohler, P., Noonan, R., Lubell, K., & Stillman, L. (2008). Bridging the gap between prevention research and practice: The interactive systems framework for dissemination and implementation. American Journal of Community Psychology, 41, 171–181.
Acknowledgments
The authors wish to thank Deborah Chilcoat, Gina Desiderio, and Milagros Garrido-Fishbein from Healthy Teen Network (HTN) for their many contributions to the PSBA project and editorial assistance with this article. We also thank Chris Zahniser (GEARS, Inc.) for her assistance with the evaluation and the Adolescent Reproductive Health Team for their support and encouragement throughout the PSBA project. We are grateful to the Centers for Disease Control and Prevention cooperative agreement # U65DP324964‐03 which supported HTN’s work on the PSBA project. The findings and conclusions in this paper are those of the authors and do not necessarily represent the views of the CDC.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Ray, M.L., Wilson, M.M., Wandersman, A. et al. Using a Training-of-Trainers Approach and Proactive Technical Assistance to Bring Evidence Based Programs to Scale: An Operationalization of the Interactive Systems Framework’s Support System. Am J Community Psychol 50, 415–427 (2012). https://doi.org/10.1007/s10464-012-9526-6
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10464-012-9526-6