The Journal of Primary Prevention

, Volume 34, Issue 3, pp 173–191 | Cite as

Intervening with Practitioners to Improve the Quality of Prevention: One-Year Findings from a Randomized Trial of Assets-Getting To Outcomes

  • Matthew Chinman
  • Joie Acosta
  • Patricia Ebener
  • Q. Burkhart
  • Patrick S. Malone
  • Susan M. Paddock
  • Michael Clifford
  • Maryann Corsello
  • Tim Duffey
  • Sarah Hunter
  • Margaret Jones
  • Michel Lahti
  • Andrea Phillips
  • Susan Savell
  • Peter C. Scales
  • Nancy Tellett-Royce
Original Paper

Abstract

There continues to be a gap in prevention outcomes achieved in research trials versus those achieved in “real-world” practice. This article reports interim findings from a randomized controlled trial evaluating Assets-Getting To Outcomes (AGTO), a two-year intervention designed to build prevention practitioners’ capacity to implement positive youth development–oriented practices in 12 community coalitions in Maine. A survey of coalition members was used to assess change on individual practitioners’ prevention capacity between baseline and one year later. Structured interviews with 32 program directors (16 in the intervention group and 16 in the control group) were used to assess changes in programs’ prevention practices during the same time period. Change in prevention capacity over time did not differ significantly between the intervention and control groups. However, in secondary analyses of only those assigned to the AGTO intervention, users showed greater improvement in their self-efficacy to conduct Assets–based programming and increases in the frequency with which they engaged in AGTO behaviors, whereas among non-users, self-efficacy to conduct Assets–based programming declined. Interview ratings showed improvement in several key areas of performance among intervention programs. Improvement was associated with the number of technical assistance hours received. These results suggest that, after one year, AGTO is beginning to improve the capacity of community practitioners who make use of it.

Keywords

Technical assistance Implementation support Capacity building 

References

  1. Acosta, J., & Chinman, M. (2011). Building capacity for outcomes measurement in human service organizations: The “Getting To Outcomes” (GTO) method. In J. L. Magnabosco & R. W. Manderscheid (Eds.), Outcomes measurement in the human services: Cross-cutting issues and methods (2nd ed.). Washington, DC: NASW Press.Google Scholar
  2. Ajzen, I., & Fishbein, M. (1977). Attitude-behavior relations: A theoretical analysis and review of empirical research. Psychological Bulletin, 84, 888–918.CrossRefGoogle Scholar
  3. Backer, T. E. (2001). Finding the balance: Program fidelity and adaptation in substance abuse prevention: A state-of-the-art review. Rockville, MD: Substance Abuse and Mental Health Administration.Google Scholar
  4. Bandura, A. (2004). Health promotion by social cognitive means. Health Education & Behavior, 31, 143–164.CrossRefGoogle Scholar
  5. Battelle. (2008). Annual findings report: Drug-free communities support program national evaluation. Washington, DC: Executive Office of the President Office of Administration Office of National Drug Control Policy.Google Scholar
  6. Benson, P. L. (2002). Toward asset-building communities: How does change occur? In R. M. Lerner & P. L. Benson (Eds.), Developmental assets and asset-building communities: Implications for research policy and practice. Norwell, MA: Kluwer Academic.Google Scholar
  7. Benson, P. L., & Scales, P. C. (2011). Developmental assets. In R. J. R. Levesque (Ed.), Encyclopedia of adolescence. New York, NY: Springer.Google Scholar
  8. Benson, P. L., Scales, P. C., Leffert, N., & Roehlkepartain, E. R. (1999). A fragile foundation: The state of developmental assets among American youth. Minneapolis, MN: Search Institute.Google Scholar
  9. Benson, P. L., Scales, P. C., & Mannes, M. (2003). Developmental strengths and their sources: Implications for the study and practice of community building. In R. M. Lerner, F. Jacobs, & D. Wertlieb (Eds.), Handbook of applied developmental science. Applying developmental science for youth and families (Vol. 1, pp. 369–406). Thousand Oaks, CA: Sage.Google Scholar
  10. Benson, P. L., Scales, P. C., Hamilton, S. F., & Sesma, A. (2006). Positive youth development: Theory, research, and applications. In W. Damon, & R. M. Lerner (Eds.), Handbook of child psychology (6th ed., pp. 894–941). New York, NY: Wiley.Google Scholar
  11. Botvin, G. J., Baker, E., Dusenbury, L., Botvin, E. M., & Diaz, T. (1995). Long-term follow-up results of a randomized drug abuse prevention trial in a white middle-class population. Journal of the American Medical Association, 273, 1106–1112.PubMedCrossRefGoogle Scholar
  12. Bryt, T., Bishop, J., & Carlin, J. B. (1993). Bias, prevalence and kappa. Journal of Clinical Epidemiology, 46, 423–429.CrossRefGoogle Scholar
  13. Butterfoss, F. D., Goodman, R. M., & Wandersman, A. (1993). Community coalitions for prevention and health promotion. Health Education Research, 8, 315–330.PubMedCrossRefGoogle Scholar
  14. Cantor, D., Crosse, S., Hagen, C. A., Mason, M. J., Siler, A. J., & von Glatz, A. (2001). A closer look at drug and violence prevention efforts in American schools: Report on the study on school violence and prevention. Washington, DC: U.S. Department of Education, Planning and Evaluation Service.Google Scholar
  15. Chinman, M., Imm, P., & Wandersman, A. (2004). Getting To Outcomes 2004: Promoting accountability through methods and tools for planning, implementation, and evaluation (No. TR-TR101). Retrieved June 1, 2011, from http://www.rand.org/publications/TR/TR101/.
  16. Chinman, M., Hannah, G., Wandersman, A., Ebener, P., Hunter, S., Imm, P., et al. (2005). Developing a community science research agenda for building community capacity for effective preventive interventions. American Journal of Community Psychology, 3, 143–157.CrossRefGoogle Scholar
  17. Chinman, M., Hunter, S., Ebener, P., Paddock, S., Stillman, L., Imm, P., et al. (2008). The Getting To Outcomes demonstration and evaluation: An illustration of the Prevention Support System. American Journal of Community Psychology, 41, 206–224.PubMedCrossRefGoogle Scholar
  18. Chinman, M., Tremain, B., Imm, P., & Wandersman, A. (2009). Strengthening prevention performance using technology: A formative evaluation of interactive Getting To Outcomes. The American Journal of Orthopsychiatry, 79, 469–481.PubMedCrossRefGoogle Scholar
  19. Chinman, M., Acosta, J., Ebener, P., Burkhart, Q., Clifford, M., Corsello, M., et al. (2012). Establishing and evaluating the key functions of an interactive systems framework using an assets-getting to outcomes intervention. American Journal of Community Psychology, 50, 295–310.PubMedCrossRefGoogle Scholar
  20. Cividin, T. M., & Ottoson, J. M. (1997). Linking reasons for continuing professional education participation with post-program application. Journal of Continuing Education in the Health Professions, 17, 46–55.CrossRefGoogle Scholar
  21. Collins, C., Harshbarger, C., Sawyer, R., & Hamdallah, M. (2006). The Diffusion of Effective Behavioral Interventions project: Development, implementation, and lessons learned. AIDS Education and Prevention, 18(4 Suppl. A), 5–20.Google Scholar
  22. Crosse, S., Burr, M., Cantor, D., Hagen, C. A., & Hantman, I. (2001). Wide scope, questionable quality: Drug and violence prevention efforts in American schools. Report on the study on school violence and prevention. Washington, DC: U.S. Department of Education, Planning and Evaluation Service.Google Scholar
  23. Crosse, S., Williams, B., Hagen, C. A., Harmon, M., Ristow, L., DiGaetano, R., et al. (2011). Prevalence and implementation fidelity of research-based prevention programs in public schools: Final report. Washington, DC: U.S. Department of Education, Planning and Evaluation Service.Google Scholar
  24. Donner, A., & Klar, N. (2004). Pitfalls of and controversies in cluster randomization trials. American Journal of Public Health, 94, 416–422.PubMedCrossRefGoogle Scholar
  25. Ellickson, P. L., & Bell, R. M. (1990). Drug prevention in junior high: A multi-site longitudinal test. Science, 247, 1299–1305.PubMedCrossRefGoogle Scholar
  26. Ennett, S. T., Ringwalt, C. L., Thorne, J., Rohrbach, L. A., Vincus, A., Simons-Rudolph, A., et al. (2003). A comparison of current practice in school-based substance abuse prevention programs with meta-analysis findings. Prevention Science, 4, 1–14.PubMedCrossRefGoogle Scholar
  27. Feinberg, M. E., Ridenour, T. A., & Greenberg, M. T. (2008). The longitudinal effect of technical assistance dosage on the functioning of Communities That Care prevention boards in Pennsylvania. The Journal of Primary Prevention, 29(2), 145–165.PubMedCrossRefGoogle Scholar
  28. Fetterman, D. M., & Wandersman, A. (2005). Empowerment evaluation principles in practice. New York: The Guilford Press.Google Scholar
  29. Fishbein, M., & Ajzen, I. (1974). Attitudes toward objects as predictive of single and multiple behavioral criteria. Psychological Review, 81, 59–74.CrossRefGoogle Scholar
  30. Fishbein, M., & Ajzen, I. (1975). Belief, attitude, intention, and behavior: An introduction to theory and research. Reading, MA: Addison-Wesley.Google Scholar
  31. Fisher, D., Imm, P. S., Chinman, M., & Wandersman, A. (2006). Getting To Outcomes with developmental assets: Ten steps to measuring success in youth programs and communities. Minneapolis, MN: Search Institute.Google Scholar
  32. Fixsen, D. L., Naoom, S. F., Blase, K. A., Friedman, R. M., & Wallace, F. (2005). Implementation research: A synthesis of the literature. Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network (FMHI Publication #231).Google Scholar
  33. Flaspohler, P., Duffy, J., Wandersman, A., Stillman, L., & Maras, M. A. (2008). Unpacking prevention capacity: An intersection of research-to-practice models and community-centered models. American Journal of Community Psychology, 41, 182–196.PubMedCrossRefGoogle Scholar
  34. Green, L. (2001). From research to “Best Practices” in other settings and populations. American Journal of Health Behavior, 25, 165–178.PubMedCrossRefGoogle Scholar
  35. Hall, G. E., & Hord, S. M. (2001). Implementing change. Boston, MA: Allyn and Bacon.Google Scholar
  36. Hall, G. E., & Loucks, S. F. (1977). A developmental model for determining whether the treatment is actually implemented. American Educational Research Journal, 14, 263–276.CrossRefGoogle Scholar
  37. Hall, G. E., Loucks, S. F., Rutherford, W. L., & Newlove, B. W. (1975). Levels of use of the innovation: A framework for analyzing innovation adoption. Journal of Teacher Education, 26, 52–56.CrossRefGoogle Scholar
  38. Hallfors, D., & Godette, D. (2002). Will the ‘principles of effectiveness’ improve prevention practice? Early findings from a diffusion study. Health Education Research, 17, 461–470.PubMedCrossRefGoogle Scholar
  39. Hawkins, J. D., Oesterle, S., Brown, E. C., Arthur, M. W., Abbott, R. D., Fagan, A. A., et al. (2009). Results of a type 2 translational research trial to prevent adolescent drug use and delinquency: A test of Communities that Care. Archives of Pediatric and Adolescent Medicine, 163, 789–798.CrossRefGoogle Scholar
  40. Hunter, S., Paddock, S., Ebener, P., Burkhart, Q., & Chinman, M. (2009). Promoting evidence based practices: The adoption of a prevention support system in community settings. The Journal of Community Psychology, 37, 579–593.CrossRefGoogle Scholar
  41. Jemmott, J. B., Jemmott, L. S., Fong, G. T., & McCaffree, K. (1999). Reducing HIV risk-associated sexual behavior among African American adolescents: Testing the generality of intervention effects. American Journal of Community Psychology, 27, 161–187.PubMedCrossRefGoogle Scholar
  42. Jemmott, J. B., Jemmott, L. S., Spears, H., Hewitt, N., & Cruz-Collins, M. (1992a). Self-efficacy, hedonistic expectancies, and condom-use intentions among inner-city black adolescent women: A social cognitive approach to AIDS risk behavior. Journal of Adolescent Health, 13, 512–519.PubMedCrossRefGoogle Scholar
  43. Jemmott, J. B., Jemmott, L. S., Spears, H., Hewitt, N., & Cruz-Collins, M. (1992b). Increasing condom-use intentions among sexually active inner-city black adolescent women: Effects of an AIDS prevention program. Nursing Research, 41, 273–279.PubMedGoogle Scholar
  44. Keener, D. C. (2007). Towards a science of capacity building: An examination of technical assistance following a training of prevention professionals. (Doctoral dissertation). Retrieved from ProQuest Dissertations and Theses. (Accession Order No. AAT 3272459).Google Scholar
  45. Kitson, A., Harvey, G., & McCormack, B. (1998). Enabling the implementation of evidence based practice: A conceptual framework. Quality in Health Care, 7, 149–158.PubMedCrossRefGoogle Scholar
  46. Leffert, N., Benson, P. L., Scales, P. C., Sharma, A. R., Drake, D. R., & Blyth, D. A. (1998). Developmental assets: Measurement and prediction of risk behaviors among adolescents. Applied Developmental Science, 2, 209–230.CrossRefGoogle Scholar
  47. Lehman, W. E. K., Greener, J. M., & Simpson, D. D. (2002). Assessing organizational readiness for change. Journal of Substance Abuse Treatment, 22, 197–209.PubMedCrossRefGoogle Scholar
  48. Livet, M., & Wandersman, A. (2005). Organizational functioning: Facilitating effective interventions and increasing the odds of programming success. In D. M. Fetterman & A. Wandersman (Eds.), Empowerment evaluation in practice (pp. 123–154). New York, NY: Guilford.Google Scholar
  49. Maine Office of Substance Abuse. (2006). Maine youth drug and alcohol survey local summary reports. Retrieved March 25, 2009 from http://www.maine.gov/maineosa/survey/home.php.
  50. Mihalic, S. F., & Irwin, K. (2003). Blueprints for violence prevention: From research to real-world settings—Factors influencing the successful replication of model programs. Youth Violence and Juvenile Justice, 1, 1–23.CrossRefGoogle Scholar
  51. Miller, T., & Hendrie, D. (2009). Substance abuse prevention dollars and cents: A cost-benefit analysis (DHHS Pub. No. (SMA) 07-4298). Rockville, MD: Center for Substance Abuse Prevention, Substance Abuse and Mental Health Services Administration.Google Scholar
  52. Mitchell, R. E., Stone-Wiggins, B., Stevenson, J. F., & Florin, P. (2004). Cultivating capacity: Outcomes of a statewide support system for prevention coalitions. Journal of Prevention & Intervention in the Community, 27, 67–87.CrossRefGoogle Scholar
  53. Murray, D. M. (1998). Design and analysis of group-randomized trials. Oxford, UK: Oxford University Press.Google Scholar
  54. O’Donnell, L. O., Scattergood, P., Adler, M., San Doval, A., Barker, M., et al. (2000). The role of technical assistance in the replication of effective HIV interventions. AIDS Education and Prevention, 12(Supl A), 99–111.Google Scholar
  55. Ottoson, J. M. (1997). After the applause: Exploring multiple influences on application following an adult education program. Adult Education Quarterly, 47, 92–107.CrossRefGoogle Scholar
  56. Philliber, S., & Nolte, K. (2008). Implementation science: Promoting science-based approaches to prevent teen pregnancy. Prevention Science, 9, 166–177.PubMedCrossRefGoogle Scholar
  57. Reuter, P., & Timpane, J. M. (2001). Assessing options for the safe and drug free schools and communities act (No. MR-1328). Santa Monica, CA: RAND Corporation.Google Scholar
  58. Ringwalt, C. L., Ennett, S., Vincus, A., Thorne, J., Rohrbach, L. A., & Simons-Rudolph, A. (2002). The prevalence of effective substance use prevention curricula in US middle schools. Prevention Science, 3, 257–265.PubMedCrossRefGoogle Scholar
  59. Robertson, E. B., David, S. L., & Rao, S. A. (2003). Preventing drug use among children and adolescents: A research-based guide for parents, educators, and community leaders (2nd ed.) (NIH Publication No. 04-4212[A]). Bethesda, MD: National Institute on Drug Abuse.Google Scholar
  60. Rycroft-Malone, J., Harvey, G., Seers, K., Kitson, A., McCormack, B., & Titchen, A. (2002). An exploration of the factors that influence the implementation of evidence into practice. Journal of Clinical Nursing, 13, 913–924.CrossRefGoogle Scholar
  61. Scales, P. C. (1999). Reducing risks and building developmental assets: Essential actions for promoting adolescent health. Journal of School Health, 69, 113–119.PubMedCrossRefGoogle Scholar
  62. Scales, P. C., Benson, P. L., Leffert, N., & Blyth, D. A. (2000). Contribution of Developmental Assets to the prediction of thriving among adolescents. Applied Developmental Science, 4, 27–46.CrossRefGoogle Scholar
  63. Silvia, E. S., & Thorne, J. (1997). School-based drug prevention programs: A longitudinal study in selected school districts. Research Triangle Park, NC: Research Triangle Institute.Google Scholar
  64. Spoth, R., Redmond, C., Shin, C., Greenberg, M., Clair, S., & Feinberg, M. (2007). Substance-use outcomes at 18 months past baseline: The PROSPER Community-University partnership trial. American Journal of Preventive Medicine, 32, 395–402.PubMedCrossRefGoogle Scholar
  65. Stetler, C. B., Legro, M. W., Rycroft-Malone, J., Bowman, C., Curran, G., Guihan, M., et al. (2006). Role of “external facilitation” in implementation of research findings: A qualitative evaluation of facilitation experiences in the Veterans Health Administration. Implementation Science, 1, 1–15.CrossRefGoogle Scholar
  66. Stevenson, J. F., Florin, P., Mills, D. S., & Andrade, M. (2002). Building evaluation capacity in human service organizations: A case study. Evaluation and Program Planning, 25, 233–243.CrossRefGoogle Scholar
  67. U.S. Census Bureau. (2000). State and county Quickfacts: Maine. Retrieved March 25, 2009, from http://quickfacts.census.gov.
  68. Wandersman, A. (2009). Four keys to success (theory, implementation, evaluation, and resource/system support): High hopes and challenges in participation. American Journal of Community Psychology, 43, 3–21.PubMedCrossRefGoogle Scholar
  69. Wandersman, A., & Florin, P. (2003). Community interventions and effective prevention: Bringing researchers/evaluators, funders and practitioners together for accountability. American Psychologist, 58, 441–448.PubMedCrossRefGoogle Scholar
  70. Wandersman, A., Chien, V. H., & Katz, J. (2012). Toward an evidence-based system for innovation support for implementing innovations with quality: Tools, training, technical assistance, and quality assurance/quality improvement. American Journal of Community Psychology, 50, 445–449.PubMedCrossRefGoogle Scholar
  71. Weiss, E., Miller-Anderson, R., & Lasker, R. (2002). Making the most of collaboration: Exploring the relationship between partnership synergy and partnership functioning. Health Education and Behavior, 29, 683–698.PubMedCrossRefGoogle Scholar
  72. Wenger, E., McDermott, R., & Snyder, W. M. (2002). Cultivating communities of practice. Boston, MA: Harvard Business School Press.Google Scholar
  73. Yin, R. K., Kaftarian, S. J., Yu, P., & Jansen, M. A. (1997). Outcomes from CSAP’s Community Partnership Program: Findings from the National Cross Site Evaluation. Evaluation and Program Planning, 20, 345–355.CrossRefGoogle Scholar

Copyright information

© RAND Corporation 2013

Authors and Affiliations

  • Matthew Chinman
    • 1
  • Joie Acosta
    • 1
  • Patricia Ebener
    • 1
  • Q. Burkhart
    • 1
  • Patrick S. Malone
    • 2
  • Susan M. Paddock
    • 1
  • Michael Clifford
    • 3
  • Maryann Corsello
    • 6
  • Tim Duffey
    • 3
  • Sarah Hunter
    • 1
  • Margaret Jones
    • 3
  • Michel Lahti
    • 5
  • Andrea Phillips
    • 1
  • Susan Savell
    • 6
  • Peter C. Scales
    • 7
  • Nancy Tellett-Royce
    • 7
  1. 1.RAND CorporationPittsburghUSA
  2. 2.University of South CarolinaColumbiaUSA
  3. 3.Vision Training AssociatesGorhamUSA
  4. 4.University of New EnglandPortlandUSA
  5. 5.University of Southern MainePortlandUSA
  6. 6.SpurwinkPortlandUSA
  7. 7.Search InstituteMinneapolisUSA

Personalised recommendations