Abstract
There continues to be a gap in prevention outcomes achieved in research trials versus those achieved in “real-world” practice. This article reports interim findings from a randomized controlled trial evaluating Assets-Getting To Outcomes (AGTO), a two-year intervention designed to build prevention practitioners’ capacity to implement positive youth development–oriented practices in 12 community coalitions in Maine. A survey of coalition members was used to assess change on individual practitioners’ prevention capacity between baseline and one year later. Structured interviews with 32 program directors (16 in the intervention group and 16 in the control group) were used to assess changes in programs’ prevention practices during the same time period. Change in prevention capacity over time did not differ significantly between the intervention and control groups. However, in secondary analyses of only those assigned to the AGTO intervention, users showed greater improvement in their self-efficacy to conduct Assets–based programming and increases in the frequency with which they engaged in AGTO behaviors, whereas among non-users, self-efficacy to conduct Assets–based programming declined. Interview ratings showed improvement in several key areas of performance among intervention programs. Improvement was associated with the number of technical assistance hours received. These results suggest that, after one year, AGTO is beginning to improve the capacity of community practitioners who make use of it.
Similar content being viewed by others
Notes
“Getting To Outcomes” and “GTO” are trademarks registered by the University of South Carolina and the RAND Corporation.
“Developmental Assets” is a trademark registered by the Search Institute.
Of the 60 programs initially randomized, nine folded between the time the study was planned and begun, leaving only 51 programs.
References
Acosta, J., & Chinman, M. (2011). Building capacity for outcomes measurement in human service organizations: The “Getting To Outcomes” (GTO) method. In J. L. Magnabosco & R. W. Manderscheid (Eds.), Outcomes measurement in the human services: Cross-cutting issues and methods (2nd ed.). Washington, DC: NASW Press.
Ajzen, I., & Fishbein, M. (1977). Attitude-behavior relations: A theoretical analysis and review of empirical research. Psychological Bulletin, 84, 888–918.
Backer, T. E. (2001). Finding the balance: Program fidelity and adaptation in substance abuse prevention: A state-of-the-art review. Rockville, MD: Substance Abuse and Mental Health Administration.
Bandura, A. (2004). Health promotion by social cognitive means. Health Education & Behavior, 31, 143–164.
Battelle. (2008). Annual findings report: Drug-free communities support program national evaluation. Washington, DC: Executive Office of the President Office of Administration Office of National Drug Control Policy.
Benson, P. L. (2002). Toward asset-building communities: How does change occur? In R. M. Lerner & P. L. Benson (Eds.), Developmental assets and asset-building communities: Implications for research policy and practice. Norwell, MA: Kluwer Academic.
Benson, P. L., & Scales, P. C. (2011). Developmental assets. In R. J. R. Levesque (Ed.), Encyclopedia of adolescence. New York, NY: Springer.
Benson, P. L., Scales, P. C., Leffert, N., & Roehlkepartain, E. R. (1999). A fragile foundation: The state of developmental assets among American youth. Minneapolis, MN: Search Institute.
Benson, P. L., Scales, P. C., & Mannes, M. (2003). Developmental strengths and their sources: Implications for the study and practice of community building. In R. M. Lerner, F. Jacobs, & D. Wertlieb (Eds.), Handbook of applied developmental science. Applying developmental science for youth and families (Vol. 1, pp. 369–406). Thousand Oaks, CA: Sage.
Benson, P. L., Scales, P. C., Hamilton, S. F., & Sesma, A. (2006). Positive youth development: Theory, research, and applications. In W. Damon, & R. M. Lerner (Eds.), Handbook of child psychology (6th ed., pp. 894–941). New York, NY: Wiley.
Botvin, G. J., Baker, E., Dusenbury, L., Botvin, E. M., & Diaz, T. (1995). Long-term follow-up results of a randomized drug abuse prevention trial in a white middle-class population. Journal of the American Medical Association, 273, 1106–1112.
Bryt, T., Bishop, J., & Carlin, J. B. (1993). Bias, prevalence and kappa. Journal of Clinical Epidemiology, 46, 423–429.
Butterfoss, F. D., Goodman, R. M., & Wandersman, A. (1993). Community coalitions for prevention and health promotion. Health Education Research, 8, 315–330.
Cantor, D., Crosse, S., Hagen, C. A., Mason, M. J., Siler, A. J., & von Glatz, A. (2001). A closer look at drug and violence prevention efforts in American schools: Report on the study on school violence and prevention. Washington, DC: U.S. Department of Education, Planning and Evaluation Service.
Chinman, M., Imm, P., & Wandersman, A. (2004). Getting To Outcomes 2004: Promoting accountability through methods and tools for planning, implementation, and evaluation (No. TR-TR101). Retrieved June 1, 2011, from http://www.rand.org/publications/TR/TR101/.
Chinman, M., Hannah, G., Wandersman, A., Ebener, P., Hunter, S., Imm, P., et al. (2005). Developing a community science research agenda for building community capacity for effective preventive interventions. American Journal of Community Psychology, 3, 143–157.
Chinman, M., Hunter, S., Ebener, P., Paddock, S., Stillman, L., Imm, P., et al. (2008). The Getting To Outcomes demonstration and evaluation: An illustration of the Prevention Support System. American Journal of Community Psychology, 41, 206–224.
Chinman, M., Tremain, B., Imm, P., & Wandersman, A. (2009). Strengthening prevention performance using technology: A formative evaluation of interactive Getting To Outcomes. The American Journal of Orthopsychiatry, 79, 469–481.
Chinman, M., Acosta, J., Ebener, P., Burkhart, Q., Clifford, M., Corsello, M., et al. (2012). Establishing and evaluating the key functions of an interactive systems framework using an assets-getting to outcomes intervention. American Journal of Community Psychology, 50, 295–310.
Cividin, T. M., & Ottoson, J. M. (1997). Linking reasons for continuing professional education participation with post-program application. Journal of Continuing Education in the Health Professions, 17, 46–55.
Collins, C., Harshbarger, C., Sawyer, R., & Hamdallah, M. (2006). The Diffusion of Effective Behavioral Interventions project: Development, implementation, and lessons learned. AIDS Education and Prevention, 18(4 Suppl. A), 5–20.
Crosse, S., Burr, M., Cantor, D., Hagen, C. A., & Hantman, I. (2001). Wide scope, questionable quality: Drug and violence prevention efforts in American schools. Report on the study on school violence and prevention. Washington, DC: U.S. Department of Education, Planning and Evaluation Service.
Crosse, S., Williams, B., Hagen, C. A., Harmon, M., Ristow, L., DiGaetano, R., et al. (2011). Prevalence and implementation fidelity of research-based prevention programs in public schools: Final report. Washington, DC: U.S. Department of Education, Planning and Evaluation Service.
Donner, A., & Klar, N. (2004). Pitfalls of and controversies in cluster randomization trials. American Journal of Public Health, 94, 416–422.
Ellickson, P. L., & Bell, R. M. (1990). Drug prevention in junior high: A multi-site longitudinal test. Science, 247, 1299–1305.
Ennett, S. T., Ringwalt, C. L., Thorne, J., Rohrbach, L. A., Vincus, A., Simons-Rudolph, A., et al. (2003). A comparison of current practice in school-based substance abuse prevention programs with meta-analysis findings. Prevention Science, 4, 1–14.
Feinberg, M. E., Ridenour, T. A., & Greenberg, M. T. (2008). The longitudinal effect of technical assistance dosage on the functioning of Communities That Care prevention boards in Pennsylvania. The Journal of Primary Prevention, 29(2), 145–165.
Fetterman, D. M., & Wandersman, A. (2005). Empowerment evaluation principles in practice. New York: The Guilford Press.
Fishbein, M., & Ajzen, I. (1974). Attitudes toward objects as predictive of single and multiple behavioral criteria. Psychological Review, 81, 59–74.
Fishbein, M., & Ajzen, I. (1975). Belief, attitude, intention, and behavior: An introduction to theory and research. Reading, MA: Addison-Wesley.
Fisher, D., Imm, P. S., Chinman, M., & Wandersman, A. (2006). Getting To Outcomes with developmental assets: Ten steps to measuring success in youth programs and communities. Minneapolis, MN: Search Institute.
Fixsen, D. L., Naoom, S. F., Blase, K. A., Friedman, R. M., & Wallace, F. (2005). Implementation research: A synthesis of the literature. Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network (FMHI Publication #231).
Flaspohler, P., Duffy, J., Wandersman, A., Stillman, L., & Maras, M. A. (2008). Unpacking prevention capacity: An intersection of research-to-practice models and community-centered models. American Journal of Community Psychology, 41, 182–196.
Green, L. (2001). From research to “Best Practices” in other settings and populations. American Journal of Health Behavior, 25, 165–178.
Hall, G. E., & Hord, S. M. (2001). Implementing change. Boston, MA: Allyn and Bacon.
Hall, G. E., & Loucks, S. F. (1977). A developmental model for determining whether the treatment is actually implemented. American Educational Research Journal, 14, 263–276.
Hall, G. E., Loucks, S. F., Rutherford, W. L., & Newlove, B. W. (1975). Levels of use of the innovation: A framework for analyzing innovation adoption. Journal of Teacher Education, 26, 52–56.
Hallfors, D., & Godette, D. (2002). Will the ‘principles of effectiveness’ improve prevention practice? Early findings from a diffusion study. Health Education Research, 17, 461–470.
Hawkins, J. D., Oesterle, S., Brown, E. C., Arthur, M. W., Abbott, R. D., Fagan, A. A., et al. (2009). Results of a type 2 translational research trial to prevent adolescent drug use and delinquency: A test of Communities that Care. Archives of Pediatric and Adolescent Medicine, 163, 789–798.
Hunter, S., Paddock, S., Ebener, P., Burkhart, Q., & Chinman, M. (2009). Promoting evidence based practices: The adoption of a prevention support system in community settings. The Journal of Community Psychology, 37, 579–593.
Jemmott, J. B., Jemmott, L. S., Fong, G. T., & McCaffree, K. (1999). Reducing HIV risk-associated sexual behavior among African American adolescents: Testing the generality of intervention effects. American Journal of Community Psychology, 27, 161–187.
Jemmott, J. B., Jemmott, L. S., Spears, H., Hewitt, N., & Cruz-Collins, M. (1992a). Self-efficacy, hedonistic expectancies, and condom-use intentions among inner-city black adolescent women: A social cognitive approach to AIDS risk behavior. Journal of Adolescent Health, 13, 512–519.
Jemmott, J. B., Jemmott, L. S., Spears, H., Hewitt, N., & Cruz-Collins, M. (1992b). Increasing condom-use intentions among sexually active inner-city black adolescent women: Effects of an AIDS prevention program. Nursing Research, 41, 273–279.
Keener, D. C. (2007). Towards a science of capacity building: An examination of technical assistance following a training of prevention professionals. (Doctoral dissertation). Retrieved from ProQuest Dissertations and Theses. (Accession Order No. AAT 3272459).
Kitson, A., Harvey, G., & McCormack, B. (1998). Enabling the implementation of evidence based practice: A conceptual framework. Quality in Health Care, 7, 149–158.
Leffert, N., Benson, P. L., Scales, P. C., Sharma, A. R., Drake, D. R., & Blyth, D. A. (1998). Developmental assets: Measurement and prediction of risk behaviors among adolescents. Applied Developmental Science, 2, 209–230.
Lehman, W. E. K., Greener, J. M., & Simpson, D. D. (2002). Assessing organizational readiness for change. Journal of Substance Abuse Treatment, 22, 197–209.
Livet, M., & Wandersman, A. (2005). Organizational functioning: Facilitating effective interventions and increasing the odds of programming success. In D. M. Fetterman & A. Wandersman (Eds.), Empowerment evaluation in practice (pp. 123–154). New York, NY: Guilford.
Maine Office of Substance Abuse. (2006). Maine youth drug and alcohol survey local summary reports. Retrieved March 25, 2009 from http://www.maine.gov/maineosa/survey/home.php.
Mihalic, S. F., & Irwin, K. (2003). Blueprints for violence prevention: From research to real-world settings—Factors influencing the successful replication of model programs. Youth Violence and Juvenile Justice, 1, 1–23.
Miller, T., & Hendrie, D. (2009). Substance abuse prevention dollars and cents: A cost-benefit analysis (DHHS Pub. No. (SMA) 07-4298). Rockville, MD: Center for Substance Abuse Prevention, Substance Abuse and Mental Health Services Administration.
Mitchell, R. E., Stone-Wiggins, B., Stevenson, J. F., & Florin, P. (2004). Cultivating capacity: Outcomes of a statewide support system for prevention coalitions. Journal of Prevention & Intervention in the Community, 27, 67–87.
Murray, D. M. (1998). Design and analysis of group-randomized trials. Oxford, UK: Oxford University Press.
O’Donnell, L. O., Scattergood, P., Adler, M., San Doval, A., Barker, M., et al. (2000). The role of technical assistance in the replication of effective HIV interventions. AIDS Education and Prevention, 12(Supl A), 99–111.
Ottoson, J. M. (1997). After the applause: Exploring multiple influences on application following an adult education program. Adult Education Quarterly, 47, 92–107.
Philliber, S., & Nolte, K. (2008). Implementation science: Promoting science-based approaches to prevent teen pregnancy. Prevention Science, 9, 166–177.
Reuter, P., & Timpane, J. M. (2001). Assessing options for the safe and drug free schools and communities act (No. MR-1328). Santa Monica, CA: RAND Corporation.
Ringwalt, C. L., Ennett, S., Vincus, A., Thorne, J., Rohrbach, L. A., & Simons-Rudolph, A. (2002). The prevalence of effective substance use prevention curricula in US middle schools. Prevention Science, 3, 257–265.
Robertson, E. B., David, S. L., & Rao, S. A. (2003). Preventing drug use among children and adolescents: A research-based guide for parents, educators, and community leaders (2nd ed.) (NIH Publication No. 04-4212[A]). Bethesda, MD: National Institute on Drug Abuse.
Rycroft-Malone, J., Harvey, G., Seers, K., Kitson, A., McCormack, B., & Titchen, A. (2002). An exploration of the factors that influence the implementation of evidence into practice. Journal of Clinical Nursing, 13, 913–924.
Scales, P. C. (1999). Reducing risks and building developmental assets: Essential actions for promoting adolescent health. Journal of School Health, 69, 113–119.
Scales, P. C., Benson, P. L., Leffert, N., & Blyth, D. A. (2000). Contribution of Developmental Assets to the prediction of thriving among adolescents. Applied Developmental Science, 4, 27–46.
Silvia, E. S., & Thorne, J. (1997). School-based drug prevention programs: A longitudinal study in selected school districts. Research Triangle Park, NC: Research Triangle Institute.
Spoth, R., Redmond, C., Shin, C., Greenberg, M., Clair, S., & Feinberg, M. (2007). Substance-use outcomes at 18 months past baseline: The PROSPER Community-University partnership trial. American Journal of Preventive Medicine, 32, 395–402.
Stetler, C. B., Legro, M. W., Rycroft-Malone, J., Bowman, C., Curran, G., Guihan, M., et al. (2006). Role of “external facilitation” in implementation of research findings: A qualitative evaluation of facilitation experiences in the Veterans Health Administration. Implementation Science, 1, 1–15.
Stevenson, J. F., Florin, P., Mills, D. S., & Andrade, M. (2002). Building evaluation capacity in human service organizations: A case study. Evaluation and Program Planning, 25, 233–243.
U.S. Census Bureau. (2000). State and county Quickfacts: Maine. Retrieved March 25, 2009, from http://quickfacts.census.gov.
Wandersman, A. (2009). Four keys to success (theory, implementation, evaluation, and resource/system support): High hopes and challenges in participation. American Journal of Community Psychology, 43, 3–21.
Wandersman, A., & Florin, P. (2003). Community interventions and effective prevention: Bringing researchers/evaluators, funders and practitioners together for accountability. American Psychologist, 58, 441–448.
Wandersman, A., Chien, V. H., & Katz, J. (2012). Toward an evidence-based system for innovation support for implementing innovations with quality: Tools, training, technical assistance, and quality assurance/quality improvement. American Journal of Community Psychology, 50, 445–449.
Weiss, E., Miller-Anderson, R., & Lasker, R. (2002). Making the most of collaboration: Exploring the relationship between partnership synergy and partnership functioning. Health Education and Behavior, 29, 683–698.
Wenger, E., McDermott, R., & Snyder, W. M. (2002). Cultivating communities of practice. Boston, MA: Harvard Business School Press.
Yin, R. K., Kaftarian, S. J., Yu, P., & Jansen, M. A. (1997). Outcomes from CSAP’s Community Partnership Program: Findings from the National Cross Site Evaluation. Evaluation and Program Planning, 20, 345–355.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Chinman, M., Acosta, J., Ebener, P. et al. Intervening with Practitioners to Improve the Quality of Prevention: One-Year Findings from a Randomized Trial of Assets-Getting To Outcomes. J Primary Prevent 34, 173–191 (2013). https://doi.org/10.1007/s10935-013-0302-7
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10935-013-0302-7