Skip to main content
SpringerLink
Log in

Prevention System Implementation and Reach: Attitudes and Environmental Predictors in a Randomized Controlled Trial of the NORTH STAR Prevention System

  • Published:
Prevention Science Aims and scope Submit manuscript

Abstract

Although many evidence-based interventions are well-established, our understanding of how to effectively implement and sustain those interventions in real-world settings is less well understood. We investigated predictors of implementation and reach in a randomized controlled trial of the NORTH STAR prevention system. One-third of U.S. Air Force (AF) bases worldwide were randomly assigned to NORTH STAR (n = 12) or an assessment-and-feedback-only condition (n = 12). Process data regarding implementation factors were collected from Community Action Team (CAT) members and observations of CAT processes. Results from a series of regression analyses indicated that change in leadership and community support, action planning processes, and perceived approach effectiveness from pre-action planning to follow-up predicted community action plan (CAP) implementation and that changes in barriers to implementation predicted CAP reach. Pre-action planning reports of CAT member self-efficacy and perceived approach effectiveness also predicted CAP implementation at 1-year follow-up. Future directions and practice recommendations are provided.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

Notes

  1. At the time of this study, the CAT comprised representatives from the following USAF agencies: Airman and Family Support, Family Advocacy Program (i.e., military family maltreatment prevention and response), Health and Wellness Center, Mental Health, Child and Youth Programs, Chaplain. Each agency was required to provide a representative; it was up to each agency to decide how to choose a representative. Members from other Air Force elements could be appointed at the direction of the Wing Commander (e.g., Finance, Communications).

References

  • Aarons, G. A., Green, A. E., Palinkas, L. A., et al. (2012). Dynamic adaptation process to implement an evidence-based child maltreatment intervention. Implementation Science, 7, 32. https://doi.org/10.1186/1748-5908-7-32

    Article  PubMed  PubMed Central  Google Scholar 

  • Alcántara, C., Diaz, S. V., Cosenzo, L. G., Loucks, E. B., Penedo, F. J., & Williams, N. J. (2020). Social determinants as moderators of health behaviour change interventions: Scientific gaps and opportunities. Health Psychology Review. https://doi.org/10.1080/17437199.2020.1718527

    Article  PubMed  Google Scholar 

  • Ajzen, I. (1991). The Theory of Planned Behavior. Organizational Behavior and Human Decision Processes, 50, 179–211.

    Article  Google Scholar 

  • Arthur, M. W., Hawkins, J. D., Catalano, R. F., & Olson, J. J. (2002). Community key informant survey. Social Development Research Group, University of Washington.

    Google Scholar 

  • Bandura, A. (1977). Self-efficacy: Toward a unifying theory of behavioral change. Psychological Review, 84, 191. https://doi.org/10.1037/0033-295X.84.2.191

    Article  CAS  PubMed  Google Scholar 

  • Barnidge, E. K., Radvanyi, C., Duggan, K., Motton, F., Wiggs, I., Baker, E. A., & Brownson, R. C. (2013). Understanding and addressing barriers to implementation of environmental and policy interventions to support physical activity and healthy eating in rural communities. The Journal of Rural Health, 29, 97–105. https://doi.org/10.1111/j.1748-0361.2012.00431.x

    Article  PubMed  Google Scholar 

  • Basic, J. (2015). Community mobilization and readiness: Planning flaws which challenge effective implementation of “Communities that Care” (CTC) prevention system. Substance Use and Misuse, 50, 1083–1088. https://doi.org/10.3109/10826084.2015.1007655

    Article  PubMed  Google Scholar 

  • Berends, M., Bodilly, S. J., & Kirby, S. N. (2002). Facing the challenges of whole-school reform: New American Schools after a decade. Rand Corporation. MR-1498-EDU, 2002. As of February 08, 2022: https://www.rand.org/pubs/monograph_reports/MR1498.html

  • Botvin, G. J. (2004). Advancing prevention science and practice: Challenges, critical issues, and future directions. Prevention Science, 5, 69–72. https://doi.org/10.1023/B:PREV.0000013984.83251.8b

    Article  PubMed  Google Scholar 

  • Breitenstein, S. M., Laurent, S., Pabalan, L., Risser, H. J., Roper, P., Saba, M. T., & Schoeny, M. (2019). Implementation findings from an effectiveness-implementation trial of tablet-based parent training in pediatric primary care. Families, Systems, & Health. Advance online publication. https://doi.org/10.1037/fsh0000447

  • Brown, E. C., Hawkins, J. D., Rhew, I. C., Shapiro, V. B., Abbott, R. D., Oesterle, S., & Catalano, R. F. (2014). Prevention system mediation of communities that care effects on youth outcomes. Prevention Science, 15, 623–632. https://doi.org/10.1007/s11121-013-0413-7

    Article  PubMed  PubMed Central  Google Scholar 

  • Brown, L. D., Feinberg, M. E., Shapiro, V. B., & Greenberg, M. T. (2015). Reciprocal relations between coalition functioning and the provision of implementation support. Prevention Science, 16, 101–109. https://doi.org/10.1007/s11121-013-0447-x

    Article  PubMed  PubMed Central  Google Scholar 

  • Castillo, J. M., Wang, J. H., Daye, J. G., Shum, K. Z., & March, A. L. (2018). A longitudinal analysis of the relations among professional development, educators’ beliefs and perceived skills, and response-to-intervention implementation. Journal of Educational and Psychological Consultation, 28, 413–444. https://doi.org/10.1080/10474412.2017.1394864

    Article  Google Scholar 

  • Catalano, R. F., Fagan, A. A., Gavin, L. E., Greenberg, M. T., Irwin, C. E., Jr., Ross, D. A., & Shek, D. T. (2012). Worldwide application of prevention science in adolescent health. The Lancet, 379, 1653–1664. https://doi.org/10.1016/S0140-6736(12)60238-4

    Article  Google Scholar 

  • Chilenski, S. M., Frank, J., Summers, N., & Lew, D. (2019). Public health benefits 16 years after a statewide policy change: Communities that care in Pennsylvania. Prevention Science. https://doi.org/10.1007/s11121-019-01028-y

    Article  PubMed  Google Scholar 

  • Damschroder, L. J. (2020). Clarity out of chaos: Use of theory in implementation research. Psychiatry Research, 283, 112461. https://doi.org/10.1016/j.psychres.2019.06.036

    Article  PubMed  Google Scholar 

  • Damschroder, L. J., Aron, D. C., Keith, R. E., Kirsh, S. R., Alexander, J. A., & Lowery, J. C. (2009). Fostering implementation of health services research findings into practice: A consolidated framework for advancing implementation science. Implementation Science, 4, 50. https://doi.org/10.1186/1748-5908-4-50

    Article  PubMed  PubMed Central  Google Scholar 

  • Drainoni, M., Koppelman, E. A., Feldman, J. A., Walley, A. Y., Mitchell, P. M., Ellison, J., & Bernstein, E. (2016). Why is it so hard to implement change? A qualitative examination of barriers and facilitators to distribution of naloxone for overdose prevention in a safety net environment. BMC Research Notes, 9, 465. https://doi.org/10.1186/s13104-016-2268-z

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Ehrhart, M. G., Aarons, G. A., & Farahnak, L. R. (2014). Assessing the organizational context for EBP implementation: The development and validity testing of the Implementation Climate Scale (ICS). Implementation Science, 9, 157–168. https://doi.org/10.1186/s13012-014-0157-1

    Article  PubMed  PubMed Central  Google Scholar 

  • Forman, S. G., Olin, S. S., Hoagwood, K. E., Crowe, M., & Saka, N. (2009). Evidence-based interventions in schools: Developers’ views of implementation barriers and facilitators. School Mental Health, 1, 26–36. https://doi.org/10.1007/s12310-008-9002-5

    Article  Google Scholar 

  • Glasgow, R. E., Vogt, T. M., & Boles, S. M. (1999). Evaluating the public health impact of health promotion interventions: The RE-AIM framework. American Journal of Public Health, 89, 1322–1327. https://doi.org/10.2105/AJPH.89.9.1322

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Gottfredson, D. C., Cook, T. D., Gardner, F. E. M., Gorman-Smith, D., Howe, G. W., Sandler, I. N., & Zafft, K. M. (2015). Standards of evidence for efficacy, effectiveness, and scale-up research in prevention science: Next generation. Prevention Science, 16, 893–926. https://doi.org/10.1007/s11121-015-0555-x

    Article  PubMed  PubMed Central  Google Scholar 

  • Hallfors, D., & Godette, D. (2002). Will the ‘Principles of Effectiveness’ improve prevention practice? Early findings from a diffusion study. Health Education Research, 17, 461–470. https://doi.org/10.1093/her/17.4.461

    Article  CAS  PubMed  Google Scholar 

  • Hawkins, J. D., & Catalano, R. F., Jr. (1992). Communities that care: Action for drug abuse prevention. Jossey-Bass.

  • Hawkins, J. D., Catalano, R. F., & Arthur, M. W. (2002). Promoting science-based prevention in communities. Addictive Behaviors, 27, 951–976. https://doi.org/10.1016/S0306-4603(02)00298-8

    Article  PubMed  Google Scholar 

  • Henggeler, S. W., Schoenwald, S. K., Liao, J. G., Letourneau, E. J., & Edwards, D. L. (2002). Transporting efficacious treatments to field settings: The link between supervisory practices and therapist fidelity in MST programs. Journal of Clinical Child and Adolescent Psychology, 31, 155–167. https://doi.org/10.1207/S15374424JCCP3102_02

    Article  PubMed  Google Scholar 

  • Heyman, R. E., Slep, A. M. S., & Nelson, J. P. (2011). Empirically guided community intervention for partner abuse, child maltreatment, suicidality, and substance misuse. In S. MacDermid Wadsworth & D. Riggs (Eds.), Risk and Resilience in U.S. Military Families, (pp. 85–110). Springer.

  • Kim, P. Y., Toblin, R. L., Riviere, L. A., Kok, B. C., Grossman, S. H., & Wilk, J. E. (2016). Provider and nonprovider sources of mental health help in the military and the effects of stigma, negative attitudes, and organizational barriers to care. Psychiatric Services, 67, 221–226. https://doi.org/10.1176/appi.ps.201400519

    Article  PubMed  Google Scholar 

  • Labouliere, C. D., Green, K. L., Vasan, P., Cummings, A., Layman, D., Kammer, J., Rahman, M., Brown, G. K., Finnerty, M., & Stanley, B. (2021). Is the outpatient mental health workforce ready to save lives? Suicide prevention training, knowledge, self-efficacy, and clinical practices prior to the implementation of a statewide suicide prevention initiative. Suicide and Life-Threatening Behaviors, 51, 325–333. https://doi.org/10.1111/sltb.12708

    Article  Google Scholar 

  • Lo, S. B., Ryba, M. M., Brothers, B. M., & Andersen, B. L. (2019). Predicting implementation of an empirically supported treatment for cancer patients using the Theory of Planned Behavior. Health Psychology. https://doi.org/10.1037/hea0000794

    Article  PubMed  PubMed Central  Google Scholar 

  • Melgarejo, M., & Lind., T., Stadnick, N. A., Helm, J. L., & Locke, J. (2020). Strengthening capacity for implementation of evidence-based practices for autism in schools: The roles of implementation climate, school leadership, and fidelity. American Psychologist, 75, 1105–1115. https://doi.org/10.1037/amp0000649

    Article  PubMed  Google Scholar 

  • Mihalic, S., Fagan, A., Irwin, K., Ballard, D., & Elliott, D. (2004). Blueprints for violence prevention (NCJ 204274). Office of Juvenile and Delinquency Prevention.

    Google Scholar 

  • Mihalic, S. F., & Irwin, K. (2003). From research to real-world settings – Factors influencing the successful replication of model programs. Youth Violence and Juvenile Justice, 1, 307–329.

    Article  Google Scholar 

  • Muthén, L. K., & Muthén, B. O. (1998–2017). Mplus user’s guide (8th ed.). Los Angeles, CA: Muthén & Muthén.

  • Nielson, P. (2015). Making sense of implementation theories, models, and frameworks. Implementation Science, 10, 53. https://doi.org/10.1186/s13012-015-0242-0

    Article  Google Scholar 

  • Oakes, W. P., Lane, K. L., Royer, D. J., Menzies, H. M., Buckman, M. M., Brunsting, N., Cantwell, E. D., Schatschneider, C., & Lane, N. A. (2021). Elementary teachers’ self-efficacy during initial implementation of comprehensive, integrated, three-tiered models of prevention. Journal of Positive Behavior Interventions, 23, 93–105. https://doi.org/10.1177/1098300720916718

    Article  Google Scholar 

  • Panzano, P. C., & Roth, D. (2006). The decision to adopt evidence-based and other innovative mental health practices: Risky business? Psychiatric Services, 57, 1153–1161. https://doi.org/10.1176/ps.2006.57.8.1153

    Article  PubMed  Google Scholar 

  • Panzano, P. C., Seffrin, B., Chaney-Jones, S., Roth, D., Crane-Ross, D., Massatti, R. et al. (2005). The innovation diffusion and adoption research project (IDARP): Moving from the diffusion of research results to promoting the adoption of evidence-based innovations in the Ohio mental health system. New Research in Mental Health, 16.

  • Perez-Gomez, A., Mejia-Trujillo, J., Brown, E. C., & Eisenberg, N. (2016). Adaptation and implementation of a science-based prevention system in Colombia: Challenges and achievements. Journal of Community Psychology, 44, 538–545. https://doi.org/10.1002/jcop.21781

    Article  PubMed  PubMed Central  Google Scholar 

  • Poston, W. S. C., Haddock, C. K., Jahnke, S. A., Jitnarin, N., Malone, R. E., & Smith, E. A. (2016). Perspectives of US military commanders on tobacco use and tobacco control policy. Tobacco Control, 26, 254–259. https://doi.org/10.1136/tobaccocontrol-2015-052829

    Article  PubMed  Google Scholar 

  • Ransford, C. R., Greenberg, M. T., Domitrovich, C. E., Small, M., & Jacobson, L. (2009). The role of teachers’ psychological experiences and perceptions of curriculum supports on the implementation of a social and emotional learning curriculum. School Psychology Review, 38, 510–532.

    Google Scholar 

  • Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models: Applications and data analysis methods (Vol. 1). sage.

  • Rohrbach, L. A., Grana, R., Sussman, S., & Valente, T. W. (2006). Type II translation: Transporting prevention interventions from research to real-world settings. Evaluation & the Health Professions, 29, 302–333. https://doi.org/10.1177/0163278706290408

    Article  Google Scholar 

  • Rogers, E. M. (2010). Diffusion of innovations. Simon and Schuster.

    Google Scholar 

  • Saldana, L., Chamberlain, P., Wang, W., & Brown, C. H. (2012). Predicting program start-up using the stages of implementation measure. Administration and Policy in Mental Health and Mental Health Services Research, 39, 419–425. https://doi.org/10.1007/s10488-011-0363-y

    Article  PubMed  Google Scholar 

  • Schoenwald, S. K., Sheidow, A. J., Letourneau, E. J., & Liao, J. G. (2003). Transportability of multisystemic therapy: Evidence for multilevel influences. Mental Health Services Research, 5, 223–239.

    Article  Google Scholar 

  • Shapiro, V. B., Oesterle, S., & Hawkins, J. D. (2015). Relating coalition capacity to the adoption of science-based prevention in communities: Evidence from a randomized trial of Communities that Care. American Journal of Community Psychology, 55, 1–12. https://doi.org/10.1007/s10464-014-9684-9

    Article  PubMed  PubMed Central  Google Scholar 

  • Zeeck, A., Herpertz-Dahlmann, B., Friederich, H. C., Brockmeyer, T., Resmark, G., Hagenah, U., & Hartmann, A. (2018). Psychotherapeutic treatment for anorexia nervosa: A systematic review and network meta-analysis. Frontiers in Psychiatry, 9, 158. https://doi.org/10.3389/fpsyt.2018.00158

    Article  PubMed  PubMed Central  Google Scholar 

  • Slep, A. M. S. & Heyman, R. E. (Eds.). (2006). Enhancing the integrated delivery system: A guidebook to activities that work, Third Edition. New York, NY: Family Translational Research Group, New York University.

  • Slep, A. M. S. & Heyman, R. E. (2008). Public health approaches to family maltreatment prevention: Resetting family psychology’s sights from the home to the community. Journal of Family Psychology, 22, 518–528.

  • Slep, A. M. S., Heyman, R. E., Lorber, M. F., & Linkh, D. J. (2021). The impact of NORTH STAR on suicidality, substance problems, intimate partner violence, and child abuse. Military Medicine, 186(3–4), e351–e358. https://doi.org/10.1093/milmed/usaa380

  • Spoth, R., Rohrbach, L. A., Greenberg, M., Leaf, P., Brown, C. H., Fagan, A., & Hawkins, J. D. (2013). Addressing core challenges for the next generation of type 2 translation research and systems: The translation science to population impact (TSci Impact) framework. Prevention Science, 14, 319–351. https://doi.org/10.1007/s11121-012-0362-6

    Article  PubMed  PubMed Central  Google Scholar 

  • Stanley, J. (1971). Reliability. In R. L. Thorndike (Ed.), Educational measurement. Second edition. Washington, DC: American Council on Education

  • Trejo, L. L., Steele, N. M., & Jordan, P. (2018). Using the RE-AIM model of health promotion to implement a military women’s health promotion program for austere settings. Military Medicine, 183, 538–546. https://doi.org/10.1093/milmed/usx230

    Article  Google Scholar 

  • Turner, K. M. T., Nicholson, J. M., & Sanders, M. R. (2011). The role of practitioner self-efficacy, training, program and workplace factors on the implementation of an evidence-based parenting intervention in primary care. Journal of Primary Prevention, 32, 95–112. https://doi.org/10.1007/s10935-011-0240-1

    Article  PubMed  Google Scholar 

  • Whiteman, J. A., Snyder, D. A., & Ragland, J. J. (2001). The value of leadership in implementing and maintaining a successful health promotion program in the naval surface force, U.S. Pacific Fleet. American Journal of Health Promotion, 15, 437–440. https://doi.org/10.4278/0890-1171-15.6.437

    Article  CAS  PubMed  Google Scholar 

  • Winstanley, E. L., Clark, A., Feinberg, J., & Wilder, C. M. (2016). Barriers to implementation of opioid overdose prevention programs in Ohio. Substance Abuse, 37, 42–46. https://doi.org/10.1080/08897077.2015.1132294

    Article  PubMed  Google Scholar 

Download references

Funding

This work was supported by the Congressionally Directed Medical Research Program (W81XWH0610165). The content is solely the responsibility of the authors and does not necessarily represent the official views of the Department of Defense, U.S. Air Force, or the National Institutes of Health.

Author information

Authors and Affiliations

  1. New York University, New York, NY, USA

    Kimberly A. Rhoades, Amy M. Smith Slep, Michael F. Lorber, Richard E. Heyman & J. Mark Eddy

  2. Ellsworth Air Force Base, Box Elder, USA

    David J. Linkh

Authors
  1. Kimberly A. Rhoades
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Amy M. Smith Slep
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Michael F. Lorber
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Richard E. Heyman
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. J. Mark Eddy
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. David J. Linkh
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kimberly A. Rhoades.

Ethics declarations

Ethical Approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. The study was approved by the Institutional Review Board at New York University and the United States Army Medical Research and Development Command Human Research Protection Office.

Informed Consent

Informed consent was obtained for all individual participants in the study.

Conflict of Interest

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rhoades, K.A., Slep, A.M.S., Lorber, M.F. et al. Prevention System Implementation and Reach: Attitudes and Environmental Predictors in a Randomized Controlled Trial of the NORTH STAR Prevention System. Prev Sci 23, 1426–1437 (2022). https://doi.org/10.1007/s11121-022-01352-w

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11121-022-01352-w

Keywords

Search

Navigation