Skip to main content

Fidelity at a Distance: Assessing Implementation Fidelity of the Early Risers Prevention Program in a Going-to-Scale Intervention Trial

Prevention Science volume 9, pages 215–229 (2008)Cite this article

Abstract

The present study examined the feasibility of an innovative technology designed to assess implementation fidelity of the Early Risers conduct problems prevention program across 27 geographically dispersed school sites. A multidimensional construct of fidelity was used to assess the quantity of services provided (exposure), the degree to which program strategies conformed to the manual (adherence), and how well implementers delivered the program (quality of delivery). The measurement technology featured a fidelity monitoring system that required (a) weekly reporting on a web-based documentation system to assess program exposure and adherence, and (b) five annually administered telephone interviews with a technical assistant to assess quality of program implementation. The results showed that the fidelity monitoring system was feasible, with all sites achieving 100% compliance in completion of their required on-line reporting and on average over 80% of the required teleconference interviews. User feedback indicated satisfaction with the web-based program. The system was successful in measuring multiple indices of fidelity. The strengths and limitations of measuring fidelity at a distance with web-based and teleconferencing technologies are discussed.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

43,34 €

Price includes VAT (Finland)
Tax calculation will be finalised during checkout.

Notes

  1. The Early Risers program has evolved over a 10-year period and has been continually refined through feedback from efficacy and effectiveness research trials. The goals of the program, conceptual theory of change, and delivery system have remained constant over that time. However, the labels used to identify the components have changed to more accurately reflect the specific target of the intervention (e.g., child skills, child school support, parent skills, and family support). In addition, within some components, curriculum changes have been made based on scientific evidence, feasibility, and acceptability.

  2. There has been confusion between the terms, exposure and dose (or dosage) in the literature of fidelity of program implementation. Whereas dose may indicate the amount of program content received by participants, we define exposure as the amount of program content delivered by program implementers to participants, regardless of whether a given participant actually attended the program component.

References

  • August, G. J., Bloomquist, M. L., Realmuto, G. M., & Hektner, J. M. (2007). The Early Risers “Skills for Success” Program: A targeted intervention for preventing conduct problems and substance abuse in aggressive elementary school children. In P. Tolan, J. Szapocznik, & S. Sambrano (Eds.), Preventing youth substance abuse (pp. 137–158). Washington, DC: American Psychological Association.

    Chapter  Google Scholar 

  • Blakeley, C. H., Mayer, J. P., Gottschalk, R. G., Schmitt, N., Davidson, W. S., Roitman, D. B., et al. (1987). The fidelity-adaptation debate: Implications for the implementation for public sector social programs. American Journal of Community Psychology, 15, 253–268, doi:10.1007/BF00922697.

    Article  Google Scholar 

  • Botvin, G. J., Baker, E., Dusenberry, L., Tortu, S., & Botvin, E. M. (1990). Preventing adolescent drug abuse through a multimodal cognitive-behavioral approach: Results of a 3-year study. Journal of Consulting and Clinical Psychology, 58, 437–446, doi:10.1037/0022-006X.58.4.437.

    PubMed  Article  CAS  Google Scholar 

  • Dane, A. V., & Schneider, B. H. (1998). Program integrity in primary and early secondary prevention: Are implementation efforts out of control? Clinical Psychology Review, 18, 23–45, doi:10.1016/S0272-7358(97)00043-3.

    PubMed  Article  CAS  Google Scholar 

  • Dumas, J. E., Lynch, A. M., Laughlin, J. E., Smith, E. P., & Prinz, R. J. (2001). Promoting intervention fidelity: Conceptual issues, methods, and preliminary results from the Early Alliance Prevention Trial. American Journal of Preventive Medicine, 20, 38–47, doi:10.1016/S0749-3797(00)00272-5.

    PubMed  Article  CAS  Google Scholar 

  • Durlak, J. A. (1997). Successful prevention programs for children and adolescents. New York: Plenum.

    Google Scholar 

  • Dusenbury, L., Branningan, R., Falco, M., & Hansen, W. B. (2003). A review of research on fidelity of implementation: Implications for drug abuse prevention in school settings. Health Education Research, 18, 237–256, doi:10.1093/her/18.2.237.

    PubMed  Article  Google Scholar 

  • Greenberg, M. T., & Kusche, C. A. (1998). Promoting Alternative Thinking Strategies, Book 10: Blueprint for violence prevention. University of Colorado: Institute of Behavioral Sciences.

    Google Scholar 

  • Gresham, F. M., MacMillian, D. L., Beebe-Frankenberger, M. E., & Bocian, K. M. (2000). Treatment integrity in learning disabilities intervention research: Do we really know how treatments are implemented? Learning Disabilities Research and Practice, 15, 198–205, doi:10.1207/SLDRP1504_4.

    Article  Google Scholar 

  • Hansen, W. B., Graham, J. W., Wolkenstein, B. H., & Rohrbach, L. A. (1991). Journal of Studies on Alcohol, 52, 568–579.

    PubMed  CAS  Google Scholar 

  • Hill, L. G., Maucione, K., & Hood, B. K. (2007). A focused approach to assessing program fidelity. Prevention Science, 8, 25–34, doi:10.1007/s11121-006-0051-4.

    PubMed  Article  Google Scholar 

  • Kam, C. M., Greenberg, M. T., & Walls, C. T. (2003). Examining the role of implementation quality in school-based prevention using the PATHS curriculum. Prevention Science, 4, 55–63, doi:10.1023/A:1021786811186.

    PubMed  Article  Google Scholar 

  • Kendall, P. C., & Beidas, R. S. (2007). Smoothing the trail for dissemination of evidence-based practices for youth: Flexibility within fidelity. Professional Psychology, Research and Practice, 38, 13–20, doi:10.1037/0735-7028.38.1.13.

    Article  Google Scholar 

  • Lillehoj, C. J., Griffin, K. W., & Spoth, R. (2004). Program provider and observer ratings of school-based preventive intervention implementation: Agreement and relation to youth outcomes. Health Education and Behavior, 31, 242–257, doi:10.1177/1090198103260514.

    Article  Google Scholar 

  • Moncher, F., & Prinz, R. (1991). Treatment fidelity in outcome studies. Clinical Psychology Review, 11, 247–266, doi:10.1016/0272-7358(91)90103-2.

    Article  Google Scholar 

  • Ogden, T., Forgatch, M. S., Askeland, E., Patterson, G. R., & Bullock, B. M. (2005). Implementation of parent management training at the national level: The Case of Norway. Journal of Social Work Practice, 19, 317–329, doi:10.1080/02650530500291518.

    Article  Google Scholar 

  • Pentz, M. A., Trebow, E. A., Hansen, W. B., MacKinnon, D. P., Dqyer, J. H., Johnson, C. A., et al. (1990). Effects of program implementation on adolescent drug use behavior: The Midwestern Prevention Project (MPP). Evaluation Review, 14, 264–289, doi:10.1177/0193841X9001400303.

    Article  Google Scholar 

  • Rohrbach, L. A., Graham, J. W., & Hansen, W. B. (1993). Diffusion of a school-based substance abuse prevention program: Predictors of program implementation. Preventive Medicine, 22, 237–260, doi:10.1006/pmed.1993.1020.

    PubMed  Article  CAS  Google Scholar 

  • Small, M., Domitrovich, C., Camplese, K., & Prabhu, V. (2006). Interactive system for program implementation and real-time evaluation: Using technology to improve implementation quality of EBIs. Paper presented at the annual convention of the Society for Prevention Research, San Antonio, TX, May.

  • Sobol, D., Rorbach, L., Dent, C., Gleason, L., Brannon, B., Johnson, C. A., et al. (1989). The integrity of smoking prevention curriculum delivery. Health Education Research, 4, 59–67, doi:10.1093/her/4.1.59.

    Article  Google Scholar 

  • Yeaton, W. H., & Sechrest, L. (1981). Critical dimensions in the choice and maintenance of successful treatments: Strength, integrity, and effectiveness. Journal of Consulting and Clinical Psychology, 49, 156–167, doi:10.1037/0022-006X.49.2.156.

    PubMed  Article  CAS  Google Scholar 

Download references

Author information

Affiliations

  1. Division of Child and Adolescent Psychiatry, University of Minnesota Medical School, F256/2B West, 2450 Riverside Avenue, Minneapolis, MN, 55454, USA

    Chih-Yuan S. Lee, Gerald J. August, George M. Realmuto, Jason L. Horowitz, Michael L. Bloomquist & Bonnie Klimes-Dougan

Authors
  1. Chih-Yuan S. Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Gerald J. August
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. George M. Realmuto
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jason L. Horowitz
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Michael L. Bloomquist
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Bonnie Klimes-Dougan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chih-Yuan S. Lee.

Additional information

This study was supported by a grant to Gerald J. August from the National Institute on Drug Abuse (DA016067). The authors would like to thank Ms. Sarah Coleman, the project manager, for her major contribution to this effort.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Lee, CY.S., August, G.J., Realmuto, G.M. et al. Fidelity at a Distance: Assessing Implementation Fidelity of the Early Risers Prevention Program in a Going-to-Scale Intervention Trial. Prev Sci 9, 215–229 (2008). https://doi.org/10.1007/s11121-008-0097-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11121-008-0097-6

Keyword

  • Prevention
  • Implementation fidelity
  • Exposure
  • Adherence
  • Quality of delivery