Skip to main content
SpringerLink
Log in

Measuring Fidelity in Research Studies: A Field Guide to Developing a Comprehensive Fidelity Measurement System

  • Published:
Child and Adolescent Social Work Journal Aims and scope Submit manuscript

Abstract

An accurate assessment of fidelity, combined with a high degree of fidelity to the intervention, is critical to the reliability, validity, replicability, and scale-up of the results of an intervention research study. However, extant measures of fidelity are infrequently applicable to the program or intervention being studied, and the literature lacks guidance on the specific process of developing a system to measure fidelity in a manualized intervention. This article describes a five-step process to define the scope, identify components, develop tools, monitor fidelity, and analyze outcomes to develop a comprehensive fidelity measurement system for an intervention. The process describes the components, measures and key decisions that form a comprehensive fidelity measurement system. In addition, the process is illustrated by a case study of the development of a fidelity measurement system for a research study testing Pathways Triple P, a behavioral parent-training program, with a population of child welfare-involved families. Pathways Triple P is a common, manualized intervention and the process described in this article can be generalized to other manualized interventions. The implications and requirements for accurately assessing and monitoring fidelity in research studies and practice are discussed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  • Aarons, G. A., Sommerfeld, D. H., Hecht, D. B., Silovsky, J. F., & Chaffin, M. J. (2009). The impact of evidence-based practice implementation and fidelity monitoring on staff turnover: Evidence for a protective effect. Journal of Consulting and Clinical Psychology, 77(2), 270.

    Article  PubMed  PubMed Central  Google Scholar 

  • Altman, D. G. (1991). Some common problems in medical research. Practical Statistics for Medical Research, 1, 396–403.

    Google Scholar 

  • Beidas, R. S., Benjamin, C. L., Puleo, C. M., Edmunds, J. M., & Kendall, P. C. (2010). Flexible applications of the coping cat program for anxious youth. Cognitive and Behavioral Practice, 17(2), 142–153.

    Article  PubMed  PubMed Central  Google Scholar 

  • Bellg, A. J., Borrelli, B., Resnick, B., Hecht, J., Minicucci, D. S., Ory, M., ... Czajkowski, S. (2004). Enhancing treatment fidelity in health behavior change studies: Best practices and recommendations from the NIH behavior change consortium. Health Psychology, 23(5), 443.

    Article  PubMed  Google Scholar 

  • Carroll, C., Patterson, M., Wood, S., Booth, A., Rick, J., & Balain, S. (2007). A conceptual framework for implementation fidelity. Implementation Science, 2(40), 1–9.

    Google Scholar 

  • Cross, W., & West, J. (2011). Examining implementer fidelity: Conceptualising and measuring adherence and competence. Journal of Children’s Services, 6(1), 18–33.

    Article  PubMed  PubMed Central  Google Scholar 

  • Dane, A. V., & Schneider, B. H. (1998). Program integrity in primary and early secondary prevention: Are implementation effects out of control? Clinical Psychology Review, 18(1), 23–45.

    Article  PubMed  Google Scholar 

  • Dobson, D., & Cook, T. J. (1980). Avoiding type III error in program evaluation: Results from a field experiment. Evaluation and Program Planning, 3(4), 269–276.

    Article  Google Scholar 

  • Dumas, J. E., Lynch, A. M., Laughlin, J. E., Phillips Smith, E., & Prinz, R. J. (2001). Promoting intervention fidelity: Conceptual issues, methods, and preliminary results from the EARLY ALLIANCE prevention trial. American Journal of Preventive Medicine, 20(1), 38–47.

    Article  PubMed  Google Scholar 

  • Dusenbury, L., Brannigan, R., Falco, M., & Hansen, W. B. (2003). A review of research on fidelity of implementation: Implications for drug abuse prevention in school settings. Health Education Research, 18(2), 237–256.

    Article  PubMed  Google Scholar 

  • Forgatch, M. S., Patterson, G. R., & DeGarmo, D. S. (2006). Evaluating fidelity: Predictive validity for a measure of competent adherence to the oregon model of parent management training. Behavior Therapy, 36(1), 3–13.

    Article  Google Scholar 

  • Gearing, R. E., El-Bassel, N., Ghesquiere, A., Baldwin, S., Gillies, J., & Ngeow, E. (2011). Major ingredients of fidelity: A review and scientific guide to improving quality of intervention research implementation. Clinical Psychology Review, 31(1), 79–88.

    Article  PubMed  Google Scholar 

  • Hallgren, K. A. (2012). Computing inter-rater reliability for observational data: An overview and tutorial. Tutorials in Quantitative Methods for Psychology, 8(1), 23.

    Article  PubMed  PubMed Central  Google Scholar 

  • Hamilton, J. D., Kendall, P. C., Gosch, E., Furr, J. M., & Sood, E. (2008). Flexibility within fidelity. Journal of the American Academy of Child & Adolescent Psychiatry, 47(9), 987–993.

    Article  Google Scholar 

  • Hodge, D. R., & Gillespie, D. F. (2007). Phrase completion scales: A better measurement approach than likert scales? Journal of Social Service Research, 33(4), 1–12.

    Article  Google Scholar 

  • Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics 33, 159–174.

    Article  PubMed  Google Scholar 

  • Madson, M. B., & Campbell, T. C. (2006). Measures of fidelity in motivational enhancement: A systematic review. Journal of Substance Abuse Treatment, 31(1), 67–73.

    Article  PubMed  Google Scholar 

  • Mazzucchelli, T. G., & Sanders, M. R. (2010). Facilitating practitioner flexibility within an empirically supported intervention: Lessons from a system of parenting support. Clinical Psychology: Science and Practice, 17(3), 238–252.

    Google Scholar 

  • Miller, W. R., & Rollnick, S. (2014). The effectiveness and ineffectiveness of complex behavioral interventions: Impact of treatment fidelity. Contemporary Clinical Trials, 37(2), 234–241.

    Article  PubMed  Google Scholar 

  • Moncher, F. J., & Prinz, R. J. (1991). Treatment fidelity in outcome studies. Clinical Psychology Review, 11(3), 247–266.

    Article  Google Scholar 

  • Proctor, E., Silmere, H., Raghavan, R., Hovmand, P., Aarons, G., Bunger, A., .... Hensley, M. (2011). Outcomes for implementation research: Conceptual distinctions, measurement challenges, and research agenda. Administration and. Policy in Mental Health and Mental Health Services Research, 38(2), 65–76.

    Article  PubMed  Google Scholar 

  • Sanders, M. R., Cann, W., & Markie-Dadds, C. (2003a). The triple P-positive parenting programme: a universal population-level approach to the prevention of child abuse. Child Abuse Review, 12(3), 155–171.

    Article  Google Scholar 

  • Sanders, M.R., Markie-Dadds, C., & Turner, K. (2001). Practitioner’s manual for standard triple P. Brisbane: Triple P International Pty. Ltd.

    Google Scholar 

  • Sanders, M.R., Markie-Dadds, C., & Turner, K. (2003b). Every parent's family workbook. Brisbane: Triple P International Pty. Ltd.

    Google Scholar 

  • Sanders, M. R., & Pidgeon, A. M. (2005). Pracitioner’s manual for pathways triple P. Brisbane: Triple P International Pty. Ltd.

    Google Scholar 

  • Sanders, M. R., Pidgeon, A. M., Gravestock, F., Connors, M. D., Brown, S., & Young, R. W. (2004). Does parental attributional retraining and anger management enhance the effects of the triple P-positive parenting program with parents at risk of child maltreatment? Behavior Therapy, 35(3), 513–535.

    Article  Google Scholar 

  • Sanders, M. R., Turner, K. M. T., & Markie-Dadds, C. (2002). The development and dissemination of the triple P-positive parenting program: A multilevel, evidence-based system of parenting and family support. Prevention Science, 3(3), 173–189. doi:10.1023/A:1019942516231.

    Article  PubMed  Google Scholar 

  • Schoenwald, S. K. (2011). It’s a bird, it’s a plane, it’s… fidelity measurement in the real world. Clinical Psychology: Science and Practice, 18(2), 142–147.

    Google Scholar 

  • Schoenwald, S. K., & Garland, A. F. (2013). A review of treatment adherence measurement methods. Psychological assessment, 25(1), 146.

    Article  PubMed  Google Scholar 

  • Schoenwald, S. K., Garland, A. F., Chapman, J. E., Frazier, S. L., Sheidow, A. J., & Southam-Gerow, M. A. (2011). Toward the effective and efficient measurement of implementation fidelity. Administration and Policy in Mental Health and Mental Health Services Research, 38(1), 32–43.

    Article  PubMed  PubMed Central  Google Scholar 

  • Seay, K. D., Byers, K., Feely, M., Lanier, P. Maguire-Jack, K., & McGill, T. (2015). Scaling up: Replicating promising interventions with fidelity. In D. Daro, A. Cohn Donnelly, L. Huang, & B. Powell (Eds.), Advances in child abuse prevention knowledge: The perspective of new leadership (pp. 179–201). New York: Springer.

    Chapter  Google Scholar 

  • Shrout, P. E., & Fleiss, J. L. (1979). Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin, 86(2), 420.

    Article  PubMed  Google Scholar 

  • Turner, K. M. T., Markie-Dadds, C., & Sanders, M. R. (2002). Facilitator’s manual for group triple P. Brisbane: Families International Publishing.

    Google Scholar 

Download references

Acknowledgements

This research was funded by a grant from the The Eunice Kennedy Shriver National Institute of Child Health and Human Development (1RO1HD061454) to Washington University in St. Louis. This work was also supported by the National Institute on Drug Abuse (F31DA034442, K. Seay, PI; 5T32DA015035), a Doris Duke Fellowship, and a Fahs-Beck Doctoral Dissertation Grant. The work was supported by The Washington University Institute of Clinical and Translational Sciences grant UL1 TR000448 from the National Center for Advancing Translational Sciences. TL1 Trainee, subaward TL1 TR000449. Points of view, opinions and content are solely the responsibility of the authors and do not necessarily represent the official views of the NIH, the NICHD or NIDA.

Author information

Authors and Affiliations

  1. University of Connecticut School of Social Work, Hartford, CT, USA

    Megan Feely

  2. College of Social Work, University of South Carolina, Columbia, SC, USA

    Kristen D. Seay

  3. University of North Carolina School of Social Work, Chapel Hill, NC, USA

    Paul Lanier

  4. Brown School at Washington University in St. Louis, St. Louis, MO, USA

    Wendy Auslander & Patricia L. Kohl

Authors
  1. Megan Feely
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kristen D. Seay
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Paul Lanier
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Wendy Auslander
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Patricia L. Kohl
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Megan Feely.

Ethics declarations

Ethical Approval

All procedures performed in this study involving human participants were in accordance with the ethical standards of the institutional research committee, approved by the institutional review board and are in compliance with the 1964 Helsinki declaration and its later amendments.

Informed Consent

Informed consent was obtained from all individual participants included in the study.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Feely, M., Seay, K.D., Lanier, P. et al. Measuring Fidelity in Research Studies: A Field Guide to Developing a Comprehensive Fidelity Measurement System. Child Adolesc Soc Work J 35, 139–152 (2018). https://doi.org/10.1007/s10560-017-0512-6

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10560-017-0512-6

Keywords

Search

Navigation