Abstract
Ensuring high levels of procedural fidelity during behavior-analytic interventions is a crucial component of providing effective behavior-analytic services. However, few resources are available to help guide practitioners through measuring procedural fidelity. In fact, most published behavior-analytic research on procedural fidelity analyzes a single treatment procedure, which might not completely reflect the process of monitoring and addressing the procedural fidelity of a robust treatment package that might be necessary in clinical settings. The purpose of this article is to guide behavior analysts through the process of creating and using procedural fidelity measurement systems, with a focus on direct observation of implementation as a means of fidelity data collection. This process consists of six steps: (1) task analyze treatment procedures into measurable units; (2) assign measures to each treatment component; (3) plan the direct observation; (4) collect procedural fidelity data; (5) analyze and interpret procedural fidelity data; and (6) take action to improve procedural fidelity. Each step is described and discussed in the article.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data Sharing
Not applicable
Notes
High PF refers to the implementation of a treatment closely matching the expectations of the treatment. Low PF refers to the implementation of a treatment diverging from the expectations of the treatment.
Exceeding an intended count or time would only be considered an error if a cap is set for that treatment component. For example, Example 2 specifies that Aiden should be given a 3–4-min enriched break. Because a range is specified, it is clear that anything below or above that amount of time would be considered an error. Example 2 also specifies that Aiden should be encouraged/praised at least five times through the interval. Thus, instances of encouragement/praise that exceed the intended amount would not violate the treatment protocol and, therefore, would not be considered a PF error.
References
Arkoosh, M. K., Derby, K. M., Wacker, D. P., Berg, W., McLaughlin, T. F., & Barretto, A. (2007). A descriptive evaluation of long-term treatment integrity. Behavior Modification, 31(6), 880–895. https://doi.org/10.1177/0145445507302254
Behavior Analyst Certification Board. (2020). Ethics code for behavior analysts. https://bacb.com/wp-content/ethics-code-for-behavior-analysts/
Behavior Analyst Certification Board. (2022a). Registered behavior technician handbook. https://www.bacb.com/wp-content/uploads/2022/01/RBTHandbook_230823-a.pdf
Behavior Analyst Certification Board. (2022b). Board certified assistant behavior analyst handbook. https://www.bacb.com/wp-content/uploads/2022/01/BCaBAHandbook_230623-a.pdf
Behavior Analyst Certification Board. (2022c). Board certified behavior analyst handbook. https://www.bacb.com/wp-content/uploads/2022/01/BCBAHandbook_230804-a.pdf
Brand, D., Henley, A. J., DiGennaro Reed, F. D., Gray, E., & Crabbs, B. (2019). A review of published studies involving parametric manipulations of treatment integrity. Journal of Behavioral Education, 28(1), 1–26. https://doi.org/10.1007/s10864-018-09311-8
Carroll, R. A., Kodak, T., & Fisher, W. W. (2013). An evaluation of programmed treatment-integrity errors during discrete-trial instruction. Journal of Applied Behavior Analysis, 46(2), 379–394. https://doi.org/10.1002/jaba.49
Clayton, M., & Headley, A. (2019). The use of behavioral skills training to improve staff performance of discrete trial training. Behavioral Interventions, 34(1), 136–143. https://doi.org/10.1002/bin.1656
Collier-Meek, M. A., Fallon, L. M., & Gould, K. (2018). How are treatment integrity data assessed? Reviewing the performance feedback literature. School Psychology Quarterly, 33(4), 517. https://doi.org/10.1037/spq0000239
Collier-Meek, M. A., Sanetti, L. M., Fallon, L., & Chafouleas, S. (2020). Exploring the influences of assessment method, intervention steps, intervention sessions, and observation timing on treatment fidelity estimates. Assessment for Effective Intervention, 46(1), 3–13. https://doi.org/10.1177/1534508419857228
Cook, J. E., Subramaniam, S., Brunson, L. Y., Larson, N. A., Poe, S. G., & St. Peter, C. C. (2015). Global measures of treatment integrity may mask important errors in discrete-trial training. Behavior Analysis in Practice, 8(1), 37–47. https://doi.org/10.1007/s40617-014-0039-7
Cooper, J. O., Heron, T. E., & Heward, W. L. (2019). Applied behavior analysis (3rd ed.). Pearson Education.
Falakfarsa, G., Brand, D., Jones, L., Godinez, E. S., Richardson, D. C., Hanson, R. J., Velazquez, S. D., & Wills, C. (2021). Treatment integrity reporting in Behavior Analysis in Practice 2008–2019. Behavior Analysis in Practice, 15(2), 443–453. https://doi.org/10.1007/s40617-021-00573-9
Fallon, L. M., Cathcart, S. C., & Sanetti, L. M. H. (2020). Assessing parents’ treatment fidelity: A survey of practitioners in home settings. Focus on Autism & Other Developmental Disabilities, 35(1), 15–25. https://doi.org/10.1177/1088357619866192
Fryling, M. J., Wallace, M. D., & Yassine, J. N. (2012). Impact of treatment integrity on intervention effectiveness. Journal of Applied Behavior Analysis, 45(2), 449–453. https://doi.org/10.1901/jaba.2012.45-449
Gresham, F. M. (1989). Assessment of treatment integrity in school consultation and prereferral intervention. School Psychology Review, 18(1), 37–50. https://doi.org/10.1080/02796015.1989.12085399
Gresham, F. M., MacMillan, D. L., Beebe-Frankenberger, M. E., & Bocian, K. M. (2000). Treatment integrity in learning disabilities intervention research: Do we really know how treatments are implemented? Learning Disabilities Research & Practice, 15(4), 198–205. https://doi.org/10.1207/SLDRP1504_4
Hagermoser Sanetti, L. M., & Fallon, L. M. (2011). Treatment integrity assessment: How estimates of adherence, quality, and exposure influence interpretation of implementation. Journal of Educational & Psychological Consultation, 21(3), 209–232. https://doi.org/10.1080/10474412.2011.595163
Hagermoser Sanetti, L. M., & Kratochwill, T. R. (2008). Treatment integrity in behavioral consultation: Measurement, promotion, and outcomes. International Journal of Behavioral Consultation & Therapy, 4(1), 95–114. https://doi.org/10.1037/h0100835
Han, J. B., Bergmann, S., Brand, D., Wallace, M. D., St. Peter, C. C., Feng, J., & Long, B. P. (2022). Trends in reporting procedural integrity: A comparison. Behavior Analysis in Practice, 16, 388–398. https://doi.org/10.1007/s40617-022-00741-5
Ingham, P., & Greer, R. D. (1992). Changes in student and teacher responses in observed and generalized settings as a function of supervisor observations. Journal of Applied Behavior Analysis, 25(1), 153–164. https://doi.org/10.1901/jaba.1992.25-153
Johnston, J. M., Pennypacker, H. S., & Green, G. (2020). Strategies and tactics of behavioral research and practice (4th ed.). Routledge/Taylor & Francis Group.
Kazdin, A. E. (1982). Observer effects: Reactivity of direct observation. New Directions for Methodology of Social & Behavioral Science, 14, 5–19. https://psycnet.apa.org/record/1983-22374-001.
Kodak, T., Bergmann, S., & Waite, M. (2022). Strengthening the procedural fidelity research-to-practice loop in animal behavior. Journal of the Experimental Analysis of Behavior, 118(2), 215–236. https://doi.org/10.1002/jeab.780
LeBlanc, L. A., Taylor, B. A., & Marchese, N. V. (2020). The training experiences of behavior analysts: Compassionate care and therapeutic relationships with caregivers. Behavior Analysis in Practice, 13(2), 387–393. https://doi.org/10.1007/s40617-019-00368-z
Morris, C., & Hollins, N. A. (2022). On the uncanny similarities between police brutality and client mistreatment. Behavior Analysis in Practice, 15(4), 1106–1111. https://doi.org/10.1007/s40617-021-00576-6
Noell, G. H. (2007). Research examining the relationships among consultation process, treatment integrity, and outcomes. In W. P. Erchul & S. M. Sheridan (Eds.), Handbook of research in school consultation: Empirical foundations for the field. Lawrence Erlbaum Associates.
Noell, G. H., Duhon, G. J., Gatti, S. L., & Connell, J. E. (2002). Consultation, follow-up, and implementation of behavior management interventions in general education. School Psychology Review, 31(2), 217–234. https://doi.org/10.1080/02796015.2002.12086152
Noell, G. H., Witt, J. C., Slider, N. J., Connell, J. E., Gatti, S. L., Williams, K. L., Resetar, J. L., & Duhon, G. J. (2005). Treatment implementation following behavioral consultation in schools: A comparison of three follow-up strategies. School Psychology Review, 34(1), 87–106. https://doi.org/10.1080/02796015.2005.12086277
Novack, M. N., & Dixon, D. R. (2019). Predictors of burnout, job satisfaction, and turnover in behavior technicians working with individuals with autism spectrum disorder. Review Journal of Autism & Developmental Disorders, 6(4), 413–421. https://doi.org/10.1007/s40489-019-00171-0
Pipkin, C. S. P., Vollmer, T. R., & Sloman, K. N. (2010). Effects of treatment integrity failures during differential reinforcement of alternative behavior: A translational model. Journal of Applied Behavior Analysis, 43(1), 47–70. https://doi.org/10.1901/jaba.2010.43-47
Reid, D. H., Rosswurm, M., & Rotholz, D. A. (2018). No less worthy: Recommendations for behavior analysts treating adults with intellectual and developmental disabilities with dignity. Behavior Analysis in Practice, 11(1), 71–79. https://doi.org/10.1007/s40617-017-0203-y
Reid, D. H., Parsons, M. B., & Green, C. W. (2021). The supervisor’s guidebook: Evidence-based strategies for promoting work quality and enjoyment among human service staff. Thomas.
Rohrer, J. L., Marshall, K. B., Suzio, C., & Weiss, M. J. (2021). Soft skills: The case for compassionate approaches or how behavior analysis keeps finding its heart. Behavior Analysis in Practice, 14(4), 1135–1143. https://doi.org/10.1007/s40617-021-00563-x
Sanetti, L. M. H., & Collier-Meek, M. A. (2014). Increasing the rigor of procedural fidelity assessment: An empirical comparison of direct observation and permanent product review methods. Journal of Behavioral Education, 23, 60–88. https://doi.org/10.1007/s10864-013-9179-z
Sanetti, L. M. H., & Collier-Meek, M. A. (2019). Increasing implementation science literacy to address the research-to-practice gap in school psychology. Journal of School Psychology, 76, 33–47. https://doi.org/10.1016/j.jsp.2019.07.008
Shahan, T. A., & Greer, B. D. (2021). Destructive behavior increases as a function of reductions in alternative reinforcement during schedule thinning: A retrospective quantitative analysis. Journal of the Experimental Analysis of Behavior, 116(2), 243–248. https://doi.org/10.1002/jeab.708
St. Peter, C. C., Brand, D., Jones, S. H., Wolgemuth, H. R., & Lipien, L. (2023). On a persisting curious double standard in behavior analysis: Behavioral scholars’ perspectives on procedural fidelity. Journal of Applied Behavior Analysis, 56(2), 336–351. https://doi.org/10.1002/jaba.974
St. Peter Pipkin, C., Vollmer, T. R., & Sloman, K. N. (2010). Effects of treatment integrity failures during differential reinforcement of alternative behavior: A translational model. Journal of Applied Behavior Analysis, 43(1), 47–70. https://doi.org/10.1901/jaba.2010.43-47
Suhrheinrich, J., Dickson, K. S., Chan, N., Chan, J. C., Wang, T., & Stahmer, A. C. (2020). Fidelity assessment in community programs: an approach to validating simplified methodology. Behavior Analysis in Practice, 13, 29–39. https://doi.org/10.1007/s40617-019-00337-6
Taylor, B. A., LeBlanc, L. A., & Nosik, M. R. (2019). Compassionate care in behavior analytic treatment: Can outcomes be enhanced by attending to relationships with caregivers? Behavior Analysis in Practice, 12(3), 654–666. https://doi.org/10.1007/s40617-018-00289-3
Vollmer, T. R., Sloman, K. N., & St. Peter Pipkin, C. (2008). Practical implications of data reliability and treatment integrity monitoring. Behavior Analysis in Practice, 1, 4–11. https://doi.org/10.1007/BF03391722
Ward-Horner, J., & Sturmey, P. (2012). Component analysis of behavior skills training in functional analysis. Behavioral Interventions, 27(2), 75–92. https://doi.org/10.1002/bin.1339
Weyman, J. R., & Sy, J. R. (2018). Effects of neutral and enthusiastic praise on the rate of discrimination acquisition. Journal of Applied Behavior Analysis, 51(2), 335–344. https://doi.org/10.1002/jaba.440
Wilder, D. A., Atwell, J., & Wine, B. (2006). The effects of varying levels of treatment integrity on child compliance during treatment with a three-step prompting procedure. Journal of Applied Behavior Analysis, 39(3), 369–373. https://doi.org/10.1901/jaba.2006.144-05
Williams, D. E., & Vollmer, T. E. (2015). Essential components of written behavior treatment plans. Research in Developmental Disabilities, 36, 323–327. https://doi.org/10.1016/j.ridd.2014.10.003
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflicts of Interest
We have no known conflict of interest to disclose.
Research Involving Human Participants and/or Animals
Not applicable.
Informed Consent
Not applicable.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Morris, C., Jones, S.H. & Oliveira, J.P. A Practitioner’s Guide to Measuring Procedural Fidelity. Behav Analysis Practice (2024). https://doi.org/10.1007/s40617-024-00910-8
Accepted:
Published:
DOI: https://doi.org/10.1007/s40617-024-00910-8