Abstract
Functional analysis (FA) is an integral component of behavioral assessment and treatment given that clinicians design behavioral treatments based on FA results. Unfortunately, the interrater reliability of FA data interpretation by visual analysis can be inconsistent, potentially leading to ineffective treatment implementation. Hall et al. (2020) recently developed automated nonparametric statistical analysis (ANSA) to facilitate the interpretation of FA data and Kranak et al. (2021) subsequently extended and validated ANSA by applying it to unpublished clinical data. The results of both Hall et al. and Kranak et al. support ANSA as an emerging statistical supplement for interpreting FA data. In the present article, we show how ANSA can be applied to interpret FA data collected in clinical settings in multielement and pairwise designs. We provide a detailed overview of the calculations involved, how to use ANSA in practice, and recommendations for its implementation. A free web-based application is available at https://ansa.shinyapps.io/ansa/.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
For test conditions that include a contingency (e.g., attention, tangible, demand), however, a test condition is still differentiated so long as the last observation in the series is above response efficiency (e.g., at least two responses per min if the contingency duration was 30 s).
References
Baer, D. M. (1977). Perhaps it would be better not to know everything. Journal of Applied Behavior Analysis, 10(1), 167–172. https://doi.org/10.1901/jaba.1977.10-167.
Baer, D. M., Wolf, M. M., & Risley, T. R. (1968). Some current dimensions of applied behavior analysis. Journal of Applied Behavior Analysis, 1(1), 91–97. https://doi.org/10.1901/jaba.1968.1-91.
Barnard-Brak, L., Morales-Aleman, M., Tomeny, K., & McWilliam, R. A. (2021). Rural and racial/ethnic differences in children receiving early intervention services. Family & Community Health, 44(1), 52–58. https://doi.org/10.1097/FCH.0000000000000285.
Bartlett, S. M., Rapp, J. T., & Henrickson, M. L. (2011). Detecting false positives in multielement designs: Implications for brief assessments. Behavior Modification, 35(6), 531–552. https://doi.org/10.1177/0145445511415396.
Behavior Analyst Certification Board. (2021). BACB certificant data. https://www.bacb.com/bacb-certificant-data/
Branch, M. N., & Pennypacker, H. S. (2003). Generality and generalization of research findings. In G. J. Madden, W. V. Dube, T. D. Hackenberg, G. P. Hanley, & K. A. Lattal (Eds.), APA handbook of behavior analysis, Vol. 1. Methods and principles (pp. 171–175). American Psychological Association. https://doi.org/10.1037/13937-007.
Brogan, K. M., Rapp, J. T., & Sturdivant, B. R. (2019). Transition states in single case experimental designs. Behavior Modification. Advance online publication. https://doi.org/10.1177/0145445519839213.
Coon, J. C., & Rapp, J. T. (2018). Application of multiple baseline designs in behavior analytic research: Evidence for the influence of guidelines. Behavioral Interventions, 33(2), 160–172. https://doi.org/10.1002/bin.1510.
Cooper, J. O., Heron, T. E., & Heward, W. L. (2020). Applied behavior analysis (3rd ed.). Pearson.
Cox, A., & Friedel, J. E. (2021). Toward an automation of functional analysis interpretation: A proof of concept. Behavior Modification. Advance online publication. https://doi.org/10.1177/0145445520969188.
Cox, A. D., & Virues-Ortega, J. (2016). Interactions between behavior function and psychotropic medication. Journal of Applied Behavior Analysis, 49(1), 85–104. https://doi.org/10.1002/jaba.247.
Davis, B. J., Kahng, S., Schmidt, J., Bowman, L. G., & Boelter, E. W. (2012). Alterations to functional analysis methodology to clarify the functions of low rate, high intensity problem behavior. Behavior Analysis in Practice, 5(1), 27–39. https://doi.org/10.1007/BF03391815.
Derby, K. M., Wacker, D. P., Sasso, G., Steege, M., Northup, J., Cigrand, K., & Asmus, J. (1992). Brief functional assessment techniques to evaluate aberrant behavior in an outpatient setting: A summary of 79 cases. Journal of Applied Behavior Analysis, 25(3), 713–721. https://doi.org/10.1901/jaba.1992.25-713.
Devine, S. L., Rapp, J. T., Testa, J. R., Henrickson, M. L., & Schnerch, G. (2011). Detecting changes in simulated events using partial-interval recording and momentary time sampling III: Evaluating sensitivity as a function of session length. Behavioral Interventions, 26(2), 103–124. https://doi.org/10.1002/bin.328.
Diller, J. W., Barry, R. J., & Gelino, B. W. (2016). Visual analysis of data in a multielement design. Journal of Applied Behavior Analysis, 49(4), 980–985. https://doi.org/10.1002/jaba.325.
Edgemon, A. K., Rapp, J. T., Brogan, K. M., Richling, S. M., Hamrick, S. A., Peters, R. J., & O’Rourke, S. A. (2020). Behavioral skills training to increase interview skills of adolescent males in a juvenile residential treatment facility. Journal of Applied Behavior Analysis, 53(4), 2303–2318. https://doi.org/10.1002/jaba.707.
Enloe, K., & Rapp, J. T. (2014). Effects of noncontingent social interaction on immediate and subsequent engagement in vocal and motor stereotypy in children with autism. Behavior Modification, 38(3), 374–391. https://doi.org/10.1177/0145445513514081.
Falligant, J. M., Kranak, M. P., McNulty, M. K., Schmidt, J. D., Hausman, N. L., & Rooker, G. W. (2021). Prevalence of renewal of problem behavior: Replication and extension to an inpatient setting. Journal of Applied Behavior Analysis, 54(1), 367–373. https://doi.org/10.1002/jaba.740.
Falligant, J. M., Kranak, M. P., Schmidt, J. D., & Rooker, G. W. (2020). Correspondence between fail-safe k and dual-criteria methods: Analysis of data series stability. Perspectives on Behavior Science, 43(2), 303–319. https://doi.org/10.1007/s40614-020-00255-x.
Falligant, J. M., & Vetter, J. A. (2020). Quantifying false positives in simulated events using partial interval recording and momentary time sampling with dual-criteria methods. Behavioral Interventions, 35(2), 281–294. https://doi.org/10.1002/bin.1707.
Fisch, G. S. (2001). Evaluating data from behavioral analysis: Visual inspection or statistical models? Behavioral Processes, 54, 137–154.
Fisher, W. W., Kelley, M. E., & Lomas, J. E. (2003). Visual aids and structured criteria for improving visual inspection and interpretation of single-case designs. Journal of Applied Behavior Analysis, 36(3), 387–406. https://doi.org/10.1901/jaba.2003.36-387.
Fisher, W. W., Kuhn, D. E., & Thompson, R. H. (1998). Establishing discriminative control of responding using functional and alternative reinforcers during functional communication training. Journal of Applied Behavior Analysis, 31(4), 543–560. https://doi.org/10.1901/jaba.1998.31-543.
Fisher, W. W., & Lerman, D. C. (2014). It has been said that, “There are three degrees of falsehoods: Lies, damn lies, and statistics.” Journal of School Psychology, 52(2), 243–248. https://doi.org/10.1016/j.jsp.2014.01.001
Gardner, A. W., Spencer, T. D., Boelter, E. W., DuBard, M., & Jennett, H. K. (2012). A systematic review of brief functional analysis methodology with typically developing children. Education & Treatment of Children, 35(2), 313–332. https://doi.org/10.1353/etc.2012.0014.
Gerow, S., Rispoli, M., Gregori, E., & Sanchez, L. (2019). Parent-implemented trial-based functional analysis for young children with ASD. Focus on Autism & Other Developmental Disabilities, 34(1), 29–40. https://doi.org/10.1177/1088357618755695.
Greer, B. D., Fisher, W. W., Saini, V., Owen, T. M., & Jones, J. K. (2016). Functional communication training during reinforcement schedule thinning: An analysis of 25 applications. Journal of Applied Behavior Analysis, 49(1), 105–121. https://doi.org/10.1002/jaba.265.
Hagopian, L. P. (2020). The consecutive controlled case series: Design, data-analytics, and reporting methods supporting the study of generality. Journal of Applied Behavior Analysis, 53(2), 596–619. https://doi.org/10.1002/jaba.691.
Hagopian, L. P., Fisher, W. W., Thompson, R. H., Owen-DeSchryver, J., Iwata, B. A., & Wacker, D. P. (1997). Toward the development of structured criteria for interpretation of functional analysis data. Journal of Applied Behavior Analysis, 30(2), 313–325. https://doi.org/10.1901/jaba.1997.30-313.
Hagopian, L. P., Rooker, G. W., Jessel, J., & DeLeon, I. G. (2013). Initial functional analysis outcomes and modifications in pursuit of differentiation: A summary of 176 inpatient cases. Journal of Applied Behavior Analysis, 46(1), 88–100. https://doi.org/10.1002/jaba.25.
Hains, A. H., & Baer, D. M. (1989). Interaction effects in multielement designs: Inevitable, desirable, and ignorable. Journal of Applied Behavior Analysis, 22(1), 57–69. https://doi.org/10.1901/jaba.1989.22-57.
Hall, S. S., Barnett, R. P., & Hustyi, K. M. (2018). Examining the influence of social-environmental variables on self-injurious behavior in adolescent boys with fragile X syndrome. Journal of Intellectual Disability Research, 62(12), 1072–1085. https://doi.org/10.1111/jir.12489.
Hall, S. S., Pollard, J. S., Monlux, K. D., & Baker, J. D. (2020). Interpreting functional analysis outcomes using automated nonparametric statistical analysis. Journal of Applied Behavior Analysis, 53(2), 1177–1191. https://doi.org/10.1002/jaba.689.
Hanley, G. P. (2012). Functional assessment of problem behavior: Dispelling myths, overcoming implementation obstacles, and developing new lore. Behavior Analysis in Practice, 5(1), 54–72. https://doi.org/10.1007/BF03391818.
Hanley, G. P., Iwata, B. A., & McCord, B. E. (2003). Functional analysis of problem behavior: A review. Journal of Applied Behavior Analysis, 36(2), 147–185. https://doi.org/10.1901/jaba.2003.36-147.
Harrington, M., & Velicer, W. F. (2015). Comparing visual and statistical analysis in single-case studies using published studies. Multivariate Behavioral Research, 50(2), 162–183. https://doi.org/10.1080/00273171.2014.973989.
Holehan, K. M., Dozier, C. L., Diaz de Villegas, S. C., Jess, R. L., Goddard, K. S., & Foley, E. A. (2020). A comparison of isolated and synthesized contingencies in functional analyses. Journal of Applied Behavior Analysis, 53(3), 1559–1578. https://doi.org/10.1002/jaba.700.
Horner, R. H., Carr, E. G., Halle, J., McGee, G., Odom, S., & Wolery, M. (2005). The use of single-subject research to identify evidence-based practice in special education. Exceptional Children, 71(2), 165–179. https://doi.org/10.1177/001440290507100203.
Iwata, B. A., Dorsey, M. F., Slifer, K. J., Bauman, K. E., & Richman, G. S. (1994a). Toward a functional analysis of self-injury. Journal of Applied Behavior Analysis, 27(2), 215–240. https://doi.org/10.1901/jaba.1994.27-215.
Iwata, B. A., & Dozier, C. L. (2008). Clinical application of functional analysis methodology. Behavior Analysis in Practice, 1(1), 3–9. https://doi.org/10.1007/BF03391714.
Iwata, B. A., Pace, G. M., Cowdery, G. E., & Miltenberger, R. G. (1994b). What makes extinction work: An analysis of procedural form and function. Journal of Applied Behavior Analysis, 27(1), 131–144. https://doi.org/10.1901/jaba.1994.27-131.
Johnston, J. M., & Pennypacker, H. S. (2009). Strategies and tactics of behavioral research (3rd ed.). Routledge/Taylor & Francis Group.
Kahng, S., Chung, K., Gutshall, K., Pitts, S. C., Kao, J., & Girolami, K. (2010). Consistent visual analyses of intrasubject data. Journal of Applied Behavior Analysis, 43 (1), 35–45. https://doi.org/10.1901/jaba/2010.43-35.
Kranak, M. P., Falligant, J. M., & Hausman, N. L. (2021). Application of automated nonparametric statistical analysis in clinical contexts. Journal of Applied Behavior Analysis, 54(2), 824–833. https://doi.org/10.1002/jaba.789.
Kyonka, E. G., Mitchell, S. H., & Bizo, L. A. (2019). Beyond inference by eye: Statistical and graphing practices in JEAB, 1992-2017. Journal of the Experimental Analysis of Behavior, 111(2), 155–165. https://doi.org/10.1002/jeab.509.
Lanovaz, M. J., Cardinal, P., & Francis, M. (2019). Using a visual structured criterion for the analysis of alternating-treatment designs. Behavior Modification, 43(1), 115–131. https://doi.org/10.1177/0145445517739278.
Lanovaz, M. J., Giannakakos, A. R., & Destras, O. (2020). Machine learning to analyze single-case data: A proof of concept. Perspectives on Behavior Science, 43(1), 21–38. https://doi.org/10.1007/s40614-020-00244-0.
Lanovaz, M. J., Huxley, S. C., & Dufour, M. M. (2017). Using the dual-criteria methods to supplement visual inspection: An analysis of nonsimulated data. Journal of Applied Behavior Analysis, 50(3), 662–667. https://doi.org/10.1002/jaba.394.
Lanovaz, M. J., & Rapp, J. T. (2016). Using single-case experiments to support evidence-based decisions: How much is enough? Behavior Modification, 40(3), 377–395. https://doi.org/10.1177/0145445515613584.
Lanovaz, M. J., Rapp, J. T., & Maciw, I. (2014). Assessment and treatment of stereotypy in an individual with Cornelia de Lange syndrome and deafblindness. Journal of Intellectual & Developmental Disability, 39(4), 375–380. https://doi.org/10.3109/13668250.2014.953464.
LeBlanc, L. A., & Luiselli, J. K. (2016). Refining supervisory practices in the field of behavior analysis: Introduction to the special section on supervision. Behavior Analysis in Practice, 9(4), 271–273. https://doi.org/10.1007/s40617-016-0156-6.
Ledford, J. R., & Gast, D. L. (Eds.). (2018). Single case research methodology: Applications in special education and behavioral sciences. Routledge.
Luna, O., & Rapp, J. T. (2020). Increasing praise delivery within dorms of a juvenile justice facility. Behavior Modification. Advance online publication. https://doi.org/10.1177/0145445520982976.
Monlux, K. D., Pollard, J. S., Bujanda Rodriguez, A. Y., & Hall, S. S. (2019). Telehealth delivery of function-based behavioral treatment for problem behaviors exhibited by boys with fragile X syndrome. Journal of Autism & Developmental Disorders, 29, 2461–2475. https://doi.org/10.1007/s10803-019-03963-9.
Moore, J. W., Edwards, R. P., Sterling-Turner, H. E., Riley, J., DuBard, M., & McGeorge, A. (2002). Teacher acquisition of functional analysis methodology. Journal of Applied Behavior Analysis, 35(1), 73–77. https://doi.org/10.1901/jaba.2002.35-73.
Mueller, M. M., Nkosi, A., & Hine, J. F. (2011). Functional analysis in public schools: A summary of 90 functional analyses. Journal of Applied Behavior Analysis, 44(4), 807–818. https://doi.org/10.1901/jaba.2011.44-807.
Muething, C., Call, N., Pavlov, A., Ringdahl, J., Gillespie, S., Clark, S., & Mevers, J. L. (2020). Prevalence of renewal of problem behavior during context changes. Journal of Applied Behavior Analysis, 53(3), 1485–1493. https://doi.org/10.1002/jaba.672.
Neil, N., & Jones, E. A. (2012). Communication intervention for individuals with Down syndrome: Systematic review and meta-analysis. Developmental Neurorehabilitation, 21(1), 1–12. https://doi.org/10.1080/17518423.2016.1212947.
Ninci, J. (2021). Single-case data analysis: A practitioner guide for accurate and reliable decisions. Behavior Modification. Advance online publication. https://doi.org/10.1177/0145445519867054.
Ninci, J., Vannest, K. J., Willson, V., & Zhang, N. (2015). Interrater agreement between visual analysts of single-case data: A meta-analysis. Behavior Modification, 39(4), 510–541. https://doi.org/10.1177/0145445515581327.
Novotny, M. A., Sharp, K. J., Rapp, J. T., Jelinski, J. D., Lood, E. A., & Steffes, A. K. (2014). False positives with visual analysis for nonconcurrent multiple baseline designs and ABAB designs: Preliminary findings. Research in Autism Spectrum Disorders, 8(8), 933–943. https://doi.org/10.1016/j.rasd.2014.04.009.
Querim, A. C., Iwata, B. A., Roscoe, E. M., Schlichenmeyer, K. J., Ortega, J. V., & Hurl, K. E. (2013). Functional analysis screening for problem behavior maintained by automatic reinforcement. Journal of Applied Behavior Analysis, 46(1), 47–60. https://doi.org/10.1002/jaba.26.
Rapp, J. T., Colby-Dirksen, A. M., Michalski, D. N., Carroll, R. A., & Lindenberg, A. M. (2008). Detecting changes in simulated events using partial-interval recording and momentary time sampling. Behavioral Interventions, 23(4), 237–269. https://doi.org/10.1002/bin.269.
Rapp, J. T., Cook, J. L., McHugh, C., & Mann, K. R. (2017). Decreasing stereotypy using NCR and DRO with functionally matched stimulation: Effects on targeted and non-targeted stereotypy. Behavior Modification, 41(1), 45–83. https://doi.org/10.1177/0145445516652370.
Rapp, J. T., Swanson, G., Sheridan, S. M., Enloe, K. A., Maltese, D., Sennott, L. A., Shrader, L., Carroll, R. A., Richling, S. M., Long, E. S., & Lanovaz, M. J. (2013). Immediate and subsequent effects of matched and unmatched stimuli on targeted vocal stereotypy and untargeted motor stereotypy. Behavior Modification, 37(4), 543–567. https://doi.org/10.1177/0145445512461650.
Retzlaff, B. J., Phillips, L. A., Fisher, W. W., Hardee, A. M., & Fuhrman, A. M. (2020). Using e-learning modules to teach ongoing-visual inspection of functional analyses. Journal of Applied Behavior Analysis, 53(4), 2126–2138. https://doi.org/10.1002/jaba.719.
Roane, H. S., Fisher, W. W., Kelley, M. E., Mevers, J. L., & Bouxsein, K. J. (2013). Using modified visual-inspection criteria to interpret functional analysis outcomes. Journal of Applied Behavior Analysis, 46(1), 130–146. https://doi.org/10.1002/jaba.13.
Rodriguez, N. M., Thompson, R. H., Schlichenmeyer, K., & Stocco, C. S. (2012). Functional analysis and treatment of arranging and ordering by individuals with an autism spectrum disorder. Journal of Applied Behavior Analysis, 45(1), 1–22. https://doi.org/10.1901/jaba.2012.45-1.
Rooker, G. W., Jessel, J., Kurtz, P. F., & Hagopian, L. P. (2013). Functional communication training with and without alternative reinforcement and punishment: An analysis of 58 applications. Journal of Applied Behavior Analysis, 46(4), 708–722. https://doi.org/10.1002/jaba.76.
Roscoe, E. M., Carreau, A., MacDonald, J., & Pence, S. T. (2008). Further evaluations of leisure items in the attention condition of functional analyses. Journal of Applied Behavior Analysis, 41(3), 351–364. https://doi.org/10.1901/jaba.2008.41-351.
Roscoe, E. M., Rooker, G. W., Pence, S. T., Longworth, L. J., & Zarcone, J. (2009). Assessing the utility of a demand assessment for functional analysis. Journal of Applied Behavior Analysis, 42(4), 819–825. https://doi.org/10.1901/jaba.2009.42-819.
Saini, V., Fisher, W. W., & Retzlaff, B. J. (2018). Predictive validity and efficiency of ongoing visual inspection criteria for interpreting functional analyses. Journal of Applied Behavior Analysis, 51(2), 303–320. https://doi.org/10.1002/jaba.450.
Saini, V., Fisher, W. W., Retzlaff, B. J., & Keevy, M. (2020). Efficiency in functional analysis of problem behavior: A quantitative and qualitative review. Journal of Applied Behavior Analysis, 53(1), 44–66. https://doi.org/10.1002/jaba.583.
Schlichenmeyer, K. J., Roscoe, E. M., Rooker, G. W., Wheeler, E. E., & Dube, W. V. (2013). Idiosyncratic variables that affect functional analysis outcomes: A review (2011–2010). Journal of Applied Behavior Analysis, 46(1), 339–348. https://doi.org/10.1002/jaba.12.
Schmidt, J. D., Kranak, M. P., Goetzel, A. L., Kaur, J., & Rooker, G. W. (2020). A clinical demonstration of correlational and experimental analyses of precursor behavior. Behavior Analysis in Practice, 13(4), 966–971. https://doi.org/10.1007/s40617-020-00452-9.
Sham, E., & Smith, T. (2014). Publication bias in studies of an applied behavior-analytic intervention: An initial analysis. Journal of Applied Behavior Analysis, 47(3), 663–678. https://doi.org/10.1002/jaba.146.
Siegel, S. (1957). Nonparametric statistics. The American Statistician, 11(3), 13–19 http://www.jstor.org/stable/2685679.
Thomason-Sassi, J. L., Iwata, B. A., Neidert, P. L., & Roscoe, E. (2011). Response latency as an index of response strength during functional analyses of problem behavior. Journal of Applied Behavior Analysis, 44(1), 51–67. https://doi.org/10.1901/jaba.2011.44-51.
Tiger, J. H., & Effertz, H. M. (2020). On the validity of data produced by isolated and synthesized contingencies during the functional analysis of problem behavior. Journal of Applied Behavior Analysis. Advance online publication. https://doi.org/10.1002/jaba.792.
Vollmer, T. R., Iwata, B. A., Duncan, B. A., & Lerman, D. C. (1993). Extensions of multielement functional analyses using reversal-type designs. Journal of Developmental & Physical Disabilities, 5(4), 311–325. https://doi.org/10.1007/BF01046388.
Wacker, D. P., Lee, J. F., Padilla Dalmau, Y. C., Kopelman, T. G., Lindgren, S. D., Kuhle, J., Pelzel, K. E., Dyson, S., Schieltz, K. M., & Waldron, D. B. (2013). Conducting functional analyses of problem behavior via telehealth. Journal of Applied Behavior Analysis, 46(1), 31–46. https://doi.org/10.1002/jaba.29.
Watkins, N., & Rapp, J. T. (2014). Environmental enrichment and response cost: Immediate and subsequent effects on stereotypy. Journal of Applied Behavior Analysis, 47(1), 186–191. https://doi.org/10.1002/jaba.97.
Wolfe, K., Seaman, M. A., & Drasgow, E. (2016). Interrater agreement on the visual analysis of individual tiers and functional relations in multiple baseline designs. Behavior Modification, 40(6), 852–873. https://doi.org/10.1177/0145445516644699.
Wolfe, K., Seaman, M. A., Drasgow, E., & Sherlock, P. (2018). An evaluation of the agreement between the conservative dual-criterion method and expert visual analysis. Journal of Applied Behavior Analysis, 51(2), 345–351. https://doi.org/10.1002/jaba.453.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflicts of interest
The authors report no conflicts of interest and adhered to all applicable ethical standards.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
The authors thank Joe Baker for programming the web-based ANSA application and the reviewers for their comments on earlier versions of this article.
Supplementary Information
ESM 1
(CSV 290 bytes)
Rights and permissions
About this article
Cite this article
Kranak, M.P., Hall, S.S. Implementing Automated Nonparametric Statistical Analysis on Functional Analysis Data: A Guide for Practitioners and Researchers. Perspect Behav Sci 45, 53–75 (2022). https://doi.org/10.1007/s40614-021-00290-2
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s40614-021-00290-2