Abstract
Implementation of evidence-based practices for autistic students must be informed by robust research literature. Randomized-controlled trials (RCTs) are often considered a “gold standard” methodology for determining the effectiveness of interventions. However, the complex nature of schools presents challenges for implementing successful RCTs. We review common challenges that researchers may face when implementing RCTs of interventions for autistic students in school settings. Additionally, we provide recommendations to researchers for avoiding and addressing such challenges, based on our own experience conducting a school-based RCT. Limitations and directions for future research are also discussed.
Similar content being viewed by others
Notes
We adopt the term autistic rather than using person-first language (i.e., “student with ASD”) reflecting preferences among autistic individuals and the autistic community (Bottema-Beutel et al., 2021).
References
Alibali, M. W., & Nathan, M. J. (2010). Conducting research in schools: A practical guide. Journal of Cognition and Development, 11(4), 397–407.
Alexander, J. L., Ayres, K. M., & Smith, K. A. (2015). Training teachers in evidence-based practice for individuals with autism spectrum disorder: A review of the literature. Teacher Education and Special Education, 38(1), 13–27.
American Psychiatric Association. (2013). Diagnostic and statistical manual of mental health disorders (5th ed.). Arlington, VA: Author.
Anderman, C., Cheadle, A., Curry, S., Diehr, P., Shultz, L., & Wagner, E. (1995). Selection bias related to parental consent in school-based survey research. Evaluation Review, 19(6), 663–674.
Anderson, C. M., Iovannone, R., Smith, T., Levato, L., Martin, R., Cavanaugh, B., Hochheimer, S., Wang, H., & Iadarola, S. (2021). Thinking small to think big: Modular Approach for Autism Programming in Schools (MAAPS). Journal of Autism and Developmental Disorders, 51(1), 193–211. https://doi.org/10.1007/s10803-020-04532-1
Anderson, C. M., Smith, T., & Iovannone, R. (2018). Building capacity to support students with autism spectrum disorder: a modular approach to intervention. Education and Treatment of Children, 41(1), 107–138. https://www.jstor.org/stable/26535257
Barry, L., Holloway, J., & McMahon, J. (2020). A scoping review of the barriers and facilitators to the implementation of interventions in autism education. Research in Autism Spectrum Disorders, 78, 101617.
Beahm, L. A., Yan, X., & Cook, B. G. (2021). Where Do Teachers Go for Behavior Management Strategies? Education and Treatment of Children, 44(3), 201–213.
Befort, C., Lynch, R., James, R. L., Carroll, S. L., Nollen, N., & Davis, A. (2008). Perceived barriers and benefits to research participation among school administrators. Journal of School Health, 78(11), 581–586.
Bonnell, K. J., Hargiss, C. L., & Norland, J. E. (2018). Challenges conducting research with adolescents in public schools. Natural Sciences Education, 47(1), 1–7.
Bottema-Beutel, K., Kapp, S. K., Lester, J. N., Sasson, N. J., & Hand, B. N. (2021). Avoiding ableist language: Suggestions for autism researchers. Autism in Adulthood, 3(1), 18–29.
Bowker, A., D’Angelo, N. M., Hicks, R., & Wells, K. (2011). Treatments for autism: Parental choices and perceptions of change. Journal of Autism and Developmental Disorders, 41(10), 1373–1382.
Brondino, N., Fusar-Poli, L., Miceli, E., Di Stefano, M., Damiani, S., Rocchetti, M., & Politi, P. (2019). Prevalence of medical comorbidities in adults with autism spectrum disorder. Journal of General Internal Medicine, 34(10), 1992–1994.
Brookman-Frazee, L., Stadnick, N., Chlebowski, C., Baker-Ericzén, M., & Ganger, W. (2018). Characterizing psychiatric comorbidity in children with autism spectrum disorder receiving publicly funded mental health services. Autism, 22(8), 938–952.
Callahan, K., Hughes, H. L., Mehta, S., Toussaint, K. A., Nichols, S. M., Ma, P. S., ... & Wang, H. T. (2017). Social validity of evidence-based practices and emerging interventions in autism. Focus on Autism and Other Developmental Disabilities, 32(3), 188–197.
Camden, C., Shikako-Thomas, K., Nguyen, T., Graham, E., Thomas, A., Sprung, J., ... & Russell, D. J. (2015). Engaging stakeholders in rehabilitation research: a scoping review of strategies used in partnerships and evaluation of impacts. Disability and rehabilitation, 37(15), 1390–1400.
Chown, N., Robinson, J., Beardon, L., Downing, J., Hughes, L., Leatherland, J., ... & MacGregor, D. (2017). Improving research about us, with us: a draft framework for inclusive autism research. Disability & Society, 32(5), 720–734.
Constantino, J. N., & Gruber, C. P. (2012). The social responsiveness scale (2nd ed.). Western Psychological Services.
Dababnah, S., Shaia, W. E., Kim, I., & Magaña, S. (2021). Parents taking action: Adapting a peer-to-peer program for parents raising black children with autism. Inclusion, 9(3), 205–224.
Daniel, G. R., Wang, C., & Berthelsen, D. (2016). Early school-based parent involvement, children’s self-regulated learning and academic achievement: An Australian longitudinal study. Early Childhood Research Quarterly, 36, 168–177.
Detrich, R. (1999). Increasing treatment fidelity by matching interventions to contextual variables within the educational setting. School Psychology Review, 28(4), 608–620.
Devilly, G. J., & McFarland, A. C. (2009). When wait lists are not feasible, nothing is a thing that does not need to be done. Journal of Consulting and Clinical Psychology, 77(6), 1159–1168.
DiGennaro, F., Martens, B. K., & McIntyre, L. L. (2005). Increasing treatment integrity through negative reinforcement: Effects on teacher and student behavior. School Psychology Review, 34(2), 220–231. https://doi.org/10.1080/02796015.2005.12086284
Drahota, A., Sadler, R., Hippensteel, C., Ingersoll, B., & Bishop, L. (2020). Service deserts and service oases: Utilizing geographic information systems to evaluate service availability for individuals with autism spectrum disorder. Autism, 24(8), 2008–2020.
Feeley, N., Cossette, S., Côté, J., Héon, M., Stremler, R., Martorella, G., & Purden, M. (2009). The importance of piloting an RCT intervention. Canadian Journal of Nursing Research Archive, 84–99.
Felzmann, H. (2009). Ethical issues in school-based research. Research Ethics Review, 5(3), 104–109.
Ferguson, J. L., Cihon, J. H., Leaf, J. B., Van Meter, S. M., McEachin, J., & Leaf, R. (2019). Assessment of social validity trends in the journal of applied behavior analysis. European Journal of Behavior Analysis, 20(1), 146–157.
Fletcher-Watson, S., Adams, J., Brook, K., Charman, T., Crane, L., Cusack, J., ... & Pellicano, E. (2019). Making the future together: Shaping autism research through meaningful participation. Autism, 23(4), 943–953.
Fountain, C., Winter, A. S., & Bearman, P. S. (2012). Six developmental trajectories characterize children with autism. Pediatrics, 129(5), e1112–e1120.
Gadke, D. L., Kratochwill, T. R., & Gettinger, M. (2021). Incorporating feasibility protocols in intervention research. Journal of School Psychology, 84, 1–18.
Gartland, D., Riggs, E., Giallo, R., Glover, K., Casey, S., Muyeen, S., ... Brown, S. J. (2021). Using Participatory Methods to Engage Diverse Families in Research about Resilience in Middle Childhood. Journal of Health Care for the Poor and Underserved 32(4), 1844–1871.
Gearing, R. E., El-Bassel, N., Ghesquiere, A., Baldwin, S., Gillies, J., & Ngeow, E. (2011). Major ingredients of fidelity: A review and scientific guide to improving quality of intervention research implementation. Clinical Psychology Review, 31(1), 79–88.
Gersten, R., Baker, S., & Lloyd, J. W. (2000). Designing high-quality research in special education: Group experimental design. The Journal of Special Education, 34(1), 2–18.
Gopalan, M., Rosinger, K., & Ahn, J. B. (2020). Use of quasi-experimental research designs in education research: Growth, promise, and challenges. Review of Research in Education, 44(1), 218–243.
Gowen, E., Taylor, R., Bleazard, T., Greenstein, A., Baimbridge, P., & Poole, D. (2019). Guidelines for conducting research studies with the autism community. Autism Policy & Practice, 2(1 A new beginning), 29.
Greenberg, M. T. (2004). Current and future challenges in school-based prevention: The researcher perspective. Prevention Science, 5(1), 5–13.
Gresham, F., & Elliott, S. N. (2008). Social skills improvement system (SSIS) rating scales. Pearson Education Inc.
Harris, P. A., Taylor, R., Thielke, R., Payne, J., Gonzalez, N., & Conde, J. G. (2009). Research electronic data capture (REDCap)—a metadata-driven methodology and workflow process for providing translational research informatics support. Journal of Biomedical Informatics, 42(2), 377–381.
Harrison, P., & Oakland, T. (2015). Adaptive behavior assessment system (3rd ed.). Western Psychological Services.
Hein, S., & Weeland, J. (2019). Introduction to the special issue. Randomized controlled trials (RCTs) in clinical and community settings: Challenges, alternatives, and supplementary designs. New Directions for Child and Adolescent Development, 2019(167), 7–15.
IDEA. (2004). Individuals with disabilities education improvement act, 20, U. S. C. § 1400 et seq.
Iadarola, S., Pellecchia, M., Stahmer, A., Lee, H. S., Hauptman, L., Hassrick, E. M., ... & Smith, T. (2020). Mind the gap: an intervention to support caregivers with a new autism spectrum disorder diagnosis is feasible and acceptable. Pilot and Feasibility Studies, 6, 1–13.
Iovannone, R., Iadarola, S., Hodges, S., Haynes, R., Stark, C., McFee, K., Grace, Sheri, & Anderson, C. M. (2019). An extra set of hands: A qualitative analysis of stakeholder perspectives on implementation of a modular approach to school adoption of evidence-based interventions for students with autism spectrum disorders. International Journal of Positive Behavioural Support, 9(2), 25–40.
Kasari, C. (2002). Assessing change in early intervention programs for children with autism. Journal of Autism and Developmental Disorders, 32(5), 447–461.
Kasari, C., & Smith, T. (2013). Interventions in schools for children with autism spectrum disorder: Methods and recommendations. Autism, 17(3), 254–267.
Kerns, C. M., Kendall, P. C., Berry, L., Souders, M. C., Franklin, M. E., Schultz, R. T., ... & Herrington, J. (2014). Traditional and atypical presentations of anxiety in youth with autism spectrum disorder. Journal of Autism and Developmental Disorders, 44(11), 2851–2861.
Koegel, L., Matos-Freden, R., Lang, R., & Koegel, R. (2012). Interventions for children with autism spectrum disorders in inclusive school settings. Cognitive and Behavioral Practice, 19(3), 401–412.
Kraemer, H. C., Wilson, G. T., Fairburn, C. G., & Agras, W. S. (2002). Mediators and moderators of treatment effects in randomized clinical trials. Archives of General Psychiatry, 59(10), 877–883.
Lord, C., Charman, T., Havdahl, A., Carbone, P., Anagnostou, E., Boyd, B., ... & McCauley, J. B. (2021). The Lancet Commission on the future of care and clinical research in autism. The Lancet.
Lord, C., Rutter, M., DiLavore, P. C., Risi, S., Gotham, K., & Bishop, S.L. (2012). Autism Diagnostic Observation Schedule, 2nd Edition (ADOS-2). Torrance, CA: Western Psychological Services.
Maenner, M. J., Warren, Z., Williams, A., Amoakohene, E., Bakian, A., Bilder, D., Durkin, M., Fitzgerald, R., Furnier, S., Hughes, M., Ladd-Acosta, C., McArthur, D., Pas, E., Salinas, A., Vehorn, A., Williams, S., Esler, A., Grzybowski, A., Hall-Lande, J., … & Shaw, K. (2023). Prevalence and characteristics of autism spectrum disorder among children aged 8 years—Autism and Developmental Disabilities Monitoring Network, 11 sites, United States, 2020. MMWR. Surveillance Summaries, 72.
Maddox, B. B., Rump, K. M., Stahmer, A. C., Suhrheinrich, J., Rieth, S. R., Nahmias, A. S., ... & Mandell, D. S. (2020). Concordance between a US educational autism classification and the Autism Diagnostic Observation Schedule. Journal of Clinical Child & Adolescent Psychology, 49(4), 469–475.
Martin, R. J., Anderson, C. M., Gould, K., Morganelli, M., & Kleinert, W. L. (2021a). A descriptive secondary analysis of evidence-based interventions for students with autism spectrum disorder. Contemporary School Psychology, 25(1), 75–85.
Martin, R. J., Cavanaugh, B., Levato, L., Fontechia, K., Hochheimer, S., Iadarola, S., Iovannone, R., Smith, T. & Anderson, C. M. (2021b). Modular Approach to Autism Programs in Schools (MAAPS): A Feasibility Study. Contemporary School Psychology, 1–11. https://doi.org/10.1007/s40688-021-00397-y
Milton, D. E. (2014). Autistic expertise: A critical reflection on the production of knowledge in autism studies. Autism, 18(7), 794–802.
Mire, S. S., Truong, D. M., Sakyi, G. J., Ayala-Brittain, M. L., Boykin, J. D., Stewart, C. M., ... & McKee, S. L. (2023). A Systematic Review of Recruiting and Retaining Sociodemographically Diverse Families in Neurodevelopmental Research Studies. Journal of Autism and Developmental Disorders, 1–15.
Mohr, D. C., Spring, B., Freedland, K. E., Beckner, V., Arean, P., Hollon, S. D., ... & Kaplan, R. (2009). The selection and design of control conditions for randomized controlled trials of psychological interventions. Psychotherapy and Psychosomatics, 78(5), 275–284.
Morris, C., Detrick, J. J., & Peterson, S. M. (2021). Participant assent in behavior analytic research: Considerations for participants with autism and developmental disabilities. Journal of Applied Behavior Analysis, 54(4), 1300–1316.
Noell, G. H., Witt, J. C., Gilbertson, D. N., Ranier, D. D., & Freeland, J. T. (1997). Increasing teacher intervention implementation in general education settings through consultation and performance feedback. School Psychology Quarterly, 12(1), 77–88. https://doi.org/10.1037/h0088949
Odom, S. L. (2021). Education of students with disabilities, science, and randomized controlled trials. Research and Practice for Persons with Severe Disabilities, 46(3), 132–145.
Owens, J. S., & Murphy, C. E. (2004). Effectiveness research in the context of school-based mental health. Clinical Child and Family Psychology Review, 7(4), 195–209.
Park, E. Y., & Shin, M. (2020). A meta-analysis of special education teachers’ burnout. SAGE Open, 10(2), 2158244020918297.
Pellecchia, M., Nuske, H. J., Straiton, D., Hassrick, E. M., Gulsrud, A., Iadarola, S., Vejnoska, S. F., Bullen, B., Haine-Schlagel, R., Kasari, C., Mandell, D. S., Smith, T., & Stahmer, A. C. (2018). Strategies to engage underrepresented parents in child intervention services: A review of effectiveness and co-occurring use. Journal of Child and Family Studies, 27(10), 3141–3154.
Pham, A. V., N. Goforth, A., N. Aguilar, L., Burt, I., Bastian, R., & Diaków, D. M. (2021). Dismantling Systemic Inequities in School Psychology: Cultural Humility as a Foundational Approach to Social Justice. School Psychology Review, 1–18.
Reed, F. D. D., & Codding, R. S. (2014). Advancements in procedural fidelity assessment and intervention: Introduction to the special issue. Journal of Behavioral Education, 23(1), 1–18.
Ruble, L., McGrew, J. H., & Toland, M. D. (2012). Goal attainment scaling as an outcome measure in randomized controlled trials of psychosocial interventions in autism. Journal of Autism and Developmental Disorders, 42(9), 1974–1983.
Sandbank, M., Bottema-Beutel, K., Crowley, S., Cassidy, M., Dunham, K., Feldman, J. I., Crank, J., Albarran, S. A., Raj, S., Mahbub, P., & Woynaroski, T. G. (2020). Project AIM: Autism intervention meta-analysis for studies of young children. Psychological Bulletin, 146(1) 1–29. https://doi.org/10.1037/bul0000215
Sandbank, M., Chow, J., Bottema‐Beutel, K., & Woynaroski, T. (2021). Evaluating evidence‐based practice in light of the boundedness and proximity of outcomes: Capturing the scope of change. Autism Research.
Schenk, Y. A., Martin, R. J., Kleinert, W. L., Quigley, S. P., & Langone, S. R. (2018). Unsubstantiated Interventions for Autism Spectrum Disorder. Handbook of Parent-Child Interaction Therapy for Children on the Autism Spectrum, 87–106.
Schutz, L. E., Rivers, K. O., & Ratusnik, D. L. (2008). The role of external validity in evidence-based practice for rehabilitation. Rehabilitation Psychology, 53(3), 294.
Shadish, W. R. (2011). Randomized controlled studies and alternative designs in outcome studies: Challenges and opportunities. Research on Social Work Practice, 21(6), 636–643.
Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and Quasi-Experimental Designs for Generalized Causal Inference. Houghton Mifflin.
Shefcyk, A. (2015). Count us in: addressing gender disparities in autism research. Autism, 19(2), 131–132.
Spencer, T. D., Detrich, R., & Slocum, T. A. (2012). Evidence-based practice: A framework for making effective decisions. Education and Treatment of Children, 35(2), 127–151.
Spreckley, M., & Boyd, R. (2009). Efficacy of applied behavioral intervention in preschool children with autism for improving cognitive, language, and adaptive behavior: A systematic review and meta-analysis. The Journal of Pediatrics, 154(3), 338–344.
Stahmer, A. C., Aranbarri, A., Drahota, A., & Rieth, S. (2017). Toward a more collaborative research culture: Extending translational science from research to community and back again. Autism, 21(3), 259–261.
Steinbrenner, J. R., Hume, K., Odom, S. L., Morin, K. L., Nowell, S. W., Tomaszewski, B., ... & Savage, M. N. (2020). Evidence-Based Practices for Children, Youth, and Young Adults with Autism. FPG Child Development Institute.
Strain, P. S., Barton, E. E., & Dunlap, G. (2012). Lessons learned about the utility of social validity. Education and Treatment of Children, 183–200.
Turan, Y., & Meadan, H. (2011). Social validity assessment in early childhood special education. Young Exceptional Children, 14(3), 13–28.
Valli, L., Stefanski, A., & Jacobson, R. (2018). School-community partnership models: Implications for leadership. International Journal of Leadership in Education, 21(1), 31–49. https://doi.org/10.1080/13603124.2015.1124925
Wagner, A., Lecavalier, L., Arnold, L. E., Aman, M. G., Scahill, L., Stigler, K. A., ... & Vitiello, B. (2007). Developmental disabilities modification of the Children’s Global Assessment Scale. Biological Psychiatry, 61(4), 504–511.
Walker, H. M., & Severson, H. H. (1990). Systematic screening for behavior disorders (SSBD): User’s guide and technical manual. Longmont, CO: Sopris West.
Walmsley, J., Strnadova, I., & Johnson, K. (2018). The added value of inclusive research. Journal of Applied Research in Intellectual Disabilities, 31(5), 751–759.
West, E. A., Travers, J. C., Kemper, T. D., Liberty, L. M., Cote, D. L., McCollow, M. M., & Stansberry Brusnahan, L. L. (2016). Racial and ethnic diversity of participants in research supporting evidence-based practices for learners with autism spectrum disorder. The Journal of Special Education, 50(3), 151–163.
West, S. G., & Thoemmes, F. (2010). Campebell’s and Rubin’s perspectives on causal inference. Psychological Methods, 15(1), 18.
Wilder, D. A., Ertel, H. M., & Cymbal, D. J. (2020). A review of recent research on the manipulation of response effort in applied behavior analysis. Behavior Modification, 0145445520908509.
Wolf, M. M. (1978). Social validity: The case for subjective measurement or how applied behavior analysis is finding its heart. Journal of Applied Behavior Analysis, 11(2), 203–214.
Wong, C., Odom, S. L., Hume, K. A., Cox, A. W., Fettig, A., Kucharczyk, S., ... & Schultz, T. R. (2015). Evidence-based practices for children, youth, and young adults with autism spectrum disorder: A comprehensive review. Journal of Autism and Developmental Disorders, 45(7), 1951–1966.
Yoder, P. J., Bottema-Beutel, K., Woynaroski, T., Chandrasekhar, R., & Sandbank, M. (2013). Social communication intervention effects vary by dependent variable type in preschoolers with autism spectrum disorders. Evidence-Based Communication Assessment and Intervention, 7(4), 150–174.
Funding
This research was supported by research grants from the United States Department of Education’s Institute of Education Sciences (R324A150032, Smith; R324A200154, Anderson).
Author information
Authors and Affiliations
Contributions
All authors contributed to the conception and preparation of this manuscript, and all authors read and approved the final draft. The authors would like to acknowledge the late Dr. Tristram Smith for providing critical guidance and inspiration during the initial conception of this manuscript.
Corresponding author
Ethics declarations
Conflicts of Interest
The authors have no other known conflicts of interest to report.
Ethical Approval
This conceptual article did not involve human subjects. However, the study protocols for our feasibility study and subsequent RCT of MAAPS were approved and overseen by the Institutional Review Boards of the coordinating center and all implementation sites.
Informed Consent
Parents of all participating students, and all participating teachers provided informed consent to participate. Additionally, student assent was sought when appropriate (i.e., student developmental level and communicative abilities were such that it could be provided).
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Martin, R.J., Iadarola, S., Iovannone, R. et al. Conducting a Randomized Controlled Trial to Evaluate a Comprehensive School-Based Intervention for Students on the Autism Spectrum: Challenges and Mitigation Strategies. J Dev Phys Disabil (2023). https://doi.org/10.1007/s10882-023-09918-4
Accepted:
Published:
DOI: https://doi.org/10.1007/s10882-023-09918-4