Skip to main content

Advertisement

SpringerLink
Log in

A Systematic Review of Sequential Multiple-Assignment Randomized Trials in Educational Research

  • Review Article
  • Published:
Educational Psychology Review Aims and scope Submit manuscript

Abstract

The purpose of this systematic review is to describe the state of the art of sequential multiple-assignment randomized trials in education research. An iterative, systematic search strategy yielded thirteen reports for synthesis. We coded eligible reports for study characteristics, population, intervention, outcomes, SMART design components, overall findings, and study quality. Of the thirteen included reports, nine were completed studies at either the full or the pilot design stage, and four were published protocols. All studies measured educational and/or psychosocial outcomes, few studies included measures of achievement, and studies were primarily conducted outside of the classroom setting. We evaluate the current uses of SMARTs in education research and discuss the promise of this design and how it is well-suited for the dynamic nature of education research, diverse student populations, and the field of educational research.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

  • Almirall, D., Compton, S. N., Gunlicks-Stoessel, M., Duan, N., & Murphy, S. A. (2012). Designing a pilot sequential multiple assignment randomized trial for developing an adaptive treatment strategy. Statistics in Medicine, 31(17), 1887–1902.

    Article  Google Scholar 

  • Almirall, D., DiStefano, C., Chang, Y. C., Shire, S., Kaiser, A., Lu, X., ... & Kasari, C. (2016). Longitudinal effects of adaptive interventions with a speech-generating device in minimally verbal children with ASD. Journal of Clinical Child & Adolescent Psychology, 45(4), 442–456.

  • Almirall, D., Kasari, C., McCaffrey, D. F., & Nahum-Shani, I. (2018). Developing optimized adaptive interventions in education. Journal of Research on Educational Effectiveness, 11(1), 27–34.

    Article  Google Scholar 

  • August, G. J., Piehler, T. F., & Bloomquist, M. L. (2016). Being “SMART” about adolescent conduct problems prevention: Executing a SMART pilot study in a juvenile diversion agency. Journal of Clinical Child & Adolescent Psychology, 45(4), 495–509.

    Article  Google Scholar 

  • August, G. J., Piehler, T. F., & Miller, F. G. (2018). Getting “SMART” about implementing multi-tiered systems of support to promote school mental health. Journal of School Psychology, 66, 85–96.

    Article  Google Scholar 

  • Barlow, D. H., & Hersen, M. (1973). Single-case experimental designs: Uses in applied clinical research. Archives of General Psychiatry, 29(3), 319–325.

    Article  Google Scholar 

  • Chow JC, Granger KL, Broda MD, & Peterson N (2020). Predictive role of classroom management in literacy development in preschool children at risk for EBD. Behavioral Disorders, 0198742920972322.

  • Chow, J. C., & Hampton, L. H. (2019). Sequential multiple-assignment randomized trials: Developing and evaluating adaptive interventions in special education. Remedial and Special Education, 40(5), 267–276.

    Article  Google Scholar 

  • Chow, J. C. (2020). Commentary–classroom motivation and learning disabilities: Consensus points and recommendations. Learning Disabilities: A Multidisciplinary Journal, 25(2), 54–60.

    Google Scholar 

  • Chronis-Tuscano, A., Wang, C. H., Strickland, J., Almirall, D., & Stein, M. A. (2016). Personalized treatment of mothers with ADHD and their young at-risk children: A SMART pilot. Journal of Clinical Child & Adolescent Psychology, 45(4), 510–521.

    Article  Google Scholar 

  • Collins, L. M., Murphy, S. A., & Bierman, K. L. (2004). A conceptual framework for adaptive preventive interventions. Prevention Science, 5(3), 185–196.

    Article  Google Scholar 

  • Cook, T. D., Campbell, D. T., & Shadish, W. (2002). Experimental and quasi-experimental designs for generalized causal inference. Houghton Mifflin.

    Google Scholar 

  • Crank, J. E., Sandbank, M., Dunham, K., Crowley, S., Bottema-Beutel, K., Feldman, J., & Woynaroski, T. G. (2021). Understanding the effects of naturalistic developmental behavioral interventions: A project AIM meta-analysis. Autism Research, 14(4), 817–834.

    Article  Google Scholar 

  • Cronbach LJ, & Snow RE (1977). Aptitudes and instructional methods: A handbook for research on interactions. Irvington.

  • Czyz, E. K., King, C. A., Prouty, D., Micol, V. J., Walton, M., & Nahum-Shani, I. (2021). Adaptive intervention for prevention of adolescent suicidal behavior after hospitalization: a pilot sequential multiple assignment randomized trial. Journal of Child Psychology and Psychiatry, and Allied Disciplines, 62(8), 1019–1031. https://doi.org/10.1111/jcpp.13383

    Article  Google Scholar 

  • Fairchild, A. J., & MacKinnon, D. P. (2009). A general model for testing mediation and moderation effects. Prevention Science, 10(2), 87–99.

    Article  Google Scholar 

  • Fatori, D., de Bragança Pereira, C. A., Asbahr, F. R., Requena, G., Alvarenga, P. G., de Mathis, M. A., & Miguel, E. C. (2018). Adaptive treatment strategies for children and adolescents with obsessive-compulsive disorder: A sequential multiple assignment randomized trial. Journal of Anxiety Disorders, 58, 42–50.

    Article  Google Scholar 

  • Fuchs, D., & Fuchs, L. S. (2006). Introduction to response to intervention: What, why, and how valid is it? Reading Research Quarterly, 41(1), 93–99.

    Article  Google Scholar 

  • Fuchs, D., & Fuchs, L. S. (2019). On the importance of moderator analysis in intervention research: An introduction to the special issue. Exceptional Children, 85(2), 126–128.

    Article  Google Scholar 

  • Gunlicks-Stoessel, M., Mufson, L., Westervelt, A., Almirall, D., & Murphy, S. (2016). A pilot SMART for developing an adaptive treatment strategy for adolescent depression. Journal of Clinical Child & Adolescent Psychology, 45(4), 480–494.

    Article  Google Scholar 

  • Gunlicks-Stoessel, M., Mufson, L., Bernstein, G., Westervelt, A., Reigstad, K., Klimes-Dougan, B., & Vock, D. (2019). Critical decision points for augmenting interpersonal psychotherapy for depressed adolescents: A pilot sequential multiple assignment randomized trial. Journal of the American Academy of Child & Adolescent Psychiatry, 58(1), 80–91.

    Article  Google Scholar 

  • Guy, W. (1976) Clinical Global Impressions, ECDEU Assessment Manual for Psychopharmacology, revised (DHEW Publ. No. ADM 76-338). National Institute of Mental Health, Rockville, 218–222.

  • Hampton, L. H., & Chow, J. C. (2021).Deeply tailoring adaptive interventions: Enhancing knowledge generation of SMARTs in special education. Remedial and Special Education. https://doi.org/10.1177/07419325211030669

  • Hampton LH, & Rodriguez EM (2021). Preemptive interventions for infants and toddlers with a high likelihood for autism: A systematic review and meta-analysis. Autism, 13623613211050433.

  • Hayes, A. F. (2017). Introduction to mediation, moderation, and conditional process analysis: A regression-based approach. Guilford publications.

  • Heppen, J. B., Kurki, A., & Brown, S. (2020). Can texting parents improve attendance in elementary school? A test of an adaptive messaging strategy. Appendix. NCEE 2020-006a. National Center for Education Evaluation and Regional Assistance.

  • Higgins, J. P. T., Altman, D. G., Gøtzsche, P. C., Jüni, P., Moher, D., Oxman, A. D., ... & Sterne, J. A. C. (2011). The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials. BMJ (Online), 343(7829).

  • Kasari, C., Kaiser, A., Goods, K., Nietfeld, J., Mathy, P., Landa, R., & Almirall, D. (2014). Communication interventions for minimally verbal children with autism: A sequential multiple assignment randomized trial. Journal of the American Academy of Child & Adolescent Psychiatry, 53(6), 635–646.

    Article  Google Scholar 

  • Kasari C, Shire S, Shih W, & Almirall D (2021). Getting SMART about social skills interventions for students with ASD in inclusive classrooms. Exceptional Children, 00144029211007148.

  • Kilbourne, A. M., Almirall, D., Eisenberg, D., Waxmonsky, J., Goodrich, D. E., Fortney, J. C., & Kyle, J. (2014). Protocol: Adaptive Implementation of Effective Programs Trial (ADEPT): Cluster randomized SMART trial comparing a standard versus enhanced implementation strategy to improve outcomes of a mood disorders program. Implementation Science, 9(1), 1–14.

    Article  Google Scholar 

  • Kilbourne, A. M., Smith, S. N., Choi, S. Y., Koschmann, E., Liebrecht, C., Rusch, A., ... & Almirall, D. (2018). Adaptive School-based Implementation of CBT (ASIC): clustered-SMART for building an optimized adaptive implementation intervention to improve uptake of mental health interventions in schools. Implementation Science, 13(1), 1–15.

  • Kim, J. S., Asher, C. A., Burkhauser, M., Mesite, L., & Leyva, D. (2019). Using a sequential multiple assignment randomized trial (SMART) to develop an adaptive K–2 literacy intervention with personalized print texts and app-based digital activities. AERA Open, 5(3), 2332858419872701.

    Article  Google Scholar 

  • Ledford, J. R., & Gast, D. L. (Eds.). (2018). Single case research methodology. Routledge.

  • MacKinnon, D. P., Fairchild, A. J., & Fritz, M. S. (2007). Mediation analysis. Annual Review of Psychology, 593–614.

  • Murphy, S. A., Lynch, K. G., Oslin, D., McKay, J. R., & TenHave, T. (2007). Developing adaptive treatment strategies in substance abuse research. Drug and alcohol dependence, 88, S24–S30.

    Article  Google Scholar 

  • Nahum-Shani, I., Qian, M., Almirall, D., Pelham, W. E., Gnagy, B., Fabiano, G. A., & Murphy, S. A. (2012). Experimental design and primary data analysis methods for comparing adaptive interventions. Psychological Methods, 17(4), 457–477.

    Article  Google Scholar 

  • Nahum-Shani I and Almirall D (2019). An introduction to adaptive interventions and SMART designs in education (NCSER 2020–001). U.S. Department of Education. Washington, DC: National Center for Special Education Research. Retrieved [date] from https://ies.ed.gov/ncser/pubs/.

  • O’Keefe, V. M., Haroz, E. E., Goklish, N., Ivanich, J., Cwik, M. F., & Barlow, A. (2019). Employing a sequential multiple assignment randomized trial (SMART) to evaluate the impact of brief risk and protective factor prevention interventions for American Indian youth suicide. BMC Public Health, 19(1), 1675.

    Article  Google Scholar 

  • Page, T. F., Pelham, W. E., III., Fabiano, G. A., Greiner, A. R., Gnagy, E. M., Hart, K. C., & Pelham, W. E., Jr. (2016). Comparative cost analysis of sequential, adaptive, behavioral, pharmacological, and combined treatments for childhood ADHD. Journal of Clinical Child & Adolescent Psychology, 45(4), 416–427.

    Article  Google Scholar 

  • Pelham, W. E., Jr., Fabiano, G. A., Waxmonsky, J. G., Greiner, A. R., Gnagy, E. M., Pelham, W. E., III., & Karch, K. (2016). Treatment sequencing for childhood ADHD: A multiple-randomization study of adaptive medication and behavioral interventions. Journal of Clinical Child & Adolescent Psychology, 45(4), 396–415.

    Article  Google Scholar 

  • Peterson, B. S., West, A. E., Weisz, J. R., Mack, W. J., Kipke, M. D., Findling, R. L., ... & Weersing, R. (2021). A Sequential Multiple Assignment Randomized Trial (SMART) study of medication and CBT sequencing in the treatment of pediatric anxiety disorders. BMC Psychiatry, 21(1), 1–38.

  • Raudenbush, S. W., & Schwartz, D. (2020). Randomized experiments in education, with implications for multilevel causal inference. Annual Review of Statistics and Its Application, 7, 177–208.

    Article  Google Scholar 

  • Roberts, M. Y., Curtis, P. R., Sone, B. J., & Hampton, L. H. (2019). Association of parent training with child language development: A systematic review and meta-analysis. JAMA Pediatrics, 173(7), 671–680.

    Article  Google Scholar 

  • Schoenfelder, E. N., Chronis-Tuscano, A., Strickland, J., Almirall, D., & Stein, M. A. (2019). Piloting a sequential, multiple assignment, randomized trial for mothers with attention-deficit/hyperactivity disorder and their at-risk young children. Journal of Child and Adolescent Psychopharmacology, 29(4), 256–267.

    Article  Google Scholar 

  • Snow, R. E. (1980). Aptitude, learner control, and adaptive instruction. Educational Psychologist, 15(3), 151–158.

    Article  Google Scholar 

  • Speece, D. L. (1990). Aptitude-treatment interactions: Bad rap or bad idea? The Journal of Special Education, 24(2), 139–149.

    Article  Google Scholar 

  • Sugai, G., & Horner, R. R. (2006). A promising approach for expanding and sustaining school-wide positive behavior support. School Psychology Review, 35(2), 245–259.

    Article  Google Scholar 

  • Thall, P. F., Logothetis, C., Pagliaro, L. C., Wen, S., Brown, M. A., Williams, D., & Millikan, R. E. (2007). Adaptive therapy for androgen-independent prostate cancer: a randomized selection trial of four regimens. Journal of the National Cancer Institute, 99(21), 1613–1622.

    Article  Google Scholar 

  • Yan, X., Matchar, D. B., Sivapragasam, N., Ansah, J. P., Goel, A., & Chakraborty, B. (2021). Sequential Multiple Assignment Randomized Trial (SMART) to identify optimal sequences of telemedicine interventions for improving initiation of insulin therapy: A simulation study. BMC medical research methodology, 21(1), 1–11.

Download references

Author information

Authors and Affiliations

  1. College of Education, University of Maryland, 3942 Campus Dr, College Park, MD, 20742, USA

    Jason C. Chow

  2. College of Behavioral & Social Sciences, University of Maryland, 7343 Preinkert Dr, College Park, MD, 20742, USA

    Jason C. Chow

  3. College of Education, The University of Texas at Austin, 1912 Speedway, Stop D5000, Austin, TX, 78712, USA

    Lauren H. Hampton

Authors
  1. Jason C. Chow
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lauren H. Hampton
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jason C. Chow.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chow, J.C., Hampton, L.H. A Systematic Review of Sequential Multiple-Assignment Randomized Trials in Educational Research. Educ Psychol Rev 34, 1343–1369 (2022). https://doi.org/10.1007/s10648-022-09660-x

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10648-022-09660-x

Keywords

Search

Navigation