Clinically Significant and Reliable Change: Comparing an Evidence-based Intervention to Usual Care
Multiple Family Groups (MFG) is an evidence-based behavioral parent training developed with a specific focus on increasing engagement and decreasing treatment barriers for families of children with disruptive behavior problems within high-risk communities. Previous studies have demonstrated the effectiveness of MFG in improving oppositional behavior at the group-level compared to services as usual (SAU). However, information is lacking regarding intervention effectiveness on an individual-level (i.e., clinical significance).
The reliable change index and clinical cutoff score method was utilized to determine whether MFG produced clinically meaningful changes compared to SAU for both child- and parent-level outcomes in a sample of 320 youth aged 7 to 11-years-old.
A significantly greater percentage of children in the MFG group experienced clinically meaningful change in problem behaviors compared to the SAU group, (p = 0.003, 95% [CI]: 1.610–18.481). A significantly greater number of parents in MFG also demonstrated clinically meaningful change in parental experience of stress compared to SAU, (p = 0.01, 95% [CI]: 1.255–14.704).
Findings suggested clinically significant and reliable improvements in child problem behaviors and decreases in parental perceived stress for families in MFG compared to SAU. Nevertheless, analyses demonstrated that both MFG and SAU resulted in few families obtaining clinically significant or reliable change in their functioning. Ongoing assessments and deeper understanding of intervention effect are needed to better service families in need. Both group- and individual-level comparisons should be considered when examining the effects of a treatment as they may provide a nuanced understanding of evidence-based interventions.
KeywordsClinical significance Behavioral parent training Disruptive behavior disorders Usual care Child functioning
M.M.V.: Led paper development, data analysis, and writing of the manuscript. A.R.: Collaborated with writing of the manuscript, assisted in data analysis. S.P.: Collaborated with writing of the manuscript, wrote part of the introduction. Y.S.: Assisted in data analysis, wrote part of the results. S.P.S. and E.C.: Collaborated in writing and editing of the manuscript. A.C.: Designed and executed main study, collaborated in editing of final manuscript.
Compliance with Ethical Standards
Conflict of Interest
The authors declare that they have no conflict of interest.
All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee (IRB approval received by the Mount Sinai School of Medicine) and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.
Informed consent was obtained from all individual participants included in the original study.
- Acri, M., Chacko, A., Gopalan, G., & McKay, M. (2018). Engaging families in treatment for child behavior disorders: a synthesis of the literature. In J. E. Lochman & W. Matthys (Eds), The Wiley handbook of disruptive and impulse-control disorders (pp. 393–409). Hoboken, NJ: John Wiley & Sons Inc.Google Scholar
- Barkham, M., Connell, J., Stiles, W. B., Miles, J. N., Margison, F., Evans, C., & Mellor-Clark, J. (2006). Dose-effect relations and responsive regulation of treatment duration: the good enough level. Journal of Consulting and Clinical Psychology, 74(1), 160–167.PubMedCrossRefPubMedCentralGoogle Scholar
- Chacko, A., Alan, C., Uderman, J., Cornwell, M., Anderson, L., & Chimiklis, A. (2015a). Training parents of children with ADHD. In R. Barkley (Ed.), Attention deficit hyperactivity disorder: a handbook for diagnosis and treatment (pp. 513–536). New York, NY: Guilford Press.Google Scholar
- Chacko, A., Granski, M., Horn, E. P., Levy, M. D., Dahl, V., Lacks, R. S., & Ramakrishnan, A. (2018b). Prevention of disruptive behavior problems in children. In M. M. Martel (Ed.), Developmental pathways to disruptive, impulse-control and conduct disorders (pp. 347–380). London, UK: Academic Press.CrossRefGoogle Scholar
- Chacko, A., Isham, A., Cleek, A. F., & McKay, M. M. (2016a). Using mobile health technology to improve behavioral skill implementation through homework in evidence-based parenting intervention for disruptive behavior disorders in youth: study protocol for intervention development and evaluation. Pilot and Feasibility Studies, 2(1), 57–68.PubMedPubMedCentralCrossRefGoogle Scholar
- Chacko, A., Wymbs, B. T., Chimiklis, A., Wymbs, F. A., & Pelham, W. E. (2012). Evaluating a comprehensive strategy to improve engagement to group-based behavioral parent training for high-risk families of children with ADHD. Journal of Abnormal Child Psychology, 40(8), 1351–1362.PubMedCrossRefPubMedCentralGoogle Scholar
- Deater-Deckard, K. (2008). Parenting stress. New Haven, CT: Yale University Press.Google Scholar
- Deater-Deckard, K. D. & Panneton, R. (Eds) (2017). Parental stress and early child development. Cham, CH: Springer.Google Scholar
- Devilly, G.J. (2005). ClinTools Software for Windows: Version 4 (computer programme). Psytek Ltd. www.clintools.com.
- Fabiano, G. A., Pelham, J., William, E., Waschbusch, D. A., Gnagy, E. M., Lahey, B. B., & Burrows-MacLean, L. (2006). A practical measure of impairment: psychometric properties of the impairment rating scale in samples of children with attention deficit hyperactivity disorder and two school-based samples. Journal of Clinical Child & Adolescent Psychology, 35(3), 369–385.CrossRefGoogle Scholar
- Gopalan, G., Bornheimer, L. A., Acri, M., Winters, A., O’Brien, K. H., & Chacko, A. (2018). Multiple family group service delivery model for children with disruptive behavior disorders: Impact on caregiver stress and depressive symptoms. Journal of Emotional and Behavioral Disorders, 26(3), 182–192.PubMedCrossRefPubMedCentralGoogle Scholar
- Gopalan, G., Chacko, A., Franco, L., Dean-Assael, K. M., Rotko, L. E., Marcus, S. M., & McKay, M. M. (2015a). Multiple family groups for children with disruptive behavior disorders: Child outcomes at 6-month follow-up. Journal of Child and Family Studies, 24(9), 2721–2733.PubMedCrossRefPubMedCentralGoogle Scholar
- Gresham, F. M. (2015). Disruptive behavior disorders: evidence-based practice for assessment and interventions. New York, NY: Guilford Press.Google Scholar
- Health Resources and Services Administration [HRSA] (2016). National projections of supply and demand for selected behavioral health practitioners: 2013–2025. Department of Health and Human Services. https://bhw.hrsa.gov/sites/default/files/bhw/health-workforce-analysis/research/projections/behavioral-health2013-2025.pdf. Accessed 23 Jan 2019.
- Hoagwood, K., & Kolko, D. J. (2009). Introduction to the special section on practice contexts: A glimpse into the nether world of public mental health services for children and families. Administration and Policy in Mental Health and Mental Health Services Research, 36(1), 35–36.PubMedCrossRefPubMedCentralGoogle Scholar
- IBM Corp. (2017). IBM SPSS statistics for windows, version25.0. Armonk, NY: IBM Corp. Released.Google Scholar
- Sheldrick, R. C., Kendall, P. C., & Heimberg, R. G. (2001). The clinical significance of treatments: a comparison of three treatments for conduct disordered children. Clinical Psychology: Science and Practice, 8(4), 418–430.Google Scholar
- Warren, J. S., Nelson, P. L., Mondragon, S. A., Baldwin, S. A., & Burlingame, G. M. (2010). Youth psychotherapy change trajectories and outcomes in usual care: community mental health versus managed care settings. Journal of Consulting and Clinical Psychology, 78(2), 144–155.PubMedCrossRefPubMedCentralGoogle Scholar
- Weisz, J. R., Kuppens, S., Ng, M. Y., Eckshtain, D., Ugueto, A. M., Vaughn-Coaxum, R., & Weersing, V. R. (2017). What five decades of research tells us about the effects of youth psychological therapy: a multilevel meta-analysis and implications for science and practice. American Psychologist, 72(2), 79–117.PubMedCrossRefPubMedCentralGoogle Scholar