A Supervisor-Targeted Implementation Approach to Promote System Change: The R3 Model

  • Lisa SaldanaEmail author
  • Patricia Chamberlain
  • Jason Chapman
Original Paper


Opportunities to evaluate strategies to create system-wide change in the child welfare system (CWS) and the resulting public health impact are rare. Leveraging a real-world, system-initiated effort to infuse the use of evidence-based principles throughout a CWS workforce, a pilot of the R3 model and supervisor-targeted implementation approach is described. The development of R3 and its associated fidelity monitoring was a collaboration between the CWS and model developers. Outcomes demonstrate implementation feasibility, strong fidelity scale measurement properties, improved supervisor fidelity over time, and the acceptability and perception of positive change by agency leadership. The value of system-initiated collaborations is discussed.


Child welfare system R3 Supervisor System-initiated 



The authors would like to thank the New York City Administration for Children’s Services and the NYC supervisors and caseworkers for their collaboration in creating this model. We would also like to thank Courtenay Padgett for her project direction and Katie Lewis for her editorial assistance. This research was supported by the National Institute on Drug Abuse (P50 DA035763; R01 DA 032634; and 1R01DA040416-01A1) and the National Institute of Mental Health (R01 MH097748).


  1. Aarons, G. A., Horowitz, J. D., Dlugosz, L. R., & Ehrhart, M. G. (2012). The role of organizational processes in dissemination and implementation research. In R. C. Brownson, G. A. Colditz, & E. K. Proctor (Eds.), Dissemination and implementation research in health: Translating science to practice (pp. 128–153). New York: Oxford University Press.CrossRefGoogle Scholar
  2. Aarons, G. A., Hurlburt, M., & Horwitz, S. M. (2011). Advancing a conceptual model of evidence-based practice implementation in public service sectors. Administration and Policy in Mental Health and Mental Health Services Research, 38(1), 4–23.CrossRefPubMedGoogle Scholar
  3. Adoption and Safe Families Act of 1997, Pub. L. No. 105-89, 111, Stat. 2115 (1997)Google Scholar
  4. Altman, J. C. (2008). Engaging families in child welfare services: Worker versus client perspectives. Child Welfare, 87(3), 41–61.PubMedGoogle Scholar
  5. Bandura, A. (1971). Vicarious and self-reinforcement processes. In R. Glaser (Ed.), The nature of reinforcement (pp. 228–278). New York: Academic Press.Google Scholar
  6. Beidas, R. S., & Kendall, P. C. (2010). Training therapists in evidence-based practice: A critical review of studies from a systems-contextual perspective. Clinical Psychology: Science and Practice, 17(1), 1–30.Google Scholar
  7. Bond, T. G., & Fox, C. M. (2007). Applying the Rasch model: Fundamental measurement in the human sciences (2nd ed.). Mahwah, NJ: Lawrence Erlbaum Associates.Google Scholar
  8. Chamberlain, P., Price, J., Reid, J. B., & Landsverk, J. (2008). Cascading implementation of a foster and kinship parent intervention. Child Welfare, 87(5), 27–48.PubMedPubMedCentralGoogle Scholar
  9. Cornerstones For Kids. (2006). The human services workforce initiative: Relationship between staff turnover, child welfare system functioning and recurrent child abuse. Houston, TX: Cornerstones For Kids.Google Scholar
  10. Falander, C. A., & Shafranske, E. P. (2004). Clinical supervision. Washington, DC: American Psychological Association.Google Scholar
  11. Forgatch, M. S., Patterson, G. R., & Gewirtz, A. H. (2013). Looking forward: The promise of widespread implementation of parent training programs. Perspectives on Psychological Science, 8, 682–694.CrossRefPubMedPubMedCentralGoogle Scholar
  12. Institutes of Medicine. (2014). New directions in child abuse and neglect research. Washington, DC: National Academies Press.Google Scholar
  13. Leve, L. D., Pears, K. C., & Fisher, P. A. (2002). Competence in early development. In J. B. Reid, G. R. Patterson, & J. Snyder (Eds.), Antisocial behavior in children and adolescents: A developmental analysis and model for intervention (pp. 45–64). Washington, DC: American Psychological Association.CrossRefGoogle Scholar
  14. Linacre, J. M. (2002). Optimizing rating scale category effectiveness. Journal of Applied Measurement, 3, 85–106.PubMedGoogle Scholar
  15. Linacre, J. M. (2015). WINSTEPS Rasch measurement computer program [Computer software and manual]. Beaverton, OR: Scholar
  16. Marsh, J. C., Ryan, J. P., Choi, S., & Testa, M. F. (2006). Integrated services for families with multiple problems: Obstacles to family reunification. Children and Youth Services Review, 28, 1074–1087.CrossRefGoogle Scholar
  17. O’Connor, B. P. (2000). SPSS and SAS programs for determining the number of components using parallel analysis and Velicer’s MAP test. Behavior Research Methods, Instruments, & Computers, 32, 396–402.CrossRefGoogle Scholar
  18. Patterson, G. R., Reid, J. B., & Dishion, T. J. (1992). A social learning approach. IV. Antisocial boys. Eugene, OR: Castalia.Google Scholar
  19. Perepletchikova, F., Treat, T. A., & Kazdin, A. E. (2007). Treatment integrity in psychotherapy research: Analysis of the studies and examination of the associated factors. Journal of Consulting and Clinical Psychology, 75, 829–841.CrossRefPubMedGoogle Scholar
  20. Price, J. M., Chamberlain, P., Landsverk, J., Reid, J., Leve, L. D., & Laurent, H. (2008). Effects of a foster parent training intervention on placement changes of children in foster care. Child Maltreatment, 13, 64–75.CrossRefPubMedPubMedCentralGoogle Scholar
  21. Ramsey, E., Walker, H. M., Shinn, M., O’Neill, R. E., & Stieber, S. (1989). Parent management practices and school adjustment. School Psychology Review, 18, 513–525.Google Scholar
  22. Rasch, G. (1960). Probabilistic models for some intelligence and attainment tests. Copenhagen: The Danish Institute of Educational Research.Google Scholar
  23. Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models: Applications and data analysis methods. Thousand Oaks, CA: Sage.Google Scholar
  24. Reise, S. P. (2010). Bifactor models and rotations: Exploring the extent to which multidimensional data yield univocal scale scores. Journal of Personality Assessment, 92, 544–559.CrossRefPubMedPubMedCentralGoogle Scholar
  25. Reise, S. P. (2012). The rediscovery of bifactor measurement models. Multivariate Behavioral Research, 47, 667–696.CrossRefPubMedPubMedCentralGoogle Scholar
  26. Schoenwald, S. K., Chapman, J. E., & Garland, A. F. (2014). Capturing fidelity in dissemination and implementation science. In R. S. Beidas & P. C. Kendall (Eds.), Dissemination and implementation of evidence-based practices in child and adolescent mental health (pp. 44–60). New York: Oxford University Press.Google Scholar
  27. Schoenwald, S. K., Sheidow, A. J., & Chapman, J. E. (2009). Clinical supervision in treatment transport: Effects on adherence and outcomes. Journal of Consulting and Clinical Psychology, 77, 410–421.CrossRefPubMedPubMedCentralGoogle Scholar
  28. Schumacker, R. E., & Smith, E. V, Jr. (2007). Reliability: A Rasch perspective. Educational and Psychological Measurement, 67, 394–409.CrossRefGoogle Scholar
  29. Shim, M. (2010). Factors influencing child welfare employee’s turnover: Focusing on organizational culture and climate. Children and Youth Services Review, 32, 847–856.CrossRefGoogle Scholar
  30. Smith, E. V, Jr. (2002). Detecting and evaluating the impact of multidimensionality using item fit statistics and principal component analysis of residuals. Journal of Applied Measurement, 3, 205–231.PubMedGoogle Scholar
  31. U.S. Department of Health and Human Services. (2014). Child welfare outcomes 2009–2012: Report to Congress. Washington, DC: U.S. Government Printing Office.Google Scholar
  32. Wright, B. D., & Masters, G. N. (1982). Rating scale analysis. Chicago: Institute for Objective Measurement.Google Scholar
  33. Wright, B. D., & Mok, M. (2000). Rasch models overview. Journal of Applied Measurement, 1, 83–106.PubMedGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2016

Authors and Affiliations

  1. 1.Oregon Social Learning CenterEugeneUSA

Personalised recommendations