Advertisement

Leveraging Technology to Facilitate Teachers’ Use of a Targeted Classroom Intervention: Evaluation of the Daily Report Card.Online (DRC.O) System

  • Julie Sarno OwensEmail author
  • John D. McLennan
  • Chelsea L. Hustus
  • Rebecca Haines-Saah
  • Sarah Mitchell
  • Clifton S. Mixon
  • Ayanna Troutman
Original Paper

Abstract

Historically, teachers’ uptake and implementation of empirically supported classroom interventions have involved substantial face-to-face consultation. However, most schools do not have the resources to provide this intensive level of support and many teachers may not need it. Thus, evaluation of alternative supports is warranted. In this pilot study, we evaluated the feasibility and effectiveness of an interactive web-based technology [i.e., the Daily Report Card.Online (DRC.O)] designed to facilitate teachers’ use of the DRC with minimal external support and examined individual teacher characteristics associated with DRC use. Elementary school teachers (N = 54) were given access to the DRC.O Web site and asked to use it to implement a DRC with one child. With regard to implementation, 16% were short-term adopters (less than 1 month) and 39% were long-term adopters (1–7 months of use). On average, short-term adopters adhered to 37% of data entry procedures, whereas long-term adopters adhered to 74% of data entry procedures. Higher teacher stress was associated with shorter use and lower adherence. Web site analytics revealed that, on average, long-term adopters completed all steps of DRC development in less than 1 h and spent only 3 min per day engaged in data entry for progress monitoring. The magnitude of change in student target behaviors and Strengths and Difficulties Questionnaire scores was moderate to large. These results reveal the feasibility and promise of the DRC.O and generate hypotheses for future research. Implications for additional evaluation of technology-driven implementation supports for teachers are discussed.

Keywords

Daily report card Technology Adoption Implementation Teachers Classroom management 

Notes

Acknowledgements

The authors express their gratitude to the educators, administrators, and families in the participating school districts for their collaboration.

Funding

This study was funded by the PolicyWise for Children & Families, Alberta, Canada (Grant No. 1609SM). The opinions expressed are those of the authors and do not represent views of PolicyWise.

Compliance with Ethical Standards

Conflict of interest

The first author received no financial benefit from the DRC.O program, but is the primary developer of the Web site. Other authors have no conflicts of interest to disclose.

Ethical Standard

Procedures were approved by the Institutional Review Boards at both universities and within all school districts. All procedures were performed in accordance with the ethical standards as laid down in the 1964 Declaration of Helsinki and its later amendments or comparable ethical standards.

Informed Consent

Informed consent was obtained from all individual participants included in the study.

References

  1. Aarons, G. A., Hurlburt, M., & Horwitz, S. M. (2011). Advancing a conceptual model of evidence-based practice implementation in public service sectors. Administration and Policy in Mental Health and Mental Health Services Research, 38(1), 4–23.CrossRefGoogle Scholar
  2. Alberta Government. (2013). A transformation in progress: Alberta’s K-12 education workforce 20112012. Retrieved at https://open.alberta.ca/publications/9781460106884 on December 8, 2018.
  3. Atkins, M. S., Frazier, S. L., Leathers, S. J., Graczyk, P. A., Talbott, E., Jakobsons, L., et al. (2008). Teacher key opinion leaders and mental health consultation in low-income urban schools. Journal of Consulting and Clinical Psychology, 76(5), 905–908.CrossRefGoogle Scholar
  4. Becker, K. D., Bohnenkamp, J., Domitrovich, C., Keperling, J. P., & Ialongo, N. S. (2014). Online training for teachers delivering evidence-based preventative interventions. School Mental Health, 6(4), 225–236.CrossRefGoogle Scholar
  5. Becker, K. D., Bradshaw, C. P., Domitrovich, C., & Ialongo, N. S. (2013). Coaching teachers to improve implementation of the good behavior game. Administration and Policy in Mental Health and Mental Health Services Research, 40, 482–493.CrossRefGoogle Scholar
  6. Benner, G. J., Kutash, K., Nelson, J. R., & Fisher, M. B. (2013). Closing the achievement gap of youth with emotional and behavioral disorders through multi-tiered systems of support. Education and Treatment of Children, 36, 15–29.CrossRefGoogle Scholar
  7. Bishop, D. C., Giles, S. M., & Bryant, K. S. (2005). Teacher receptiveness toward web-based training and support. Teaching and Teacher Education, 21(1), 3–14.CrossRefGoogle Scholar
  8. Canadian Teachers’ Federation. (2018). Teacher voices on work-life balance. Available from https://www.ctf-fce.ca/en/Pages/Issues/Teachers-Survey-Infographic.aspx. Retrieved on September 27, 2018.
  9. Carey, R., Kleiman, G., Russell, M., Douglas Venable, J. D., & Louie, J. (2008). Online courses for math teachers: Comparing self-paced and facilitated cohort approaches. The Journal of Technology, Learning and Assessment, 7(3), 4–35.Google Scholar
  10. Coles, E. K., Owens, J. S., Serrano, V. J., Slavec, J., & Evans, S. W. (2015). From consultation to student outcomes: The role of teacher knowledge, skills, and beliefs in increasing integrity in classroom management strategies. School Mental Health, 7(1), 34–48.CrossRefGoogle Scholar
  11. Collier-Meek, M. A., Sanetti, L. M., & Boyle, A. M. (2018). Barriers to implementing classroom management and behavior support plans: An exploratory investigation. Psychology in the Schools, 56, 5–17.CrossRefGoogle Scholar
  12. Conroy, M. A., Sutherland, K. S., Algina, J. J., Wilson, R. E., Martinez, J. R., & Whalon, K. J. (2015). Measuring teacher implementation of the BEST in CLASS intervention program and corollary child outcomes. Journal of Emotional and Behavioral Disorders, 23, 144–155.CrossRefGoogle Scholar
  13. Domitrovich, C. E., Pas, E. T., Bradshaw, C. P., Becker, K. D., Keperling, J. P., Embry, D. D., et al. (2015). Individual and school organizational factors that influence implementation of the PAX good behavior game intervention. Prevention Science, 16(8), 1064–1074.CrossRefGoogle Scholar
  14. Epstein, M., Atkins, M., Culinan, D., Kutash, K., & Weaver, R. (2008). Reducing behavior problems in the elementary school classroom. IES Practice Guide (NCEE 2008-012). U.S. Department of Education, National Center for Education Statistics. Washington, DC: U.S. Government Printing Office.Google Scholar
  15. Fabiano, G. A., Vujnovic, R., Pelham, W. E., Waschbusch, D. A., Massetti, G. M., Yu, J., et al. (2010). Enhancing the effectiveness of special education programming for children with ADHD using a daily report card. School Psychology Review, 39, 219–239.Google Scholar
  16. Farley-Ripple, E., & Buttram, J. (2015). The development of capacity for data use: The role of teacher networks in an elementary school. Teachers College Record, 117(4), 1–34.Google Scholar
  17. Fimian, M. J. (1988). Teacher stress inventory. Brandon, VT: Clinical Psychology Publishing Company.Google Scholar
  18. Fimian, M. J., & Krupicka, W. M. (1987). Occupational stress and receipt of professional counseling in special education. Perceptual and Motor Skills, 65, 995–999.CrossRefGoogle Scholar
  19. Forness, S. R., Freeman, S. F., Paparella, T., Kauffman, J. M., & Walker, H. M. (2012). Special education implications of point and cumulative prevalence for children with emotional or behavioral disorders. Journal of Emotional and Behavioral Disorders, 20(1), 4–18.CrossRefGoogle Scholar
  20. Frank, J. L., & Kratochwill, T. R. (2014). School-based problem-solving consultation. In W. P. Erchul & S. M. Sheridan (Eds.), Handbook of research in school consultation (2nd ed., pp. 18–39). New York: Routledge.Google Scholar
  21. Girio, E. L., & Owens, J. S. (2009). Teacher acceptability of evidence-based and promising treatments for children with attention-deficit/hyperactivity disorder. School Mental Health, 1, 16–25.  https://doi.org/10.1007/s12310-008-9001-6.CrossRefGoogle Scholar
  22. Goodman, R. (2001). Psychometric properties of the strengths and difficulties questionnaire. Journal of the American Academy of Child & Adolescent Psychiatry, 40(11), 1337–1345.CrossRefGoogle Scholar
  23. Hart, K. C., Fabiano, G. A., Evans, S. W., Manos, M. J., Hannah, J. N., & Vujnovic, R. K. (2017). Elementary and middle school teachers’ self-reported use of positive behavioral supports for children with ADHD: A national survey. Journal of Emotional and Behavioral Disorders, 25(4), 246–256.CrossRefGoogle Scholar
  24. Holdaway, A. S., Owens, J. S., Evans, S. W., Coles, E. K., Egan, T. E., & Himawan, L. K. (2018, October). Incremental benefits of a daily report card over time for youth with or at-risk for ADHD: Replication and extension. Poster presented at the Annual Conference on Advancing School Mental Health, Las Vegas, NV.Google Scholar
  25. Ingersoll, R. M. (2001). Teacher turnover and teacher shortages: An organizational analysis. American Educational Research Journal, 38(3), 499–534.CrossRefGoogle Scholar
  26. Kelly, M. S., & Lueck, C. (2011). Adopting a data-driven public health framework in schools: Results from a multi-disciplinary survey on school-based mental health practice. Advances in School Mental Health Promotion, 4(4), 5–12.CrossRefGoogle Scholar
  27. Kratochwill, T. R., Elliott, S. N., & Busse, R. T. (1995). Behavior consultation: A five-year evaluation of consultant and client outcomes. School Psychology Quarterly, 10(2), 87–117.CrossRefGoogle Scholar
  28. Long, A. C., Sanetti, L. M. H., Collier-Meek, M. A., Gallucci, J., Altschaefl, M., & Kratochwill, T. R. (2016). An exploratory investigation of teachers’ intervention planning and perceived implementation barriers. Journal of School Psychology, 55, 1–26.CrossRefGoogle Scholar
  29. Martinussen, R., Tannock, R., & Chaban, P. (2011). Teachers’ reported use of instructional and behavior management practices for students with behavior problems: Relationship to role and level of training in ADHD. Child & Youth Care Forum, 40(3), 193–210.CrossRefGoogle Scholar
  30. McLean, L., & Connor, C. M. (2015). Depressive symptoms in third-grade teachers: Relations to classroom quality and student achievement. Child Development, 86(3), 945–954.CrossRefGoogle Scholar
  31. McLeod, B. D., Sutherland, K. S., Martinez, R. G., Conroy, M. A., Snyder, P. A., & Southam-Gerow, M. A. (2017). Identifying common practice elements to improve social, emotional, and behavioral outcomes of young children in early childhood classrooms. Prevention Science, 18(2), 204–213.CrossRefGoogle Scholar
  32. Mixon, C. S., Owens, J. S., Hustus, C., Serrano, V. J., & Holdaway, A. S. (2019). Evaluating the impact of online professional development on teachers’ use of a targeted behavioral classroom intervention. School Mental Health, 11(1), 115–128.CrossRefGoogle Scholar
  33. Owens, J. S., Coles, E. K., & Evans, S. W. (2014, September). The role of teacher knowledge, skills, and beliefs in the implementation of classroom management skills. Presentation at the Annual School Mental Health Research Summit, Pittsburgh, PA.Google Scholar
  34. Owens, J. S., Coles, E. K., Evans, S. W., Himawan, L. K., Girio-Herrera, E., Holdaway, A. S., et al. (2017). Using multi-component consultation to increase the integrity with which teachers implement behavioral classroom interventions: A pilot study. School Mental Health, 9, 218–234.CrossRefGoogle Scholar
  35. Owens, J. S., Evans, S. W., Coles, E. K., Himawan, L. K., Holdaway, A. S., Mixon, C., et al. (2018). Consultation for classroom management and targeted interventions: examining benchmarks for teacher practices that produce desired change in student behavior. Journal of Emotional and Behavioral Disorders.  https://doi.org/10.1177/106342661879544.Google Scholar
  36. Owens, J. S., Holdaway, A. S., Zoromski, A. K., Evans, S. W., Himawan, L. K., Girio-Herrera, E., et al. (2012). Incremental benefits of a daily report card intervention over time for youth with disruptive behavior. Behavior Therapy, 43, 848–861.CrossRefGoogle Scholar
  37. Owens, J. S., Murphy, C. E., Richerson, L., Girio, E. L., & Himawan, L. K. (2008). Science to practice in underserved communities: The effectiveness of school mental health programming. Journal of Clinical Child and Adolescent Psychology, 37, 434–447.CrossRefGoogle Scholar
  38. Parker, R. I., Vannest, K. J., Davis, J. L., & Sauber, S. B. (2011). Combining nonoverlap and trend for single-case research: Tau-U. Behavior Therapy, 42(2), 284–299.CrossRefGoogle Scholar
  39. Pianta, R. C., Mashburn, A. J., Downer, J. T., Hamre, B. K., & Justice, L. (2008). Effects of web-mediated professional development resources on teacher-child interactions in pre-kindergarten classrooms. Early Childhood Research Quarterly, 23, 431–451.CrossRefGoogle Scholar
  40. Pyle, K., & Fabiano, G. A. (2017). Daily report card intervention and attention deficit hyperactivity disorder: A meta-analysis of single-case studies. Exceptional Children, 83(4), 378–395.Google Scholar
  41. Russell, M., Carey, R., Kleiman, G., & Venable, J. D. (2009). Face-to-face and online professional development for mathematics teachers: A comparative study. Journal of Asynchronous Learning Networks, 13(2), 71–87.Google Scholar
  42. Sanetti, L. M. H., & Kratochwill, T. R. (2009). Toward developing a science of treatment integrity: Introduction to the special series. School Psychology Review, 38(4), 445–459.Google Scholar
  43. Solomon, B. G., Klein, S. A., & Politylo, B. C. (2012). The effect of performance feedback on teachers’ treatment integrity: A meta-analysis of the single-case literature. School Psychology Review, 41(2), 160–175.Google Scholar
  44. Troutman, A. C., Owens, J. S., Monopoli, W. J., Evans, S. W., & Coles, E. C. (2018, October). School, teacher, and student factors that predict use of effective classroom management practices. Poster presented at the Annual Conference on Advancing School Mental Health, Las Vegas, NV.Google Scholar
  45. Vannest, K. J., Davis, J. L., Davis, C. R., Mason, B. A., & Burke, M. D. (2010). Effective intervention for behavior with a daily behavior report card: A meta-analysis. School Psychology Review, 39(4), 654–672.Google Scholar
  46. Vannest, K. J., & Ninci, J. (2015). Evaluating intervention effects in single-case research designs. Journal of Counseling & Development, 93(4), 403–411.CrossRefGoogle Scholar
  47. Wehby, J. H., Maggin, D. M., Partin, T. C. M., & Robertson, R. (2012). The impact of working alliance, social validity, and teacher burnout on implementation fidelity of the good behavior game. School Mental Health, 4(1), 22–33.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  • Julie Sarno Owens
    • 1
    Email author
  • John D. McLennan
    • 2
  • Chelsea L. Hustus
    • 1
  • Rebecca Haines-Saah
    • 3
  • Sarah Mitchell
    • 4
  • Clifton S. Mixon
    • 1
    • 6
  • Ayanna Troutman
    • 5
  1. 1.Ohio UniversityAthensUSA
  2. 2.CHEO-RIUniversity of CalgaryOttawaCanada
  3. 3.University of CalgaryCalgaryCanada
  4. 4.University of CalgaryCalgaryCanada
  5. 5.Spelman CollegeAtlantaUSA
  6. 6.Ochsner Hospital for ChildrenJeffersonUSA

Personalised recommendations