Abstract
The present study used a community partnered research method to develop and pilot a classroom-focused measurement feedback system (MFS) for school mental health providers to support teachers’ use of effective universal and target classroom practices related to student emotional and behavioral issues. School personnel from seven urban elementary and middle school classrooms participated. Phase I involved development and refinement of the system through a baseline needs assessment and rapid-cycle feedback. Phase II involved detailed case study analysis of pre-to-post quantitative and implementation process data. Results suggest that teachers who used the dashboard along with consultation showed improvement in observed classroom organization and emotional support. Results also suggest that MFS use was tied closely to consultation dose, and that broader support at the school level was critical. Classroom-focused MFSs are a promising tool to support classroom improvement, and warrant future research focused on their effectiveness and broad applicability.
Similar content being viewed by others
References
Aarons, G. A., Hurlburt, M., & Horwitz, S. M. C. (2011). Advancing a conceptual model of evidence-based practice implementation in public service sectors. Administration and Policy in Mental Health and Mental Health Services Research, 38(1), 4–23.
APA Presidential Task Force on Evidence-Based Practice. (2006). Evidence-based practice in psychology. American Psychologist, 61(4), 271–285.
Bambrick-Santoyo, B. (2010). Driven by Data: A practical guide to improve instruction. San Francisco: Jossey-Bass.
Becker, K. D., Bradshaw, C. P., Domitrovich, C., & Ialongo, N. S. (2013a). Coaching teachers to improve implementation of the good behavior game. Administration and Policy in Mental Health and Mental Health Services Research, 40(6), 482–493.
Becker, K. D., Darney, D., Domitrovich, C., Keperling, J. P., & Ialongo, N. S. (2013b). Supporting universal prevention programs: A two-phased coaching model. Clinical Child and Family Psychology Review, 16(2), 213–228.
Berwick, D. M. (1989). Continuous improvement as an ideal in health care. New England Journal of Medicine, 320(1), 53–56.
Bickman, L., Kelley, S. D., Breda, C., de Andrade, A. R., & Riemer, M. (2011). Effects of routine feedback to clinicians on mental health outcomes of youths: results of a randomized trial. Psychiatric Services, 62(12), 1423–1429.
Borntrager, C., & Lyon, A. R. (2014). Client progress monitoring and feedback in school-based mental health. Cognitive and Behavioral Practice, 22, 74–88.
Bourdon, K. H., Goodman, R., Rae, D. S., Simpson, G., & Koretz, D. S. (2005). The Strengths and Difficulties Questionnaire: U.S. normative data and psychometric properties. Journal of the American Academy of Child and Adolescent Psychiatry, 44, 557–564.
Brown, J. L., Jones, S. M., LaRusso, M. D., & Aber, J. L. (2010). Improving classroom quality: Teacher influences and experimental impacts of the 4Rs program. Journal of Educational Psychology, 102(1), 153.
Bruns, E. J., & Hoagwood, K. E. (2008). State implementation of evidence-based practice for youths, Part I: Responses to the state of the evidence. Journal of the American Academy of Child and Adolescent Psychiatry, 47(4), 369–373.
Bruns, E. J., Hoagwood, K. E., & Hamilton, J. D. (2008). State implementation of evidence-based practice for youths, part I: Responses to the state of the evidence. Journal of the Academy of Child and Adolescent Psychiatry, 47(4), 369–373.
Bryk, A. S., Gomez, L., & Grunow, A. (2010). Getting ideas into action: Building networked improvement communities in education. Retrieved August 21, 2013, from http://www.carnegiefoundation.org/spotlight/webinar-bryk-gomez-building-networkedimprovement-communities-in-education.
Bryk, A. S., & Schneider, B. L. (2002). Trust in schools: A core resource for improvement. New York: Russell Sage.
Buntin, M. B., Burke, M. F., Hoaglin, M. C., & Blumenthal, D. (2011). The benefits of health information technology: a review of the recent literature shows predominantly positive results. Health Affairs, 30(3), 464–471.
Burchinal, M., Vandergrift, N., Pianta, R., & Mashburn, A. (2010). Threshold analysis of association between child care quality and child outcomes for low-income children in pre-kindergarten programs. Early Childhood Research Quarterly, 25(2), 166–176.
Cappella, E., Hamre, B. K., Kim, H. Y., Henry, D. B., Frazier, S. L., Atkins, M. S., & Schoenwald, S. K. (2012). Teacher consultation and coaching within mental health practice: Classroom and child effects in urban elementary schools. Journal of Consulting and Clinical Psychology, 80(4), 597–610.
Chorpita, B. F., Bernstein, A., & Daleiden, E. L. (2008). Driving with roadmaps and dashboards: Using information resources to structure the decision models in service organizations. Administration and Policy in Mental Health and Mental Health Services Research, 35(1–2), 114–123.
Chorpita, B. F., & Daleiden, E. L. (2009). Mapping evidence-based treatments for children and adolescents: Application of the distillation and matching model to 615 treatments from 322 randomized trials. Journal of Consulting and Clinical Psychology, 77(3), 566.
Cresswell, K., & Sheikh, A. (2013). Organizational issues in the implementation and adoption of health information technology innovations: An interpretative review. International journal of medical informatics, 82(5), e73–e86.
Deming, W. E. (1986). Out of the crisis. Cambridge: MIT Press.
DiGennaro, F. D., Martens, B. K., & Kleinmann, A. E. (2007). A comparison of performance feedback procedures on teachers’ treatment implementton integrity and students’ inappropriate behavior in special education classrooms. Journal of Applied Behavior Analysis, 40(3), 447–461.
Domitrovich, C. E., Bradshaw, C. P., Poduska, J. M., Hoagwood, K., Buckley, J. A., Olin, S., & Ialongo, N. S. (2008). Maximizing the implementation quality of evidence-based preventive interventions in schools: A conceptual framework. Advances in School Mental Health Promotion, 1(3), 6–27.
Domitrovich, C. E., Cortes, R. C., & Greenberg, M. T. (2007). Improving young children’s social and emotional competence: A randomized trial of the preschool “PATHS” curriculum. The Journal of Primary Prevention, 28(2), 67–91.
Duchaine, E. L., Jolivette, K., & Fredrick, L. D. (2011). The effect of teacher coaching with performance feedback on behavior-specific praise in inclusion classrooms. Education and Treatment of Children, 34(2), 209–227.
Duhon, G. J., Mesmer, E. M., Gregerson, L., & Witt, J. C. (2009). Effects of public feedback during RTI team meetings on teacher implementation integrity and student academic performance. Journal of School Psychology, 47(1), 19–37.
Duncan, K., & Pozehl, B. (2000). Effects of performance feedback on patient pain outcomes. Clinical Nursing Research, 9(4), 379–397.
Embry, D. D. (2002). The Good Behavior Game: A best practice candidate as a universal behavioral vaccine. Clinical Child and Family Psychology Review, 5(4), 273–297.
Evertson, C. M., & Weinstein, C. S. (2006). Handbook of classroom management: Research, practice, and contemporary issues. Mahwah: Lawrence Erlbaum Associates Publishers.
Fixsen, D., Blase, K., Metz, A., & Van Dyke, M. (2013). Statewide implementation of evidence-based programs. Exceptional Children, 79(2), 213–230.
Gamoran, A., Secada, W. G., & Marrett, C. B. (2000). The organizational context of teaching and learning. In M. Hallinan (Ed.), Handbook of the Sociology of Education (pp. 37–63). USA: Springer.
Goodman, R. (1999). The extended version of the strengths and difficulties questionnaire as a guide to child psychiatric caseness and consequent burden. Journal of Child Psychology and Psychiatry, 40, 791–801.
Goodman, R., Meltzer, H., & Bailey, V. (1998). The strengths and difficulties questionnaire: A pilot study on the validity of the self-report version. European Child and Adolescent Psychiatry, 7, 125–130.
Goodman, R., & Scott, S. (1999). Comparing the Strengths and Difficulties Questionnaire and the Child Behavior Checklist: Is small beautiful? Journal of Abnormal Child Psychology, 27, 17–24.
Hamre, B. K., Pianta, R. C., Downer, J. T., DeCoster, J., Mashburn, A. J., Jones, S. M., & Hamagami, A. (2013). Teaching though interactions: Testing a developmental framework of teacher effectiveness across 4000 classrooms. Elementary School Journal, 113(4), 461–487.
Hawkins, S. M., & Heflin, L. J. (2010). Increasing secondary teachers’ behavior-specific praise using a video self-modeling and visual performance feedback intervention. Journal of Positive Behavior Interventions, 13(2), 97–108.
Jensen-Doss, A., & Hawley, K. M. (2010). Understanding barriers to evidence-based assessment: Clinician attitudes toward standardized assessment tools. Journal of Clinical Child & Adolescent Psychology, 39(6), 885–896.
Juran, J. M. (1951). Quality control handbook. New York: McGraw-Hill.
Kellermann, A. L., & Jones, S. S. (2013). What it will take to achieve the as-yet-unfulfilled promises of health information technology. Health Affairs, 32(1), 63–68.
Lambert, M. J., Hansen, N. B., & Finch, A. E. (2001a). Patient-focused research: Using patient outcome data to enhance treatment effects. Journal of Consulting and Clinical Psychology, 69(2), 159.
Lambert, M. J., Harmon, C., Slade, K., Whipple, J. L., & Hawkins, E. J. (2005). Providing feedback to psychotherapists on their patients’ progress: clinical results and practice suggestions. Journal of Clinical Psychology, 61(2), 165–174.
Lambert, M. J., Whipple, J. L., Smart, D. W., Vermeersch, D. A., & Nielsen, S. L. (2001b). The effects of providing therapists with feedback on patient progress during psychotherapy: Are outcomes enhanced? Psychotherapy Research, 11(1), 49–68.
Mashburn, A. J., Pianta, R. C., Hamre, B. K., Downer, J. T., Barbarin, O. A., Bryant, D., & Howes, C. (2008). Measures of classroom quality in prekindergarten and children’s development of academic, language, and social skills. Child Development, 79(3), 732–749.
National Council on Teacher Quality. (2012). State of the States 2012: Teacher Effectiveness Policies. NCTQ State Teacher Policy Yearbook Brief. Washington, DC: National Council on Teacher Quality.
Office of Head Start. (2014). USe of the Classrom Assessment Scoring System (CLASS) in Head Start Retrieved October 1, 2014, from http://eclkc.ohs.acf.hhs.gov/hslc/tta-system/teaching/eecd/Assessment/Classroom.Assessment.Scoring.System%28CLASS%29/ClassroomAssessm.htm.
Owens, J. S., Lyon, A. R., Brandt, N. E., Warner, C. M., Nadeem, E., Spiel, C., & Wagner, M. (2014). Implementation science in school mental health: Key constructs in a developing research Agenda. School Mental Health, 6(2), 99–111.
Park, S., Hironaka, S., Carver, P., & Nordstrum, L. (2013). Continuous improvement in education. Carnegie Foundation for the Advancement of Teaching.
Pianta, R. C., La Paro, K. M., & Hamre, B. K. (2008a). Classroom assessment scoring system. Paul H. Brookes Publishing Company.
Pianta, R. C., Mashburn, A. J., Downer, J. T., Hamre, B. K., & Justice, L. (2008b). Effects of web-mediated professional development resources on teacher–child interactions in pre-kindergarten classrooms. Early Childhood Research Quarterly, 23(4), 431–451.
PracticeWise. (2014). PracticeWise. Retrieved September 15, 2014, from http://www.practicewise.com.
Reddy, L. A., Newman, E., De Thomas, C. A., & Chun, V. (2009). Effectiveness of school-based prevention and intervention programs for children and adolescents with emotional disturbance: A meta-analysis. Journal of School Psychology, 47(2), 77–99.
Reinke, W. M., Lewis-Palmer, T., & Martin, E. (2007). The effect of visual performance feedback on teacher use of behavior-specific praise. Behavior Modification, 31(3), 247–263.
Reinke, W. M., Stormont, M., Webster-Stratton, C., Newcomer, L. L., & Herman, K. C. (2012). The Incredible Years teacher classroom management program: Using coaching to support generalization to real-world classroom settings. Psychology in the Schools, 49, 416–428.
Rose, D. J., & Church, R. J. (1998). Learning to teach: The acquisition and maintenance of teaching skills. Journal of Behavioral Education, 8(1), 5–35.
Singer, M. I., Anglin, T. M., Song, L., & Lunghofer, L. (1995). Adolescents’ exposure to violence and associated symptoms of psychological trauma. JAMA, 273(6), 477–482
Stamm, B. H. (2010). The concise ProQOL manual (Vol. 2007). Pocatello: ProQOL. org.
State of New Jersey Department of Education. (2013). AchieveNJ Retrieved August 21, 2013, from http://www.state.nj.us/education/AchieveNJ/.
Stecker, P. M., Lembke, E. S., & Foegen, A. (2008). Using progress-monitoring data to improve instructional decision making. Preventing School Failure: Alternative Education for Children and Youth, 52(2), 48–58.
Tuckman, B. W., & Yates, D. (1980). Evaluating the student feedback strategy for changing teacher style. The Journal of Educational Research, 74(2), 74–77.
U.S. Department of Health and Human Services. (2003). New Freedom Commission on Mental Health: Achieving the Promise: Transforming Mental Health Care in America. Final Report. In DHHS (Ed.). Rockville, MD: Department of Health and Human Services.
Wells, K. B., Staunton, A., Norris, K. C., Bluthenthal, R., Chung, B., Gelberg, L., & Wong, M. (2006). Building an academic-community partnered network for clinical services research: the Community Health Improvement Collaborative (CHIC). Ethnicity and Disease, 16(1 Suppl 1), S3–S17.
Yin, R. K. (2009). Case Study Research: Design and Methods (4th ed.). Thousand Oaks: Sage Publications.
Yoon, J. S. (2002). Teacher characteristics as predictors of teacher-student relationships: Stress, negative affect, and self-efficacy. Social Behavior and Personality: An International Journal, 30(5), 485–493.
Ysseldyke, J., & Tardrew, S. (2007). Use of a progress monitoring system to enable teachers to differentiate mathematics instruction. Journal of Applied School Psychology, 24(1), 1–28.
Acknowledgments
Writing of this paper was supported by NIMH 5K01MH83694 (EN). The authors would like to acknowledge the contributions of all of our school district collaborators on this project, and the support of the PracticeWise, LLC team in the development of this manuscript.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Nadeem, E., Cappella, E., Holland, S. et al. Development and Piloting of a Classroom-Focused Measurement Feedback System. Adm Policy Ment Health 43, 379–393 (2016). https://doi.org/10.1007/s10488-015-0651-z
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10488-015-0651-z