Skip to main content

Advertisement

SpringerLink
Log in

The observation of special education classrooms by school personnel

  • Original Paper
  • Published:
Learning Environments Research Aims and scope Submit manuscript

Abstract

Classroom observation is an accountability practice which promotes the evaluation of teachers’ capacity to meet standards, improve teaching practices, and enhance student learning outcomes. Prior research has revealed that these practices are not without bias: the reliability of observation can be challenged because of classroom or observer characteristics. Unlike general education learning environments, much less is known about the reliability of observing classrooms that service students with disabilities. In this study, we evaluated the reliability of special-education classroom observation by school personnel. Using a systematic scorer design modeled after the MET Project, we examined different combinations of observers (special educator, school leader, paraprofessional, and researcher) and lessons to test the impact of prior exposure to a teacher on scoring the RELATE Tool for Special Education Classroom Observation (RELATE). Analyses of RELATE scorer reliability, comparison of school personnel and researchers on their scores’ aggregates and variance, and the implications for generalization and special education classroom observation are discussed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  • Allen, J., Gregory, A., Mikami, A., Lun, J., Hamre, B., & Pianta, R. (2013). Observations of effective teacher–student interactions in secondary school classrooms: Predicting student achievement with the classroom assessment scoring system—Secondary. School Psychology Review, 42(1), 76–98.

    Article  Google Scholar 

  • Barnes, T., Cipriano, C., Flynn, L., Rivers, S., & Xu, W. (2019). Validating the Recognizing Excellence in Learning and Teaching (RELATE) Tool for special education classrooms. Journal of Experimental Education, 87(3), 415–429. https://doi.org/10.1080/00220973.2018.1465383

    Article  Google Scholar 

  • Bell, C. A., Gitomer, D. H., McCaffrey, D. F., Hamre, B. K., Pianta, R. C., & Qi, Y. (2012). An argument approach to observation protocol validity. Educational Assessment, 17(2–3), 62–87.

    Article  Google Scholar 

  • Berger, J. (2014). Word of mouth and interpersonal communication: A review and directions for future research. Journal of Consumer Psychology, 24(4), 586–607.

    Article  Google Scholar 

  • Billingsley, B. S. (2004). Special education teacher retention and attrition: A critical analysis of the research literature. The Journal of Special Education, 38, 39–55. https://doi.org/10.1177/00224669040380010401

    Article  Google Scholar 

  • Borko, H. (2004). Professional development and teacher learning: Mapping the terrain. Educational Researcher, 33(8), 3–15. https://doi.org/10.3102/0013189X033008003

    Article  Google Scholar 

  • Brophy, J. (2003). Teaching problem students. Guilford Press.

    Google Scholar 

  • Brownell, M. T., & Jones, N. D. (2015). Teacher evaluation in special education: Approaches, supporting research, and challenges confronted. Journal of Special Education Leadership, 28, 63–73.

    Google Scholar 

  • Bush, G. W. (2001). No child left behind: Executive summary. U.S. Department of Education.

    Google Scholar 

  • Casabianca, J. M., Lockwood, J. R., & McCaffrey, D. F. (2015). Trends in classroom observation scores. Educational and Psychological Measurement, 75(2), 311–337.

    Article  Google Scholar 

  • Cavalier, A. R., Ferretti, R. P., & Hodges, A. E. (1997). Self-management within a classroom token economy for students with learning disabilities. Research in Developmental Disabilities, 18, 167–178. https://doi.org/10.1016/S0891-4222(96)00045-5

    Article  Google Scholar 

  • Chambers, D. (2015). Working with teaching assistants and other support staff for inclusive education. Emerald Group Publishing Limited.

    Book  Google Scholar 

  • Chang, L. (1994). A psychometric evaluation of 4-point and 6-point Likert-type scales in relation to reliability and validity. Applied Psychological Measurement, 18, 205–216.

    Article  Google Scholar 

  • Cicchetti, D. V., Showalter, D., & Tyrer, P. J. (1985). The effect of number of rating scale categories on levels of interater reliability: A Monte Carlo investigation. Applied Psychological Measurement, 9, 31–36.

    Article  Google Scholar 

  • Cipriano, C., Barnes, T., Bertoli, M., & Rivers, S. (2018). Applicability of the classroom assessment scoring system in special education classrooms serving students with emotional behavioural disorders. Emotional and Behavioural Difficulties, 23(4), 343–360. https://doi.org/10.1080/13632752.2018.1461454

    Article  Google Scholar 

  • Clarke, L. S., Embury, D. C., Jones, R. E., & Yssel, N. (2014). Supporting students with disabilities during school crises: A teacher’s guide. Teaching Exceptional Children, 46, 169–178. https://doi.org/10.1177/0040059914534616

    Article  Google Scholar 

  • Council for Exceptional Children. (2015). What every special educator must know: Professional ethics and standards. Author.

    Google Scholar 

  • Crowe, C., Rivers, S., & Bertoli, M. (2017). Mind the gap: Accountability, observation, and special education. Assessment in Education: Principles, Policy, and Practice, 24(1), 21–43. https://doi.org/10.1080/0969594X.2015.1114913

    Article  Google Scholar 

  • Danielson, C., & McGreal, T. L. (2000). Teacher evaluation to enhance professional practice. Association for Supervision and Curriculum Development.

    Google Scholar 

  • Danielson, C. (2012). Observing classroom practice. Educational Leadership, 70(3), 32–37.

    Google Scholar 

  • Desimone, L. M. (2009). Improving impact studies of teachers’ professional development: Toward better conceptualizations and measures. Educational Researcher, 38, 181–199. https://doi.org/10.3102/0013189X08331140

    Article  Google Scholar 

  • Ferguson, M. (2016). Washington view ESSA as more than the latest acronym on education’s block. Phi Delta Kappan, 97(6), 72–73. https://doi.org/10.1177/0031721716636879

    Article  Google Scholar 

  • Ferguson, E., & Cox, T. (1993). Exploratory factor analysis: A users’ guide. International Journal of Selection and Assessment, 1, 84–94. https://doi.org/10.1111/j.1468-2389.1993.tb00092

    Article  Google Scholar 

  • Fisher, M., & Pleasants, S. L. (2012). Roles, responsibilities, and concerns of paraeducators: Findings from a statewide survey. Remedial and Special Education, 33(5), 287–297.

    Article  Google Scholar 

  • Friend, M., Cook, L., Hurley-Chamberlain, D. A., & Shamberger, C. (2010). Coteaching: Anillustrationofthecomplexityofcollaborationinspecialeducation. Journal of Educational and Psychological Consultation, 20(1), 9e27. https://doi.org/10.1080/10474410903535380

    Article  Google Scholar 

  • Gabriel, R. (2015). Not whether, but how: Asking the right questions in teacher performance assessment. Language Arts, 93(2), 120.

    Google Scholar 

  • Gage, N. A., Lierheimer, K. S., & Goran, L. G. (2012). Characteristics of students with high incidence disabilities broadly defined. Journal of Disability Policy Studies, 23, 168–178. https://doi.org/10.1177/1044207311425385

    Article  Google Scholar 

  • Gal, E., Schreur, N., & Engel-Yeger, B. (2010). Inclusion of children with disabilities: Teachers’ attitudes and requirements for environmental accommodations. International Journal of Special Education, 25, 89–99.

    Google Scholar 

  • Ghere, G., & York-Barr, J. (2007). Paraprofessional turnover and retention in inclusive programs: Hidden costs and promising practices. Remedial and Special Education, 28(1), 21–32.

    Article  Google Scholar 

  • Giangreco, M. F., Suter, J. C., & Doyle, M. B. (2010). Paraprofessionals in inclusive schools: A review of recent research. Journal of Educational and Psychological Consultation, 20, 41–57. https://doi.org/10.1080/10474410903535356

    Article  Google Scholar 

  • Hampden-Thompson, G., Diehl, J., & Kinukawa, A. (2007). Description and employment criteria of instructional paraprofessionals (No. NCES 2007-008). National Center for Educational Statistics.

    Google Scholar 

  • Haynes, M. (2014). Onthepathtoequity:Improvingtheeffectivenessof beginningteachers. Alliance for Excellent Education.

    Google Scholar 

  • Hickey, D. T., & Schafer, N. J. (2006). Design-based, participation-centered approaches to classroom management. In C. M. Evertson & C. S. Weinstein (Eds.), Handbook of classroom management: Research, practice, and contemporary issues (pp. 281–308). Lawrence Erlbaum Associates Inc.

    Google Scholar 

  • Ho, A. D., & Kane, T. J. (2013). The reliability of classroom observations by school personnel. Seattle, WA: Measures of Effective Teaching Project, Bill and Melinda Gates Foundation.

    Google Scholar 

  • Idol, L. (2006). Toward inclusion of special education students in general education: A program evaluation of eight schools. Remedial and Special Education, 27(2), 77–94.

    Article  Google Scholar 

  • Joe, J. N., Tocci, C. M., Holtzman, S. L., & Williams, J. C. (2013). Foundations of observation: Considerations for developing a classroom observation system that helps districts achieve consistent and accurate scores (policy and practice brief). MET Project, Bill & Melinda Gates Foundation.

    Google Scholar 

  • Johnson, E. S., & Semmelroth, C. L. (2012). Examining interrater agreement analyses of a pilot special education observation tool. Journal of Special Education Apprenticeship, 1, 1–19.

    Google Scholar 

  • Jones, N. D. (2015). Using test scores from students with disabilities in teacher evaluation. Educational Measurement: Issues and Practice, 34, 28–38.

    Article  Google Scholar 

  • Jones, N. D., Brownell, M. T., & Bell, C. (2015). Validating an observation protocol for the evaluation of special educators. Retrieved from https://ies.ed.gov/funding/grantsearch/details.asp?ID=1631.

  • Kane, T. J., McCaffrey, D. F., Miller, T., & Staiger, D. O. (2013). Have we identified effective teachers? Validating measures of effective teaching using random assignment (research paper). MET Project, Bill & Melinda Gates Foundation.

    Google Scholar 

  • Kane, T. J., & Staiger, D. O. (2012). Gathering feedback for teaching. Seattle, WA: Measures of Effective Teaching Project, Bill and Melinda Gates Foundation.

    Google Scholar 

  • Lawson, J. E., & Cruz, R. A. (2018). Evaluating special educators’ classroom performance: Does rater “type” matter? Assessment for Effective Intervention, 43(4), 227–240.

    Article  Google Scholar 

  • Mastropieri, M. A., & Scruggs, T. E. (2006). The inclusive classroom: Strategies for effective instruction (3rd ed.). Prentice-Hall.

    Google Scholar 

  • Milanowski, A. (2017). Lower performance evaluation practice ratings for teachers of disadvantaged students: Bias or reflection of reality? AERA Open, 3(1), 2332858416685550.

    Article  Google Scholar 

  • National Center for Special Education Research. (2006). Facts from NLTS2: School behavior and disciplinary experiences of youth with disabilities. Retrieved January 4, 2013, from www.nlts2.org/fact_sheets/nlts2_fact_sheet_2006_03.pdf.

  • National Education Association. (2018). ESP careers: Paraeducators. Retrieved from http://www.nea.org/home/64522.htm.

  • Newman, L., Wagner, M., Huang, T., Shaver, D., Knokey, A. M., Yu, J., & Cameto, R. (2011a). Secondaryschoolprograms andperformanceof studentswith disabilities:A special topic report of findings from the National Longitudinal Transition Study-2(NLTS2). National Center for Special Education Research.

    Google Scholar 

  • Newman, L., Wagner, M., Knokey, A. M., Marder, C., Nagle, K., Shaver, D., & Wei, X. (2011b). The post-high school outcomes of young adults with disabilities up to 8 years after high school: A report from the National Longitudinal Transition Study-2 (NLTS2) (NCSER 2011-3005). National Center for Special Education Research.

    Google Scholar 

  • Nichols, A. S., & Sosnowsky, F. L. P. (2002). Burnout among special education teachers in self-contained cross-categorical classrooms. Teacher Education and Special Education, 25(1), 71–86.

    Article  Google Scholar 

  • Pianta, R. C., & Hamre, B. K. (2009). Conceptualization, measurement, and improvement of classroom processes: Standardized observation can leverage capacity. Educational Researcher, 38(2), 109–119. https://doi.org/10.3102/0013189X09332374

    Article  Google Scholar 

  • Pickett, A. L., Likins, M., & Wallace, T. (2003). The employment and preparation of paraeducators: The state of the art—2003. Retrieved from http://www.nrcpara.org/sites/default/files/state-of-the-art.pdf.

  • Reddy, L. A., Fabiano, G. A., & Jimerson, S. R. (2013). Assessment of general education teachers’ Tier 1 classroom practices: Contemporary science, practice, and policy. School Psychology Quarterly, 28(4), 273.

    Article  Google Scholar 

  • Ronfeldt, M., Loeb, S., & Wyckoff, J. (2013). How teacher turnover harms student achievement. American Educational Research Journal, 50(1), 4–36. https://doi.org/10.3102/0002831212463813

    Article  Google Scholar 

  • Semmelroth, C. L., & Johnson, E. (2014). Measuring rater reliability on a special education observation tool. Assessment for Effective Intervention, 39, 131–145. https://doi.org/10.1177/1534508413511488

    Article  Google Scholar 

  • Shavelson, R. J., & Webb, N. M. (1991). Generalizability theory: A primer. Sage.

    Google Scholar 

  • Tabachnick, B. G., & Fidell, L. S. (2007). Using multivariate statistics. Allyn & Bacon/Pearson Education.

    Google Scholar 

  • Tindal, G., Heath, B., Hollenbeck, K., Almond, P., & Harniss, M. (1998). Accommodating students with disabilities on large-scale tests: An experimental study. Exceptional Children, 64, 439–450. https://doi.org/10.1177/001440299806400401

    Article  Google Scholar 

  • US Department of Education. (2015). Percentage distribution of students 6 to 21 years old served under Individuals with Disabilities Education Act (IDEA), Part B, by educational environment and type of disability: Selected years, fall 1989 through fall 2013 (Table 204.60). Retrieved from http://www2.ed.gov/programs/osepidea/618-data/state-leveldatafiles/index.html#bcc

  • Walsh, S. (2014). What’s happening in DC? UCONN Center for Excellence in Developmental Disabilities. Storrs, CT: Lecture conducted from University of Connecticut, March 2014.

  • Webber, J., Scheuermann, B., McCall, C., & Coleman, M. (1993). Research on self-monitoring as a behavior management technique in special education classrooms: A descriptive review. Remedial and Special Education, 14, 38–56. https://doi.org/10.1177/074193259301400206

    Article  Google Scholar 

  • Wragg, T. (2002). An introduction to classroom observation (2nd ed.). Routledge.

    Book  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Yale Child Study Center, Yale School of Medicine, 350 George Street, New Haven, CT, 06520-8376, USA

    Christina Cipriano

  2. Department of Human Development and Family Sciences, University of Delaware, 111 Alison Hall West, Newark, DE, 19716, USA

    Tia N. Barnes

Authors
  1. Christina Cipriano
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tia N. Barnes
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Christina Cipriano.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Cipriano, C., Barnes, T.N. The observation of special education classrooms by school personnel. Learning Environ Res 25, 235–253 (2022). https://doi.org/10.1007/s10984-021-09359-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10984-021-09359-w

Keywords

Search

Navigation