# Classroom observation and mathematics education research

- 156 Downloads
- 1 Citations

## Abstract

Classroom observations have become an integral part of research related to mathematics education. In this qualitative study, we describe the current state of the mathematics education field with regard to the use of classroom observation. The research question was: *How is classroom observation being used to measure instructional quality in mathematics education research*? In all, 114 peer-reviewed manuscripts published between 2000 and 2015 that involved classroom observation as part of an empirical study were examined using a cross-comparative methodology. Seventy (61%) did not use a formalized classroom observation protocol (COP), 21 (18%) developed their own COP, and 23 (20%) used a previously developed COP. Of the implemented COPs, 44% have published validity evidence in a peer-reviewed journal. We perceive the great variety of research approaches for classroom observation as necessary and potentially challenging in moving mathematics education forward with respect to research on instructional contexts.

## Keywords

Classroom observation Instruction Qualitative Validity## Notes

### Acknowledgements

We would like to share our sincere appreciation to Timothy Folger, Maria Nielsen, and Davis Gerber at Bowling Green State University, and Dan Chibnall at Drake University for their assistance throughout this project.

## References

- American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (2014).
*Standards for educational and psychological testing*. Washington, DC: American Educational Research Association.Google Scholar - American Educational Research Association, American Psychological Association, National Council on Measurement in Education, Joint Committee on Standards for Educational and Psychological Testing (US). (1999).
*Standards for educational and psychological testing*. Washington, DC: American Educational Research Association.Google Scholar - Arbaugh, F., Lannin, J., Jones, D. L., & Park-Rogers, M. (2006). Examining instructional practices in Core-Plus lessons: Implications for professional development.
*Journal of Mathematics Teacher Education,**9*(6), 517–550.CrossRefGoogle Scholar - Baker, J. A. (1999). Teacher-student interaction in urban at-risk classrooms: Differential behavior, relationship quality, and student satisfaction with school.
*The Elementary School Journal,**100*(1), 57–70.CrossRefGoogle Scholar - Ball, D. L., & Rowan, B. (2004). Introduction: Measuring instruction.
*The Elementary School Journal,**105,*3–10.CrossRefGoogle Scholar - Berry, III, R. Q., Rimm-Kaufman, S. E., Ottmar, E. M., Walkowiak, T. A., & Merritt, E. (2010).
*The Mathematics Scan (M*-*Scan): A measure of mathematics instructional quality*. Unpublished measure, University of Virginia.Google Scholar - Blank, R. K., Porter, A., & Smithson, J. (2001).
*New tools for analyzing teaching, curriculum and standards in Mathematics & Science. Results from Survey of Enacted Curriculum Project. Final Report*. Council of Chief State School Officers, Attn: Publications, One Massachusetts Avenue, NW, Suite 700, Washington, DC 20001-1431.Google Scholar - Borasi, R., Fonzi, J., Smith, C., & Rose, B. J. (1999). Beginning the process of rethinking mathematics instruction: A professional development program.
*Journal of Mathematics Teacher Education,**2,*49–78.CrossRefGoogle Scholar - Bostic, J. (2017). Moving forward: Instruments and opportunities for aligning current practices with testing standards.
*Investigations in Mathematics Learning,**9*(3), 109–110.CrossRefGoogle Scholar - Bostic, J. (2018). Improving test development reporting practices. In L. Venenciano & A. Sanogo (Eds.),
*Proceedings of the 45th Annual Meeting of the Research Council on Mathematics Learning*(pp. 57–64). Baton Rouge, LA.Google Scholar - Bostic, J., Lesseig, K., Sherman, M., & Boston, M. (2017).
*Classroom observation protocols: Choose your own tool*. Research report presented at the National Council of Teachers of Mathematics Research Conference, San Antonio, TX.Google Scholar - Bostic, J., Matney, G., & Sondergeld, T. (2019). A lens on teachers’ promotion of the Standards for Mathematical Practice.
*Investigations in Mathematics Learning,**11*(1), 69–82.CrossRefGoogle Scholar - Boston, M. D. (2012a). Assessing the quality of mathematics instruction.
*Elementary School Journal,**113,*76–104.CrossRefGoogle Scholar - Boston, M. (2012b). Assessing instructional quality in mathematics.
*The Elementary School Journal,**113*(1), 76–104.CrossRefGoogle Scholar - Boston, M., Bostic, J., Lesseig, K., & Sherman, M. (2015a). A comparison of mathematics classroom observation protocols.
*Mathematics Teacher Educator,**3,*154–175.CrossRefGoogle Scholar - Boston, M., Bostic, J., Lesseig, K., & Sherman, M. (2015b). Classroom Observation tools to support the work of mathematics teacher educators.
*Invited Manuscript for Mathematics Teacher Educator,**3,*154–175.CrossRefGoogle Scholar - Boston, M. D., & Smith, M. S. (2009). Transforming secondary mathematics teaching: Increasing the cognitive demands of instructional tasks used in teachers’ classrooms.
*Journal for Research in Mathematics Education,**40,*119–156.Google Scholar - Boston, M. D., & Wilhelm, A. G. (2015). Middle school mathematics instruction in instructionally-focused urban districts.
*Urban Education,**52*(7), 829–861.CrossRefGoogle Scholar - Briars, D. J., & Resnick, L. B. (2000).
*Standards, assessments… and what else? The essential elements of standards*-*based school improvement*. Center for the Study of Evaluation, National Center for Research on Evaluation, Standards, and Student Testing, Graduate School of Education & Information Studies, University of California, Los Angeles.Google Scholar - Brigham Young University Department of Mathematics Education. (2008). Report on venue study. Retrieved from https://nctm.confex.com/nctm/…/BYU%20Study%20for%20Journal%20Rankings.pdf.
- Brophy, J. (1986). Teacher influences on student achievement.
*American Psychologist,**41,*1069.CrossRefGoogle Scholar - Brown, J. C., & Crippen, K. J. (2016). The growing awareness inventory: Building capacity for culturally responsive science and mathematics with a structured observation protocol.
*School Science and Mathematics,**116*(3), 127–138.CrossRefGoogle Scholar - Capobianco, B. M., & Rupp, M. (2014). STEM teachers’ planned and enacted attempts at implementing engineering design-based instruction.
*School Science and Mathematics,**114*(6), 258–270.CrossRefGoogle Scholar - Charalambous, C. Y., & Praetorius, A. K. (2018). Studying mathematics instruction through different lenses: Setting the ground for understanding instructional quality more comprehensively.
*ZDM Mathematics Education,**50,*355–366. https://doi.org/10.1007/s11858-018-0914-8.CrossRefGoogle Scholar - Danielson, C. (2013).
*The framework for teaching: Evaluation instrument*. Princeton, NJ: Danielson Group.Google Scholar - Doabler, C. T., Baker, S. K., Kosty, D. B., Smolkowski, K., Clarke, B., Miller, S. J., et al. (2015). Examining the association between explicit mathematics instruction and student mathematics achievement.
*The Elementary School Journal,**115*(3), 303–333.CrossRefGoogle Scholar - Donovan, M. S., & Bransford, J. D. (2005).
*How students learn: History, mathematics, and science in the classroom*. Committee on How People Learn: A Targeted Report for Teachers National Research Council. Washington, DC: National Academies Press.Google Scholar - Farmer, J. D., Gerretston, H., & Lassak, M. (2003). What teachers take from professional development: Cases and implications.
*Journal of Mathematics Teacher Education,**6,*331–360.CrossRefGoogle Scholar - Fraivillig, J. L., Murphy, L. A., & Fuson, K. C. (1999). Advancing children’s mathematical thinking in everyday mathematics classrooms.
*Journal for Research in Mathematics Education,**30,*148–170.CrossRefGoogle Scholar - Franke, M. L., Carpenter, T. P., Levi, L., & Fennema, E. (2001). Capturing teachers’ generative change: A follow-up study of professional development in mathematics.
*American Educational Research Journal,**38,*653–689.CrossRefGoogle Scholar - Franke, M., Kazemi, E., & Battey, D. (2007). Understanding teaching and classroom practice in mathematics. In F. Lester Jr. (Ed.),
*Second handbook of research on mathematics teaching and learning*(pp. 225–256). Charlotte, NC: Information Age Publishing.Google Scholar - Good, T., & Brophy, J. (1994).
*Looking in classrooms*(6th ed., pp. 209–262). New York: Harper Collins College Publishers.Google Scholar - Hall, A., & Miro, D. (2016). A study of student engagement in project-based learning across multiple approaches to STEM education programs.
*School Science and Mathematics,**116*(6), 310–319.CrossRefGoogle Scholar - Hamre, B., Pianta, R., Burchinal, M., Field, S., LoCasale-Crouch, J., Downer, J., et al. (2012). A course on effective teacher-child interactions: Effects on teacher beliefs, knowledge, and observed practice.
*American Educational Research Journal,**49*(1), 88–123.CrossRefGoogle Scholar - Hiebert, J., Stigler, J. W., Jacobs, J. K., Givvin, K. B., Garnier, H., Smith, M., et al. (2005). Mathematics teaching in the United States today (and tomorrow): Results from the TIMSS 1999 video study.
*Educational Evaluation and Policy Analysis,**27,*111–132.CrossRefGoogle Scholar - Hill, H. C., Charalambous, C. Y., & Kraft, M. A. (2012). When rater reliability is not enough: Teacher observation systems and a case for the generalizability study.
*Educational Researcher,**41*(2), 56–64.CrossRefGoogle Scholar - Hill, H., & Shih, J. (2009). Examining the quality of statistical mathematics education research.
*Journal of Research in Mathematics Education,**40*(3), 241–250.Google Scholar - Jackson, K., Garrison, A., Wilson, J., Gibbons, L., & Shahan, E. (2013). Exploring relationships between setting up complex tasks and opportunities to learn in concluding whole-class discussions in middle-grades mathematics instruction.
*Journal for Research in Mathematics Education,**44*(4), 646–682.CrossRefGoogle Scholar - James, L. R., Demaree, R. G., & Wolf, G. (1993). Rwg: An assessment of within-group interrater agreement.
*Journal of Applied Psychology,**78,*306.CrossRefGoogle Scholar - Jong, C., Pedulla, J. J., Reagan, E. M., Salomon-Fernandez, Y., & Cochran-Smith, M. (2010). Exploring the link between reformed teaching practices and pupil learning in elementary school mathematics.
*School Science and Mathematics,**110*(6), 309–326.CrossRefGoogle Scholar - Judson, E. (2013). Development of an instrument to assess and deliberate on the integration of mathematics into student-centered science learning.
*School Science and Mathematics,**113*(2), 56–68.CrossRefGoogle Scholar - Kane, M. T. (2006). Valildation. In R. L. Brennan, National Council on Measurement in Education, & American Council on Education (Eds.),
*Educational measurement*. Westport, CT: Praeger Publishers.Google Scholar - Kane, M. T. (2016). Validation strategies: Delineating and validating proposed interpretations and uses of test scores. In S. Lane, M. Raymond, & T. M. Haladyna (Eds.),
*Handbook of test development*(Vol. 2nd). New York, NY: Routledge.Google Scholar - Kane, T. J. & Staiger, D. O. (2012). Gathering Feedback for Teaching: Combining high-quality observations with student surveys and achievement gains. Research paper. MET Project. Bill and Melinda Gates Foundation.Google Scholar
- Kapitula, L., & Umland, K. (2011). A validity argument approach to evaluating teacher value-added scores.
*American Educational Research Journal,**48*(3), 794–831.CrossRefGoogle Scholar - Kersting, N. B., Sutton, T., Kalinec-Craig, C., Stoehr, K. J., Heshmati, S., Lozano, G., et al. (2016). Further exploration of the classroom video analysis (CVA) instrument as a measure of usable knowledge for teaching mathematics: Taking a knowledge system perspective.
*ZDM Mathematics Education,**48*(1–2), 97–109.CrossRefGoogle Scholar - Khan, S., & VanWynsberghe, R. (2008). Cultivating the under-mined: Cross-case analysis as knowledge mobilization.
*Forum: Qualitative Social Research,**9,*1–21.Google Scholar - Le, V., Lockwood, J. R., Stecher, B. M., Hamilton, L. S., & Martinez, J. F. (2009). A longitudinal investigation of the relationship between teachers’ self-reports of reform-oriented instruction and mathematics and science achievement.
*Educational Evaluation and Policy Analysis,**31,*200–220.CrossRefGoogle Scholar - Learning Mathematics for Teaching Project. (2011). Measuring the mathematical quality of instruction.
*Journal of Mathematics Teacher Education,**14,*25–47.CrossRefGoogle Scholar - Lubienski, S. T. (2008). On” gap gazing” in mathematics education: The need for gaps analyses.
*Journal for Research in Mathematics Education,**39,*350–356.Google Scholar - Marshall, J., Smart, J., & Horton, R. (2010). The design and validation of EQUIP: An instrument to assess inquiry-based instruction.
*International Journal of Science & Mathematics Education,**8*(2), 299–321.CrossRefGoogle Scholar - Marshall, J. C., Smart, J., Lotter, C., & Sirbu, C. (2011). Comparative analysis of two inquiry observational protocols: Striving to better understand the quality of teacher-facilitated inquiry-based instruction.
*School Science and Mathematics,**111*(6), 306–315.CrossRefGoogle Scholar - Matsumura, L. C., Garnier, H., Slater, S. C., & Boston, M. (2008). Toward measuring instructional interactions ‘at-scale’.
*Educational Assessment,**13,*267–300.CrossRefGoogle Scholar - McCaffrey, D. F., Hamilton, L. S., Stecher, B. M., Klein, S. P., Bugliari, D., & Robyn, A. (2001). Interactions among instructional, practices, curriculum, and student achievement: The case of Standards-based high school mathematics.
*Journal for Research in Mathematics Education,**32,*493–517.CrossRefGoogle Scholar - McCaslin, M., Good, T. L., Nichols, S., Zhang, J., Wiley, C. R., Bozack, A. R., et al. (2006). Comprehensive school reform: An observational study of teaching in grades 3 through 5.
*The Elementary School Journal,**106*(4), 313–331.CrossRefGoogle Scholar - Mendez, E. P., Sherin, M. G., & Louis, D. A. (2007). Multiple perspectives on the development of an eighth-grade mathematical discourse community.
*The Elementary School Journal,**108*(1), 41–61.CrossRefGoogle Scholar - Morrell, P. D., Wainwright, C., & Flick, L. (2004). Reform teaching strategies used by student teachers.
*School Science and Mathematics,**104*(5), 199–213.CrossRefGoogle Scholar - Morrone, A. S., Harkness, S. S., D’ambrosio, B., & Caulfield, R. (2004). Patterns of instructional discourse that promote the perception of mastery goals in a social constructivist mathematics course.
*Educational Studies in Mathematics,**56*(1), 19–38.CrossRefGoogle Scholar - National Council of Teachers of Mathematics. (2014).
*Principles to actions: Ensuring mathematical success for all*. Reston, VA: Author.Google Scholar - Newton, K. J. (2009). Instructional practices related to prospective elementary school teachers’ motivation for fractions.
*Journal of Mathematics Teacher Education,**12*(2), 89–109.CrossRefGoogle Scholar - Ottmar, E. R., Rimm-Kaufman, S. E., Berry, R. Q., & Larsen, R. A. (2013). Does the responsive classroom approach affect the use of standards-based mathematics teaching practices? Results from a randomized controlled trial.
*The Elementary School Journal,**113*(3), 434–457.CrossRefGoogle Scholar - Peters Burton, E., Kaminsky, S. E., Lynch, S., Behrend, T., Han, E., Ross, K., et al. (2014). Wayne School of Engineering: Case study of a rural inclusive STEM-Focused High School.
*School Science and Mathematics,**114*(6), 280–290.CrossRefGoogle Scholar - Pianta, R. C., Belsky, J., Vandergrift, N., Houts, R., & Morrison, F. J. (2008). Classroom effects on children’s achievement trajectories in elementary school.
*American Educational Research Journal,**45*(2), 365–397.CrossRefGoogle Scholar - Ross, S. M., Smith, L. J., & Alberg, M. (1998).
*The school observation measure (SOM VC)*. Memphis: Center for Research in Educational Policy, The University of Memphis.Google Scholar - Rubel, L. H., & Chu, H. (2012). Reinscribing urban: Teaching high school mathematics in low income, urban communities of color.
*Journal of Mathematics Teacher Education,**15*(1), 39–52.CrossRefGoogle Scholar - Santagata, R., & Barbieri, A. (2005). Mathematics teaching in Italy: A cross-cultural video analysis.
*Mathematical Thinking and Learning,**7*(4), 291–312.CrossRefGoogle Scholar - Sawada, D., Piburn, M. D., Judson, E., Turley, J., Falconer, K., Benford, R., et al. (2002). Measuring reform practices in science and mathematics classrooms: The reformed teaching observation protocol.
*School Science and Mathematics,**102,*245–253.CrossRefGoogle Scholar - Saxe, G. B., Gearhart, M., & Seltzer, M. (1999). Relations between classroom practices and student learning in the domain of fractions.
*Cognition and instruction,**17,*1–24.CrossRefGoogle Scholar - Schifter, D. A., & Simon, M. (1992). Assessing teachers’ development of a constructivist view of mathematics learning.
*Teacher and Teacher Education,**8,*187–197.CrossRefGoogle Scholar - Schlesinger, L., & Jentsch, A. (2016). Theoretical and methodological challenges in measuring instructional quality in mathematics education using classroom observations.
*ZDM Mathematics Education,**48*(1–2), 29–40.CrossRefGoogle Scholar - Schoen, H. L., Cebulla, K. J., Finn, K. F., & Fi, C. (2003). Teacher variables that relate to student achievement when using a standards-based curriculum.
*Journal for Research in Mathematics Education,**34,*228–259.CrossRefGoogle Scholar - Schoenfeld, A. H. (2013). Classroom observations in theory and practice.
*ZDM Mathematics Education,**45*(4), 607–621.CrossRefGoogle Scholar - Sears, R., & Chavez, O. (2014). Opportunities to engage with proof: the nature of proof tasks in two geometry textbooks and its influence on enacted lessons.
*ZDM Mathematics Education,**46,*767–780.CrossRefGoogle Scholar - Simon, M. A., & Shifter, D. (1991). Towards a constructivist perspective: An intervention study of mathematics teacher development.
*Educational Studies in Mathematics,**22,*309–331.CrossRefGoogle Scholar - Slavin, R. E., Lake, C., & Groff, C. (2009). Effective programs in middle and high school mathematics: A best evidence synthesis.
*Review of Educational Research,**79,*839–911.CrossRefGoogle Scholar - Smith, M. (2014). Tools as a catalyst for practitioners’ thinking.
*Mathematics Teacher Educator,**3,*3–7.CrossRefGoogle Scholar - Steenbergen-Hu, S., & Cooper, H. (2013). A meta-analysis of the effectiveness of intelligent tutoring systems on K–12 students’ mathematical learning.
*Journal of Educational Psychology,**105,*970–987.CrossRefGoogle Scholar - Swafford, J. O., Jones, G. A., & Thornton, C. A. (1997). Increased knowledge in geometry and instructional practice.
*Journal for Research in Mathematics Education,**28,*467–483.CrossRefGoogle Scholar - Tarr, J. E., Reys, R. E., Reys, B. J., Chavez, O., Shih, J., & Osterlind, S. (2008). The impact of middle grades mathematics curricula on student achievement and the classroom learning environment.
*Journal for Research in Mathematics Education,**39,*247–280.Google Scholar - Toerner, G., & Arzarello, F. (2012). Grading mathematics education research journals.
*Newletter of the European Mathematical Society,**86,*52–54.Google Scholar - U.S. Department of Education, Institute of Education Sciences, What Works Clearinghouse. (2012, February).
*High School Math intervention report: I CAN Learn*^{®}. Retrieved from http://whatworks.ed.gov. - U.S. Department of Education, Institute of Education Sciences, What Works Clearinghouse. (2013, January).
*High School Mathematics intervention report: Carnegie Learning Curricula and Cognitive Tutor*^{®}. Retrieved from http://whatworks.ed.gov. - Valli, L., & Croninger, R. (2002).
*High quality teaching of foundational skills in mathematics and reading (# 0115389)*. Washington: National Science Foundation Interdisciplinary Educational Research Initiative.Google Scholar - Wainwright, C., Morrell, P. D., Flick, L., & Schepige, A. (2004). Observation of reform teaching in undergraduate level mathematics and science courses.
*School Science and Mathematics,**104*(7), 322–335.CrossRefGoogle Scholar - Walkington, C., Arora, P., Ihorn, S., Gordon, J., Walker, M., Abraham, L., & Marder, M. (2012).
*Development of the UTeach observation protocol: A classroom observation instrument to evaluate mathematics and science teachers from the UTeach preparation program*. Unpublished paper. Southern Methodist University.Google Scholar - Walkowiak, T., Berry, R., Meyer, J., Rimm-Kaufman, S., & Ottmar, E. (2014). Introducing an observational measure of standards-based mathematics teaching practices: Evidence of validity and score reliability.
*Educational Studies in Mathematics,**85,*109–128.CrossRefGoogle Scholar - Wasserman, N., & Walkington, C. (2014). Exploring links between beginning UTeachers’ beliefs and observed classroom practices.
*Teacher Education & Practice,**27*(2/3), 376–401.Google Scholar - Wilhelm, A. G., & Kim, S. (2015). Generalizing from observations of mathematics teachers’ instructional practice using the instructional quality assessment.
*Journal for Research in Mathematics Education,**46*(3), 270–279.CrossRefGoogle Scholar - Williams, S., & Leatham, K. (2017). Journal quality in mathematics education.
*Journal for Research in Mathematics Education,**48*(4), 369–396.CrossRefGoogle Scholar