Design and Development of an International Mathematics-Focused Observation Protocol



Observations of what teachers do in classrooms is one part of the FIRSTMATH research plan. Since videos are not feasible or permissible in all locations in all countries, the FIRSTMATH team needed a live observation instrument. This chapter reports on key challenges in implementing a live observation instrument including validity evidence to support the use of the instrument for FIRSTMATH purposes across multiple countries in multiple languages; the training and knowledge necessary for observers to achieve reliable observation data; and access to classrooms of novice teachers to actually conduct the observations.


  1. Aguirre, J., & Zavala, M. (2013). Making culturally responsive mathematics teaching explicit: A lesson analysis tool. Pedagogies: An International Journal, 8, 163–190. Scholar
  2. Allen, J. P., Pianta, R. C., Gregory, A., Mikami, A. Y., & Lun, J. (2011). An interaction-based approach to enhancing secondary school instruction and student achievement. Science, 333(6045), 1034–1037.Google Scholar
  3. Baumert, J., Kunter, M., Blum, W., Brunner, M., Voss, T., Jordan, A., Klusmann, U., Krauss, S., Neubrand, M. & Tsai, Y. M. (2010). Teachers’ mathematical knowledge, cognitive activation in the classroom, and student progress. American Educational Research Journal, 47(1), 133-180.Google Scholar
  4. Blazar, D. (2015). Effective teaching in elementary mathematics: Identifying classroom practices that support student achievement. Economics of Education Review, 48, 16–29.Google Scholar
  5. Bodovski, K., & Farkas, G. (2007). Mathematics growth in early elementary school: The roles of beginning knowledge, student engagement, and instruction. The Elementary School Journal, 108(2), 115–130.Google Scholar
  6. Boonen, T., Van Damme, J., & Onghena, P. (2013). Teacher effects on student achievement in first grade: Which aspects matter most? School Effectiveness and School Improvement, 25(1), 126–152.Google Scholar
  7. Borasi, R. (1987). Exploring mathematics through the analysis of errors. For the Learning of Mathematics, 7(3), 2–8.Google Scholar
  8. Boston, M., Bostic, J., Lesseig, K., & Sherman, M. (2015). A comparison of mathematics classroom observation protocols. Mathematics Teacher Educator, 3(2).
  9. Bruns, B., & Luque, J. (2014). Great teachers: How to raise pupil learning in Latin America and the Caribbean (advance edition). Washington, DC: World Bank. License: Creative Commons Attribution CC BY 3.0 IGO.Google Scholar
  10. Danielson, C. (2012). Teacher evaluation: What’s fair? What’s effective? Educational Leadership, 70(3), 32–37.Google Scholar
  11. Desimone, L. M., Smith, T. M., & Frisvold, D. E. (2009). Survey measures of classroom instruction: Comparing student and teacher reports. Educational Policy, 24(2), 267–329.Google Scholar
  12. Desimone, L., Smith, T., & Phillips, K. (2013). Linking student achievement growth to professional development participation and changes in instruction: A longitudinal study of elementary students and teachers in Title I schools. Teachers College Record, 115(5), 1–46.Google Scholar
  13. Frank, K.A., Bieda, K., Salloum, S., Ye, Y., Hu, S., & Youngs, P. (2017). Brass tacks and networks: The network effects of substantive planning on beginning elementary teachers’ ambitious mathematics instruction [Paper presentation]. American Educational Research Association Annual Meeting, San Antonio, TX.Google Scholar
  14. Garet, M. S., Wayne, A., Stancavage, F., Taylor, J., Eaton, M., Walters, K., et al. (2011). Middle school mathematics professional development impact study: Findings after the second year of implementation (NCEE 2011-4024). Washington, DC: U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance.
  15. Gleason, J., Livers, S., & Zelkowski, J. (2017). Mathematics classroom observation protocol for practices (MCOP2): A validation study. Investigations in Mathematics Learning, 9(3), 111–129. Scholar
  16. Gojak, L. M. (2013, January 8). The power of a good mistake. NCTM summing up (messages from the president). Reston, VA: National Council of Teachers of Mathematics.
  17. Guarino, C., Dieterle, S., Bargagliotti, A., & Mason, W. (2013). What can we learn about effective early mathematics teaching? A framework for estimating causal effects using longitudinal survey data. Journal of Research on Educational Effectiveness, 6(2), 164–198.Google Scholar
  18. Hastie, T., Tibshirani, R., & Friedman, J. (2009). The elements of statistical learning (2nd ed.). New York, NY: Springer.CrossRefGoogle Scholar
  19. Horizon Research, Inc. (2000). Inside the classroom observation and analytic protocol
  20. Junker, B., Weisberg, Y, Matsumura, L. C., Crosson, A., Wolf, M. K., Levison, A., & Resnick, L. (2006). Overview of the instructional quality of assessment (CSE Tech. Rep. No. 671). Los Angeles, CA: Center for the Study of Evaluation, National Center for Research on Evaluation, Standards, and Pupil Testing (CRESST), University of California.
  21. Kane, M. (2013). The argument-based approach to validation. School Psychology Review, 42(4), 448–457.Google Scholar
  22. Kersting, N., Givvin, K. B., Thompson, B. J., & Stigler, J. (2012). Measuring usable knowledge. American Educational Research Journal, 49(3), 568–589.Google Scholar
  23. Kimball, S. M., White, B., Milanowski, A., & Borman, G. (2004). Examining the relationship between teacher evaluation and student assessment results in Washoe County. Peabody Journal of Education, 79(4), 54–78.Google Scholar
  24. Koellner, K., & Jacobs, J. (2015). Knowledge, instruction, and student achievement distinguishing models of professional development: The case of an adaptive model’s impact on teachers’ on behalf of: American Association of Colleges for Teacher Education (AACTE). Journal of Teacher Education, 66(1), 51–67.Google Scholar
  25. Lai, C.-F. (2012). Behavior research and teaching. Technical Report 1012. Eugene, OR: University of Oregon.Google Scholar
  26. Learning Mathematics for Teaching Project. (2010). Measuring the mathematical quality of instruction. Journal of Mathematics Teacher Education, 14, 25–47.CrossRefGoogle Scholar
  27. Marzano, R. J. (Ed.). (2010). On excellence in teaching. Bloomington, IN: Solution Tree Press.Google Scholar
  28. Mikami, Y., Allen, J. P., Pianta, R. C., & Lun, J. (2011). Effects of a teacher professional development intervention on peer relationships in secondary classrooms. School Psychology Review, 40(3), 367–385.Google Scholar
  29. Mullens, J. E., & Gayler, K. (1999). Measuring classroom instructional processes: Using survey and case study fieldtest results to improve item construction (Working Paper No. 1999-08). Washington, DC: National Center for Education Statistics, U.S. Department of Education.
  30. National Council of Teachers of Mathematics (NCTM). (2000). Principles and standards for school mathematics
  31. National Council of Teachers of Mathematics (NCTM). (2014). Principles to actions: Ensuring mathematical success for all
  32. Ottmar, E., Rimm-Kaufman, S., Larsen, R. A., & Berry, R. Q. (2015). Mathematical knowledge for teaching, standards-based mathematics teaching practices, and student achievement in the context of the responsive classroom approach. American Educational Research Journal, 52(4), 787–821.Google Scholar
  33. Santagata, R., & Guarino, J. (2011). Using video to teach future teachers to learn from teaching. ZDM: The International Journal on Mathematics Education, 43(1), 133–145.Google Scholar
  34. Sawada, D., Piburn, M., Judson, E., Turley, J., Falconer, K., Benford, R., & Bloom, I. (2002). Measuring reform practices in science and mathematics classrooms: The reformed teaching observation protocol. School Science and Mathematics, 102(6), 245–253.Google Scholar
  35. Smith, J. B., Lee, V., & Newman, F. M. (2001). Instruction and achievement in Chicago elementary schools. Chicago, IL: Consortium on Chicago School Research.Google Scholar
  36. Smith, M. K., Jones, F. H. M., Gilbert, S. L., & Wieman, C. E. (2013). The Classroom Observation Protocol for Undergraduate STEM (COPUS): A new instrument to characterize university STEM classroom practices. CBE Life Sciences Education, 12, 618–627.CrossRefGoogle Scholar
  37. Stallings, J. A., Knight, S. L., & Markham, D. (2014). Using the Stallings Observation System to investigate time on task in four countries. Washington, DC: World Bank. License: CC BY 3.0 IGOGoogle Scholar
  38. Stallings Snapshot Observation Manual (2001). College Station, TX: World Bank Group.Google Scholar
  39. Tatto, M. T., Burn, K., Menter, I., Mutton, T., & Thompson, I. (2018). Learning to teaching in England and the United States. New York, NY: Routledge.Google Scholar
  40. Wenglinsky, H. (2002). The link between teacher classroom practices and student academic performance. Education Policy Analysis Archives, 10(12), 

Copyright information

© The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.Mary Lou Fulton Teachers CollegeArizona State UniversityTempeUSA
  2. 2.Educational PsychologyUniversity of MinnesotaMinneapolisUSA
  3. 3.College of EducationMichigan State UniversityEast LansingUSA
  4. 4.Center for Science, Mathematics & Computer EducationUniversity of Nebraska-LincolnLincolnUSA
  5. 5.Faculty of Mathematics and Informatics, the University of SofiaInstitute of Mathematics and Informatics, Bulgarian Academy of SciencesSofiaBulgaria
  6. 6.Oakland Community CollegeEast LansingUSA

Personalised recommendations