Beyond Knowledge: Measuring Primary Teachers’ Subject-Specific Competences in and for Teaching Mathematics with Items Based on Video Vignettes

  • Imke Knievel
  • Anke M. LindmeierEmail author
  • Aiso Heinze


Teachers’ subject-specific cognition is seen as an important factor for the quality of instruction and, accordingly, student learning. However, in-depth research on these relations can only be carried out if a sound theoretical model for subject-specific teacher cognition (knowledge and competence/practical skills) and—in the case of a quantitative research approach—corresponding measures are available. The subject-specific cognition can be modeled as basic professional knowledge (BK) complemented by two further components of reflective competence (RC) and action-related competence (AC) with a close connection to professional demands. In order to implement these subject-specific demands rigorously, we developed innovative standardized measures for primary mathematics teachers. In particular, we argue that video-based items that are implemented in a speed condition and rated as holistic observations are well suited to realize the assessment of action-related competence. This article gives a detailed insight into the test development as well as the coding and scoring procedure and focuses on validation efforts. The study is based on the data of 85 in-service primary mathematics teachers and shows the viability of the approach. Classical scale analyses as well as confirmatory factor analyses and the comparison of different models as well as teacher groups (mathematics certified vs. non-mathematics certified teachers) give evidence for the validity and reliability of the measures.


Mathematics education Primary teachers Standardized assessment Teacher competence Teacher knowledge Video vignettes 


  1. AERA, APA & NCME (1999). Standards for educational and psychological testing. Washington, DC: AERA.Google Scholar
  2. Ball, D. L. (2000). Bridging practices: Intertwining content and pedagogy in teaching and learning to teach. Journal of Teacher Education, 51(3), 241–247.CrossRefGoogle Scholar
  3. Ball, D. L., Thames, M. H. & Phelps, G. (2008). Content knowledge for teaching: What makes it special? JTE, 59(5), 389–407.Google Scholar
  4. Baumert, J., Kunter, M., Blum, W., Brunner, M., Voss, T., Jordan, A. & Tsai, Y. M. (2010). Teachers’ mathematical knowledge, cognitive activation in the classroom, and student progress. American Educational Research Journal, 47(1), 133–180.Google Scholar
  5. Berliner, D. C. (2001). Learning about and learning from expert teachers. International Journal of Educational Research, 35(5), 463–482.CrossRefGoogle Scholar
  6. Blömeke, S. & Delaney, S. (2012). Assessment of teacher knowledge across countries: A review of the state of research. ZDM, 44(3), 223–247.CrossRefGoogle Scholar
  7. Bromme, R. (2001). Teacher expertise. In N. J. Smelser & P. B. Baltes (Eds.), International encyclopedia of the social & behavioral sciences (pp. 15459–15465). Oxford, UK: Pergamon.Google Scholar
  8. Burns, R. B. (1984). The process and context of teaching: A conceptual framework. Evaluation in Education, 8, 95–112.Google Scholar
  9. Calderhead, J. (1996). Teachers: Beliefs and knowledge. In D. C. Berliner & R. C. Calfee (Eds.), Handbook of educational psychology (pp. 709–725). Mahwah, NJ: Routledge.Google Scholar
  10. Carpenter, T. P. & Fennema, E. (1999). Children’s mathematics: Cognitively guided instruction. Portmouth, England: Heinemann.Google Scholar
  11. Döhrmann, M., Kaiser, G. & Blömeke, S. (2012). The conceptualisation of mathematics competencies in the international teacher education study TEDS-M. ZDM, 44(3), 325–340.CrossRefGoogle Scholar
  12. Fenstermacher, G. D. (1994). The knower and the known: The nature of knowledge in research on teaching. Review of Research in Education, 20(1), 3–56.CrossRefGoogle Scholar
  13. Fuson, C. K., Wearne, D., Hiebert, J. C., Murray, H. G., Olivier, A. I., Carpenter, T. P. & Fennema, E. (1997). Children’s conceptual structure for multidigit numbers and methods of multidigit addition and subtraction. Journal for Research in Mathematics Education, 28(2), 130–162.Google Scholar
  14. Graham, J. W., Olchowski, A. E. & Gilreath, T. D. (2007). How many imputations are really needed? Some practical clarifications of multiple imputation theory. Prevention Science, 8(3), 206–213.CrossRefGoogle Scholar
  15. Grimmet, P. P. & Mackinnon, A. M. (1992). Craft knowledge and the education of teachers. Review of Research in Education, 18, 385–456.Google Scholar
  16. Hair, J. F. (1998). Multivariate data analysis (5th ed.). Upper Saddle River, NJ: Prentice Hall.Google Scholar
  17. Hill, H., Ball, D. L., Blunk, M., Goffney, I. M. & Rowan, B. (2007). Validating the ecological assumption: The relationship of measure scores to classroom teaching and student learning. Measurement, 5(2–3), 107–118.Google Scholar
  18. Hill, H. C., Ball, D. L. & Schilling, S. G. (2008). Unpacking pedagogical content knowledge: Conceptualizing and measuring teachers’ topic-specific knowledge of students. Journal for Research in Mathematics Education, 39(4), 372–400.Google Scholar
  19. Hill, H. C., Rowan, B. & Ball, D. L. (2005). Effects of teachers’ mathematical knowledge for teaching on student achievement. American Education Research Journal, 42(2), 371–406.Google Scholar
  20. Hu, L.-T. & Bentler, P. M. (1998). Fit indices in covariance structure modeling: Sensitivity to underparameterized model misspecification. Psychological Methods, 3(4), 424–453.CrossRefGoogle Scholar
  21. Kersting, N. B., Givvin, K. B., Thompson, B., Santagata, R. & Stigler, J. W. (2012). Measuring usable knowledge: Teachers’ analyses of mathematics classroom videos predict teaching quality and student learning. American Education Research Journal, 49(3), 568–589.Google Scholar
  22. Koeppen, K., Hartig, J., Klieme, E. & Leutner, D. (2008). Current issues in competence modeling and assessment. Journal of Psychology, 216(2), 61–73.Google Scholar
  23. König, J., Blömeke, S., Klein, P., Suhl, U., Busse, A. & Kaiser, G. (2014). Is teachers’ general pedagogical knowledge a premise for noticing and interpreting classroom situations? A video-based assessment approach. Teaching and Teacher Education, 38, 76–88.CrossRefGoogle Scholar
  24. Krauss, S., Brunner, M., Kunter, M., Baumert, J., Blum, W. & Neubrand, M. (2008). Pedagogical content knowledge of secondary mathematics teachers. Journal of Educational Psychology, 100(3), 716–725.CrossRefGoogle Scholar
  25. Kunter, M., Klusmann, U., Baumert, J., Richter, D., Voss, T. & Hachfeld, A. (2013). Professional competence of teachers: Effects on instructional quality and student development. Journal of Educational Psychology, 105(3), 805–820.CrossRefGoogle Scholar
  26. Lindmeier, A. (2013). Video-vignettenbasierte standardisierte Erhebung von Lehrerkognitionen [Video vignette-based standardized survey of teacher cognition]. In U. Riegel & K. Macha (Eds.), Videobasierte Kompetenzforschung in den Fachdidaktiken (pp. 45–62). Münster, Germany: Waxmann.Google Scholar
  27. Lindmeier, A. (2011). Modeling and measuring knowledge and competencies of teachers: A threefold domain-specific structure model for mathematics. Münster, Germany: Waxmann.Google Scholar
  28. Little, T. D., Cuningham, W. A. & Shahar, G. (2002). To parcel or not to parcel. Exploring the question, weighing the merits. Structural Equation Modelling, 9(2), 151–173.CrossRefGoogle Scholar
  29. Marsh, H. W., Hau, K.-T., Balla, J. R. & Grayson, D. (1998). Is more ever too much? The number of indicators per factor in confirmatory factor analysis. Multivariate Behavioral Research, 33(2), 181–220.CrossRefGoogle Scholar
  30. Muthén, L. K. & Muthén, B. O. (1998–2012). Mplus user guide (7th ed.). Los Angeles, CA: Muthén & Muthén.Google Scholar
  31. National Council of Teachers of Mathematics (2000). Principles and standards for school mathematics. Reston, VA: Author.Google Scholar
  32. Osburn, H. G. (2000). Coefficient alpha and related internal consistency reliability coefficients. Psychological Methods, 5(3), 343–355.CrossRefGoogle Scholar
  33. Oser, F., Salzmann, P. & Heinzer, S. (2009). Measuring the competence-quality of vocational teachers: An advocatory approach. Empirical Research in Vocational Education and Training, 1(1), 65–83.Google Scholar
  34. Padberg, F. & Benz, C. (2011). Didaktik der Arithmetik [Teaching of Arithmetic]. Heidelberg, Germany: Spektrum.Google Scholar
  35. Polanyi, M. (1967). The tacit dimension. New York, NY: Anchor Books.Google Scholar
  36. Raftery, A. E. (1995). Bayesian model selection in social research. Sociological Methodology, 25, 111–163.CrossRefGoogle Scholar
  37. Renkl, A. & Mandl, H. (1996). Inert knowledge: Analysis and remedies. Educational Psychologist, 31(2), 115–121.CrossRefGoogle Scholar
  38. Richter, D., Kuhl, P. & Reimers, H. (2012). Aspekte der Aus- und Fortbildung von Lehrkräften in der Primarstufe [Aspects of the basic and further training of teachers at the primary level]. In P. Stanat, H. A. Pant, K. Böhme & D. Richter (Eds.), Kompetenzen von Schülerinnen und Schülern am Ende der vierten Jahrgangstufe in den Fächern Deutsch und Mathematik (pp. 237–251). Münster, Germany: Waxmann.Google Scholar
  39. Rohaan, E. J., Taconis, R. & Jochems, W. M. (2009). Measuring teachers’ pedagogical content knowledge in primary technology education. Research in Science & Technological Education, 27(3), 327–338.CrossRefGoogle Scholar
  40. Sabers, D. S., Cushing, K. S. & Berliner, D. C. (1991). Differences among teachers in a task characterized by simultaneity, multidimensional, and immediacy. American Education Research Journal, 28(1), 63–88.Google Scholar
  41. Schmelzing, S., van Driel, J. H., Jüttner, M., Brandenbusch, S., Sandmann, A. & Neuhaus, B. (2013). Development, evaluation, and validation of a paper-and-pencil test for measuring two components of biology teachers’ pedagogical development, concerning the “cardiovascular system”. International Journal of Science and Mathematics Education, 11(6), 1369–1390.CrossRefGoogle Scholar
  42. Schön, D. (1983). The reflective practitioner: How professionals think in action. New York, NY: Basic Books.Google Scholar
  43. Schoenfeld, A. H. (2008). On modeling teachers’ in-the-moment decision making. In A. H. Schoenfeld (Ed.), A study of teaching. Multiple lenses, multiple views (no. 14) (p. 45). Reston, VA: NCTM.Google Scholar
  44. Seidel, T. & Stürmer, K. (2014). Modeling and measuring the structure of professional vision in preservice teachers. American Education Research Journal, 51(4), 739–771.CrossRefGoogle Scholar
  45. Senk, S. L., Tatto, M. T., Reckase, M., Rowley, G., Peck, R. & Bankov, K. (2014). Knowledge of future primary teachers for teaching mathematics: An international comparative study. In S. Blömeke, F.-J. Hsieh, G. Kaiser & W. H. Schmidt (Eds.), International perspectives on teacher knowledge, beliefs and opportunities to learn (pp. 61–90). Dodrecht, Netherlands: Springer.Google Scholar
  46. Shulman, L. S. (1986). Those who understand: Knowledge growth in teaching. Educational Researcher, 15, 4–14.CrossRefGoogle Scholar
  47. Sherin, M. G. & van Es, E. A. (2009). Effects of video club participation on teachers’ professional vision. Journal of Teacher Education, 60(1), 20–37.CrossRefGoogle Scholar
  48. van Driel, J. H., Beijaard, D. & Verloop, N. (2001). Professional development and reform in science education: The role of teachers’ practical knowledge. Journal of Research in Science Teaching, 38(2), 137–158.CrossRefGoogle Scholar
  49. Vogt, F. & Rogalla, M. (2009). Developing adaptive teaching competency through coaching. Teaching and Teacher Education, 25(8), 1051–1060.CrossRefGoogle Scholar
  50. Voss, T., Kunter, M. & Baumert, J. (2011). Assessing teacher candidates’ general pedagogical/psychological knowledge: Test construction and validation. Journal of Educational Psychology, 103(4), 952–969.CrossRefGoogle Scholar
  51. Wahl, D. (2002). Mit Training vom trägen Wissen zum kompetenten Handeln? [Training the sluggish knowledge to a competent action] Zeitschrift für Pädagogik, 48(2), 227–241.Google Scholar
  52. Weinert, F. (2001). Concept of competence: A conceptual clarification. In D. Rychen & L. Salyanik (Eds.), Defining and selecting key competencies (pp. 45–65). Göttingen, Germany: Hogrefe.Google Scholar

Copyright information

© Ministry of Science and Technology, Taiwan 2015

Authors and Affiliations

  1. 1.IPN—Leibniz Institute for Science and Mathematics EducationKielGermany

Personalised recommendations