Advertisement

About the Complexities of Video-Based Assessments: Theoretical and Methodological Approaches to Overcoming Shortcomings of Research on Teachers’ Competence

  • Gabriele Kaiser
  • Andreas BusseEmail author
  • Jessica Hoth
  • Johannes König
  • Sigrid Blömeke
Article

Abstract

Research on the evaluation of the professional knowledge of mathematics teachers (comprising for example mathematical content knowledge, mathematics pedagogical content knowledge and general pedagogical knowledge) has become prominent in the last decade; however, the development of video-based assessment approaches is a more recent topic. This paper follows the call for more situated and performance-related ways to assess teacher competence. We discuss the theoretical and methodological challenges connected to the development of such instruments and exemplify these by an instrument developed within the follow-up study of the international “Teacher Education and Development Study in Mathematics (TEDS-M)”, called TEDS-FU. Drawing on the novice-expert framework from cognitive psychology allows analysing the structure and development of mathematics teachers’ professional competence. More recent concepts on teacher noticing of classroom situations and students’ activities are incorporated into this video-based evaluation instrument, which is described in detail in this paper, by assessing perceptual, interpretative and decision-making skills. Reliability and validity concerns remain an issue of such assessments for which solutions are proposed. Overall, the paper shows that a more comprehensive evaluation of teachers’ competence comprising cognitive-affective and situated facets is possible and has been achieved.

Keywords

Teacher competence Teacher knowledge Video-based assessment Mathematics teacher education Noticing Perception skills Decision-making Performance assessment 

References

  1. Adler, J., Ball, D. L., Krainer, K., Lin, F.-L. & Novotna, J. (2005). Mirror images of an emerging field: researching mathematics teacher education. Educational Studies in Mathematics, 60(3), 359–381.CrossRefGoogle Scholar
  2. Ball, D. L. & Bass, H. (2000). Interweaving content and pedagogy in teaching and learning to teach: Knowing and using mathematics. In J. Boaler (Ed.), Multiple perspectives on mathematics teaching and learning (pp. 83–104). London, England: Ablex.Google Scholar
  3. Ball, D. L., Thames, M. H. & Phelps, G. (2008). Content knowledge for teaching: What makes it special? Journal of Teacher Education, 59(5), 389–407.CrossRefGoogle Scholar
  4. Berliner, D. C. (2001). Learning about and learning from expert teachers. International Journal of Educational Research, 35(5), 463–482.CrossRefGoogle Scholar
  5. Blömeke, S. & Delaney, S. (2012). Assessment of teacher knowledge across countries: A review of the state of research. ZDM – The International Journal on Mathematics Education, 44(3), 223–247.Google Scholar
  6. Blömeke, S., Hsieh, F.-J., Kaiser, G., & Schmidt, W. H. (Eds.). (2014). International Perspectives on Teacher Knowledge, Beliefs and Opportunities to Learn. Dordrecht, The Netherlands: Springer.Google Scholar
  7. Blömeke, S., König, J., Busse, A.,  Suhl, U., Benthien, J., Döhrmann, M. & Kaiser, G. (2014). Von der Lehrerausbildung in den Beruf: Fachbezogenes Wissen als Voraussetzung einer genauen Wahrnehmung und Analyse von Unterricht [The transition from teacher training to the teaching profession: Subject-specific knowledge as a prerequisite for perception, interpretation and decision-making in the classroom]. Zeitschrift für Erziehungswissenschaft, 17(3), 509–542.Google Scholar
  8. Blömeke, S., Gustafsson, J.-E. & Shavelson, R. (2015). Beyond dichotomies: Competence viewed as a continuum. Zeitschrift für Psychologie, 223, 3–13.Google Scholar
  9. Blömeke, S., König, J., Suhl, U., Hoth, J. & Döhrmann, M. (2015). Wie situationsbezogen ist die Kompetenz von Lehrkräften? Zur Generalisierbarkeit von videobasierten Performanztests [How strongly does teacher performance depend on differences in classroom situations? On the generalizability of video-based performance assessments]. Zeitschrift für Pädagogik 3.Google Scholar
  10. Blum, W. (2011). Can modelling be taught and learnt? Some answers from empirical research. In G. Kaiser, W. Blum, R. Borromeo Ferri & G. Stillman (Eds.), Trends in teaching and learning of mathematical modelling (pp. 15–30). Dordrecht, The Netherlands: Springer.Google Scholar
  11. Carter, K., Cushing, K., Sabers, D., Stein, P. & Berliner, D. C. (1988). Expert-novice differences in perceiving and processing visual information. Journal of Teacher Education, 39, 25–31.CrossRefGoogle Scholar
  12. Chi, M. T. H. (2011). Theoretical perspectives, methodological approaches, and trends in the study of expertise. In Y. Li & G. Kaiser (Eds.), Expertise in mathematics instruction: An international perspective (pp. 17–39). New York, NY: Springer.Google Scholar
  13. Clausen, M., Reusser, K. & Klieme, E. (2003). Unterrichtsqualität auf der Basis hoch-inferenter Unterrichtsbeurteilungen. Ein Vergleich zwischen Deutschland und der deutschsprachigen Schweiz. Unterrichtswissenschaft, 31(2), 122–141.Google Scholar
  14. Depaepe, F., Verschaffel, L. & Kelchtermanns, G. (2013). Pedagogical content knowledge: A systematic review of the way in which the concept has pervaded mathematical educational research. Teaching and Teacher Education, 34, 12–25.CrossRefGoogle Scholar
  15. Döhrmann, M., Kaiser, G. & Blömeke, S. (2014). The conceptualisation of mathematics competencies in the international teacher education study TEDSM. In S. Blömeke , F.J. Hsieh, G. Kaiser, & W. H. Schmidt (Eds.). International Perspectives on Teacher Knowledge, Beliefs and Opportunities to Learn (pp. 431456). Dordrecht, The Netherlands: Springer.Google Scholar
  16. Even, R. & Ball, D. L. (Eds.). (2008). The professional education and development of teachers of mathematics. The 15th ICMI Study. New York, NY: Springer.Google Scholar
  17. Goldman, R., Pea, R., Barron, B. & Denny, S. J. (Eds.). (2007). Video research in the learning sciences. Mahwah, NJ: Lawrence Erlbaum.Google Scholar
  18. Hiebert, J., Gallimore, R., Garnier, H., Givven, K. B., Hollingsworth, H., Jacobs, J., Chui, A. M.-Y., Wearne, D., Smith, M., Manaster, A., Tseng, E., Etterbeek, W., Manaster, C., Gonzales, P. & Stigler, J. W. (2003). Teaching mathematics in seven countries: Results from the TIMSS 1999 Video Study. Washington, D.C.: U.S. Department of Education, National Center for Education Statistics.Google Scholar
  19. Jacobs, V. R., Lamb, L. L. & Philipp, R. A. (2010). Professional noticing of children’s mathematical thinking. Journal for Research in Mathematics Education, 2(41), 169–202.Google Scholar
  20. Kaiser, G.,  Benthien, J., Döhrmann, M., König, J. & Blömeke, S. (2013). Expert Ratings as an Instrument for Validating Results of Video-Based Testing. In A. M. Lindmeier & A. Heinze (Eds.), Proceedings of the 37th Conference of the International Group for the Psychology of Mathematics Education PME in Kiel/Germany (Vol. 5, pp. 83). Kiel, German: PME.Google Scholar
  21. Kane, M. (1992). An argument-based approach to validation. Psychological Bulletin, 112, 527–535.CrossRefGoogle Scholar
  22. Kersting, N. B. (2008). Using video clips of mathematics classroom instruction as item prompts to measure Teachers’ knowledge of teaching mathematics. Educational and Psychological Measurement, 68(5), 845–861.CrossRefGoogle Scholar
  23. König, J., Blömeke, S., Klein, P., Suhl, U., Busse, A. & Kaiser, G. (2014). Is teachers’ general pedagogical knowledge a premise for noticing and interpreting classroom situations? A video-based assessment approach. Teaching and Teacher Education, 38, 76–88.Google Scholar
  24. Kunter, M., Baumert, J., Blum, W., Klusmann, U., Krauss, S. & Neubrand, M. (Eds.). (2013). Cognitive activation in the mathematics classroom and professional competence of teachers. Results from the COACTIV project. New York, NY: Springer.Google Scholar
  25. Kunter, M. & Voss, T. (2013). The model of instructional quality in COACTIV: A multicriteria analysis. In M. Kunter, J. Baumert, W. Blum, U. Klusmann, S. Krauss & M. Neubrand (Eds.), Cognitive activation in the mathematics classroom and professional competence of teachers. Results from the COACTIV project (pp. 79–96). New York, NY: Springer.Google Scholar
  26. Li, Y. & Kaiser, G. (Eds.) (2011). Expertise in Mathematics Instruction: An International Perspective. New York, NY: Springer.Google Scholar
  27. Livingston, C. & Borko, H. (1989). Expert-novice differences in teaching: A cognitive analysis and implications for teacher education. Journal of Teacher Education, 40, 36–42.CrossRefGoogle Scholar
  28. Rowland, T. & Ruthven, K. (Eds.). (2011). Mathematical knowledge in teaching. Dordrecht, The Netherlands: Springer.Google Scholar
  29. Sabers, D. S., Cushing, K. S. & Berliner, D. C. (1991). Differences among teachers in a task characterized by simultaneity, multidimensionality, and immediacy. American Educational Research Journal, 28, 63–88.Google Scholar
  30. Santagata, R. & Guarino, J. (2011). Using video to teach future teachers to learn from teaching. ZDM – The International Journal on Mathematics Education, 41(1), 133–146.CrossRefGoogle Scholar
  31. Schoenfeld, A. H. (2011). How we think. A theory of goal-oriented decision making and its educational applications. New York, NY: Routledge.Google Scholar
  32. Schoenfeld, A. H. & Kilpatrick, J. (2008). Toward a theory of proficiency in teaching mathematics. In D. Tirosh & T. Wood (Eds.), International handbook of mathematics teacher education: Vol. 2. Tools and processes in mathematics teacher education (pp. 321–354). Rotterdam, The Netherlands: Sense Publishers.Google Scholar
  33. Sherin, M. G., Jacobs, V. R. & Philipp, R. A. (Eds.). (2011). Mathematics teacher noticing. Seeing through Teachers’ eyes. New York, NY: Routledge.Google Scholar
  34. Shulman, L. S. (1986). Those who understand: Knowledge growth in teaching. Educational Researcher, 15(2), 4–14.CrossRefGoogle Scholar
  35. Shulman, L. S. (1987). Knowledge and teaching: Foundations of the new reform. Harvard Educational Review, 57, 1–21.Google Scholar
  36. Van Es, E. (2011). A framework for learning to notice student thinking. In M. G. Sherin, V. R. Jacobs & R. A. Philipp (Eds.), Mathematics teacher noticing. Seeing through teachers’ eyes (pp. 134–151). New York, NY: Routledge.Google Scholar
  37. Van Es, E. A. & Sherin, M. G. (2002). Learning to notice: Scaffolding new teachers‘interpretations of classroom interactions. Journal of Technology and Teacher Education, 10(4), 571–596.Google Scholar
  38. Weinert, F. E. (2001). Concept of competence: A conceptual clarification. In D. S. Rychen & L. H. Salganik (Eds.), Defining and selecting key competencies (pp. 45–66). Göttingen, German: Hogrefe.Google Scholar
  39. Wetzel, E. & Carstensen, C. H. (2014). Reversed thresholds in partial credit models: A reason for collapsing categories? Assessment. Advance online publication. doi: 10.1177/1073191114530775.
  40. Wood, T. (Ed.). (2008). The international handbook of mathematics teacher education. Rotterdam, The Netherlands: Sense.Google Scholar

Copyright information

© Springer Science + Business Media B.V. 2015

Authors and Affiliations

  • Gabriele Kaiser
    • 1
  • Andreas Busse
    • 1
    Email author
  • Jessica Hoth
    • 2
  • Johannes König
    • 3
  • Sigrid Blömeke
    • 4
  1. 1.Fakultät für ErziehungswissenschaftUniversität HamburgHamburgGermany
  2. 2.Institut für Didaktik der Mathematik und des SachunterrichtsUniversität VechtaVechtaGermany
  3. 3.Universität zu KölnKölnGermany
  4. 4.Centre for Educational Measurement (CEMO)University of OsloOsloNorway

Personalised recommendations