Skip to main content

Advertisement

SpringerLink
Log in

About the Complexities of Video-Based Assessments: Theoretical and Methodological Approaches to Overcoming Shortcomings of Research on Teachers’ Competence

  • Published:
International Journal of Science and Mathematics Education Aims and scope Submit manuscript

Abstract

Research on the evaluation of the professional knowledge of mathematics teachers (comprising for example mathematical content knowledge, mathematics pedagogical content knowledge and general pedagogical knowledge) has become prominent in the last decade; however, the development of video-based assessment approaches is a more recent topic. This paper follows the call for more situated and performance-related ways to assess teacher competence. We discuss the theoretical and methodological challenges connected to the development of such instruments and exemplify these by an instrument developed within the follow-up study of the international “Teacher Education and Development Study in Mathematics (TEDS-M)”, called TEDS-FU. Drawing on the novice-expert framework from cognitive psychology allows analysing the structure and development of mathematics teachers’ professional competence. More recent concepts on teacher noticing of classroom situations and students’ activities are incorporated into this video-based evaluation instrument, which is described in detail in this paper, by assessing perceptual, interpretative and decision-making skills. Reliability and validity concerns remain an issue of such assessments for which solutions are proposed. Overall, the paper shows that a more comprehensive evaluation of teachers’ competence comprising cognitive-affective and situated facets is possible and has been achieved.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Notes

  1. TEDS-FU was funded by the German Research Foundation (DFG, KA 797/18-1). The views expressed in the paper are those of the authors and do not necessarily reflect those of the DFG.

References

  • Adler, J., Ball, D. L., Krainer, K., Lin, F.-L. & Novotna, J. (2005). Mirror images of an emerging field: researching mathematics teacher education. Educational Studies in Mathematics, 60(3), 359–381.

    Article  Google Scholar 

  • Ball, D. L. & Bass, H. (2000). Interweaving content and pedagogy in teaching and learning to teach: Knowing and using mathematics. In J. Boaler (Ed.), Multiple perspectives on mathematics teaching and learning (pp. 83–104). London, England: Ablex.

  • Ball, D. L., Thames, M. H. & Phelps, G. (2008). Content knowledge for teaching: What makes it special? Journal of Teacher Education, 59(5), 389–407.

    Article  Google Scholar 

  • Berliner, D. C. (2001). Learning about and learning from expert teachers. International Journal of Educational Research, 35(5), 463–482.

    Article  Google Scholar 

  • Blömeke, S. & Delaney, S. (2012). Assessment of teacher knowledge across countries: A review of the state of research. ZDM – The International Journal on Mathematics Education, 44(3), 223–247.

  • Blömeke, S., Hsieh, F.-J., Kaiser, G., & Schmidt, W. H. (Eds.). (2014). International Perspectives on Teacher Knowledge, Beliefs and Opportunities to Learn. Dordrecht, The Netherlands: Springer.

  • Blömeke, S., König, J., Busse, A.,  Suhl, U., Benthien, J., Döhrmann, M. & Kaiser, G. (2014). Von der Lehrerausbildung in den Beruf: Fachbezogenes Wissen als Voraussetzung einer genauen Wahrnehmung und Analyse von Unterricht [The transition from teacher training to the teaching profession: Subject-specific knowledge as a prerequisite for perception, interpretation and decision-making in the classroom]. Zeitschrift für Erziehungswissenschaft, 17(3), 509–542.

  • Blömeke, S., Gustafsson, J.-E. & Shavelson, R. (2015). Beyond dichotomies: Competence viewed as a continuum. Zeitschrift für Psychologie, 223, 3–13.

  • Blömeke, S., König, J., Suhl, U., Hoth, J. & Döhrmann, M. (2015). Wie situationsbezogen ist die Kompetenz von Lehrkräften? Zur Generalisierbarkeit von videobasierten Performanztests [How strongly does teacher performance depend on differences in classroom situations? On the generalizability of video-based performance assessments]. Zeitschrift für Pädagogik 3.

  • Blum, W. (2011). Can modelling be taught and learnt? Some answers from empirical research. In G. Kaiser, W. Blum, R. Borromeo Ferri & G. Stillman (Eds.), Trends in teaching and learning of mathematical modelling (pp. 15–30). Dordrecht, The Netherlands: Springer.

  • Carter, K., Cushing, K., Sabers, D., Stein, P. & Berliner, D. C. (1988). Expert-novice differences in perceiving and processing visual information. Journal of Teacher Education, 39, 25–31.

    Article  Google Scholar 

  • Chi, M. T. H. (2011). Theoretical perspectives, methodological approaches, and trends in the study of expertise. In Y. Li & G. Kaiser (Eds.), Expertise in mathematics instruction: An international perspective (pp. 17–39). New York, NY: Springer.

  • Clausen, M., Reusser, K. & Klieme, E. (2003). Unterrichtsqualität auf der Basis hoch-inferenter Unterrichtsbeurteilungen. Ein Vergleich zwischen Deutschland und der deutschsprachigen Schweiz. Unterrichtswissenschaft, 31(2), 122–141.

    Google Scholar 

  • Depaepe, F., Verschaffel, L. & Kelchtermanns, G. (2013). Pedagogical content knowledge: A systematic review of the way in which the concept has pervaded mathematical educational research. Teaching and Teacher Education, 34, 12–25.

    Article  Google Scholar 

  • Döhrmann, M., Kaiser, G. & Blömeke, S. (2014). The conceptualisation of mathematics competencies in the international teacher education study TEDSM. In S. Blömeke , F.J. Hsieh, G. Kaiser, & W. H. Schmidt (Eds.). International Perspectives on Teacher Knowledge, Beliefs and Opportunities to Learn (pp. 431456). Dordrecht, The Netherlands: Springer.

  • Even, R. & Ball, D. L. (Eds.). (2008). The professional education and development of teachers of mathematics. The 15th ICMI Study. New York, NY: Springer.

  • Goldman, R., Pea, R., Barron, B. & Denny, S. J. (Eds.). (2007). Video research in the learning sciences. Mahwah, NJ: Lawrence Erlbaum.

  • Hiebert, J., Gallimore, R., Garnier, H., Givven, K. B., Hollingsworth, H., Jacobs, J., Chui, A. M.-Y., Wearne, D., Smith, M., Manaster, A., Tseng, E., Etterbeek, W., Manaster, C., Gonzales, P. & Stigler, J. W. (2003). Teaching mathematics in seven countries: Results from the TIMSS 1999 Video Study. Washington, D.C.: U.S. Department of Education, National Center for Education Statistics.

    Google Scholar 

  • Jacobs, V. R., Lamb, L. L. & Philipp, R. A. (2010). Professional noticing of children’s mathematical thinking. Journal for Research in Mathematics Education, 2(41), 169–202.

    Google Scholar 

  • Kaiser, G.,  Benthien, J., Döhrmann, M., König, J. & Blömeke, S. (2013). Expert Ratings as an Instrument for Validating Results of Video-Based Testing. In A. M. Lindmeier & A. Heinze (Eds.), Proceedings of the 37th Conference of the International Group for the Psychology of Mathematics Education PME in Kiel/Germany (Vol. 5, pp. 83). Kiel, German: PME.

  • Kane, M. (1992). An argument-based approach to validation. Psychological Bulletin, 112, 527–535.

    Article  Google Scholar 

  • Kersting, N. B. (2008). Using video clips of mathematics classroom instruction as item prompts to measure Teachers’ knowledge of teaching mathematics. Educational and Psychological Measurement, 68(5), 845–861.

    Article  Google Scholar 

  • König, J., Blömeke, S., Klein, P., Suhl, U., Busse, A. & Kaiser, G. (2014). Is teachers’ general pedagogical knowledge a premise for noticing and interpreting classroom situations? A video-based assessment approach. Teaching and Teacher Education, 38, 76–88.

  • Kunter, M., Baumert, J., Blum, W., Klusmann, U., Krauss, S. & Neubrand, M. (Eds.). (2013). Cognitive activation in the mathematics classroom and professional competence of teachers. Results from the COACTIV project. New York, NY: Springer.

  • Kunter, M. & Voss, T. (2013). The model of instructional quality in COACTIV: A multicriteria analysis. In M. Kunter, J. Baumert, W. Blum, U. Klusmann, S. Krauss & M. Neubrand (Eds.), Cognitive activation in the mathematics classroom and professional competence of teachers. Results from the COACTIV project (pp. 79–96). New York, NY: Springer.

  • Li, Y. & Kaiser, G. (Eds.) (2011). Expertise in Mathematics Instruction: An International Perspective. New York, NY: Springer.

  • Livingston, C. & Borko, H. (1989). Expert-novice differences in teaching: A cognitive analysis and implications for teacher education. Journal of Teacher Education, 40, 36–42.

    Article  Google Scholar 

  • Rowland, T. & Ruthven, K. (Eds.). (2011). Mathematical knowledge in teaching. Dordrecht, The Netherlands: Springer.

  • Sabers, D. S., Cushing, K. S. & Berliner, D. C. (1991). Differences among teachers in a task characterized by simultaneity, multidimensionality, and immediacy. American Educational Research Journal, 28, 63–88.

    Google Scholar 

  • Santagata, R. & Guarino, J. (2011). Using video to teach future teachers to learn from teaching. ZDM – The International Journal on Mathematics Education, 41(1), 133–146.

    Article  Google Scholar 

  • Schoenfeld, A. H. (2011). How we think. A theory of goal-oriented decision making and its educational applications. New York, NY: Routledge.

  • Schoenfeld, A. H. & Kilpatrick, J. (2008). Toward a theory of proficiency in teaching mathematics. In D. Tirosh & T. Wood (Eds.), International handbook of mathematics teacher education: Vol. 2. Tools and processes in mathematics teacher education (pp. 321–354). Rotterdam, The Netherlands: Sense Publishers.

  • Sherin, M. G., Jacobs, V. R. & Philipp, R. A. (Eds.). (2011). Mathematics teacher noticing. Seeing through Teachers’ eyes. New York, NY: Routledge.

  • Shulman, L. S. (1986). Those who understand: Knowledge growth in teaching. Educational Researcher, 15(2), 4–14.

    Article  Google Scholar 

  • Shulman, L. S. (1987). Knowledge and teaching: Foundations of the new reform. Harvard Educational Review, 57, 1–21.

    Google Scholar 

  • Van Es, E. (2011). A framework for learning to notice student thinking. In M. G. Sherin, V. R. Jacobs & R. A. Philipp (Eds.), Mathematics teacher noticing. Seeing through teachers’ eyes (pp. 134–151). New York, NY: Routledge.

  • Van Es, E. A. & Sherin, M. G. (2002). Learning to notice: Scaffolding new teachers‘interpretations of classroom interactions. Journal of Technology and Teacher Education, 10(4), 571–596.

    Google Scholar 

  • Weinert, F. E. (2001). Concept of competence: A conceptual clarification. In D. S. Rychen & L. H. Salganik (Eds.), Defining and selecting key competencies (pp. 45–66). Göttingen, German: Hogrefe.

  • Wetzel, E. & Carstensen, C. H. (2014). Reversed thresholds in partial credit models: A reason for collapsing categories? Assessment. Advance online publication. doi:10.1177/1073191114530775.

  • Wood, T. (Ed.). (2008). The international handbook of mathematics teacher education. Rotterdam, The Netherlands: Sense.

Download references

Author information

Authors and Affiliations

  1. Fakultät für Erziehungswissenschaft, Universität Hamburg, Binderstr. 34, 20146, Hamburg, Germany

    Gabriele Kaiser & Andreas Busse

  2. Institut für Didaktik der Mathematik und des Sachunterrichts, Universität Vechta, Driverstraße 22, 49377, Vechta, Germany

    Jessica Hoth

  3. Universität zu Köln, Gronewaldstr. 2, 50931, Köln, Germany

    Johannes König

  4. Centre for Educational Measurement (CEMO), University of Oslo, Niels Henrik Abels Hus, Moltke Moes vei 35, Postbox 1161 Blindern, Oslo, 0318, Norway

    Sigrid Blömeke

Authors
  1. Gabriele Kaiser
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Andreas Busse
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jessica Hoth
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Johannes König
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Sigrid Blömeke
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Andreas Busse.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kaiser, G., Busse, A., Hoth, J. et al. About the Complexities of Video-Based Assessments: Theoretical and Methodological Approaches to Overcoming Shortcomings of Research on Teachers’ Competence. Int J of Sci and Math Educ 13, 369–387 (2015). https://doi.org/10.1007/s10763-015-9616-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10763-015-9616-7

Keywords

Search

Navigation