Advertisement

Where Low and High Inference Data Converge: Validation of CLASS Assessment of Mathematics Instruction Using Mobile Eye Tracking with Expert and Novice Teachers

  • Kai S. CortinaEmail author
  • Kevin F. Miller
  • Ryan McKenzie
  • Alanna Epstein
Article

Abstract

Classroom observation research and research on teacher expertise are similar in their reliance on observational data with high-inference procedure to assess the quality of instruction. Expertise research usually uses low-inference measures like eye tracking to identify qualitative difference between expert and novice behaviors and cognition. In this study, we used mobile eye-tracking technology to create a low inference quality indicator for the comparison of experienced and student teachers. The distribution of visual fixations on students was measured using Gini coefficients based on the observation of van den Bogert, van Bruggen, Kostons, and Jochems (Teacher and Teacher Education, 37, 208–216, 2014) that expert teachers show better classroom monitoring. Results confirm that student teachers have a higher Gini coefficient than experienced teachers indicating weaker classroom monitoring. However, the Gini coefficient did not correlate in the predicted way with trained observer coding of video footage of the same classrooms using the Classroom Assessment Scoring System (CLASS) (Pianta, Hamre, Haynes, Mintz, & La Paro, 2007) although the mean differences in behavioral management were higher for the experienced teachers as expected. The CLASS dimension Quality of Feedback was significantly related to the Gini coefficient as an interaction with expertise: Only for novice teachers that a high quality of feedback was negatively associated with monitoring of the classroom.

Keywords

Mobile eye tracking Classroom observational assessment CLASS Expert–novice paradigm Quality of feedback 

References

  1. Aiken, L. S. & West, S. G. (1991). Multiple regression: Testing and interpreting interactions. Newbury Park, CA: Sage.Google Scholar
  2. Allen, J. P., Pianta, R. C., Gregory, A., Mikami, A. Y. & Lun, J. (2011). An interaction-based approach to enhancing secondary school instruction and student achievement. Science, 333, 1034–1037.CrossRefGoogle Scholar
  3. Ben-Chaim, D. & Zoller, U. (2001). Self-perception versus students’ perception of teachers’ personal style in college science and mathematics courses. Research in Science Education, 31(3), 437–454. doi: 10.1023/A:1013172329170.CrossRefGoogle Scholar
  4. Berliner, D. C. (2001). Learning about and learning from expert teachers. International Journal of Educational Research, 35, 463–482.CrossRefGoogle Scholar
  5. Blömeke, S., Felbrich, A., Müller, C., Kaiser, G. & Lehmann, R. (2008). Effectiveness of teacher education. ZDM Mathematics Education, 40, 719–734.CrossRefGoogle Scholar
  6. Borko, H. & Livingston, C. (1989). Cognition and improvisation: Differences in mathematics instruction by expert and novice teachers. American Educational Research Journal, 26, 473–498.CrossRefGoogle Scholar
  7. Brophy, J. E & Good, T. L. (1969). Teacher–child dyadic interaction: A manual for coding classroom behavior. Report Series No. 27. Retrieved from ERIC database. (ED042688)Google Scholar
  8. Brophy, J. E. & Good, T. L. (1974). Teacher-student relationships: Causes and consequences. New York, NY: Holt, Rinehart and Winston.Google Scholar
  9. Brophy, J. E. & Good, T. L. (1986). Teacher behavior and student achievement. In M. C. Wittrock (Ed.), Handbook of research on teaching (3rd ed., pp. 328–375). New York, NY: MacMillan.Google Scholar
  10. Bromme, R. (2001). Teacher expertise. In N. J. Smelser, P. B. Baltes & F. E. Weinert (Eds.), International encyclopedia of the behavioral sciences: Education (pp. 15459–15465). London, England: Pergamon.CrossRefGoogle Scholar
  11. Bromme, R. & Dobslaw, G. (2003). Teachers’ instructional quality and their explanation of students’ understanding. In M. Kompf & P. Denicolo (Eds.), Teacher thinking twenty years on: Revisiting persisting problems and advances in education (pp. 25–36). Liss, NL: Swets & Zeitlinger.Google Scholar
  12. Campbell, R. J., Kyriakides, L., Muijs, R. D. & Robinson, W. (2003). Differential teacher effectiveness: Towards a model for research and teacher appraisal. Oxford Review of Education, 29, 347–362.CrossRefGoogle Scholar
  13. Casabianca, J. M., McCaffrey, D. F., Gitomer, D. H., Bell, C. A., Hamre, B. K. & Pianta, R. B. (2013). Effect of observation mode on measures of secondary mathematics teaching. Educational and Psychological Measurement, 73(5), 757–783. doi: 10.1177/0013164413486987.CrossRefGoogle Scholar
  14. Curby, T. W., Rimm-Kaufman, S. E. & Abry, T. (2013). Do emotional support and classroom organization earlier in the year set the stage of higher quality instruction? Journal of School Psychology, 51, 557–569.CrossRefGoogle Scholar
  15. Doyle, W. (1983). Academic work. Review of Educational Research, 53, 159–199.CrossRefGoogle Scholar
  16. Duchowski, A. T. (2007). Eye tracking methodology: Theory and practice. London, England: Springer.Google Scholar
  17. Ericsson, K. A. (2006). Development of professional expertise: Toward measurement of expert performance and design of optimal learning environments. Cambridge, UK: Cambridge University Press.Google Scholar
  18. Flanders, N. A. (1970). Analyzing teaching behavior. Reading, MA: Addison-Wesley.Google Scholar
  19. Fraser, B. J. (1991). Two decades of classroom environment research. In B. J. Fraser & H. J. Walberg (Eds.), Educational environments: Evaluation, antecedents and consequences (pp. 3–27). Elmsford, NY: Pergamon Press.Google Scholar
  20. Gegenfurtner, A., Lehtinen, E. & Säljö, R. (2011). Expertise differences in the comprehension of visualizations: A meta-analysis of eye-tracking research in professional domains. Educational Psychology Review, 23, 523–552.CrossRefGoogle Scholar
  21. Hafen, C. A., Allen, J. P., Mikami, A. Y., Gregory, A., Hamre, B. & Pianta, R. C. (2012). The pivotal role of adolescent autonomy in secondary school classrooms. Journal of Youth and Adolescence, 41, 245–255.CrossRefGoogle Scholar
  22. Hill, H. C., Rowan, B. & Ball, D. B. (2005). Effect of teachers’ mathematical knowledge for teaching on student achievement. Americal Educational Research Journal, 42, 371-406. Jarodzka, H., Scheiter, K., Gerjets, P., & Van Gog, T. (2010). In the eyes of the beholder: How experts and novices interpret dynamic stimuli. Learning and Instruction, 20, 146–154.Google Scholar
  23. Kunter, M. & Baumert, J. (2006). Who is the expert? Construct and criteria validity of student and teacher ratings of instruction. Learning Environments Research, 9, 231–251.CrossRefGoogle Scholar
  24. Miller, K. F. & Zhou, X. (2007). Learning from classroom video: What makes it compelling and what makes it hard. In R. Goldman, R. Pea, B. Barron & S. J. Derry (Eds.), Video research in the learning sciences (pp. 321–334). Mahwah, NJ: Lawrence Erlbaum.Google Scholar
  25. Miller, K. F. & Correa, C. (2010, June). Attention in the classroom. Teacher eye movements as an index of situation awareness. Poster presented at the 5. IES research conference, National Harbor, MD.Google Scholar
  26. Muijs, D. & Reynolds, D. (2001). Effective teaching: Evidence and practice. London, England: Sage.Google Scholar
  27. Pianta, R. C., Hamre, B. K., Haynes, N. J., Mintz, S. L. & La Paro, K. M. (2007). Classroom assessment scoring system manual, middle/secondary version. Charlottesville, VA: University of Virginia.Google Scholar
  28. Pianta, R. C., La Paro, K. M. & Hamre, B. K. (2008). Classroom assessment scoring system, manual, K–3. Baltimore, MD: Brookes.Google Scholar
  29. Polikoff, M. S. & Porter, A. C. (2014). Instructional alignment as a measure of teaching quality. Educational Evaluation and Policy Analysis. Advance online publication. doi: 10.3102/0162373714531851.
  30. Sabers, D. S., Cushing, K. S. & Berliner, D. C. (1991). Differences among teachers in a task characterized by simultaneity, multidimensionality, and immediacy. American Educational Research Journal, 28(1), 63–88.Google Scholar
  31. Sadler, T. D. (2006). “I won’t last three weeks”: Preservice science teachers reflect on their student-teaching experiences. Journal of Science Teacher Education, 17(3), 217–241.CrossRefGoogle Scholar
  32. Seidel, T. & Shavelson, R. J. (2007). Teaching effectiveness research in the past decade: The role of theory and research design in disentangling meta-analysis results. Review of Educational Research, 77, 454–499.CrossRefGoogle Scholar
  33. Shulman, L. S. (1986). Those who understand: Knowledge growth in teaching. Educational Researcher, 15, 4-14.Google Scholar
  34. van Gog, T., Kester, L., Nievelstein, F., Giesbers, B. & Paas, F. (2009). Uncovering cognitive processes: Different techniques that can contribute to cognitive load research and instruction. Computers in Human Behavior, 25, 325–331.CrossRefGoogle Scholar
  35. van den Bogert, N., van Bruggen, J., Kostons, D. & Jochems, W. (2014). First steps into understanding teachers’ visual perception of classroom events. Teacher and Teacher Education, 37, 208–216.CrossRefGoogle Scholar
  36. Wang, Z., Miller, K. F., & Cortina, K. S. (2013). Using the LENA in teacher training: promoting student involvement through automated feedback. Unterrichtswissenschaft, 41, 290-305.Google Scholar
  37. Yamamoto, T. & Imai-Matsumura, K. (2012). Teachers’ gaze and awareness of students’ behavior: Using an eye tracker. Innovative Teaching, 2. Retrieved from http://www.amsciepub.com/doi/full/10.2466/01.IT.2.6.

Copyright information

© Springer Science + Business Media B.V. 2015

Authors and Affiliations

  • Kai S. Cortina
    • 1
    Email author
  • Kevin F. Miller
    • 1
  • Ryan McKenzie
    • 1
  • Alanna Epstein
    • 1
  1. 1.Department of PsychologyUniversity of MichiganAnn ArborUSA

Personalised recommendations