Advertisement

ZDM

, Volume 44, Issue 3, pp 427–441 | Cite as

A validation study of the use of mathematical knowledge for teaching measures in Ireland

  • Seán Delaney
Original Article

Abstract

Researchers who study mathematical knowledge for teaching (MKT) are interested in how teachers deploy their mathematical knowledge in the classroom to enhance instruction and student learning. However, little data exists on how teachers’ scores on the US-developed measures relate to classroom instruction in other countries. This article documents a validation study of Irish teachers’ scores on measures of MKT that were adapted for use in Ireland. A validity argument is made identifying elemental, structural and ecological assumptions. The argument is evaluated using qualitative and quantitative data to analyse inferences related to the three assumptions. The data confirmed the elemental assumption but confirming the structural and ecological assumptions was more difficult. Only a weak association was found between teachers’ MKT scores and the mathematical quality of instruction. Possible reasons for this are outlined and challenges in validating the use of measures are identified.

Keywords

Mathematical knowledge for teaching Measures Mathematical quality of instruction Validity Validation study Cross-cultural Elementary school 

Abbreviations

3-D

Three-dimensional

CCK

Common content knowledge

CK

Content knowledge

COACTIV

Cognitive activation in the classroom

IRT

Item response theory

KCS

Knowledge of content and students

KCT

Knowledge of content and teaching

MKT

Mathematical knowledge for teaching

MQI

Mathematical quality of instruction

OECD

Organisation for Economic Co-Operation and Development

PISA

Programme for International Student Assessment

SCK

Specialized content knowledge

TEDS-M

Teacher Education Study in Mathematics

TIMSS

Trends in International Mathematics and Science Study

US

United States

Notes

Acknowledgments

This article is based on the author’s doctoral dissertation submitted to the University of Michigan. The author acknowledges his doctoral advisor, Deborah Loewenberg Ball for her support, as well as other members of his doctoral committee: Magdalene Lampert, Heather C Hill, and Kathleen M. Sutcliffe. He also acknowledges the contribution of Gabriele Kaiser and three anonymous reviewers for comments which substantially improved the article.

References

  1. An, S., Kulm, G., & Wu, Z. (2004). The pedagogical content knowledge of middle school, mathematics teachers in China and the US. Journal of Mathematics Teacher Education, 7, 145–172.CrossRefGoogle Scholar
  2. Andrews, P. (2011). The cultural location of teachers’ mathematical knowledge: Another hidden variable in mathematics education research? In T. Rowland & K. Ruthven (Eds.), Mathematical knowledge in teaching (pp. 99–118). London and New York: Springer Science + Business Media.CrossRefGoogle Scholar
  3. Ball, D. L., & Bass, H. (2003). Toward a practice-based theory of mathematical knowledge for teaching. Edmonton, AB: Paper presented at the annual meeting of the Canadian Mathematics Education Study Group.Google Scholar
  4. Ball, D. L., Thames, M. H., & Phelps, G. (2008). Content knowledge for teaching: What makes it special? Journal of Teacher Education, 59(5), 389–407.CrossRefGoogle Scholar
  5. Baumert, J., Kunter, M., Blum, W., Brunner, M., Voss, T., Jordan, A., et al. (2010). Teachers’ mathematical knowledge, cognitive activation in the classroom, and student progress. American Educational Research Journal, 47(1), 133–180.CrossRefGoogle Scholar
  6. Blömeke, S., Felbrich, A., Müller, C., Kaiser, G., & Lehmann, R. (2008). Effectiveness of teacher education: State of research, measurement issues and consequences for future studies. ZDM: The international journal on mathematics education, 40, 719–734.Google Scholar
  7. Blunk, M. L., & Hill, H. C. (2007). The mathematical quality of instruction (MQI) video coding tool: Results from validation and scale building. Paper presented at the American Educational Association Annual Conference, Chicago, IL.Google Scholar
  8. Bock, R. D., Thissen, D., & Zimowski, M. F. (1997). IRT estimation of domain scores. Journal of Educational Measurement, 34(3), 197–211.CrossRefGoogle Scholar
  9. Cronbach, L. J. (1971). Test validation. In R. L. Thorndike (Ed.), Educational measurement (2nd ed., pp. 443–507). Washington, DC: American Council on Education.Google Scholar
  10. Delaney, S. (2008). Adapting and using US measures to study Irish teachers’ mathematical knowledge for teaching. Ph.D., Unpublished doctoral dissertation, University of Michigan, Ann Arbor, MI.Google Scholar
  11. Delaney, S., Ball, D. L., Hill, H. C., Schilling, S. G., & Zopf, D. (2008). Mathematical knowledge for teaching: Adapting US measures for use in Ireland. Journal of Mathematics Teacher Education, 11(3), 171–197.CrossRefGoogle Scholar
  12. Gorsuch, R. L. (1983). Factor analysis (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum Associates.Google Scholar
  13. Hambleton, R. K., & Kanjee, A. (1995). Increasing the validity of cross-cultural assessments: Use of improved methods for test adaptations. European Journal of Psychological Assessment, 11(3), 147–157.CrossRefGoogle Scholar
  14. Hill, H. C., Ball, D. L., & Schilling, S. G. (2008). Unpacking pedagogical content knowledge: Conceptualizing and measuring teachers’ topic-specific knowledge of students. Journal for Research in Mathematics Education, 39(4), 372–400.Google Scholar
  15. Hill, H. C., Blunk, M. L., Charalambous, C. Y., Lewis, J., Phelps, G. C., Sleep, L., & Ball, D. L. (2008). Mathematical knowledge for teaching and the mathematical quality of instruction: An exploratory study. Cognition and Instruction, 26(4), 430–511.Google Scholar
  16. Hill, H. C., Dean, C., & Goffney, I. M. (2007). Assessing elemental and structural validity: Data from teachers, non-teachers, and mathematicians. Measurement: Interdisciplinary Research and Perspectives, 5(2), 81–92.CrossRefGoogle Scholar
  17. Hill, H. C., Schilling, S. G., & Ball, D. L. (2004). Developing measures of teachers’ knowledge for teaching. The Elementary School Journal, 105(1), 11–30.CrossRefGoogle Scholar
  18. Hitchcock, J. H., Nastasi, B. K., Dai, D. Y., Newman, J., Jayasena, A., Bernstein-Moore, R., et al. (2005). Illustrating a mixed-method approach for validating culturally specific constructs. Journal of School Psychology, 43, 259–278.CrossRefGoogle Scholar
  19. Kane, M. T. (2004). Certification testing as an illustration of argument-based validation. Measurement: Interdisciplinary Research and Perspectives, 2(3), 135–170.CrossRefGoogle Scholar
  20. Kane, M. T. (2006). Validation. In R. L. Brennan (Ed.), Educational measurement: Fourth edition (4th ed., pp. 17–64). Westport, CT: American Council on Education and Praeger Publishers.Google Scholar
  21. Kane, M. T. (2008). Terminology, emphasis, and utility in validation. Educational Researcher, 37(2), 76–82.CrossRefGoogle Scholar
  22. Krauss, S., Baumert, J., & Blum, W. (2008). Secondary mathematics teachers’ pedagogical content knowledge and content knowledge: validation of the COACTIV constructs. ZDM: The International Journal on Mathematics Education, 40, 873–892.CrossRefGoogle Scholar
  23. Learning Mathematics for Teaching. (2006). A coding rubric for measuring the quality of mathematics in instruction (Technical Report LMT 1.06). Ann Arbor, MI: University of Michigan, School of Education. See http://sitemaker.umich.edu/lmt/files/lmt-mqi_description_of_codes.pdf.
  24. Learning Mathematics for Teaching. (2011). Measuring the mathematical quality of instruction. Journal of Mathematics Teacher Education, 14(1), 25–47.CrossRefGoogle Scholar
  25. Li, Y., & Even, R. (2011). Approaches and practices in developing teachers’ expertise in mathematics instruction: An introduction. ZDM: The international journal on mathematics education, 43, 759–762.Google Scholar
  26. Lissitz, R. W., & Samuelsen, K. (2007). A suggested change in terminology and emphasis regarding validity and education. Educational Researcher, 36(8), 437–448.CrossRefGoogle Scholar
  27. Ma, L. (1999). Knowing and teaching elementary mathematics. Mahwah, New Jersey: Lawrence Erlbaum Associates Inc.Google Scholar
  28. Messick, S. (1989). Meaning and values in test validation: The science and ethics of assessment. Educational Researcher, 18(2), 5–11.Google Scholar
  29. Morris, A. K., Hiebert, J., & Spitzer, S. M. (2009). Mathematical knowledge for teaching in planning and evaluating instruction: What can preservice teachers learn? Journal for Research in Mathematics Education, 40(5), 491–529.Google Scholar
  30. Muthén, L. K., & Muthén, B. O. (1998–2007). MPlus (Version 5). Los Angeles: Muthén & Muthén.Google Scholar
  31. OECD. (2009). PISA 2006: Technical report. Paris: OECD.Google Scholar
  32. Organisation for Economic Co-operation and Development (OECD). (2004). Reviews of national policies for education: Chile. Paris: Author.Google Scholar
  33. Organisation for Economic Co-operation and Development (OECD). (2008). Reviews of national policies for education: South Africa. Paris: Author.Google Scholar
  34. Patton, M. Q. (2002). Qualitative research and evaluation methods (3rd ed.). Thousand Oaks, CA: Sage Publications, Inc.Google Scholar
  35. Pepin, B. (2011). How educational systems and cultures mediate teacher knowledge: ‘Listening’ in English, French and German classrooms. In T. Rowland & K. Ruthven (Eds.), Mathematical knowledge in teaching (pp. 119–137). London and New York: Springer Science + Business Media.CrossRefGoogle Scholar
  36. Rowland, T., Huckstep, P., & Thwaites, A. (2005). Elementary teachers’ mathematics subject knowledge: The knowledge quartet and the case of Naomi. Journal of Mathematics Teacher Education, 8(3), 255–281.CrossRefGoogle Scholar
  37. Schilling, S. G. (2002). ORDFAC software. Ann Arbor, MI: Author.Google Scholar
  38. Schilling, S. G., Blunk, M. L., & Hill, H. C. (2007). Test validation and the MKT measures: Generalizations and conclusions. Measurement: Interdisciplinary Research and Perspectives, 5(2), 118–128.CrossRefGoogle Scholar
  39. Schilling, S. G., & Hill, H. C. (2007). Assessing measures of mathematical knowledge for teaching: A validity argument approach. Measurement: Interdisciplinary Research and Perspectives, 5(2), 70–80.CrossRefGoogle Scholar
  40. Schmidt, W. H., Houang, R., Cogan, L., Blömeke, S., Tatto, M. T., Hsieh, F. J., et al. (2008). Opportunity to learn in the preparation of mathematics teachers: its structure and how it varies across six countries. ZDM: The international journal on mathematics education, 40, 735-747.Google Scholar
  41. Schmidt, W. H., Tatto, M. T., Bankov, K., Blömeke, S., Cedillo, T., Cogan, L., et al. (2007). The preparation gap: Teacher education for middle school mathematics in six countries (MT21 Report). East Lansing, MI: Center for Research in Mathematics and Science Education, Michigan State University.Google Scholar
  42. Shulman, L. S. (1986). Those who understand: Knowledge growth in teaching. Educational Researcher, 15(2), 4–14.Google Scholar
  43. Shulman, L. S. (1987). Knowledge and teaching: Foundations of the new reform. Harvard Educational Review, 57(1), 1–22.Google Scholar
  44. Sireci, S. G. (2007). On validity theory and test validation. Educational Researcher, 36(8), 477–481.CrossRefGoogle Scholar
  45. Stigler, J. W., & Hiebert, J. (1999). The teaching gap: Best ideas from the world’s teachers for improving education in the classroom. New York: The Free Press.Google Scholar
  46. Straus, M. A. (1969). Phenomenal identity and conceptual equivalence of measurement in cross-national comparative research. Journal of Marriage and the Family, 31(2), 233–239.CrossRefGoogle Scholar
  47. Stylianides, A. J., & Delaney, S. (2011). The cultural dimension of teachers’ mathematical knowledge. In T. Rowland & K. Ruthven (Eds.), Mathematical knowledge in teaching (Vol. 50, pp. 179–191). London, New York: Springer Science + Business Media.CrossRefGoogle Scholar
  48. Tatto, M. T., Schwille, J., Senk, S. L., Ingvarson, L., Peck, R., & Rowley, G. (2008). Teacher education and development study in mathematics (TEDS-M): Policy, practice, and readiness to teach primary and secondary mathematics conceptual framework. East Lansing, MI: Teacher Education and Development International Study Center, College of Education, Michigan State University.Google Scholar
  49. Turner, F., & Rowland, T. (2011). The knowledge quartet as an organising framework for developing and deepening teachers’ mathematical knowledge. In T. Rowland & K. Ruthven (Eds.), Mathematical knowledge in teaching (Vol. 50, pp. 195–212). London and New York: Springer Science + Business.CrossRefGoogle Scholar
  50. van de Vijver, F., & Leung, K. (1997). Methods and data analysis for cross-cultural research. Thousand Oaks, CA: Sage Publications, Inc.Google Scholar
  51. Yang, X., & Leung, F. (2011). Mathematics teaching expertise development approaches and practices: Similarities and differences between Western and Eastern countries. ZDM: The international journal on mathematics education, 43, 1007–1015.Google Scholar
  52. Zimowski, M. F., Muraki, E., Mislevy, R. J., & Bock, R. D. (2003). Bilog-MG 3.0; Item analysis and test scoring with binary logistic models for multiple groups. Mooresville, IN: Scientific Software International.Google Scholar

Copyright information

© FIZ Karlsruhe 2012

Authors and Affiliations

  1. 1.Coláiste MhuireMarino Institute of EducationDublinIreland

Personalised recommendations