Springer Nature is making SARS-CoV-2 and COVID-19 research free. View research | View latest news | Sign up for updates

Modelling mathematics problem solving item responses using a multidimensional IRT model

Abstract

This research examined students’ responses to mathematics problem-solving tasks and applied a general multidimensional IRT model at the response category level. In doing so, cognitive processes were identified and modelled through item response modelling to extract more information than would be provided using conventional practices in scoring items. More specifically, the study consisted of two parts. The first part involved the development of a mathematics problem-solving framework that was theoretically grounded, drawing upon research in mathematics education and cognitive psychology. The framework was then used as the basis for item development. The second part of the research involved the analysis of the item response data. It was demonstrated that multidimensional IRT models were powerful tools for extracting information from a limited number of item responses. A problem-solving profile for each student could be constructed from the results of IRT scaling.

This is a preview of subscription content, log in to check access.

References

  1. Adams, R. J., Wilson, M. R., & Wang, W. (1997). The multidimensional random coefficients multinomial logit model.Applied Psychological Measurement, 21, 1–23.

  2. Adams, R. J., & Wu, M. L. (2002).PISA 2000 technical report. Paris: OECD.

  3. Bond, T.G., & Fox, C. M. (2007)Applying the Rasch model: Fundamental measurement in the human sciences (2nd ed). Mahwah, NJ: Lawrence Erlbaum.

  4. Bonotto, C. (2003). Suspension of sense-making in mathematical word problem solving: A possible remedy. Retrieved August 16, 2003, from http://math.unipa.it/~grim/Jbonotto

  5. Cai, J., & Silver, E. A. (1995). Solution processes and interpretations of solutions in solving division-with-remainder story problems: Do Chinese and U.S. students have similar difficulties?Journal for Research in Mathematics Education, 26, 491–497.

  6. Carroll, J. B. (1945). The effect of difficulty and chance success on correlations between items or between tests.Psychometrika, 10, 1–19.

  7. Carroll, J. B. (1993).Human cognitive abilities. A survey of factor-analytic studies. Cambridge: Cambridge University Press.

  8. Carroll, J. B. (1996). Mathematical abilities: Some results from factor analysis. In R. J. Sternberg & B.-Z. Talia (Eds.),The nature of mathematical thinking. Mahwah, NJ: Lawrence Erlbaum.

  9. Collis, K. F., & Romberg, T. A. (1992).Collis-Romberg mathematical problem solving profiles. Melbourne: Australian Council for Educational Research.

  10. Cornish, G., & Wines, R. (1977).Mathematics profiles series: Operations test teachers handbook. Melbourne: Australian Council for Educational Research.

  11. De Lange, J. (1996). Using and applying mathematics in education. In A. J. Bishop, K. Clements, C. Keitel, J. Kilpatrick, & C. Laborde (Eds.),International handbook of mathematics education (pp. 49–98). Dordrecht, The Netherlands: Kluwer.

  12. Ellerton, N. F., & Clarkson, P. C. (1996). Language factors in mathematics teaching and learning. In A. J. Bishop, K. Clements, C. Keitel, J. Kilpatrick, & C. Laborde (Eds.),International handbook of mathematics education (pp.987–1033). Dordrecht, The Netherlands: Kluwer.

  13. Embretsons, S. E. (1991). A multidimensional latent trait model for measuring learning and change.Psychometrika, 56, 495–515.

  14. Embretson, S. E. (1997). Multicomponent response models. In W. J. van der Linden & R. K. Hambleton (Eds.),Handbook of modern item response theory. New York: Springer-Verlag

  15. Fredriksen, J., Mislevy, R. J., & Bejar, I. (Eds.) (1991).Test theory for a new generation of tests. Hillsdale, NJ: Lawrence Erlbaum.

  16. Gravemeijer, K. (1999). How emergent models may foster the constitution of formal mathematics.Mathematical Thinking and Learning: An International Journal, 1(2), 155–177.

  17. Hambleton, R. K., & Rovinelli, R. J. (1986). Assessing the dimensionality of a set of test items.Applied Psychological Measurement, 10, 287–302.

  18. Heim, A. W. (1975).Psychological testing. London: Oxford University Press.

  19. Jöreskog, K. G., & Sörbom, D. (1979).Advances in factor analysis and structural equation models. Cambridge, MA: Abt.

  20. Lincare, J. M. (1998). Detecting multidimensionality: Which residual data-type works best?Journal of Outcome Measurement, 2(3), 266–283.

  21. Malone, J. A., Douglas, G. A., Kissane, B. V., & Mortlock, R. S. (1980). Measuring problem-solving ability. In S. Krulik & R. E. Reys (Eds.),Problem solving in school mathematics (pp. 204–215). Reston, VA: NCTM.

  22. Masters, G. N. (1982). A Rasch model for partial credit scoring.Psychometrika, 47, 149–174.

  23. Masters, G. N. & Doig, B. A. (1992). Understanding children’s mathematics: Some assessment tools. In G. Leder (Ed.),Assessment and learning of mathematics. Melbourne: Australian Council of Educational Research.

  24. Mayer, R. E., & Hegarty, M. (1996). In R. J. Sternberg & B.-Z. Talia (Eds.),The nature of mathematical thinking. Mahwah, NJ: Lawrence Erlbaum.

  25. McDonald, R. P., & Ahlawat, K. S. (1974). Difficulty factors in binary data.British Journal of Mathematical and Statistical Psychology, 27, 82–99.

  26. Nandakumar, R. (1994). Assessing latent trait unidimensionality of a set of items — Comparison of different approaches.Journal of Educational Measurements, 31, 1–18.

  27. National Council of Teachers of Mathematics. (1989).Curriculum and evaluation standards for school mathematics. Reston, VA: NCTM.

  28. Nesher, P. (1980). The stereotyped nature of school word problems.For the Learning of Mathematics, 1(1), 41–48.

  29. Newman, M. A. (1977). An analysis of sixth-grade pupils’ errors on written mathematical tasks.Victorian Institute for Educational Research Bulletin, 39, 31–43.

  30. Newman, M. A. (1983).Strategies for diagnosis and remediation. Sydney: Harcourt, Brace Jovanovich.

  31. OECD (2003).The PISA 2003 assessment framework. Paris: OECD.

  32. Polya, G. (1973).How to solve it: A new aspect of mathematical method. Princeton, NJ: Princeton University Press.

  33. Rasch, G. (1960).Probabilistic models for some intelligence and attainment tests. Copenhagen, Denmark: Danish Institute for Educational Research.

  34. Romberg, T., & de Lange, J. (1998).Mathematics in context. Chicago: Britannica Mathematics System.

  35. Schoenfeld, A. H. (1983). Episodes and executive decisions in mathematical problem solving. In R. Lesh & M. Landau, M. (Eds.),Acquisition of mathematics concepts and processes (pp. 345–395). New York: Academic.

  36. Schoenfeld, A. H. (1991). On mathematics as sense-making: An informal attack on the unfortunate divorce of formal and informal mathematics. In J. E. Voss, D. N. Perkins, & J. W. Segal (Eds.),Informal reasoning and education (pp. 311–343). Hillsdale, NJ: Lawrence Erlbaum.

  37. Silver, E. A. (1982). Knowledge organisation and mathematical problem solving. In F. K. Lester & J. Garafalo (Eds.),Mathematical problem solving: Issues in research. Philadelphia, PA: Franklin Institute Press.

  38. Smith, R. M., & Miao, C. Y. (1994). Assessing unidimensionality for Rasch measurement. In M. Wilson (Ed.),Objective measurement: Theory into practice (Vol. 2, pp. 316–327) Norwood, NJ: Ablex.

  39. Stacey, K., Groves, S., Bourke, S., & Doig, B. (1993).Profiles of problem solving. Melbourne: Australian Council for Educational Research.

  40. Treffers, A. (1986).Three dimensions. Dordrecht, The Netherlands: Reidel.

  41. Verhelst, N. D., Glas, C. A. W., & de Vries, H. H. (1997). A steps model to analyze partial credit. In W. J. van der Linden & R. K. Hambleton (Eds.),Handbook of modern item response theory. New York: Springer-Verlag.

  42. Verhelst, N. D. (2001).Some thoughts on reliability. Unpublished manuscript.

  43. Verschaffel, L., Greer, B. & de Corte E. (2000). Making sense of word problems. Lisse, Switzerland: Swets & Zeitlinger.

  44. Wang, W. (1998). Rasch analysis of distractors in multiple-choice items.Journal of Outcome Measurement, 2(1), 43–65.

  45. Whimbey, A., & Lochhead, J. (1991).Problem solving and comprehension (5th ed.) Hillsdale, NJ: Lawrence Erlbaum.

  46. Willmott, A. S., & Fowles, D. E. (1974).The objective interpretation of test performance. Windsor, UK: National Foundation for Educational Research Publishing.

  47. Linacre, M. J., & Wright, B. D. (2000). WINSTEPS Rasch measurement computer program [Computer software]., Chicago: MESA Press.

  48. Wright, B. D., & Masters, G. N. (1982).Rating scale analysis. Chicago: MESA Press.

  49. Wu, M. L. (2004).The application of item response theory to measure problem-solving proficiencies. Unpublished doctoral dissertation, The University of Melbourne.

  50. Wu, M. L., Adams, R. J., & Wilson, M. R. (1998). ConQuest: Multi-aspect test software [Computer software]. Melbourne: Australian Council for Educational Research.

Download references

Author information

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Wu, M., Adams, R. Modelling mathematics problem solving item responses using a multidimensional IRT model. Math Ed Res J 18, 93–113 (2006). https://doi.org/10.1007/BF03217438

Download citation

Keywords

  • Item Response
  • Item Response Theory
  • Item Response Theory Model
  • Realistic Mathematic Education
  • Information Processing Approach