Mathematics Education Research Journal

, Volume 18, Issue 2, pp 93–113 | Cite as

Modelling mathematics problem solving item responses using a multidimensional IRT model

  • Margaret Wu
  • Raymond Adams


This research examined students’ responses to mathematics problem-solving tasks and applied a general multidimensional IRT model at the response category level. In doing so, cognitive processes were identified and modelled through item response modelling to extract more information than would be provided using conventional practices in scoring items. More specifically, the study consisted of two parts. The first part involved the development of a mathematics problem-solving framework that was theoretically grounded, drawing upon research in mathematics education and cognitive psychology. The framework was then used as the basis for item development. The second part of the research involved the analysis of the item response data. It was demonstrated that multidimensional IRT models were powerful tools for extracting information from a limited number of item responses. A problem-solving profile for each student could be constructed from the results of IRT scaling.


Item Response Item Response Theory Item Response Theory Model Realistic Mathematic Education Information Processing Approach 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Adams, R. J., Wilson, M. R., & Wang, W. (1997). The multidimensional random coefficients multinomial logit model.Applied Psychological Measurement, 21, 1–23.CrossRefGoogle Scholar
  2. Adams, R. J., & Wu, M. L. (2002).PISA 2000 technical report. Paris: OECD.Google Scholar
  3. Bond, T.G., & Fox, C. M. (2007)Applying the Rasch model: Fundamental measurement in the human sciences (2nd ed). Mahwah, NJ: Lawrence Erlbaum.Google Scholar
  4. Bonotto, C. (2003). Suspension of sense-making in mathematical word problem solving: A possible remedy. Retrieved August 16, 2003, from Scholar
  5. Cai, J., & Silver, E. A. (1995). Solution processes and interpretations of solutions in solving division-with-remainder story problems: Do Chinese and U.S. students have similar difficulties?Journal for Research in Mathematics Education, 26, 491–497.CrossRefGoogle Scholar
  6. Carroll, J. B. (1945). The effect of difficulty and chance success on correlations between items or between tests.Psychometrika, 10, 1–19.CrossRefGoogle Scholar
  7. Carroll, J. B. (1993).Human cognitive abilities. A survey of factor-analytic studies. Cambridge: Cambridge University Press.CrossRefGoogle Scholar
  8. Carroll, J. B. (1996). Mathematical abilities: Some results from factor analysis. In R. J. Sternberg & B.-Z. Talia (Eds.),The nature of mathematical thinking. Mahwah, NJ: Lawrence Erlbaum.Google Scholar
  9. Collis, K. F., & Romberg, T. A. (1992).Collis-Romberg mathematical problem solving profiles. Melbourne: Australian Council for Educational Research.Google Scholar
  10. Cornish, G., & Wines, R. (1977).Mathematics profiles series: Operations test teachers handbook. Melbourne: Australian Council for Educational Research.Google Scholar
  11. De Lange, J. (1996). Using and applying mathematics in education. In A. J. Bishop, K. Clements, C. Keitel, J. Kilpatrick, & C. Laborde (Eds.),International handbook of mathematics education (pp. 49–98). Dordrecht, The Netherlands: Kluwer.Google Scholar
  12. Ellerton, N. F., & Clarkson, P. C. (1996). Language factors in mathematics teaching and learning. In A. J. Bishop, K. Clements, C. Keitel, J. Kilpatrick, & C. Laborde (Eds.),International handbook of mathematics education (pp.987–1033). Dordrecht, The Netherlands: Kluwer.Google Scholar
  13. Embretsons, S. E. (1991). A multidimensional latent trait model for measuring learning and change.Psychometrika, 56, 495–515.CrossRefGoogle Scholar
  14. Embretson, S. E. (1997). Multicomponent response models. In W. J. van der Linden & R. K. Hambleton (Eds.),Handbook of modern item response theory. New York: Springer-VerlagGoogle Scholar
  15. Fredriksen, J., Mislevy, R. J., & Bejar, I. (Eds.) (1991).Test theory for a new generation of tests. Hillsdale, NJ: Lawrence Erlbaum.Google Scholar
  16. Gravemeijer, K. (1999). How emergent models may foster the constitution of formal mathematics.Mathematical Thinking and Learning: An International Journal, 1(2), 155–177.CrossRefGoogle Scholar
  17. Hambleton, R. K., & Rovinelli, R. J. (1986). Assessing the dimensionality of a set of test items.Applied Psychological Measurement, 10, 287–302.CrossRefGoogle Scholar
  18. Heim, A. W. (1975).Psychological testing. London: Oxford University Press.Google Scholar
  19. Jöreskog, K. G., & Sörbom, D. (1979).Advances in factor analysis and structural equation models. Cambridge, MA: Abt.Google Scholar
  20. Lincare, J. M. (1998). Detecting multidimensionality: Which residual data-type works best?Journal of Outcome Measurement, 2(3), 266–283.Google Scholar
  21. Malone, J. A., Douglas, G. A., Kissane, B. V., & Mortlock, R. S. (1980). Measuring problem-solving ability. In S. Krulik & R. E. Reys (Eds.),Problem solving in school mathematics (pp. 204–215). Reston, VA: NCTM.Google Scholar
  22. Masters, G. N. (1982). A Rasch model for partial credit scoring.Psychometrika, 47, 149–174.CrossRefGoogle Scholar
  23. Masters, G. N. & Doig, B. A. (1992). Understanding children’s mathematics: Some assessment tools. In G. Leder (Ed.),Assessment and learning of mathematics. Melbourne: Australian Council of Educational Research.Google Scholar
  24. Mayer, R. E., & Hegarty, M. (1996). In R. J. Sternberg & B.-Z. Talia (Eds.),The nature of mathematical thinking. Mahwah, NJ: Lawrence Erlbaum.Google Scholar
  25. McDonald, R. P., & Ahlawat, K. S. (1974). Difficulty factors in binary data.British Journal of Mathematical and Statistical Psychology, 27, 82–99.Google Scholar
  26. Nandakumar, R. (1994). Assessing latent trait unidimensionality of a set of items — Comparison of different approaches.Journal of Educational Measurements, 31, 1–18.Google Scholar
  27. National Council of Teachers of Mathematics. (1989).Curriculum and evaluation standards for school mathematics. Reston, VA: NCTM.Google Scholar
  28. Nesher, P. (1980). The stereotyped nature of school word problems.For the Learning of Mathematics, 1(1), 41–48.Google Scholar
  29. Newman, M. A. (1977). An analysis of sixth-grade pupils’ errors on written mathematical tasks.Victorian Institute for Educational Research Bulletin, 39, 31–43.Google Scholar
  30. Newman, M. A. (1983).Strategies for diagnosis and remediation. Sydney: Harcourt, Brace Jovanovich.Google Scholar
  31. OECD (2003).The PISA 2003 assessment framework. Paris: OECD.Google Scholar
  32. Polya, G. (1973).How to solve it: A new aspect of mathematical method. Princeton, NJ: Princeton University Press.Google Scholar
  33. Rasch, G. (1960).Probabilistic models for some intelligence and attainment tests. Copenhagen, Denmark: Danish Institute for Educational Research.Google Scholar
  34. Romberg, T., & de Lange, J. (1998).Mathematics in context. Chicago: Britannica Mathematics System.Google Scholar
  35. Schoenfeld, A. H. (1983). Episodes and executive decisions in mathematical problem solving. In R. Lesh & M. Landau, M. (Eds.),Acquisition of mathematics concepts and processes (pp. 345–395). New York: Academic.Google Scholar
  36. Schoenfeld, A. H. (1991). On mathematics as sense-making: An informal attack on the unfortunate divorce of formal and informal mathematics. In J. E. Voss, D. N. Perkins, & J. W. Segal (Eds.),Informal reasoning and education (pp. 311–343). Hillsdale, NJ: Lawrence Erlbaum.Google Scholar
  37. Silver, E. A. (1982). Knowledge organisation and mathematical problem solving. In F. K. Lester & J. Garafalo (Eds.),Mathematical problem solving: Issues in research. Philadelphia, PA: Franklin Institute Press.Google Scholar
  38. Smith, R. M., & Miao, C. Y. (1994). Assessing unidimensionality for Rasch measurement. In M. Wilson (Ed.),Objective measurement: Theory into practice (Vol. 2, pp. 316–327) Norwood, NJ: Ablex.Google Scholar
  39. Stacey, K., Groves, S., Bourke, S., & Doig, B. (1993).Profiles of problem solving. Melbourne: Australian Council for Educational Research.Google Scholar
  40. Treffers, A. (1986).Three dimensions. Dordrecht, The Netherlands: Reidel.Google Scholar
  41. Verhelst, N. D., Glas, C. A. W., & de Vries, H. H. (1997). A steps model to analyze partial credit. In W. J. van der Linden & R. K. Hambleton (Eds.),Handbook of modern item response theory. New York: Springer-Verlag.Google Scholar
  42. Verhelst, N. D. (2001).Some thoughts on reliability. Unpublished manuscript.Google Scholar
  43. Verschaffel, L., Greer, B. & de Corte E. (2000). Making sense of word problems. Lisse, Switzerland: Swets & Zeitlinger.Google Scholar
  44. Wang, W. (1998). Rasch analysis of distractors in multiple-choice items.Journal of Outcome Measurement, 2(1), 43–65.Google Scholar
  45. Whimbey, A., & Lochhead, J. (1991).Problem solving and comprehension (5th ed.) Hillsdale, NJ: Lawrence Erlbaum.Google Scholar
  46. Willmott, A. S., & Fowles, D. E. (1974).The objective interpretation of test performance. Windsor, UK: National Foundation for Educational Research Publishing.Google Scholar
  47. Linacre, M. J., & Wright, B. D. (2000). WINSTEPS Rasch measurement computer program [Computer software]., Chicago: MESA Press.Google Scholar
  48. Wright, B. D., & Masters, G. N. (1982).Rating scale analysis. Chicago: MESA Press.Google Scholar
  49. Wu, M. L. (2004).The application of item response theory to measure problem-solving proficiencies. Unpublished doctoral dissertation, The University of Melbourne.Google Scholar
  50. Wu, M. L., Adams, R. J., & Wilson, M. R. (1998). ConQuest: Multi-aspect test software [Computer software]. Melbourne: Australian Council for Educational Research.Google Scholar

Copyright information

© Mathematics Education Research Group of Australasia Inc. 2006

Authors and Affiliations

  • Margaret Wu
    • 1
  • Raymond Adams
    • 1
  1. 1.Assessment Research CentreUniversity of MelbourneParkvilleAustralia

Personalised recommendations