# Modelling mathematics problem solving item responses using a multidimensional IRT model

- 384 Downloads
- 22 Citations

## Abstract

This research examined students’ responses to mathematics problem-solving tasks and applied a general multidimensional IRT model at the response category level. In doing so, cognitive processes were identified and modelled through item response modelling to extract more information than would be provided using conventional practices in scoring items. More specifically, the study consisted of two parts. The first part involved the development of a mathematics problem-solving framework that was theoretically grounded, drawing upon research in mathematics education and cognitive psychology. The framework was then used as the basis for item development. The second part of the research involved the analysis of the item response data. It was demonstrated that multidimensional IRT models were powerful tools for extracting information from a limited number of item responses. A problem-solving profile for each student could be constructed from the results of IRT scaling.

## Keywords

Item Response Item Response Theory Item Response Theory Model Realistic Mathematic Education Information Processing Approach## Preview

Unable to display preview. Download preview PDF.

## References

- Adams, R. J., Wilson, M. R., & Wang, W. (1997). The multidimensional random coefficients multinomial logit model.
*Applied Psychological Measurement, 21*, 1–23.CrossRefGoogle Scholar - Adams, R. J., & Wu, M. L. (2002).
*PISA 2000 technical report*. Paris: OECD.Google Scholar - Bond, T.G., & Fox, C. M. (2007)
*Applying the Rasch model: Fundamental measurement in the human sciences*(2^{nd}ed). Mahwah, NJ: Lawrence Erlbaum.Google Scholar - Bonotto, C. (2003). Suspension of sense-making in mathematical word problem solving: A possible remedy. Retrieved August 16, 2003, from http://math.unipa.it/~grim/JbonottoGoogle Scholar
- Cai, J., & Silver, E. A. (1995). Solution processes and interpretations of solutions in solving division-with-remainder story problems: Do Chinese and U.S. students have similar difficulties?
*Journal for Research in Mathematics Education, 26*, 491–497.CrossRefGoogle Scholar - Carroll, J. B. (1945). The effect of difficulty and chance success on correlations between items or between tests.
*Psychometrika, 10*, 1–19.CrossRefGoogle Scholar - Carroll, J. B. (1993).
*Human cognitive abilities. A survey of factor-analytic studies*. Cambridge: Cambridge University Press.CrossRefGoogle Scholar - Carroll, J. B. (1996). Mathematical abilities: Some results from factor analysis. In R. J. Sternberg & B.-Z. Talia (Eds.),
*The nature of mathematical thinking*. Mahwah, NJ: Lawrence Erlbaum.Google Scholar - Collis, K. F., & Romberg, T. A. (1992).
*Collis-Romberg mathematical problem solving profiles*. Melbourne: Australian Council for Educational Research.Google Scholar - Cornish, G., & Wines, R. (1977).
*Mathematics profiles series: Operations test teachers handbook*. Melbourne: Australian Council for Educational Research.Google Scholar - De Lange, J. (1996). Using and applying mathematics in education. In A. J. Bishop, K. Clements, C. Keitel, J. Kilpatrick, & C. Laborde (Eds.),
*International handbook of mathematics education*(pp. 49–98). Dordrecht, The Netherlands: Kluwer.Google Scholar - Ellerton, N. F., & Clarkson, P. C. (1996). Language factors in mathematics teaching and learning. In A. J. Bishop, K. Clements, C. Keitel, J. Kilpatrick, & C. Laborde (Eds.),
*International handbook of mathematics education*(pp.987–1033). Dordrecht, The Netherlands: Kluwer.Google Scholar - Embretsons, S. E. (1991). A multidimensional latent trait model for measuring learning and change.
*Psychometrika, 56*, 495–515.CrossRefGoogle Scholar - Embretson, S. E. (1997). Multicomponent response models. In W. J. van der Linden & R. K. Hambleton (Eds.),
*Handbook of modern item response theory*. New York: Springer-VerlagGoogle Scholar - Fredriksen, J., Mislevy, R. J., & Bejar, I. (Eds.) (1991).
*Test theory for a new generation of tests*. Hillsdale, NJ: Lawrence Erlbaum.Google Scholar - Gravemeijer, K. (1999). How emergent models may foster the constitution of formal mathematics.
*Mathematical Thinking and Learning: An International Journal, 1*(2), 155–177.CrossRefGoogle Scholar - Hambleton, R. K., & Rovinelli, R. J. (1986). Assessing the dimensionality of a set of test items.
*Applied Psychological Measurement, 10*, 287–302.CrossRefGoogle Scholar - Heim, A. W. (1975).
*Psychological testing*. London: Oxford University Press.Google Scholar - Jöreskog, K. G., & Sörbom, D. (1979).
*Advances in factor analysis and structural equation models*. Cambridge, MA: Abt.Google Scholar - Lincare, J. M. (1998). Detecting multidimensionality: Which residual data-type works best?
*Journal of Outcome Measurement*, 2(3), 266–283.Google Scholar - Malone, J. A., Douglas, G. A., Kissane, B. V., & Mortlock, R. S. (1980). Measuring problem-solving ability. In S. Krulik & R. E. Reys (Eds.),
*Problem solving in school mathematics*(pp. 204–215). Reston, VA: NCTM.Google Scholar - Masters, G. N. (1982). A Rasch model for partial credit scoring.
*Psychometrika, 47*, 149–174.CrossRefGoogle Scholar - Masters, G. N. & Doig, B. A. (1992). Understanding children’s mathematics: Some assessment tools. In G. Leder (Ed.),
*Assessment and learning of mathematics*. Melbourne: Australian Council of Educational Research.Google Scholar - Mayer, R. E., & Hegarty, M. (1996). In R. J. Sternberg & B.-Z. Talia (Eds.),
*The nature of mathematical thinking*. Mahwah, NJ: Lawrence Erlbaum.Google Scholar - McDonald, R. P., & Ahlawat, K. S. (1974). Difficulty factors in binary data.
*British Journal of Mathematical and Statistical Psychology, 27*, 82–99.Google Scholar - Nandakumar, R. (1994). Assessing latent trait unidimensionality of a set of items — Comparison of different approaches.
*Journal of Educational Measurements, 31*, 1–18.Google Scholar - National Council of Teachers of Mathematics. (1989).
*Curriculum and evaluation standards for school mathematics*. Reston, VA: NCTM.Google Scholar - Nesher, P. (1980). The stereotyped nature of school word problems.
*For the Learning of Mathematics, 1*(1), 41–48.Google Scholar - Newman, M. A. (1977). An analysis of sixth-grade pupils’ errors on written mathematical tasks.
*Victorian Institute for Educational Research Bulletin, 39*, 31–43.Google Scholar - Newman, M. A. (1983).
*Strategies for diagnosis and remediation*. Sydney: Harcourt, Brace Jovanovich.Google Scholar - OECD (2003).
*The PISA 2003 assessment framework*. Paris: OECD.Google Scholar - Polya, G. (1973).
*How to solve it: A new aspect of mathematical method*. Princeton, NJ: Princeton University Press.Google Scholar - Rasch, G. (1960).
*Probabilistic models for some intelligence and attainment tests*. Copenhagen, Denmark: Danish Institute for Educational Research.Google Scholar - Romberg, T., & de Lange, J. (1998).
*Mathematics in context*. Chicago: Britannica Mathematics System.Google Scholar - Schoenfeld, A. H. (1983). Episodes and executive decisions in mathematical problem solving. In R. Lesh & M. Landau, M. (Eds.),
*Acquisition of mathematics concepts and processes*(pp. 345–395). New York: Academic.Google Scholar - Schoenfeld, A. H. (1991). On mathematics as sense-making: An informal attack on the unfortunate divorce of formal and informal mathematics. In J. E. Voss, D. N. Perkins, & J. W. Segal (Eds.),
*Informal reasoning and education*(pp. 311–343). Hillsdale, NJ: Lawrence Erlbaum.Google Scholar - Silver, E. A. (1982). Knowledge organisation and mathematical problem solving. In F. K. Lester & J. Garafalo (Eds.),
*Mathematical problem solving: Issues in research*. Philadelphia, PA: Franklin Institute Press.Google Scholar - Smith, R. M., & Miao, C. Y. (1994). Assessing unidimensionality for Rasch measurement. In M. Wilson (Ed.),
*Objective measurement: Theory into practice*(Vol. 2, pp. 316–327) Norwood, NJ: Ablex.Google Scholar - Stacey, K., Groves, S., Bourke, S., & Doig, B. (1993).
*Profiles of problem solving*. Melbourne: Australian Council for Educational Research.Google Scholar - Treffers, A. (1986).
*Three dimensions*. Dordrecht, The Netherlands: Reidel.Google Scholar - Verhelst, N. D., Glas, C. A. W., & de Vries, H. H. (1997). A steps model to analyze partial credit. In W. J. van der Linden & R. K. Hambleton (Eds.),
*Handbook of modern item response theory*. New York: Springer-Verlag.Google Scholar - Verhelst, N. D. (2001).
*Some thoughts on reliability*. Unpublished manuscript.Google Scholar - Verschaffel, L., Greer, B. & de Corte E. (2000). Making sense of word problems. Lisse, Switzerland: Swets & Zeitlinger.Google Scholar
- Wang, W. (1998). Rasch analysis of distractors in multiple-choice items.
*Journal of Outcome Measurement, 2*(1), 43–65.Google Scholar - Whimbey, A., & Lochhead, J. (1991).
*Problem solving and comprehension*(5^{th}*ed.*) Hillsdale, NJ: Lawrence Erlbaum.Google Scholar - Willmott, A. S., & Fowles, D. E. (1974).
*The objective interpretation of test performance*. Windsor, UK: National Foundation for Educational Research Publishing.Google Scholar - Linacre, M. J., & Wright, B. D. (2000). WINSTEPS Rasch measurement computer program [Computer software]., Chicago: MESA Press.Google Scholar
- Wright, B. D., & Masters, G. N. (1982).
*Rating scale analysis*. Chicago: MESA Press.Google Scholar - Wu, M. L. (2004).
*The application of item response theory to measure problem-solving proficiencies*. Unpublished doctoral dissertation, The University of Melbourne.Google Scholar - Wu, M. L., Adams, R. J., & Wilson, M. R. (1998). ConQuest: Multi-aspect test software [Computer software]. Melbourne: Australian Council for Educational Research.Google Scholar