Abstract
Students’ grasp of the non-mathematical language in a mathematics story problem—such as vocabulary and syntax—may have an important effect on their problem-solving, and this may be particularly true for students with weaker language skills. However, little experimental research has examined which individual language features influence students’ performance while solving problems—much research has been correlational or has combined language features together. In the present study, we manipulated six different language features of algebra story problems—number of sentences, pronouns, word concreteness, word hypernymy, consistency of sentences, and problem topic—and examined how systematically varying readability demands impacts student performance. We examined both accuracy and response time measures, using an assignment for learning linear functions in the ASSISTments online problem-solving environment. We found little evidence that individual language features have a considerable effect on mathematics word problem solving performance for a general population of students. However, sentence consistency reduced response time and problems about motion or travel had shorter response times than problems about business or work. In addition, it appears students may benefit or be harmed by language modifications depending on their familiarity with ASSISTments. Implications for the role of language in math word problems are discussed.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Abedi, J., & Lord, C. (2001). The language factor in mathematics tests. Applied Measurement in Education, 14, 219–234. https://doi.org/10.1207/s15324818ame1403_2.
Abedi, J., Lord, C., Hofstetter, C., & Baker, E. (2000). Impact of accommodation strategies on English language learners’ test performance. Educational Measurement: Issues and Practice, 19(3), 16–26. https://doi.org/10.1111/j.1745-3992.2000.tb00034.x.
Anand, P. G., & Ross, S. M. (1987). Using computer-assisted instruction to personalize arithmetic materials for elementary school children. Journal of Educational Psychology, 79(1), 72–78. https://doi.org/10.1037//0022-0663.79.1.72.
Baker, R. S., Corbett, A. T., & Koedinger, K. R. (2007). The difficulty factors approach to the design of lessons in intelligent tutor curricula. International Journal of Artificial Intelligence in Education, 17(4), 341–369.
Baker, R. S., de Carvalho, A. M., Raspat, J., Aleven, V., Corbett, A. T., & Koedinger, K. R. (2009). Educational software features that encourage and discourage “gaming the system.” Proceedings of the 14th International Conference on Artificial Intelligence in Education, 475–482. https://doi.org/10.3233/978-1-60750-028-5-475.
Bates, D., Maechler, M., Bolker, B., & Walker, S. (2014). lme4: Linear mixed-effects models using Eigen and S4. R package version 1.1-7.
Beal, C. R., & Galan, F. C. (2015). Math word problem solving by English learners and English primary students in an intelligent tutoring system. International Journal of Learning Technology, 10, 170–184. https://doi.org/10.1504/ijlt.2015.070686.
Bernacki, M. L., & Walkington, C. (2018). The role of situational interest in personalized learning. Journal of Educational Psychology, 110(6), 864–881. https://doi.org/10.1037/edu0000250.
Chen, O., Kalyuga, S., & Sweller, J. (2017). The expertise reversal effect is a variant of the more general element interactivity effect. Educational Psychology Review, 29(2), 393–405. https://doi.org/10.1007/s10648-016-9359-1.
Chipman, S. F., Marshall, S. P., & Scott, P. A. (1991). Content effects on word problem performance: A possible source of test bias? American Educational Research Journal, 28(4), 897–915. https://doi.org/10.3102/00028312028004897.
Clinton, V., Basaraba, D. L., & Walkington, C. (2018). English learners and mathematical word problem solving: A systematic review. In D. L. Baker, D. L. Basaraba, & C. Richards-Tutor (Eds.), Second language acquisition: Methods, perspectives and challenges (pp. 171–208). New York: Nova Science Publishers.
Clinton, V., Cooper, J. L., Michaelis, J., Alibali, M. W., & Nathan, M. J. (2017). Revising visuals based on instructional design principles: Effects on cognitive load and learning. In C. Was, F. J. Sansosti, & B. J. Morris (Eds.), Eye-tracking technology applications in educational research (pp. 195–218). Hershey, PA: IGI Global.
Clinton, V., & van den Broek, P. (2012). Interest, inferences, and learning from texts. Learning and Individual Differences, 22, 650–663. https://doi.org/10.1016/j.lindif.2012.07.004.
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum.
Crossley, S. A., & McNamara, D. S. (2012). Predicting second language writing proficiency: The roles of cohesion and linguistic sophistication. Journal of Research in Reading, 35(2), 115–135. https://doi.org/10.1111/j.1467-9817.2010.01449.x.
Crossley, S. A., Skalicky, S., Dascalu, M., McNamara, D. S., & Kyle, K. (2017). Predicting text comprehension, processing, and familiarity in adult readers: New approaches to readability formulas. Discourse Processes, 54(5–6), 340–359. https://doi.org/10.1080/0163853X.2017.1296264.
Cummins, D. D., Kintsch, W., Reusser, K., & Weimer, R. (1988). The role of understanding in solving word problems. Cognitive Psychology, 20, 405–438. https://doi.org/10.1016/0010-0285(88)90011-4.
d’Ailly, H. H., Simpson, J., & MacKinnon, G. E. (1997). Where should “you” go in a math compare problem? Journal of Educational Psychology, 89(3), 562–567. https://doi.org/10.1037//0022-0663.89.3.562.
Daroczy, G., Wolska, M., Meurers, W. D., & Nuerk, H. C. (2015). Word problems: A review of linguistic and numerical factors contributing to their difficulty. Frontiers in Psychology, 6, 348–361. https://doi.org/10.3389/fpsyg.2015.00348.
Davis-Dorsey, J., Ross, S. M., & Morrison, G. R. (1991). The role of rewording and context personalization in the solving of mathematical word problems. Journal of Educational Psychology, 83(1), 61–68. https://doi.org/10.1037/0022-0663.83.1.61.
De Jong, T. (2010). Cognitive load theory, educational research, and instructional design: some food for thought. Instructional Science, 38(2), 105–134.
Doddannara, L. S., Gowda, S. M., Baker, R. S., Gowda, S. M., & De Carvalho, A. M. (2013). Exploring the relationships between design, students’ affective states, and disengaged behaviors within an ITS. In International Conference on Artificial Intelligence in Education (pp. 31–40). Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-39112-5_4.
Duffy, S. A., & Rayner, K. (1990). Eye movements and anaphor resolution: Effects of antecedent typicality and distance. Language and Speech, 33(2), 103–119. https://doi.org/10.1177/002383099003300201.
Duran, N. D., Bellissens, C., Taylor, R. S., & McNamara, D. S. (2007). Quantifying text difficulty with automated indices of cohesion and semantics. In Proceedings of the Annual Meeting of the Cognitive Science Society, Volume 29 (pp. 233–238). Cognitive Science Society, Austin, TX.
Fang, Z. (2012). Language correlates of disciplinary literacy. Topics in Language Disorders, 32(1), 19–34. https://doi.org/10.1097/tld.0b013e31824501de.
Fliessbach, K., Weis, S., Klaver, P., Elger, C. E., & Weber, B. (2006). The effect of word concreteness on recognition memory. NeuroImage, 32(3), 1413–1421. https://doi.org/10.1016/j.neuroimage.2006.06.007.
Fuchs, L. S., Fuchs, D., Compton, D. L., Hamlett, C. L., & Wang, A. Y. (2015). Is word-problem solving a form of text comprehension? Scientific Studies of Reading, 19(3), 204–223. https://doi.org/10.1080/10888438.2015.1005745.
Gernsbacher, M. A. (1989). Mechanisms that improve referential access. Cognition, 32(2), 99–156. https://doi.org/10.1016/0010-0277(89)90001-2.
Gillmor, S., Poggio, J., & Embretson, S. (2015). Effects of reducing the cognitive load of mathematics test items on student performance. Numeracy, 8(1), 1–18. https://doi.org/10.5038/1936-4660.8.1.4.
Graesser, A. C., & McNamara, D. S. (2011). Computational analyses of multilevel discourse comprehension. Topics in Cognitive Science, 3(2), 371–398. https://doi.org/10.1111/j.1756-8765.2010.01081.x.
Graesser, A. C., McNamara, D. S., & Louwerse, M. M. (2003). What do readers need to learn in order to process coherence relations in narrative and expository text. In A. P. Sweet & C. E. Snow (Eds.), Rethinking reading comprehension (pp. 82–98). New York: Guilford.
Haag, N., Heppt, B., Stanat, P., Kuhl, P., & Pant, H. A. (2013). Second language learners’ performance in mathematics: Disentangling the effects of academic language features. Learning and Instruction, 28, 24–34. https://doi.org/10.1016/j.learninstruc.2013.04.001.
Haag, N., Roppelt, A., & Heppt, B. (2015). Effects of mathematics items’ language demands for language minority students: Do they differ between grades? Learning and Individual Differences, 42, 70–76. https://doi.org/10.1016/j.lindif.2015.08.010.
Harp, S. F., & Mayer, R. E. (1998). How seductive details do their damage: A theory of cognitive interest in science learning. Journal of Educational Psychology, 90(3), 414–434. https://doi.org/10.1037//0022-0663.90.3.414.
Heffernan, N. T., & Heffernan, C. L. (2014). The ASSISTments ecosystem: Building a platform that brings scientists and teachers together for minimally invasive research on human learning and teaching. International Journal of Artificial Intelligence in Education, 24(4), 470–497. https://doi.org/10.1007/s40593-014-0024-x.
Hembree, R. (1992). Experiments and relational studies in problem solving: A meta-analysis. Journal for Research in Mathematics Education, 23(3), 242–273. https://doi.org/10.2307/749120.
Hidi, S. E. (1995). A reexamination of the role of attention in learning from text. Educational Psychology Review, 7(4), 323–350. https://doi.org/10.1007/bf02212306.
Hidi, S., & Renninger, K. A. (2006). The four-phase model of interest development. Educational Psychologist, 41(2), 111–127. https://doi.org/10.1207/s15326985ep4102_4.
Jerman, M., & Rees, R. (1972). Predicting the relative difficulty of verbal arithmetic problems. Educational Studies in Mathematics, 4(3), 306–323. https://doi.org/10.1007/BF00302580.
Kane, M. T. (2001). Current concerns in validity theory. Journal of Educational Measurement, 38(4), 319–342. https://doi.org/10.1111/j.1745-3984.2001.tb01130.x.
Kintsch, W. (1998). Comprehension: A paradigm for cognition. Cambridge, UK: Cambridge University Press.
Kintsch, W., & Greeno, J. G. (1985). Understanding and solving word arithmetic problems. Psychological Review, 92(1), 109–129. https://doi.org/10.1037/0033-295X.92.1.109.
Kintsch, W., Welsch, D., Schmalhofer, F., & Zimny, S. (1990). Sentence memory: A theoretical analysis. Journal of Memory and Language, 29(2), 133–159. https://doi.org/10.1016/0749-596x(90)90069-c.
Koedinger, K., & Nathan, M. J. (2004). The real story behind story problems: Effects of representations on quantitative reasoning. Journal of the Learning Sciences, 13(2), 129–164. https://doi.org/10.1207/s15327809jls1302_1.
Korbach, A., Brünken, R., & Park, B. (2018). Differentiating different types of cognitive load: A comparison of different measures. Educational Psychology Review, 30(2), 503–529. https://doi.org/10.1007/s10648-017-9404-8.
Leroy, G., Endicott, J. E., Kauchak, D., Mouradi, O., & Just, M. (2013). User evaluation of the effects of a text simplification algorithm using term familiarity on perception, understanding, learning, and information retention. Journal of Medical Internet Research, 15(7), e144. https://doi.org/10.2196/jmir.2569.
Loftus, E. F., & Suppes, P. (1972). Structural variables that determine problem-solving difficulty in computer assisted instruction. Journal of Educational Psychology, 63(6), 531. https://doi.org/10.1037/h0034074.
Martiniello, M. (2009). Linguistic complexity, schematic representations, and differential item functioning for English language learners in math tests. Educational Assessment, 14(3–4), 160–179. https://doi.org/10.1080/10627190903422906.
Mavilidi, M. F., Hoogerheide, V., & Paas, F. (2014). A quick and easy strategy to reduce test anxiety and enhance test performance. Applied Cognitive Psychology, 28(5), 720–726. https://doi.org/10.1002/acp.3058.
Mayer, R. E., Fennell, S., Farmer, L., & Campbell, J. (2004). A personalization effect in multimedia learning: Students learn better when words are in conversational style rather than formal style. Journal of Educational Psychology, 96(2), 389–395. https://doi.org/10.1037/0022-0663.96.2.389.
McNamara, D. S., Graesser, A. C., McCarthy, P. M., & Cai, Z. (2014). Automated evaluation of text and discourse with Coh-Metrix. Cambridge: Cambridge University Press. https://doi.org/10.1017/cbo9780511894664.006.
Mullis, I. V., Martin, M. O., & Foy, P. (2013). The impact of reading ability on TIMSS mathematics and science achievement at the fourth grade: An analysis by item reading demands. In M. O. Martin & I. V. S. Mullis (Eds.), TIMSS and PIRLS 2011: Relationships among reading, mathematics, and science achievement at the fourth grade: Implications for early learning (pp. 67–108). Chestnut Hill, MA: TIMSS & PIRLS International Study Center, Lynch School of Education, Boston College and IEA.
Nagy, W., & Townsend, D. (2012). Words as tools: Learning academic vocabulary as language acquisition. Reading Research Quarterly, 47(1), 91–108. https://doi.org/10.1002/rrq.011.
Nathan, M. J., Kintsch, W., & Young, E. (1992). A theory of algebra-word-problem comprehension and its implications for the design of learning environments. Cognition and Instruction, 9, 329–389. https://doi.org/10.1207/s1532690xci0904_2.
Nathan, M. J., & Koedinger, K. R. (2000). An investigation of teachers’ beliefs of students’ algebra development. Cognition and Instruction, 18(2), 209–237. https://doi.org/10.1207/s1532690xci1802_03.
Ngu, B. H., Phan, H. P., Hong, K. S., & Usop, H. (2016). Reducing intrinsic cognitive load in percentage change problems: The equation approach. Learning and Individual Differences, 51, 81–90. https://doi.org/10.1016/j.lindif.2016.08.029.
Noble, T., Kachchaf, R. R., & Rosebery, A. S. (2018). Perspectives from research on the linguistic features of mathematics and science test items and the performance of English Learners. In D. L. Baker & D.L. Basaraba, & C. Richards-Tutor (Eds.), Second language acquisition: Methods, perspectives and challenges (pp. 209–236). Nova Science Publishers.
Paas, F., Renkl, A., & Sweller, J. (2003). Cognitive load theory and instructional design: Recent developments. Educational Psychologist, 38(1), 1–4. https://doi.org/10.1207/s15326985ep3801_1.
Phan, H. P., Ngu, B. H., & Yeung, A. S. (2017). Achieving optimal best: Instructional efficiency and the use of cognitive load theory in mathematical problem solving. Educational Psychology Review, 29(4), 667–692. https://doi.org/10.1007/s10648-016-9373-3.
Raghubar, K. P., Barnes, M. A., & Hecht, S. A. (2010). Working memory and mathematics: A review of developmental, individual difference, and cognitive approaches. Learning and Individual Differences, 20(2), 110–122. https://doi.org/10.1016/j.lindif.2009.10.005.
Sadoski, M. (2001). Resolving the effects of concreteness on interest, comprehension, and learning important ideas from text. Educational Psychology Review, 13(3), 263–281. https://doi.org/10.1023/A:1016675822931.
Sadoski, M., Goetz, E. T., & Avila, E. (1995). Concreteness effects in text recall: Dual coding or context availability? Reading Research Quarterly, 30(2), 278–288. https://doi.org/10.2307/748038.
Sadoski, M., Goetz, E. T., & Rodriguez, M. (2000). Engaging texts: Effects of concreteness on comprehensibility, interest, and recall in four text types. Journal of Educational Psychology, 92(1), 85–95. https://doi.org/10.1037//0022-0663.92.1.85.
Scheiter, K., Gerjets, P., & Catrambone, R. (2006). Making the abstract concrete: Visualizing mathematical solution procedures. Computers in Human Behavior, 22(1), 9–25. https://doi.org/10.1016/j.chb.2005.01.009.
Schmidt, R. A., & Bjork, R. A. (1992). New conceptualizations of practice: Common principles in three paradigms suggest new concepts for training. Psychological Science, 3(4), 207–218. https://doi.org/10.1111/j.1467-9280.1992.tb00029.x.
Schraw, G., & Lehman, S. (2001). Situational interest: A review of the literature and directions for future research. Educational Psychology Review, 13(1), 23–52. https://doi.org/10.1023/a:1009004801455.
Searle, B. W., Lorton, P., & Suppes, P. (1974). Structural variables affecting CAI performance on arithmetic word problems of disadvantaged and deaf students. Educational Studies in Mathematics, 5(1), 371–384. https://doi.org/10.1007/BF00684708.
Shaftel, J., Belton-Kocher, E., Glasnapp, D., & Poggio, J. (2006). The impact of language characteristics in mathematics test items on the performance of English language learners and students with disabilities. Educational Assessment, 11, 105–126. https://doi.org/10.1207/s15326977ea1102_2.
Suppes, P., Loftus, E. F., & Jerman, M. (1969). Problem-solving on a computer-based teletype. Educational Studies in Mathematics, 2(1), 1–15. https://doi.org/10.1007/BF00368982.
Sweller, J. (1994). Cognitive load theory, learning difficulty, and instructional design. Learning and Instruction, 4(4), 295–312. https://doi.org/10.1016/0959-4752(94)90003-5.
Sweller, J. (2016). Cognitive load theory, evolutionary educational psychology, and instructional design. In D. Geary & D. Berch (Eds.), Evolutionary perspectives on child development and education (pp. 291–306). Cham: Springer. https://doi.org/10.1007/978-3-319-29986-0_12.
Sweller, J., Ayres, P. L., Kalyuga, S., & Chandler, P. A. (2003). The expertise reversal effect. Educational Psychologist, 38(1), 23–31. https://doi.org/10.1207/S15326985EP3801_4.
Sweller, J., Ayres, P., & Kalyuga, S. (2011). Measuring cognitive load. In Cognitive Load Theory: Explorations in the Learning Sciences, Instructional Systems and Performance Technologies, Vol 1 (pp. 237–242). Springer, New York. https://doi.org/10.1007/978-1-4419-8126-4_18.
Sweller, J., & Chandler, P. (1991). Evidence for cognitive load theory. Cognition and Instruction, 8(4), 351–362. https://doi.org/10.1207/s1532690xci0804_5.
Sweller, J., van Merrienboer, J., & Paas, F. (1998). Cognitive architecture and instructional design. Educational Psychology Review, 10, 251–296.
Van Gog, T., Paas, F., & Van Merriënboer, J. J. (2006). Effects of process-oriented worked examples on troubleshooting transfer performance. Learning and Instruction, 16(2), 154–164. https://doi.org/10.1016/j.learninstruc.2006.02.003.
Vilenius-Tuohimaa, P. M., Aunola, K., & Nurmi, J. E. (2008). The association between mathematical word problems and reading comprehension. Educational Psychology, 28(4), 409–426. https://doi.org/10.1080/01443410701708228.
Walkington, C. A. (2013). Using adaptive learning technologies to personalize instruction to student interests: The impact of relevant contexts on performance and learning outcomes. Journal of Educational Psychology, 105(4), 932–945. https://doi.org/10.1037/a0031882.
Walkington, C., & Bernacki, M. (2014). Motivating students by “personalizing” learning around individual interests: A consideration of theory, design, and implementation issues. In S. Karabenick & T. Urdan (Eds.), Advances in motivation and achievement (Vol. 18, pp. 139–176). Bingley: Emerald Group Publishing. https://doi.org/10.1108/s0749-742320140000018004.
Walkington, C., & Bernacki, M. (2018). Personalizing algebra to students’ interests: How student funds of knowledge moderate outcomes. International Journal of Artificial Intelligence in Education. https://doi.org/10.1007/s40593-018-0168-1.
Walkington, C., Clinton, V., Ritter, S. N., & Nathan, M. J. (2015). How readability and topic incidence relate to performance on mathematics story problems in computer-based curricula. Journal of Educational Psychology, 107, 1051–1074. https://doi.org/10.1037/edu0000036.
Walkington, C., Clinton, V., & Shivraj, P. (2018). How readability factors are differentially associated with performance for students of different backgrounds when solving mathematics word problems. American Educational Research Journal, 55(2), 362–414. https://doi.org/10.3102/0002831217737028.
Walkington, C., & Maull, K. (2011). Exploring the assistance dilemma: The case of context personalization. In L. Carlson, C. Hölscher, & T. Shipley (Eds.), Proceedings of the 33rd Annual Conference of the Cognitive Science Society (pp. 90–95). Boston, MA: Cognitive Science Society.
Walkington, C., Sherman, M., & Petrosino, A. (2012). ‘Playing the game’ of story problems: Coordinating situation-based reasoning with algebraic representation. Journal of Mathematical Behavior, 31(2), 174–195.
Wang, A. Y., Fuchs, L. S., & Fuchs, D. (2016). Cognitive and linguistic predictors of mathematical word problems with and without irrelevant information. Learning and Individual Differences, 52, 79–87. https://doi.org/10.1016/j.lindif.2016.10.015.
Wolf, M. K., & Leon, S. (2009). An investigation of the language demands in content assessments for English language learners. Educational Assessment, 14(3–4), 139–159. https://doi.org/10.1080/10627190903425883.
Wu, M. (2010). Comparing the similarities and differences of PISA 2003 and TIMSS (OECD Working Papers No. 32). Paris: OECD Publishing. https://doi.org/10.1787/5km4psnm13nx-en.
Young, J. W. (2009). A framework for test validity research on content assessments taken by English language learners. Educational Assessment, 14(3–4), 122–138. https://doi.org/10.1080/10627190903422856.
Acknowledgements
We would like to thank Neil Heffernan, Cristina Heffernan, and Korinn Ostrow for their assistance in setting up this study. We also thank NSF for their support in making ASSISTments available to conduct this research, through NSF Cyberinfrastructure Award 1440753, SI2-SSE: Adding Research Accounts to the ASSISTments Platform: Helping Researchers do Randomized Controlled Studies with Thousands of Students.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Walkington, C., Clinton, V. & Sparks, A. The effect of language modification of mathematics story problems on problem-solving in online homework. Instr Sci 47, 499–529 (2019). https://doi.org/10.1007/s11251-019-09481-6
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11251-019-09481-6