Skip to main content
SpringerLink
Log in

Using large-scale data to determine pupils’ strategies and errors in missing value number equations

  • Published:
Educational Studies in Mathematics Aims and scope Submit manuscript

Abstract

The goal of this study is to show a novel way of using large-scale data (N = 6203) to identify pupils’ strategies when solving missing value number equations. It is based on the assumption that wrong numerical results appearing more frequently than would be the case if they were consequences of random guessing can be expected to be underlain by a pupil’s rational, theoretically justifiable, solving procedure. We determined all of the numerical results suggested by the pupils in the study for three problems with the same format and classified them based on their occurrence. Possible solving procedures leading to them were inferred using a priori analysis. Next, we identified pupils’ strategies based on the anticipated solving procedures in two problems of the same format and validated them by comparing them to the pupils’ results in the third problem and to the pupils’ whole test result. We identified six strategies, some of them underlain by clearly identified procedures. Our analysis showed a high predictive power of the identified pupils’ strategies for their achievement in the mathematics test written 3 years later and in a more difficult problem of the same format. Our study has implications for test makers and teacher practice.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

Notes

  1. Say, that we have 3000 pupils. If the result of a problem has a frequency of 1% (i.e., 30 pupils), it is statistically significantly different (thus, not-random and worth attention) from the frequency of 0.5%. If the frequency is 10%, the result is statistically significantly different from frequencies lower than 8%.

  2. The CLoSE study was conducted within the project The relationships between skills, schooling, and labour market outcomes: A longitudinal study (http://czechlongitudinal.blogspot.com/p/about.html).

  3. In the Czech Republic, pupils are educated in elementary school until grade 5. In grade 6, about 9% of them enter 8-year secondary grammar schools (selective schools with entrance examinations). The rest continue at primary school.

  4. Both tests were prepared in two versions to prevent copying as in Czech schools, pupils sit at their desks by two.

  5. The other problems in both test 1 and test 2 concerned operations with numbers, dependencies, relations, data handling and geometry in plane and space. The main source of problems was the test from TIMSS. About half of the problems were multiple-choice and half were constructed. Test 1 included 24 problems and test 2 30 problems, 9 of which were used as anchor ones. The rest of the problems were more difficult in test 2 than in test 1 (compare P4 and P1–P3).

  6. The coding was straightforward, the coders simply copied the wrong NR, or wrote “correct” or “no solution”; no inferences were made and thus, no interscorer reliability analysis was needed.

  7. Two-digit numbers were much less frequent (see WO) and were categorised as “unexplained”.

  8. But it is still significantly higher than a random distribution.

References

  • Anderson, L. W., & Krathwohl, D. R. (Eds.). (2001). A taxonomy for learning, teaching, and assessing: A revision of Bloom’s taxonomy of educational objectives. New York, NY: Addison Wesley Longman.

  • Artigue, M. (2014). Didactic engineering in mathematics education. In S. Lerman (Ed.), Encyclopedia of mathematics education (pp. 159–162). Dordrecht, the Netherlands: Springer Science+Business Media. https://doi.org/10.1007/978-94-007-4978-8

  • Ayres, P. L. (2001). Systematic mathematical errors and cognitive load. Contemporary Educational Psychology, 26(2), 227–248. https://doi.org/10.1006/ceps.2000.1051

    Article  Google Scholar 

  • Banerjee, R., & Subramaniam, K. (2012). Evolution of a teaching approach for beginning algebra. Educational Studies in Mathematics, 80(3), 351–367. https://doi.org/10.1007/s10649-011-9353-y

    Article  Google Scholar 

  • Blando, J., Kelly, A., Schneider, B., & Sleeman, D. (1989). Analyzing and modeling arithmetic errors. Journal for Research in Mathematics Education, 20(3), 301–308. https://doi.org/10.2307/749518

    Article  Google Scholar 

  • Brousseau, G. (1997). In N. Balacheff, M. Cooper, R. Sutherland, & V. Warfield (Eds.), Eds. & Trans Theory of didactical situations in mathematics. Dordrecht, the Netherlands: Kluwer.

  • Caponera, E., & Russo, P. M. (2010). Performance in TIMSS mathematics test: The influence of self-concept and socio-economic background. Cadmo, 18(2), 91–103. https://doi.org/10.3280/CAD2010-002009

    Article  Google Scholar 

  • CERMAT. (2006). Závěrečná zpráva z projektu Hodnocení výsledků vzdělávání žáků 5. ročníků základních škol 2006. [Assessment of educational outcomes of Czech 5th graders 2006]. Prague, Czech Republic: CERMAT.

  • Choi, K., Choi, T., & McAninch, M. (2012). A comparative investigation of the presence of psychological conditions in high achieving eighth graders from TIMSS 2007 Mathematics. ZDM, 44(2), 189–199. https://doi.org/10.1007/s11858-012-0401-6

    Article  Google Scholar 

  • Chvál, M., Straková, J., & Procházková, I. (2015). Hodnocení výsledků vzdělávání didaktickými testy. [Assessment of educational results by didactic tests.]. Prague, Czech Republic: Česká školní inspekce.

  • Dodeen, H., Abdelfattah, F., Shumrani, S., & Hilal, M. A. (2012). The effects of teachers’ qualifications, practices, and perceptions on student achievement in TIMSS mathematics: A comparison of two countries. International Journal of Testing, 12(1), 61–77. https://doi.org/10.1080/15305058.2011.621568

    Article  Google Scholar 

  • Hall, R. D. (2002). An analysis of errors made in the solution of simple linear equations. Philosophy of Mathematics Education Journal, 15. http://academic.sun.ac.za/education/mathematics/174/ErrorsEquatins.pdf. Accessed 4 August 2019.

  • Hautamäki, J., et al. (2002). Assessing learning to learn. A framework. Helsinki, Finland: Centre for Educational Assessment, Helsinki University.

  • Hickendorff, M., Heiser, W. J., van Putten, C. M., & Verhelst, N. D. (2009). Solution strategies and achievement in Dutch complex arithmetic: Latent variable modeling of change. Psychometrika, 74, 331–350. https://doi.org/10.1007/s11336-008-9074-z

    Article  Google Scholar 

  • Jiang, C., & Chua, B. L. (2010). Strategies for solving three fraction-related word problems on speed: A comparative study between Chinese and Singaporean students. International Journal of Science and Mathematics Education, 8, 73–96. https://doi.org/10.1007/s10763-009-9163-1

    Article  Google Scholar 

  • Jones, I., Inglis, M., Gilmore, C., & Evans, R. (2013). Teaching the substitutive conception of the equals sign. Research in Mathematics Education, 15(1), 34–49. https://doi.org/10.1080/14794802.2012.756635

    Article  Google Scholar 

  • Kieran, C. (1981). Concepts associated with the equality symbol. Educational Studies in Mathematics, 12(3), 317–326. https://doi.org/10.1007/BF00311062

    Article  Google Scholar 

  • Kieran, C. (1992). The learning and teaching of school algebra. In D. A. Grouws (Ed.), Handbook of research on mathematics teaching and learning (pp. 390–419). New York, NY: Macmillan Publishing Co, Inc.

  • Kloosterman, P., Walcott, C., Brown, N. J. S., Mohr, D., Pérez, A., Dai, S., … Huang, H.-C. (2015). Using NAEP to analyze eighth-grade students’ ability to reason algebraically. In J. A. Middleton, J. Cai, & S. Hwang (Eds.), Large-scale studies in mathematics education (pp. 179–207). Berlin, Germany: Springer Science+Business Media. https://doi.org/10.1007/978-3-319-07716-1

  • Knuth, E. J., Stephens, A. C., McNeil, N. M., & Alibali, M. W. (2006). Does understanding the equal sign matter? Evidence from solving equations. Journal for Research in Mathematics Education, 37(4), 297–312.

    Google Scholar 

  • Linchevski, L., & Livneh, D. (1999). Structure sense: The relationship between algebraic and numerical contexts. Educational Studies in Mathematics, 40(2), 173–196. https://doi.org/10.1023/A:1003606308064

    Article  Google Scholar 

  • Louis, R., & Mistele, J. (2012). The differences in scores and self-efficacy by student gender in mathematics and science. International Journal of Science and Mathematics Education, 10(5), 1163–1190. https://doi.org/10.1007/s10763-011-9325-9

    Article  Google Scholar 

  • Maher, A. A., & Sigley, R. (2014). Task-based interviews in mathematics education. In S. Lerman (Ed.), Encyclopedia of mathematics education (pp. 579–582). Dordrecht, the Netherlands: Springer Science+Business Media. https://doi.org/10.1007/978-94-007-4978-8

  • Martin, M. O., Mullis, I. V. S., & Hooper, M. (Eds.). (2016). Methods and procedures in TIMSS 2015. Retrieved from Boston College, TIMSS & PIRLS International Study Center. http://timssandpirls.bc.edu/publications/timss/2015-methods.html. Accessed 20 June 2019.

  • Middleton, J. A., Cai, J., & Hwang, S. (Eds.). (2015). Large-scale studies in mathematics education. Cham, Switzerland: Springer Science+Business Media. https://doi.org/10.1007/978-3-319-07716-1

  • Nováková, H. (2013). Analýza a priori jako součást přípravy učitele na výuku [A priori analysis as a part of a teacher’s preparation for lessons]. Scientia in educatione, 4(2), 20–51. https://doi.org/10.14712/18047106.70

    Article  Google Scholar 

  • OECD (2015). PISA 2015. Codebooks for the mail files. http://www.oecd.org/pisa/data/2015database/. Accessed 20 June 2019.

  • Rendl, M., & Vondrová, N. (2014). Kritická místa v matematice u českých žáků na základě výsledků šetření TIMSS 2007. [Critical places of mathematics for Czech pupils on the basis of TIMSS 2007 results.]. Pedagogická orientace, 24(1), 22–57. https://doi.org/10.5817/PedOr2014-1-22

    Article  Google Scholar 

  • Sangwin, C. J., & Jones, I. (2016). Asymmetry in student achievement on multiple-choice and constructed-response items in reversible mathematics processes. Educational Studies in Mathematics, 94(2), 205–222. https://doi.org/10.1007/s10649-016-9725-4

    Article  Google Scholar 

  • Shulman, L. S. (1986). Those who understand: Knowledge growth in teaching. Educational Researcher, 15(2), 4–14. https://doi.org/10.3102/0013189x015002004

    Article  Google Scholar 

  • Sowder, J. T., Wearne, D., Martin, W. G., & Strutchens, M. (2004). What do 8th-grade students know about mathematics? In P. Kloosterman & F. K. Lester Jr. (Eds.), Results and interpretations of the 1990 through 2000 mathematics assessments of the National Assessment of Educational Progress (pp. 105–144). Reston, VA: National Council of Teachers of Mathematics.

    Google Scholar 

  • Tall, D., & Thomas, M. (1991). Encouraging versatile thinking in algebra using the computer. Educational Studies in Mathematics, 22(2), 125–147. https://doi.org/10.1007/BF00555720

    Article  Google Scholar 

  • Wijaya, A., Van Den Heuvel-Panhuizen, M., Doorman, M., & Robitzsch, A. (2014). Difficulties in solving context-based PISA mathematics tasks: An analysis of students’ errors. Mathematics Enthusiast [online], 11(3), 555–584.

    Google Scholar 

Download references

Funding

This research was financially supported by the grants GACR P402/12/G130 Czech Longitudinal Study in Education (CLoSE) and GACR 16-06134S Context problems as a key to the application and understanding of mathematical concepts.

Author information

Authors and Affiliations

  1. Faculty of Education, Charles University, Magdalény Rettigové 4, 116 39, Prague 1, Czech Republic

    Martin Chvál, Naďa Vondrová & Jarmila Novotná

Authors
  1. Martin Chvál
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Naďa Vondrová
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jarmila Novotná
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Martin Chvál.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chvál, M., Vondrová, N. & Novotná, J. Using large-scale data to determine pupils’ strategies and errors in missing value number equations. Educ Stud Math 106, 5–24 (2021). https://doi.org/10.1007/s10649-020-10000-5

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10649-020-10000-5

Keywords

Search

Navigation