Skip to main content

Advertisement

SpringerLink
Log in

Italian students’ results in the PISA mathematics test: does reading competence matter?

  • Published:
European Journal of Psychology of Education Aims and scope Submit manuscript

Abstract

In Italy, from the 2003 reports to the present, the National Institute for the Educational Evaluation of Instruction and Training (INVALSI) has conducted research on Programme for International Student Assessment (PISA) results in order to understand Italian students’ low achievement in mathematics. In the present paper, data from a representative sample of 15-year-old Italian students who participated in PISA 2012 were analysed. This study’s primary aim is to verify how students’ linguistic competences are associated with their performance in mathematics. For the evaluation of the impact of item reading demand on students’ performance, we selected 24 mathematics items with a high reading demand and 31 mathematics items with a low reading demand, as classified by Italian language and methodology experts. Repeated measure variance analyses were conducted. The results showed differences in function of gender: females are advantaged in mathematics items with a high reading demand, independent of their level of reading literacy. In contrast, males are advantaged in mathematics items with a low reading demand, independent of their level of reading literacy. Possible policy implications are discussed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

Notes

  1. In PISA 2012, each of 13 booklets included a subset of mathematics items, while only 9 out of these 13 booklets included reading items.

  2. The classification of the items used in this study has been performed on the Italian version. Nonetheless, to facilitate the reader, the English version of the sample items is included in the Appendix.

References

  • Abedi, J. (2000). Confounding of students’ performance and their language background variables. Research Report 143, UD 033 934.

  • Aiken Jr., L. R. (1972). Language factors in learning mathematics. Review of Educational Research, 42(3), 359–385.

    Article  Google Scholar 

  • Anstrom, K., DiCerbo, P., Butler, F., Katz, A., Millet, J., & Rivera, C. (2010). A review of the literature on academic English: implications for K-12 English language learners. Arlington, VA: The George Washington University Center for Equity and Excellence in Education.

    Google Scholar 

  • Artelt, C., Schiefele, U., & Schneider, W. (2001). Predictors of reading literacy. European Journal of Psychology of Education, 16(3), 363–383.

    Article  Google Scholar 

  • Arzarello, F., Garuti, R., & Ricci, R. (2015). The impact of PISA studies on the Italian national assessment system. In Stacey K. & Turner R. (Eds.), Assessing mathematical literacy. The PISA experience (pp.249–260). Springer International Publishing Switzerland.

  • Baker, L. (1991). Metacognition, reading and science education. Science learning: processes and applications, 2–13.

  • Boero, P., Douek, N., & Ferrari, P. L. (2008). Developing mastery of natural language: approach to theoretical aspects of mathematics. In L. D. English (Ed.), Handbook of international research in mathematics education (pp. 262–295). New York-London: Routledge.

    Google Scholar 

  • Bolger, N., & Kellaghan, T. (1990). Method of measurement and gender differences in scholastic achievement. Journal of Educational Measurement, 27(2), 165–174.

    Article  Google Scholar 

  • Bolondi, G., & Viale, M. (2014). Abilità linguistiche e discipline scientifiche: un’esperienza di formazione del corpo insegnante nel Polo dell’Emilia-Romagna del progetto “I Lincei per una nuova didattica nella scuola” [Linguistic skills and scientific disciplines: an experience of teacher training in the Emilia-Romagna region in the project “The Lincei for a new didactics in the school”], XVIII Convegno nazionale GISCEL – Educazione linguistica e apprendimento/insegnamento delle discipline matematico-scientifiche. In 27–29 marzo.

    Google Scholar 

  • Borsese A., Mallarino B., Parrachino I., Rebella I. (2014). La definizione nel processo di insegnamento con particolare riferimento all’ambito scientifico [Definition in the teaching process with particular reference to the scientific field], XVIII Convegno Nazionale GISCEL – Educazione linguistica e apprendimento/insegnamento delle discipline matematico-scientifiche, 27–29 marzo. http://www.giscel.it/sites/default/files/convegni/2014/2014-03%20OPUSCOLO_PREPRINT%20Giscel.pdf Accessed 24/04/2017.

  • Breakspear, S. (2012). The policy impact of PISA: an exploration of the normative effects of international benchmarking in school system performance. OECD Education Working Papers, (71), 0_1.

  • Burns, M. (2005). 3 lessons by Marilyn Burns: using storybooks to teach math. Instructor, 114(7), 27–30.

    Google Scholar 

  • Caponera, E., Sestito, P., & Russo, P. M. (2016). The influence of reading literacy on mathematics and science achievement. The Journal of Educational Research, 109(2), 197–204.

    Article  Google Scholar 

  • D’Amore B. (2014). Insegnamento/Apprendimento significativo della matematica nella scuola dell’infanzia [Meaningful teaching/learning of mathematics in kindergarten], Convegno nazionale “La didattica della matematica: strumenti per capire e per intervenire”, Tricase (Lecce), 3–4-5 marzo. http://www.dm.unibo.it/rsddm/it/articoli/damore/820%20Seminario%20Atti%20Convegno%20Tricase.pdf Accessed 24/04/2017.

  • Douville, P., Pugalee, D. K., & Wallace, J. D. (2003). Examining instructional practices of elementary science teachers for mathematics and literacy integration. School Science and Mathematics, 103(8), 388–396.

    Article  Google Scholar 

  • Draper, R. J. (2002). School mathematics reform, constructivism, and literacy: a case for literacy instruction in the reform-oriented math classroom. Journal of Adolescent & Adult Literacy, 45(6), 520–529.

    Google Scholar 

  • Draper, R. J., Smith, L. K., Hall, K. M., & Siebert, D. (2005). What’s more important—literacy or content? Confronting the literacy-content dualism. Action in Teacher Education, 27(2), 12–21.

    Article  Google Scholar 

  • Driver, M. K., & Powell, S. R. (2015). Symbolic and nonsymbolic equivalence tasks: the influence of symbols on students with mathematics difficulty. Learning Disabilities Research & Practice, 30(3), 127–134.

    Article  Google Scholar 

  • Duke, N. K., & Pearson, P. D. (2002). Effective practices for developing reading comprehension. In A. E. Farstrup & S. J. Samuels (Eds.), What research has to say about reading instruction (3rd ed., pp. 205–242). Newark, DE: International Reading Association.

    Google Scholar 

  • Education, Audiovisual and Culture Executive Agency (2010). Gender differences in educational outcomes. Study on the measures taken and the current situation in Europe. Brussels: Education, Audiovisual and Culture Executive Agency P9 Eurydice. Retrieved from http://eacea.ec.europa.eu/education/eurydice/documents/thematic_reports/120EN.pdf.

  • European Commission (2010). A strategy for smart, sustainable and inclusive growth. http://ec.europa.eu/eu2020/pdf/COMPLET%20EN%20BARROSO%20%20%20007%20-%20Europe%202020%20-%20EN%20version.pdf. Accessed 02 Mar 2017.

  • European Commission. (2012). EU high level group of experts on literacy. Final report. Luxembourg: Publications Office of the European Union.

    Google Scholar 

  • Grek, S. (2009). Governing by numbers: the PISA ‘effect’in Europe. Journal of Education Policy, 24(1), 23–37.

    Article  Google Scholar 

  • Grimm, K. J. (2008). Longitudinal associations between reading and mathematics achievement. Developmental Neuropsychology, 33(3), 410–426.

    Article  Google Scholar 

  • Helwig, R., Rozek-Tedesco, M. A., Tindal, G., Heath, B., & Almond, P. J. (1999). Reading as an access to mathematics problem solving on multiple-choice tests for sixth-grade students. The Journal of Educational Research, 93(2), 113–125.

    Article  Google Scholar 

  • Hickendorff, M. (2013). The language factor in elementary mathematics assessments: computational skills and applied problem solving in a multidimensional IRT framework. Applied Measurement in Education, 26(4), 253–278.

    Article  Google Scholar 

  • INVALSI (2016). Rilevazioni nazionali degli apprendimenti 2015–16. Rilevazione degli apprendimenti nelle classi II e V primaria, nella classe III (Prova nazionale) della scuola secondaria di primo grado e nella II classe della scuola secondaria di secondo grado. [National learning assessment 2015–16. Second and fifth grades of primary school, third grade of lower secondary school, and second grade of upper secondary school.] https://invalsi-areaprove.cineca.it/docs/file/08_Rapporto_Prove_INVALSI_2016.pdf. Accessed 24/04/2017.

  • Jiban, C. L., & Deno, S. L. (2007). Using math and reading curriculum-based measurements to predict state mathematics test performance: are simple one-minute measures technically adequate? Assessment for Effective Intervention, 32(2), 78–89.

    Article  Google Scholar 

  • Johnson, B. E., & Zabrucky, K. M. (2011). Improving middle and high school students’ comprehension of science texts. International Electronic Journal of Elementary Education, 4(1), 19.

    Google Scholar 

  • Kane, R. B., Byrne, M. A., & Hater, M. A. (1974). Helping children read mathematics. New York: American Book Company.

    Google Scholar 

  • Korhonen, J., Linnanmäki, K., & Aunio, P. (2012). Language and mathematical performance: a comparison of lower secondary school students with different level of mathematical skills. Scandinavian Journal of Educational Research, 56(3), 333–344.

    Article  Google Scholar 

  • Lafontaine, D., & Monseur, C. (2009). Gender gap in comparative studies of reading comprehension: to what extent do the test characteristics make a difference? European Educational Research Journal, 8(1), 69–79.

    Article  Google Scholar 

  • Liu, O. L., & Wilson, M. (2009). Gender differences in large-scale math assessments: PISA trend 2000 and 2003. Applied Measurement in Education, 22(2), 164–184.

    Article  Google Scholar 

  • Marks, G. N. (2008). Accounting for the gender gaps in student performance in reading and mathematics: evidence from 31 countries. Oxford Review of Education, 34(1), 89–109.

    Article  Google Scholar 

  • Marks, G. N., Cresswell, J., & Ainley, J. (2006). Explaining socioeconomic inequalities in student achievement: the role of home and school factors. Educational Research and Evaluation, 12(02), 105–128.

    Article  Google Scholar 

  • Marsh, H., Trautwein, U., Lüdtke, O., Köller, O., & Baumert, J. (2005). Academic self-concept, interest, grades, and standardized test scores: Reciprocal effects models of causal ordering. Child Development, 76(2), 397–416.

    Article  Google Scholar 

  • Martin, O., & Mullis, I. V. S. (Eds.). (2013). Relationships among reading, mathematics and science achievement at the fourth grade—implications for early learning. Chestnut Hill, MA: TIMSS & PIRLS International Study Center, Lynch School of Education, Boston College; Amsterdam: International Association for the Evaluation of Educational Achievement (IEA).

  • McConney, A., & Perry, L. B. (2010). Socioeconomic status, self-efficacy, and mathematics achievement in Australia: a secondary analysis. Educational Research for Policy and Practice, 9(2), 77–91.

    Article  Google Scholar 

  • MIUR (Ministero dell’Istruzione, dell’Università e della Ricerca) - Servizio Statistico. (2017). Focus “Gli immatricolati nell’a.a. 2016/2017 il passaggio dalla scuola all’università dei diplomati nel 2016.” [Enrollment in the academic year 2016/2017: the transition from school to university of high school graduates in 2016.] http://www.miur.gov.it/documents/20182/639222/FOCUS+2016-17_immatricolazioni.pdf/3a1f576c-5498-4942-962c-435a2b791a5e?version=1.0 Accessed 17 Feb 2018.

  • Moyer, P. S. (2000). Communicating mathematically: children’s literature as a natural connection. The Reading Teacher, 54(3), 246–255.

    Google Scholar 

  • Mullis, I. V., Martin, M. O., Ruddock, G. J., O'Sullivan, C. Y., & Preuschoff, C. (2009). TIMSS 2011 assessment frameworks. Amsterdam: International Association for the Evaluation of Educational Achievement.

    Google Scholar 

  • Mullis, I.V.S. Martin MO., & Foy, P. (2013). The impact of reading ability on TIMSS mathematics and science achievement at the fourth grade: an analysis by item reading demands. In Martin, M.O and Mullis I.V.S. (Eds.), Relationships among reading, mathematics and science achievement at the fourth grade—implications for early learning. Chestnut Hill, MA: TIMSS & PIRLS International Study Center, Lynch School of Education, Boston College; Amsterdam: International Association for the Evaluation of Educational Achievement (IEA).

  • Mullis, I. V. S., Martin, M. O., Foy, P., & Hooper, M. (2016). TIMSS 2015 international results in mathematics. TIMSS & PIRLS International Study Center at Boston College. http://timssandpirls.bc.edu/timss2015/international-results/ Accessed 24/04/2017.

  • Nasser, F., & Birenbaum, M. (2005). Modeling mathematics achievement of Jewish and Arab eighth graders in Israel: the effects of learner-related variables. Educational Research and Evaluation, 11(3), 277–302.

    Article  Google Scholar 

  • National Council of Teachers of Mathematics (Ed.). (2000). Principles and standards for school mathematics (Vol. 1). Reston, VA: National Council of Teachers of Mathematics.

  • Neufeld, P. (2006). Comprehension instruction in content area classes. The Reading Teacher, 59(4), 302–312.

    Article  Google Scholar 

  • OECD. (2010). PISA 2009 results: what students know and can do—student performance in reading, mathematics and science (Vol. I). Paris: OECD.

    Google Scholar 

  • OECD. (2013). PISA 2012 assessment and analytical framework: mathematics, reading, science, problem solving and financial literacy. Paris: OECD.

    Book  Google Scholar 

  • OECD. (2014a). PISA 2012 technical report. Paris: OECD.

    Google Scholar 

  • OECD (2014b). Proficiency scale construction. In OECD (Ed.), PISA 2012 technical report (pp. 287–303). Paris: OECD.

  • OECD. (2016). PISA 2015 results (volume I): excellence and equity in education. Paris: OECD.

    Google Scholar 

  • Österholm, M. (2006). Characterizing reading comprehension of mathematical texts. Educational Studies in Mathematics, 63(3), 325–346.

    Article  Google Scholar 

  • Pica, T. (2002). Subject-matter content: how does it assist the interactional and linguistic needs of classroom language learners? The Modern Language Journal, 86(1), 1–19.

    Article  Google Scholar 

  • Pozio, S. (2011). La risoluzione di prove di competenza matematica. Analisi dei risultati italiani nell'indagine OCSE (Pisa, 2003) [Solution of mathematics competence tasks. Analysis of the Italian results in the OECD PISA 2003]. Roma: Nuova Cultura.

  • Reilly, E. M. (2014). Improving girls’ interest in mathematics through writing. Language, literacy, and learning in STEM education: research methods and perspectives from applied linguistics, 1, 143.

    Article  Google Scholar 

  • Schleppegrell, M. J. (2007). The linguistic challenges of mathematics teaching and learning: a research review. Reading & Writing Quarterly, 23(2), 139–159.

    Article  Google Scholar 

  • Schmidt, W. H., Burroughs, N. A., Zoido, P., & Houang, R. T. (2015). The role of schooling in perpetuating educational inequality: an international perspective. Educational Researcher, 44(7), 371–386.

    Article  Google Scholar 

  • Siebert, D., & Draper, R. J. (2008). Why content-area literacy messages do not speak to mathematics teachers: a critical content analysis. Literacy Research and Instruction, 47(4), 229–245.

    Article  Google Scholar 

  • Snow, C. (2002). Reading for understanding: toward an R&D program in reading comprehension. Rand Corporation.

  • Stoet, G., & Geary, D. C. (2013). Sex differences in mathematics and reading achievement are inversely related: within-and across-nation assessment of 10 years of PISA data. PLoS One, 8(3), e57988.

    Article  Google Scholar 

  • Thurber, R. S., Shinn, M. R., & Smolkowski, K. (2002). What is measured in mathematics tests? Construct validity of curriculum-based mathematics measures. School Psychology Review, 31(4), 498.

    Google Scholar 

  • Wilder, G. Z., & Powell, K. (1989). Sex differences in test performance: a survey of the literature. ETS Research Report Series, 1989(1), i–50.

  • Yore, L. D., & Treagust, D. F. (2006). Current realities and future possibilities: language and science literacy—empowering research and informing instruction. International Journal of Science Education, 28(2–3), 291–314.

    Article  Google Scholar 

  • Yore, L. D., Pimm, D., & Tuan, H. L. (2007). The literacy component of mathematical and scientific literacy. International Journal of Science and Mathematics Education, 5(4), 559–589.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. INVALSI (National Institute for the Educational Evaluation of Instruction and Training), Via Ippolito Nievo 35, 00153, Rome, Italy

    Anna Maria Ajello, Elisa Caponera & Laura Palmerio

  2. Department of Psychology, Sapienza Università di Roma, Via dei Marsi, 78, 00185, Rome, Italy

    Anna Maria Ajello

Authors
  1. Anna Maria Ajello
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Elisa Caponera
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Laura Palmerio
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Laura Palmerio.

Additional information

Anna Maria Ajello. INVALSI (National Institute for the Educational Evaluation of Instruction and Training), Via Ippolito Nievo 35, 00153, Rome, Italy. E-mail: presidente@invalsi.it. Web site: http://www.invalsi.it. Department of Psychology, Sapienza Università di Roma, Via dei Marsi, 78, 00185 Rome, Italy. E-mail: annamaria.ajello@uniroma1.it. Web site: http://www.psicologia1.uniroma1.it/

Current themes of research:

Key competence and education. Economic knowledge and education. Social reasoning and interaction in classroom.

Most relevant publications in the field of Psychology of Education:

Morselli, D., & Ajello, A. (2016). Assessing the sense of initiative and entrepreneurship in vocational students using the European qualification framework. Education+ Training, 58(7/8), 797-814.

Berti, A. E., Ajello, A. M., Aprea, C., Castelli, I., Lombardi, E., Marchetti, A., ... & Valle, A. (2017). Adolescents’ and young adults’ naïve understandings of the economic crisis. Europe's Journal of Psychology, 13(1), 143.

Lombardi, E. & Ajello, A.M. (2017). Ricerche di Psicologia, Vol. 40 (1), 93-123 ISSN 0391-6081, ISSNe 1972-5620

Elisa Caponera. INVALSI (National Institute for the Educational Evaluation of Instruction and Training), Via Ippolito Nievo 35, 00153, Rome, Italy. E-mail: elisa.caponera@invalsi.it. Web site: http://www.invalsi.it

Current themes of research:

Parental involvement at school. Gender difference in mathematics achievement. School effectiveness. Equity of school system.

Most relevant publications in the field of Psychology of Education:

Russo, P. M., Biasi, V., Cipolli, C., Mallia, L., & Caponera, E. (2017). Sleep habits, circadian preference, and school performance in early adolescents. Sleep Medicine, 29, 20-22.

Caponera, E., Sestito, P., & Russo, P. M. (2016). The influence of reading literacy on mathematics and science achievement. The Journal of Educational Research, 109 (2), 197-204.

Caponera, E., & Russo, P. M. (2014). Student characteristics and mathematics achievement in TIMSS. Cadmo, 22 (2), 93-105.

Caponera, E. Russo P.M. (2010). Performance in TIMSS mathematics test: the influence of self-concept and socio-economic background. Cadmo, 18 (2), pp. 91-103.

Laura Palmerio. Department of Psychology, Sapienza Università di Roma, Via dei Marsi, 78, 00185 Rome, Italy. E-mail: laura.palmerio@invalsi.it. Web site: http://www.invalsi.it

Current themes of research:

Gender differences in achievement. Equity in education. Relations between reading literacy and mathematics. Civic and citizenship education.

Most relevant publications in the field of Psychology of Education:

Alivernini, F., Losito, B., & Palmerio, L. (2010). Le differenze nelle prestazioni degli studenti italiani in PISA 2006 e l’equità del sistema scolastico italiano: due studi. In INVALSI. PISA 2006. Approfondimenti tematici e metodologici; Roma, Armando; pp. 59-84.

Caponera, E., Losito, B., Pietracci, R., & Palmerio, L. (2016). Le differenze tra scuole nei risultati in matematica: un’analisi multilevel in funzione dei processi cognitivi indagati in PISA. In Palmerio, L. (ed.). OCSE PISA 2012. Contributi di approfondimento. Milano, FrancoAngeli; pp. 58-73.

Examples of low-/high-reading-demand items

Examples of low-/high-reading-demand items

Fig. 2
figure 2

Example of low-reading-demand item

Full size image
Fig. 3
figure 3

Example of high-reading-demand item

Full size image

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ajello, A.M., Caponera, E. & Palmerio, L. Italian students’ results in the PISA mathematics test: does reading competence matter?. Eur J Psychol Educ 33, 505–520 (2018). https://doi.org/10.1007/s10212-018-0385-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10212-018-0385-x

Keywords

Search

Navigation