Abstract
The current study analysed the relationships between socio-economic status, migration background, and non-cognitive factors and PISA reading and mathematics achievement. The results from multi-level analyses on Sweden’s PISA survey from 2018 indicate that both mathematics achievement and reading achievement are affected by SES, migration background, reading abilities, growth mindset, and the ability to master the content. Between-school level differences are explained by reading motivation and the mother’s educational level. Our findings stress the importance of both socio-demographic, socio-economic, and non-cognitive factors such as reading self-concept and growth mindset for both mathematics achievement and reading achievement.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
Introduction
International large-scale international assessments (ILSAs) such as the Programme for International Student Assessment (PISA) exert considerable policy influence and interest in the educational realm, evoke public interest, and are tantamount for secondary analysis (e.g., Dobbins & Martens, 2012; Marks & O’Connell, 2021). In many countries, including Sweden, assessments such as PISA, Progress in Reading Literacy Study (PIRLS), and Trends in International Mathematics and Science Study (TIMSS) are considered alternative measures of academic achievement to grades and national test results (Björklund et al., 2010). Mathematics and reading achievement are important for a variety of educational and occupational outcomes (Hanushek et al., 2019) and therefore knowledge about what influences these skills are important to disentangle. In Sweden, there has only been limited research on understanding what influences mathematics and reading achievement using PISA 2018 data. This study will fill this research gap.
Related literature
Considerable research on ILSAs has been conducted during the last decades. For example, based on data from TALIS (Teaching and Learning International Survey) and PISA from seven countries, Liu et al. (2022) examined the relations between teacher- and student-rated instructional quality and other school characteristics and mathematics achievement and found that socio-economic status (SES), often defined as parental education and/or home resources in this context, moderates the relationships between school level factors and mathematics achievement. Similarly, Rolfe et al. (2022) analysed educational quality and educational equality in relation to mathematics performance among Swedish students with TIMSS 2015 data. The authors identified inequities in relation to for example school emphasis on academic success (SEAS) and content coverage. However, the SEAS variable was built on a small number of teacher observations and therefore the results may be interpreted cautiously.
There are studies which use the PISA 2018 survey about other countries than Sweden on reading and mathematics achievement, as well as some recently published studies that for various reasons use older PISA surveys. For example, Depren and Depren (2022), using an algorithmic approach, analysed factors that influence high PISA reading achievement in China and Turkey. They found that meta-cognition and SES had the strongest influence within both countries. However, parts of the analysis were simplified by using only one plausible value. Gjicali and Lipnevich (2021) examined the influence of mathematics attitude variables in PISA 2012 among US students. They found that demographic factors and mathematics attitude factors explained about 21% of the variance of mathematics achievement. Lee and Borgonovi (2022) analysed the impact of SES on PISA mathematics achievement across different developmental levels and identified substantial differences between countries in that respect. In less developed countries, SES had little impact on mathematics achievement. However, the study did only analyse the data as country level aggregates and used data from PISA 2012.
Ajello et al. (2021) examined Italian students’ results in a set of PISA items, and found that girls had an advantage in mathematics items with a high reading demand. Similarly, Ding and Homer (2020) analyzed Chinese students’ PISA 2009 mathematics performance and found that reading skills have a significant impact on mathematics achievement. Moreover, their findings revealed that reading skills moderate the relationships between gender and mathematics achievement, because although boys perform better in mathematics, girls have better reading skills.
Gamazo and Martinez-Abad (2020) analysed PISA 2018 student achievement in mathematics, reading, and science by using data mining techniques. The authors found that achievement patterns in high-SES schools were partly affected by student characteristics such as metacognitive strategies and motivation, while low-SES schools were more affected by economic development at the country level.
Ma (2021) used a multigroup latent profile analysis to examine self-efficacy (SE) profiles in China, Finland, and the US. The findings suggest that high-students, across all countries, are more likely to have a high-SE profile. Moreover, China had the lowest proportion of high-SE students whereas the US had the highest proportion of high-SE students. This is an interesting finding as China (Beijing-Jiangsu-Shanghau-Zheijiang) had the highest overall PISA scores and self-efficacy in general is associated with higher academic achievement.
Recent research in the Swedish context (e.g., Boman 2022a, b, c) shows that socioeconomic (e.g., parental education, average income) and sociodemographic variables (e.g., migration background, meaning schoolchildren who were either born abroad or whose parents were born abroad) are substantially associated with grades and national test results at the aggregated school and municipality level in today’s Sweden, as well as PISA 2015. Earlier studies such as Myrberg and Rosén (2009), who used structural equation modeling to analyze data from PIRLS 2001, including Swedish students, have found associations between socioeconomic variables and reading achievement.
However, there is a dearth of studies on Swedish students’ PISA performance, especially more recent surveys (i.e., 2015, 2018), which is carried though in association with the Organization for Economic Co-operation and Development (OECD). An exception is the book Northern Lights on TIMSS and PISA (Reimer et al., 2018), but this work only focuses on PISA 2015 from the perspective of all Nordic countries. Moreover, it has less emphasis on mathematics and reading achievement compared to science achievement and focuses mostly on socioeconomic indicators and science achievement–science interest relations. This is because in the PISA 2015 assessment, science was the main subject whereas mathematics was the main subject in 2012 and reading the main subject in 2018.
Rasmusson (2016) has examined Norwegian and Swedish students’ digital reading skills based on PISA data and found substantial relationships between cultural capital and higher skills in digital as well as non-digital reading. Thus, while this study is relevant it lacks analyses on PISA 2018. Furthermore, too few studies have examined the relationships between migration background and PISA results, as well as the nexus between migration background and SES (e.g., Wiberg & Rolfsman, 2023), non-cognitive abilities such as academic self-concepts and PISA scores. Growth mindset, which means the belief that one’s own intelligence is malleable (Dweck, 2016), seems quite important for PISA achievement (e.g., Bernardo, 2022; Schleicher, 2019), and academic performance in general (Dweck, 2016) and thus more research in that respect is called for. Another related study is Boman (2022a), who analysed individual level data on Swedish students from the 2015 PISA wave, as well as cross-national data on 31 countries from the 2015 and 2018 PISA surveys. At the cross-country level, teachers’ and students’ cognitive skills were associated with higher PISA scores, whereas at the student level, SES (i.e., number of books at home), migration background, and ambition were associated with higher (SES, ambition) or lower (migration background) PISA scores. However, this study had some methodological limitations as it did not account for school level factors.
A recent contribution using PISA 2018 is Yang Hansen et al. (2022), whose study examined the contextual school effects on student achievement in the Nordic countries (Denmark, Finland, Norway, Sweden) as well as two Chinese jurisdictions, Hong Kong and Beijing-Shanghai-Jiangsu-Zhejiang. Specifically, the authors examined relations between academic self-concept and SES in PISA 2000 and 2018. Their results reveal that school academic and socioeconomic segregation increased between the two surveys in most of the studied countries. Moreover, school SES composition positively affected students’ achievement. The well-crafted study was somewhat limited by only focusing on reading achievement.
The current study
Overall, there are only a few studies that focus on Sweden’s PISA 2018 achievement and studies that include both mathematics and reading achievement. By including both subjects it is possible to discern the effect size similarities and differences in relation to major explanatory variables such as SES, migration background (MB), and non-cognitive factors (Boman, 2022a). Even if one subject constitutes the main topic in each cycle (e.g., mathematics in 2012, science in 2015, reading in 2018) most factors can be compared within the same PISA cycle (e.g., SES, migration background, gender, growth mindset). Moreover, by taking advantage of a multi-level model approach it is possible to identify both within-school and between-school differences.
The current study aims to fill the research gap in PISA 2018 reading and mathematics as it focuses on Sweden’s PISA 2018 data, with a particular emphasis on SES, migration background, and non-cognitive variables (e.g., growth mindset, self-concept of competence, reading abilities), which means the abilities that are separated from those that are measured by IQ tests and similar cognitive tests and instead measure for example a person’s degree of self-control, perseverance, as well as self-reported abilities (Boman, 2023a). Overall, we consider it problematic that too many earlier studies on the Swedish context focus on a limited number of covariates. In addition, we also regard it as pertinent to estimate more than one form of main literacy in relation to PISA. Thus, we have focused on both mathematics and reading.
The authors contribute to the literature on educational achievement and SES in the Swedish context (e.g., Gustafsson & Yang Hansen, 2018; Author, 2021), educational achievement and non-cognitive abilities such as perseverance, growth mindset, and reading self-concept among Swedish samples (e.g., Author; Rosander & Bäckström, 2014; Thorsen et al., 2021), PISA reading (e.g., Rasmusson, 2016), and PISA achievement research more in general (e.g., Reimer et al., 2018; Muench et al., 2022). Moreover, we broaden the research conducted by Ding and Homer (2020) and Ajello et al. (2021) by including both mathematics and reading, as well as by using more recent data in our analyses.
The current article aims to answer the following research questions:
-
1.
What are the relationships between SES, migration background, non-cognitive abilities, and PISA reading and mathematics?
-
2.
Are these relationships similar for PISA reading and mathematics?
PISA
PISA, which was first administered in 2000, is a triennial, on average two-hour long, low-stake standardised assessment designated to test students within countries, as well as to make international comparisons between school systems. Individuals complete test questions in three domains, reading, mathematics and science. In each cycle, one of these three subjects makes up the main subject which is given somewhat greater analytical attention compared to the others. In addition, later PISA studies (2012 onwards) have included add-ons such as problem solving and financial literacy for a certain number of countries. The questions cover a rather broad spectrum of domain-specific cognitive skills, as well as several levels of difficulty. The PISA surveys are conducted on two-stage stratified samples of students enrolled in lower-secondary or, to lesser extent, upper secondary schools. The included students are aged between 15 years and 3 months and 16 years and 2 months. The two-stage sampling strategy implies that schools are sampled first and then students are sampled within schools (OECD, 2016).
Furthermore, the aim of the PISA test is to pinpoint a broad coverage at the group level of the distribution of ability in different subject domains. Student responses to individual test items are used to impute the likelihood that such a student, and similar students, will be able to solve items at the same, higher or lower level of difficulty and with similar content. Plausible values make it possible to account for the probabilistic constitution of assessments of various levels of “unobservable” latent student abilities based on the observed response patterns. PISA scales are divided into six or more proficiency levels. Each proficiency level is related to the knowledge and skills that students demonstrate in the test (OECD, 2009).
Conceptual framework
Urie Bronfenbrenner (e.g., Bronfenbrenner, 1994) has examined the context of human environments as a microsystem, a mesosystem, an exosytem, and a macrosystem. A microsystem level covers an individual and the relations with its immediate environment, whereas a mesosystem focuses on several microsystems (e.g., families) and its developmental relations with for example the school. The exosystem includes two or more mesosystems, whereas a macro system covers patterns at the national level such as economic and political structures, cultural belief systems, and so on (Bronfenbrenner, 1994). The current study focuses on the micro and meso levels, because it is individual characteristics (e.g., SES, migration background, non-cognitive abilities) within schools, as well as differences between schools, that seem to have the largest impact (e.g., Liu et al., 2022; Marks & O’Connell, 2021; Rolfe et al., 2022) on academic achievement. However, cultural, social, economic and political forces are also relevant and hence the micro and meso systems are related to macro systems (Boman, 2020).
However, Bronfenbrenner and Morris (1998) do also stress the importance of time for individuals’ developmental processes. Because PISA can only provide a snapshot of students’ literacy levels in mathematics, reading and science (and to lesser extent other subject domains), the time dimension cannot be captured in this respect. Gustafsson (2008) stresses that only aggregated data at the national level may cover macro system changes that have an impact regarding the education system, simply because it is not the same students who take the PISA tests in consecutive surveys. On the other hand, because of the many observations and variables embedded in the data, PISA enables high-level inferences with a high degree of generalizability. Hence, PISA potentially enables the researcher to make cross-sectional generalizations that are tied to the education system. In this regard, it requires the researcher to be able to tease out factors that are the most important for academic achievement (Boman, 2023a), which are in the current study situated at the micro and meso levels.
SES and migration background
SES consists of a composite of parental education, parental income, and occupational position (e.g., Sackett et al., 2009). In some contexts, such as PISA, the number of books and other resources at home are used as indicators of SES (e.g., Tan, 2015), although research points to that economic resources are becoming increasingly more important (e.g., Wiberg & Rolfsman, 2023). Both cultural (i.e., cultural resources) and economic capital indicators such as average income are associated with SES in most national contexts (e.g., Kim et al., 2019; Sirin, 2005; Tan, 2015). It is a general pattern that high-SES students outperform their lower-SES peers (e.g., Kim, 2019; Kim et al., 2019). That might occur because high-SES parents are more involved in their children’s upbringings and have higher aspirations regarding their children’s future trajectories in education as well as the labor market. In the home environment, high-SES parents spend more time reading with and to their children which likely stimulates cognitive growth and learning (e.g., Myrberg & Rosén, 2009). Furthermore, children whose parents have higher SES do generally develop higher cognitive ability levels which are then transmitted to their children through both genes and the environment (Engelhardt et al., 2018).
In the Swedish context, SES and migration background are, in part connected factors, although they represent separate constructs (Boman, 2023a). This is because Sweden has experienced a large influx of migrant families and individuals whose SES levels are typically lower than the “native” population, on average (Holmlund et al., 2019). However, recent research shows that SES and migration background should be regarded as two separate constructs and when controlling for SES and cognitive ability, the effect sizes (β) of migration background become very small in regression models (Boman, 2023a). Nonetheless, migration background is an important factor to consider regarding achievement in PISA (e.g., Meunier, 2011; Odell et al., 2021).
Non-cognitive abilities
There are strong theoretical and empirical reasons to assume that academic achievement such as grades or PISA are influenced by non-cognitive abilities such as academic self-concept and self-efficacy, and growth mindset. Academic self-concept means the belief in one’s own academic capabilities (e.g., Bandura, 2006; Bong & Skaalvik, 2003; Zysberg & Schwabsky, 2021). Moreover, some are also related to having grit (Duckworth & Quinn, 2009) and a growth mindset (Claro et al., 2016), which means the belief that a person’s intelligence is malleable and often related to better academic achievement (Bahnik & Vranka, 2017). Hence, if the data includes different sub-forms of non-cognitive abilities (e.g., academic self-concept, growth mindset) it is pertinent to include several variables. Therefore, we have included both reading self-concept, reding motivation, master the content, perseverance, and growth mindset, alongside socio-demographic variables such SES, migration background, and gender.
Method
The PISA data sets (https://www.oecd.org/pisa/data/) include plenty of information at the student level, school level, as well as questionnaires filled in by principals and students. For example, the cognitive tests (e.g., PISA reading) may be linked to student level information on their SES, migration background, non-cognitive abilities such as reading self-concept or general ambition in the school context, and other factors.
The descriptive statistical analyses were conducted with SPSS 28 and 29, while IDB Analyzer (IEA, 2021) was used for linear regression analyses (with all ten plausible values but without a multi-level model approach). In IDB Analyzer, missing data were handled with listwise deletion. Some of the regression results are presented in Table 1 whereas the descriptive statistics are presented in Supplementary information.
PISA uses a stratified two-stage sampling design where students belong to schools. First, one selects a school and then students are selected. The chances of students to be selected is connected to the selection of schools and thus there are dependent observations. To account for the sampling design, we used multilevel analysis (e.g., Gelman & Hill, 2006, Snijders & Bosk, 2012). Multi-level models are, in educational research, preferable to standard ordinary least square regression models when the data has a nested or hierarchical structure (e.g., Lee, 2000; Snijders & Bosker, 2012). To investigate the research questions, multi-level modeling using all ten plausible values for PISA mathematics and reading was applied to the PISA data on Sweden. In accordance with Peugh (2010), we have chosen the appropriate parameter estimator, assessed the need for an MLM approach through intra-class correlations, built the level-1 model, built the level-2 model, and reported the effect sizes.
Student background factors were used as level 1 variables and level 1 variables aggregated to the school level were used as level 2 variables. Mplus (Muthén & Muthén, 2014) was used to explore different multilevel models. Level 1 and level 2 factors were constructed from the available variables in PISA 2018. At level 1, the following variables were defined.
Main independent variables
SES (number of books at home, mother’s education level), migration background, non-cognitive abilities (growth mindset, reading self-concept, reading motivation, mastery of the school content, perseverance, self-concept of competence), and gender as a control variable.
Student level 1
SES, as being part of a broader ESCS index (economic, cultural and social status), is widely used as set of variables regarding secondary analyses with PISA data. However, recent research has with regard to both validity, reliability and theoretical considerations criticized the merging of economic, educational, social and cultural indicators into a single composite measure of SES (e.g., Avvisati, 2020; Marks & O’Connell, 2021). Regarding the previous critique of several aspects of home possessions (e.g., have a television, have a car), we have included two SES indicators: number of books at home and the mother’s educational level. The reason for this is that in progressive countries like Sweden it is often the case that mothers have acquired higher levels of education, while the opposite pattern is still the case on a global scale (Marks & O’Connell, 2021). This is conducted partly in accordance with Chmielewski (2019) who stresses that at least one major SES indicator is important to include in analyses on ILSA data. A preliminary analysis with number of books at home, father’s education, and mother’s education included in the same regression model (PISA mathematics) did also show a slightly higher effect size for mother’s education. Cronbach’s alpha was α = 0.578 for books at home, father’s education, and mother’s education.
-
H1: SES A (ST013Q01TA): Number of books at home. We kept the original coding scheme where 1 = 0–10 books, 2 = 11–25 books, 3 = 26–100 books, 4 = 101–200 books, 5 = 201–500 books, 6 = more than 500 books.
-
H2: SES B (ST005Q01TA): Highest level of schooling of the mother. As the highest level represents a small value (e.g., Level 3A = 1), a negative effect size was expected.
-
H3: Migration background (MB, ST019AQ01T): This variable was dummy coded so that native students take the value 1 and non-natives students of either first- or second-generation immigrant students take the value 0. In PISA 2018 (total N = 5338), approximately 20% of students had a first- or second-generation immigrant background (see Supplementary information). To examine if MB was confounded with the two SES indicators, as well as test language or birth country of the test taker, bivariate correlations were carried through. However, MB and test language were only moderately correlated (r = 0.419). That was also the case with MB and birth country (r = -0.276).
-
H4: Self-concept of competence constitutes a composite sum score variable of all such items related to the variable ST188Q01–06 that have a high internal consistency. Cronbach’s alpha was α = 0.818. For example, the item “I usually manage one way or another” was measured on a Likert-type scale as 1 = Strongly disagree, 2 = Disagree, 3 = Agree, and 4 = Strongly agree. This item was reverse coded and then transformed into a sum score variable with the four other related items.
-
H5: Perseverance was measured through the items ST182Q03HA–ST182Q06HA. Cronbach’s alpha was α = 0.799. The four items were measured on a Likert-type scale as 1 = Strongly disagree, 2 = Disagree, 3 = Agree, and 4 = Strongly agree. These items were reverse coded and then transformed into a sum score variable with the three other related items.
-
H6: Master the content: Three items (ST208Q1–03) measure the extent to which a student masters the curricular content in general and represent a generic form of academic self-concept. These items were reverse coded and then transformed into a sum score variable. Each item consists of five response options on a Likert-type scale: 1 = Not all true for me, 2 = Slightly true of me, 3 = Moderately true of me, 4 = Very true of me, 5 = Extremely true of me, where 4 and 5 take the value 1 and the rest 0. Cronbach’s alpha was α = 0.851.
-
H7: Reading self-concept (ST161Q01–03) measures that degree to which students have faith in their own reading skills (e.g., “I am a good reader”). As such it was, like other ordinal scale variables slightly transformed into sum scores. Cronbach’s alpha was α = 0.720.
-
H8: Growth mindset is based on a single item, “Your intelligence is something about you that you can’t change very much” (ST184Q01). This parsimonious measure of growth mindset is because only one item captures growth mindset adequately in PISA 2018 (e.g., Bernardo, 2022).
-
H9: Reading motivation is measured through the related items ST160Q01–ST160Q05A. For example, one item measures the extent to which a student reads only if one has to. Cronbach’s alpha was α = 0.775.
Control variables
-
H10: Gender (dummy coded so that boys = 1, girls = 0).
Interaction terms
In order to examine a potential interaction effect between SES and migration background, and between SES and growth mindset, two interaction terms were created. These variables were mean centered in order to avoid problems which are associated with multicollinearity.
School level variables
At level 2 the following variables (H1-S10, see above) were aggregated to the school by using the breakit function in SPSS. In addition, we included three more meso level (i.e., school) variables from the school questionnaire (Sweden’s N = 223) by using the merge function in SPSS. The first is the index of proportion of teacher fully certified (PROATCE) and the second variable concerns student behavior hindering learning (STUBEHA, valid N = 215). The third variable concerns teacher behavior hindering learning (TEACHBEHA). All these three variables were valid at the school level (N = 215).
For each subject (mathematics and reading) we examined the following two models: the null model and the full two-level model. From the PISA database, we used the available ten plausible values for reading (PVREAD) and mathematics (PVMATH), respectively as dependent values. Using the null model, we could obtain estimates of the intraclass correlations (ICC) and between-school variances. Here, the ICC gives us a measure of how homogeneous the data are within the school level clusters. In the null model the dependent variable Y (PVREAD or PVMATH) for each student i was estimated as a function of the school j average.
-
Level 1 (within schools): \(Y_{ij} = \beta_{0j} + r_{ij} ,i = 1,...,N\)
-
Level 2 (between schools): \(\beta_{0j} = \gamma_{00} + \mu_{0j} ,j = 1,...,J\)
The level 1 term \(r_{ij}\) is a random error which is a unique effect for student i in school j. The level 2 term \(\gamma_{00}\) is the grand mean of reading or mathematics performance and \(u_{0j}\) is the error term representing a unique effect associated with school j. In the two-level model, the first level contained student background factors which were weighted with student weights. In the second level, aggregated student level factors were included and weighted with school weights. Gender was included as the percentages of boys and girls at each school.
-
Level 1 (within schools): \(Y_{ij} = \beta_{0j} + \beta_{1j} H_{1ij} + ...\beta_{fj} H_{fij} + r_{ij} ,i = 1,...,N\)
-
Level 2 (between schools): \(\beta_{0j} = \gamma_{00} + \gamma_{01} S_{1j} + ... + \gamma_{0l} S_{lj} + \mu_{0j} ,j = 1,...,J\)
$$\beta_{1j} = \gamma_{10} ,...,\beta_{fj} = \gamma_{fj}$$
Each analysis was performed separately for each plausible value, and then combined using the Rubin’s (1987) method which is implemented in Mplus. To decide which model to use, we used the Bayesian Information Criterion (BIC) (Schwarz, 1978), which allows the comparison of models if the sample remains constant (McCoach & Black, 2008). Missing data was handled using the Maximum Likelihood Robust (MLR) estimation in Mplus.
Robustness checks
Partly in accordance with Avvisati (2020), we consider separated SES indicators to be appropriate. Hence, we have separated parental education from home possessions and, indeed, omitted most of the home possession indicators. As our parsimonious SES indicators might be somewhat arbitrary, we conducted robustness tests in IDB Analyzer. Our linear regressions compared the effect sizes (β) of the included variables and R2 values of the two models. We used the ten plausible values for reading achievement in both models.
Specifically, the same variables as in Table 1 were included except for the SES indicators. In the models, we included both the mother and the father’s highest degree of education and included two more home possession indicators: ‘A room of your own’ (Yes/No) and ‘A computer you can you use for school work’ (Yes/No).
Results
The results of the linear regression model (Table 1) indicate that reading self-concept, SES, migration background, gender, and growth mindset explain a substantial amount of variance in reading achievement. Other non-cognitive abilities such as perseverance and self-concept of competence explain much less in comparison.
The intraclass correlation for PISA reading was 0.075, which indicates that there might be some small but nested effects of the data and which further indicated that the multi-level model technique was appropriate. However, it does also show that only 7.5% of the variance at the within-school level can be explained by school level differences. The intraclass correlation for PISA mathematics was 0.105, which shows that the multi-level model technique was even more required in that case (e.g., Musca et al., 2011), as the school level differences explained 10.5% of the variance at the within-school level.
Our results for the full model (Table 2) indicate that the results were similar for both mathematics achievement and reading achievement. Socio-demographic and socio-economic factors such as SES and migration background were statistically significant with large effect sizes. Moreover, non-cognitive factors such as reading self-concept and growth mindset are highly significant. While the effect size of reading self-concept is larger for reading achievement than mathematics achievement it is still substantial and statistically significant for mathematics.
The results also indicate that both PISA mathematics and reading achievement transcend SES and migration background, as various non-cognitive abilities also matter. On the one hand, this shows that students seem to properly assess their own reading capabilities and competence in relation to the school context (e.g., they can master the content), and on the other hand it demonstrates the significance of having a growth mindset.
However, we found no meaningful interaction effects for SES and migration background or for SES and growth mindset, which indicates that the effects of the variables are linear and discrete. It is also important to underscore that because the regression model is based on individual level data, gender does still have an effect on the individual level.
The findings of the between-school level analysis (Level 2) show that the mother’s education and reading motivation matter for both mathematics and reading achievement. The R2 values and BIC values show good model fit (Table 2). The second model with additional school level variables (see Table 8 in Supplementary information) indicates that teacher behavior (TEACHBEHA) was statistically significant at the 5% level for reading achievement. However, none of the three variables were statistically significant for mathematics achievement.
Our robustness checks show that the R2 values increased with as much as 5% when we included more variables (e.g. see Model 3, Table 1, and Model 4, Table 1, in Supplementary information). However, the R2 value was only slightly higher for the model which included both the mother and the father’s highest level of education, an ordinal scale variable of number of books at home, and two more home possession variables. The effect sizes for the variables are largely similar, except for mother’s education whose β value was lower when the father’s education was included in the model. Yet, mother’s education had a slightly larger effect size compared to the father’s education. The SES variables are roughly equal to migration background, reading abilities, reading motivation and growth mindset.
Discussion
Our results clearly show that, as expected, both SES (specifically, number of books at home) and migration background have a substantial relationship with both PISA mathematics and PISA reading achievement at the within-school level (Level 1). More specifically, SES is somewhat more important for mathematics than migration background, whereas the opposite is the case for reading achievement. The magnitude of these two variables is of a similar order.
Moreover, the absence of a statistically significant interaction effect for SES and migration background indicates that the relationships are linear and discrete. In part, that might be because the reading abilities of students with a migration background affect both mathematics and reading achievement (Boman, 2022a; Holmlund et al., 2019). These results are partly in agreement with earlier research conducted on Swedish students and contexts such as Gustafsson and Yang Hansen (2018) and Boman (2022a), but it is notable that both the SES and migration background variables become less significant or sometimes statistically insignificant when cognitive ability is included in the same regression models (Boman, 2023a). On the other hand, recent research shows that IQ and SES have a similar magnitude for aggregated PISA scores at the cross-national level (Boman, 2023b).
However, both mathematics and reading achievement were also influenced by non-cognitive factors. Some of these are domain-specific (reading self-concept) whereas others are more generic (growth mindset, academic self-concept, here indicated by the master the content variable). It might be the case that both mathematics and reading are affected by omitted variables such as general cognitive ability (Boman, 2023a) or domain-specific non-cognitive abilities such as mathematics self-concept in the case of mathematics. Overall, the factors of the most pertinence are micro factors (individual and family effects), according to Bronfenbrenner’s (1994) theoretical classification.
Notably, both reading motivation and mother’s education (another SES indicator besides the estimated number of books at home) seem to influence between-school contextual differences (Level 2) in mathematics and reading achievement. Thus, these contextual factors at the meso level (e.g., Bronfenbrenner & Morris, 1998) seem to be important for both literacy domains. Perhaps schools with lower results should strive for higher levels of reading motivation by improving the quality of reading instruction. However, micro-meso level factors such as school SES might be more difficult to change.
In accordance with earlier studies that suggest that mathematics achievement is influenced by reading abilities (Ajello et al., 2021; Ding & Homer, 2020), this pattern was also found in our analyses.
Limitations and directions for future research
The current study has several limitations. Our framework of variables focused on socio-economic, socio-demographic (migration background, gender), and various non-cognitive factors (e.g., growth mindset) but there might be other variables that also contribute to the explanation of PISA mathematics and reading achievement. For example, Marks and O’Connell (2021) stress that PISA have substantial overlaps with cognitive ability tests but as long as there are no additional cognitive ability tests imbedded within PISA (Boman, 2023a, b) researchers must account mostly for socio-economic, socio-demographic, and non-cognitive factors. Additionally, while our theoretical and empirical focus was on SES, migration background, and non-cognitive factors there might be some other specific school-level factors that slightly influence contextual between-school relations (e.g., Liu et al., 2022).
Moreover, we did not test for measurement invariance within the migrant population vis-à-vis the native population (e.g., Ding et al., 2022; Mittal et al., 2022; Odell et al., 2021) as regards SES and migration background. Furthermore, due to the cross-sectional design of the PISA data it is not possible to discern causal relationships. In addition, although we analysed the relationship between reading self-concept and PISA mathematics achievement we did not examine item-level differences or gender and reading ability interactions (c.f., Ajello et al., 2021; Ding & Homer, 2020).
Future studies are required to investigate if these aggregated between-school differences are theoretically and empirically meaningful and not just statistical artefacts (e.g., Marks & O’Connell, 2021). However, as the intraclass correlation for mathematics achievement is rather high there some to be significant school level factors that are pivotal for mathematics achievement. However, reading achievement is mostly affected by individual differences regarding the major explanatory variables. In the Swedish context specifically, migration background and SES will likely influence student achievement also in the foreseeable future (Boman, 2022a).
Contributions of the study and concluding remarks
Our study contributes to the literature on the nexus between academic achievement, SES and migration background in the Swedish context (e.g., Gustafsson & Yang Hansen, 2018), the relations between academic achievement and non-cognitive abilities such as perseverance, growth mindset, and academic self-concept among Swedish samples (e.g., Rosander & Bäckström, 2014; Thorsen et al., 2021), PISA reading (e.g., Rasmusson, 2016), and PISA achievement research more in general (e.g., Reimer et al., 2018; Muench et al., 2022). Moreover, we expand earlier studies such as Ding and Homer (2020) and Ajello et al. (2021) by including both mathematics and reading, as well as by using more recent data from PISA (2018) in our analyses. Hence, our results have implications for both the Swedish context and to some extent more general patterns with regard to PISA mathematics and reading achievement.
Data availability
The data sets are, in the first instance, available at https://www.oecd.org/pisa/data/. Our data files, as we have coded and elaborated the data, might be obtained after requests from any of the two authors.
References
Ajello, A. M., Caponera, E., & Palmerio, L. (2021). Italian students’ results in the PISA mathematics test: Does reading competence matter? European Journal of Psychology of Education, 33, 505–520.
Avvisati, F. (2020). The measure of socio-economic status in PISA: a review and some suggested improvements. Large-scale Assessments in Education, 8.
Bahnik, S., & Vranka, M. (2017). Growth mindset is not associated with scholastic aptitude in a large sample of university applicants. Personality and Individual Differences, 117, 139–143.
Bandura, A. (2006). Toward a psychology of human agency. Perspectives on Psychological Science: A Journal of the Association for Psychological Science, 1(2), 164–180.
Bernardo, A. B. I. (2022). Socioeconomic status moderates the relationship between growth mindset and learning in mathematics and science. International Journal of School & Educational Psychology, 9(2), 208–222.
Bhutoria, A., & Aljabri, N. (2022). Patterns of cognitive returns to Information and Communication Technology (ICT) use of 15-year-olds: Global evidence from a Hierarchical Linear Modeling approach using PISA 2018. Computers and Education, 181, 104447.
Björklund, A., Fredriksson, P., Gustafsson, J.-E., Öckert, B. (2010). Den svenska utbildningspolitikens arbetsmarknadseffekter. Vad säger forskningen? Available at: https://www.ifau.se/globalassets/pdf/se/2010/r1013-den-svenska-utbildningspolitikens-arbetsmarknadseffekter-vad-sager-forskningen.pdf. Accessed December 2023.
Boman, B. (2020). What makes Estonia and Singapore so good? Globalisation, Societies and Education, 18(2), 181–193.
Boman, B. (2022a). PISA achievement in Sweden from the perspective of both individual data and aggregated cross-country data. Frontiers in Education, 6, 753347.
Boman, B. (2022b). Regional differences in educational achievement among Swedish Grade 9 students. Scandinavian Journal of Educational Research, 66(4), 610–625.
Boman, B. (2022c). Regional differences in educational achievement: A replication study of municipality data. Frontiers in Education, 7.
Boman, B. (2023a). The influence of SES, cognitive and non-cognitive abilities on grades: cross-sectional and longitudinal evidence from two Swedish cohorts. European Journal of Psychology of Education, 38, 587–603.
Boman, B. (2023b). Is the SES and academic achievement relationship mediated by cognitive ability? Evidence from PISA 2018 using data for 77 countries. Frontiers in Psychology, 14.
Bong, M., & Skaalvik, E. (2003). Academic self-concept and self-efficacy: How different are they really? Educational Psychology Review, 15, 1–40.
Bronfenbrenner, U. (1994). Ecological models of human development. In International Encyclopedia of Education, 3, 2nd. Ed. Oxford: Elsevier. Reprinted in: M. Gauvain, & M. Cole (Eds)., Reading on thedevelopment of children, 2nd ed. (1993, pp. 37–43). Freeman.
Bronfenbrenner, U., & Morris, P. A. (1998). The ecology of developmental process. In R. M. Lerner (Ed.), Handbook of child psychology (Vol. 1, pp. 993–1028). Wiley.
Chmielewski, A. K. (2019). The global increase in the socioeconomic achievement gap, 1964–2015. American Sociological Review, 84(3), 517–544.
Claro, S., Paunescu, D., & Dweck, C. (2016). Growth mindset tempers the effects of poverty of academic achievement. PNAS, 113(31), 8864–8868.
Depren, S., & Depren, Ö. (2022). Cross-cultural comparisons of the factors influencing the high achievement in Turkey and China: Evidence from PISA 2018. The Asia-Pacific Education Researcher, 31, 427–437.
Ding, H., & Homer, M. (2020). Interpreting mathematics performance in PISA: Taking into account reading performance. International Journal of Educational Research, 102.
Ding, Y., Yang Hansen, K., & Klapp, A. (2022). Testing measurement invariance of mathematics self-concept in PISA using MGCFA and the alignment method. European Journal of Psychology of Education.
Dobbins, M., & Martens, K. (2012). Towards an education approach à la finlandaise? French education policy after PISA. Journal of Education Policy, 27(1), 23–43.
Duckworth, A. L., & Quinn, P. D. (2009). Development and validation of the short grit scale (Grit-S). Journal of Personality Assessment, 91(2), 166–174.
Dweck, C. S. (2016). The remarkable reach of growth mindsets. Scientific American Mind, 27(1), 36–41.
Engelhardt, L., Church, J., Harden, P., & Tucker-Drob, E. (2018). Accounting for the shared environment in cognitive abilities and academic achievement with measured socioecological contexts. Developmental Science, 22(1).
Gamazo, A., & Martinez-Abad, F. (2020). An Exploration of factors linked to academic performance in PISA 2018 through data mining techniques. Frontiers in Psychology, 11.
Gelman, A., & Hill, J. (2006). Data analysis using regression and multilevel/hierarchical models. Cambridge University Press.
Gjicali, K., & Lipnevich, A. (2021). Got math attitude? (In)direct effects of student mathematics attitudes on intentions, behavioral engagement, and mathematics performance in the U.S. PISA. Contemporary Educational Psychology, 67.
Gustafsson, J.-E. (2008). Effects of international comparative studies on educational quality on the quality of educational research. European Educational Research Journal, 7(1), 1–17.
Gustafsson, J.-E., & Yang Hansen, K. (2018). Changes in the impact of family education on student educational achievement in Sweden 1988–2014. Scandinavian Journal of Educational Research, 62(5), 719–736.
Hanushek, E. A., Piopiunik, M., & Wiederhold, S. (2019). The value of smarter teachers: International evidence on teacher cognitive skills and student performance. Journal of Human Resources, 54(4), 857–899.
Holmlund, H., Sjögren, A., & Öckert, B. (2019). SOU, Jämlikhet i möjligheter och utfall i den svenska skolan. Bilaga 7 till Långtidsutredningen 2019 (40). https://www.regeringen.se/4adad2/contentassets/23c13d7ae0ef48e4bed43b68917573d3/jamlikhet-imojligheter-och-utfall-i-den-svenska-skolan-sou-201940.pdf. Accessed December 2023.
IEA. (2021). Help manual for the IEA IDB Analyzer (Version 5.0.1). https://www.iea.nl. Acccessed December 2023.
Kim, S. W. (2019). Is socioeconomic status less predictive of achievement in East Asian countries? A systematic and meta-analytic review. International Journal of Educational Research, 97, 29–42.
Kim, S. W., Cho, H., & Kim, L. Y. (2019). Socioeconomic status and academic outcomes in developing countries: A meta-analysis. Review of Educational Research, 89(6), 875–916.
Lee, V. E. (2000). Using hierarchical linear modeling to study social contexts: The case of school effects. Educational Psychologist, 35(2), 125–141.
Lee, J., & Borgonovi, F. (2022). Relationships between family socioeconomic status and mathematics achievement in OECD and non-OECD countries. Comparative Education Review, 66(2), 199–227.
Liu, X., Valcke, M., Yang Hansen, K., & De Neve, J. (2022). Does school level instructional quality matter for school mathematics performance? Comparing teacher data across seven countries. Sustainability, 14.
Ma, Y. (2021). A cross-cultural study of student self-efficacy profiles and the associated predictors and outcomes using a multigroup latent profile analysis. Studies in Educational Evaluation, 71, 101071.
Marks, G. N., & O’Connell, M. (2021). Inadequacies in the SES–achievement model: Evidence from PISA and other studies. Review of Education, 9(3).
McCoach, B. D., & Black, A. C. (2008). Evaluation of model fit and adequacy. In A. A. O’Connell & D. B. McCoach (Eds.), Multilevel modeling of educational data (pp. 245–272). Information age publishing inc.
Meunier, M. (2011). Immigration and student achievement: Evidence from Switzerland. Economics of Education Review, 30(1), 16–38.
Mittal, O., Scherer, R., & Nilsen, T. (2022). Assessing the evidence for the comparability of socioeconomic status with and without immigrant background in Norway and Sweden. Large-scale Assessments of Education, 10(3).
Muench, R., Wieczok, O., & Dressler, J. (2022). Equity lost: Sweden and Finland in the struggle for PISA scores. European Educational Research Journal.
Musca, S. C., Kamiejeski, R., Nugier, A., Méot, A., Er-Rafiy, A., & Brauer, M. (2011). Data with hierarchical structure: impact of intraclass correlation and sample size on Type 1 error. Frontiers in Psychology, 2.
Muthén, B & Muthén, L. (2014). MPLUS (Version 7) [computer software]. Los Angeles, CA.
Myrberg, E., & Rosén, M. (2009). Direct and indirect effects of parents’ education on reading achievement among third graders in Sweden. British Journal of Educational Psychology, 79(4), 695–711.
Odell, B., Gierl, M., & Cutumisu, M. (2021). Testing measurement invariance of PISA 2015 mathematics, science, and ICT scales using the alignment method. Studies in Educational Evaluation.
OECD. (2009). PISA data analysis manual. https://www.oecd-ilibrary.org/docserver/9789264056275-en.pdf?expires=1640625956&id=id&accname=ocid195437a&checksum=F578CC594B2011913AFF4F0C3CF3EA97. Accessed December 2023.
OECD. (2016). PISA 2015 Results: Policies and Practices for Successful Schools. Retrieved from https://www.oecd-ilibrary.org/docserver/9789264267510-en.pdf?expires=1563262477&id=id&accname=guest&checksum=4E0F4F8709475E4FDC72A6E1245B869F. Accessed December 2023.
Peugh, J. L. (2010). A practical guide to multilevel modeling. Journal of School Psychology, 48(1), 85–112.
Prais, S. J. (2004). Cautions on OECD’s recent educational survey (PISA): Rejoinder to OECD’s response. Oxford Review of Education, 30(4), 569–573.
Rasmusson, M. (2016). A mutilevel analysis of Swedish and Norwegian students’ overall and digital reading performance with a focus on equity aspects of education. Large-scale Assessments of Education, 4(3).
Reimer, D., Jensen, S. S., & Kjeldsen, C. (2018). Social inequality in student performance in the Nordic countries. In Northern Lights on TIMSS and PISA 2018 (pp. 31–60). TeamNord.
Rolfe, V., Yang Hansen, K., & Strietholt, R. (2022). Integrating educational quality and educational equality into a model of mathematics performance. Studies in Educational Evaluation, 74, 101171.
Rosander, P., & Bäckström, M. (2014). Personality traits measured at baseline can predict academic performance in upper secondary school three years later. Scandinavian Journal of Psychology, 55(6), 611–618.
Rubin, D. B. (1987). Multiple imputations for non–response in surveys. Wiley.
Ruist, J. (2015). The Fiscal cost of refugee immigration: The example of Sweden. Population and Development Review, 41(4), 567–581.
Sackett, P. R., Kuncel, N. R., Arneson, J. J., Cooper, S. R., & Waters, S. D. (2009). Does socioeconomic status explain the relationship between admissions tests and post-secondary academic performance? Psychological Bulletin, 135(1), 1–22.
Schleicher, A. (2019). PISA 2018: Insights and interpretations. Available at: https://www.oecd.org/pisa/PISA%202018%20Insights%20and%20Interpretations%20FINAL%20PDF.pdf. Accessed December 2023.
Schwarz, G. (1978). Estimating the dimensions of a model. Annals of Statistics, 6, 461–464.
Sirin, S. (2005). Socioeconomic status and academic achievement: A meta-review of research. Review of Educational Research, 75(3), 417–453.
Snijders, T., & Bosker, R. J. (2012). Multilevel analysis: An introduction to basic and advanced multilevel modelling. SAGE.
Tan, C. Y. (2015). The contribution of cultural capital to students’ mathematics achievement in medium and high socioeconomic gradient economies. British Educational Research Journal, 41(6), 1050–1067.
Thorsen, C., Yang Hansen, K., & Johansson, S. (2021). The mechanisms of interest and perseverance in predicting achievement among academically resilient and non-resilient students: Evidence from Swedish longitudinal data. British Journal of Educational Psychology, 91(4), 1481–1497.
Wiberg, M., & Rolfsman, E. (2023). Students’ Self-reported Background SES Measures in TIMSS in Relation to Register SES Measures When Analysing Students’ Achievements in Sweden. Scandinavian Journal of Educatioal Research, 67(1), 69–82.
Yang Hansen, K., Radicid, J., Ding, Y., & Liu, X. (2022). Contextual effects on students’ achievement and academic self-concept in the Nordic and Chinese educational systems. Large-scale Assessments in Education, 10.
Zysberg, L., & Schwabsky, N. (2021). School climate, academic self-efficacy and student achievement. Educational Psychology, 41(4), 467–482.
Acknowledgements
The authors acknowledge two reviewers and Monica Rosén at the Department of Education and Special Education at the University of Gothenburg for important comments on earlier versions of this manuscript.
Funding
Open access funding provided by Stockholm University.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Competing interests
The authors have no competing interests.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Björn Boman, Department of Education, Stockholm University, Frescativägen 54, 106 91, Stockholm, Sweden. contact@bjornboman.com bjorn.boman@edu.su.se
Current research interests:
Academic achievement in Sweden, mostly at the secondary levels; academic achievement with aggregated or individual level PISA scores; academic achievement in East Asian school systems.
Recent publications:
Boman, B. (2022c). Educational achievement among East Asian schoolchildren: a thematic review of the literature. International Journal of Educational Research Open, 3.
Boman, B. (2023a). The influence of SES, cognitive abilities, and non-cognitive abilities on grades: longitudinal evidence from two Swedish cohorts. (2022). European Journal of Psychology of Education, 38, 587–603.
Boman, B. (2023b). Is the SES and academic achievement relationship mediated by cognitive ability? Evidence from PISA 2018 using data for 77 countries. Frontiers in Psychology, 14.
Marie Wiberg, Umeå School of Business, Economics and Statistics, Umeå University, Biblioteksgränd 6, 901 87, Umeå, Sweden. marie.wiberg@umu.se
Current research interests:
Pyschometrics; psychometrics in educational research; international large-scale assessments.
Recent publications:
Wiberg, M., & Rolfsman, E. (2021). Students’ Self-reported Background SES Measures in TIMSS in Relation to Register SES Measures When Analysing Students’ Achievements in Sweden. Scandinavian Journal of Educational Research.
Franco, V.R., Laros, J.A., & Wiberg, M. Nondecomposable item response theory models: fundamental measurement in psychometrics. Journal of mathematical psychology, 114.
Leôncio, W., Wiberg, M., & Battauz, M. Evaluating equating transformations in IRT observed-score and Kernel Equating methods. (2023). Applied psychological measurement, 47(2). 123–140.
Supplementary Information
Below is the link to the electronic supplementary material.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Boman, B., Wiberg, M. The influence of SES, migration background, and non-cognitive abilities on PISA reading and mathematics achievement: evidence from Sweden. Eur J Psychol Educ 39, 2935–2951 (2024). https://doi.org/10.1007/s10212-024-00805-w
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10212-024-00805-w