Advertisement

Trends in Socioeconomic Achievement Gaps in the Macroeconomic Context: Discussion and Future Research

  • Markus Broer
  • Yifan Bai
  • Frank Fonseca
Open Access
Chapter
  • 6.8k Downloads
Part of the IEA Research for Education book series (IEAR, volume 5)

Abstract

Using the 20 years of comparable data collected by the IEA’s Trends in International Mathematics and Science Study (TIMSS), this study aimed to identify and describe possible relationships between trends in the inequality of educational opportunities related to the differing socioeconomic status (SES) of the students. Thirteen educational systems were assessed as having sufficiently comparable data for the purposes of this research and a modified version of the TIMSS home educational resources (HER) index was developed for the analyses. Country-level indicators, such as centralization of educational systems, gross domestic product per person, government expenditure on education, a country’s Gini index, and the top 10% of earners’ share of pre-tax national income, provided additional relevant information. The research revealed some tentative patterns that may be worthy of deeper investigation. In the second decade of TIMSS (2003–2015), larger reductions in the achievement gap between low- and high-SES students tended to accompany steeper increases in the overall TIMSS performance in the corresponding education systems. There were also some indications that this may be associated with the level of centralization of an educational system and trends in country-level spending on education; this needs to be confirmed by further research as only broad connections were made with such macro-level indicators. Future research could usefully analyze the measurement invariance of the modified SES index used in the study, or investigate the potential of reconstructing the established TIMSS HER index so it can be confidently applied to analyses across all TIMSS administrations.

Keywords

Achievement gaps Centralized versus decentralized education systems Educational inequality Education spending Income inequality International large-scale assessment Macroeconomic indicators Trends in International Mathematics and Science Study (TIMSS) 

5.1 Summary of the Findings

Although the association between family SES*1 and student achievement has been thoroughly investigated in previous research (see Chap.  2), the extent of change in that association in individual education systems over time is less well known. Improving achievement among their disadvantaged students and narrowing the achievement gaps between students of low- and high-SES backgrounds is a common policy goal for many education systems. However, the lack of quantifiable measures, especially those that are easy to understand, makes it difficult to track and assess the effect of such efforts.

Twenty years of TIMSS data, from 1995 to 2015, provide researchers with a means to empirically address important research questions regarding changes in educational inequality over time. We used the TIMSS data to examine whether the inequality of educational outcomes due to SES* has changed for education systems over time and to investigate the extent to which disadvantaged students improved their academic performance over time in each education system.

Our first research question was: “How has the inequality of education outcomes due to family socioeconomic status changed for different education systems between 1995 and 2015?” We created a modified version of the TIMSS home educational resources (HER) index that was consistent over the 20-year period to define low- and high-SES* groups of students. For each educational system and assessment cycle, we calculated the achievement gap between students in these low- and high-SES* quartile groups. When examining achievement gaps in either mathematics or science between 1995 and 2015, our results suggested that Hungary, Iran, Lithuania, and Singapore experienced a significantly widening gap between low- and high-SES* students, while Norway, Slovenia, and the United States observed a significantly narrowing gap. By contrast, some educational systems observed some significant changes in one of the two decades of TIMSS (1995–2003 and 2003–2015), but not in the other. For example, Australia experienced a significant decrease in the SES* achievement gap for science between 1995 and 2003, and then a significant increase between 2003 and 2015, resulting in an overall non-significant trend over the 20-year period. Similarly, New Zealand experienced some decrease in the SES* gaps in the first decade of TIMSS, but this was followed by a significant increase in the second decade. There are many other examples where the more detailed study of trends broken down into the 1995–2003 and 2003–2015 time periods reveals interesting countervailing trends that warrant a closer look by researchers with a deep understanding of the local contexts.

Our second research question was: “To what extent have education systems managed to increase the academic performance of disadvantaged students between 1995 and 2015?” To answer this, we calculated the percentage of low-SES* students who performed at or above the TIMSS intermediate benchmark in each education system over time. It was of great importance to examine this question in conjunction with the first question because stagnant scores for low-SES* students and declines in the scores of high-SES* students are equally undesirable, but may also underlie a headline reduction in inequality. For example, in the Republic of Korea, the achievement gap in mathematics was 107 points in 2011 but declined to 84 points in 2015; this was not due to an improvement for low-SES* students but rather a decline in the performance of their high-SES* students (Fig.  4.13). In contrast, the United States showed a decreasing achievement gap for science between 1995 and 2015, which corresponded to a continuous improvement in the performance of their low-SES* students (Fig.  4.30). Ideally, education systems should strive for equality by improving the performance of all students and by aiming to improve the achievement of low-SES* students at a faster rate to reduce gaps in achievement (Mullis et al. 2016).

5.2 Relating the Findings to Country-Level Indicators in the Educational Systems and the Macroeconomic Context

To better understand our findings in the larger context in which education systems operate, we obtained macroeconomic and other indicators from the TIMSS encyclopedias, as well as data from external resources (see Table  3.6 for all sources). Our goal was to explore changes in country level indicators over time and contrast them with changes in the SES* achievement gap (Table 5.1). A few tentative patterns emerged, which merit further investigation.
Table 5.1

Change in achievement gap and macro-economic characteristics: 2003–2015

Education system

% Change in mathematics achievement gap (2003–2015)b

% Change in mathematics achievement gap (1995–2015)b

Gap (scale score points, 2015)b

Central/ local education system2

% Change in GDP per person (1995–2015)c

Total government expenditure on education as % of GDPc

% Change over second decade of TIMSS in prior indicatorc

Gini index (recent year)c

% Change in Gini index in the second decade of TIMSSc

Russian Federation

−28

−25

47

Local

250

11.15 (2012)

11

2005–2012

37.7 (2015)

−6

2003–2015

Norway

−20

−21

50

Central

114

17.03 (2013)

7

2005–2012

26.8 (2014)

−15

2004–2014

Republic of Korea

−10

−11

84

Local

120

11

2003–2012

31.6 (2012)

0

2006–2012

United States

−7

−12

80

Local

96

14.55 (2014)

2

2005–2011

41 (2013)

1

2004–2013

Slovenia

−6

−22

54

Central

95

10.00 (2013)

2

2005–2012

25.7 (2014)

4

2004–2014

Singapore

0

79

85

Central

115

19.96 (2013)

46.3 (2015)d

10

1998–2015

Sweden

9

–2

73

Local

70

14.94 (2014)

17

2005–2013

27.2 (2014)

4

2004–2014

Lithuania

15

1

80

Local

559

13.29 (2013)

37.7 (2014)

2

2004–2014

Australia

21

−17

73

Local

177

13.90 (2014)

8

2005–2013

34.7 (2010)

4

2003–2010

New Zealand

38

18

96

Local

120

17.99 (2015)

2

2005–2014

0.349 (2014)a

4a

2003–2014

Hungary

50

40

143

Local

178

9.41 (2014)

−19

2003–2014

30.4 (2015)

2

2004–2015

Hong Kong

52

18

63

Local

81

18.61 (2015)

−23

2003–2015

53.7 (2011)

25

1996–2011

Islamic Republic of Iran

74

136

106

Central

205

18.57 (2015)

−29

2005–2014

39.0 (2016)

−11

2005–2016

Notes — No information available. aData for the Gini index for New Zealand was not available, therefore we used the Gini coefficient of the Organization for Economic Cooperation and Development (OECD)

Sources bInternational Association for the Evaluation of Educational Achievement, Trends in International Mathematics and Science Study (TIMSS), 1995, 2003, and 2015 Mathematics Assessments and Encyclopedia; cThe World Bank; and dthe World Factbook (see Table  3.6 for all sources)

5.2.1 Tentative Pattern 1: Reductions in the Achievement Gap Tend to Accompany Improvements in Overall TIMSS Performance

In the second decade of TIMSS, we identified an inverse relationship between the trends in SES* achievement gaps and the TIMSS national averages for both mathematics and science. This finding was consistent with the previous literature using other cycles of TIMSS, which suggested a prominent inverse relation between the within-country dispersion of scores and the average level of scores by country (Freeman et al. 2010; Mullis et al. 2016). In other words, greater reductions in the achievement gap between low- and high-SES* students tended to accompany higher rates of increases in overall TIMSS performance over the 2003–2015 period (see Figs. 5.1 and 5.2). This is not a trivial finding, since, as discussed previously, a continuous growth in overall performance is possible without any reduction over time in the SES* achievement gap if both low and high SES* groups see the same rate of performance growth or shrinkage over time. If that were the case for the all studied education systems, the regression line would be flat.
Fig. 5.1

Difference in average mathematics score and SES* achievement gap, by education system, 2003–2015. (Note The Islamic Republic of Iran was treated as an outlier and not included when fitting the regression line)

Fig. 5.2

Difference in average science score and SES* achievement gap, by education system, 2003–2015. (Note The Islamic Republic of Iran was treated as an outlier and not included when fitting the regression line)

5.2.2 Tentative Pattern 2: Education Systems That Observed Increases in Achievement Gaps Tend to be Decentralized

While education systems that were able to reduce the SES* achievement gap could be either centralized or decentralized systems, almost all the education systems that observed increases in their SES* achievement gaps were decentralized systems, with the exception of the Islamic Republic of Iran (Table 5.1); note that this education system was also an outlier in the previous analysis.

5.2.3 Tentative Pattern 3: Education Systems That Reduced Investment in Education Tended to Observe an Increased Mathematics Achievement Gap

When examining the changes in the percentage of GDP spent on education (this indicator spanning mostly 2003–2015 figures, see Table 5.1), results suggested that those education systems that saw a reduction of investment in education over time also happened to observe a significant increase in the SES* mathematics achievement gap in the second decade of TIMSS (Fig. 5.3).
Fig. 5.3

Change in the percent of GDP spent on education and in the mathematics achievement gap, by education system, 2003–2015. (Notes Indicator “Percent of GDP Spent on Education” was obtained from the World Bank Open Data and the UNESCO Institute for Statistics (see Table  3.6). The calculation of change in the percent of GDP spent on education between 2003 and 2015 is the difference between the percentages in 2003 and 2015 divided by the percentage in 2003)

5.3 Limitations of the Study and Recommendations for Future Research

This study has several limitations that should be addressed by future research. The first limitation is that this study did not examine all potential factors that might explain the observed trends. Although we collected information on macro-level indicators in every corresponding education system over 20 years, we did not investigate empirically if those factors contributed to the changes in educational inequality that we observed. The broad connections made between macro-level changes and changes in educational inequality are descriptive. Future research exploring influential factors driving these changes would be important for understanding why some education systems were able to reduce the SES* achievement gaps and what can be learned by others. For example, multilevel modeling can be employed to test hypotheses regarding its potential association with macro-level factors. However, some of the factors may defy easy categorization and may be specific to individual educational systems or work only in the presence of other factors. Thus, there is also a place for a more contextual and qualitative understanding of the findings. Researchers with a deep understanding of local context would be in a better position to examine why and how these changes took place in their own educational system.

Second, measurement invariance of common items across years could be a concern for the SES* index used in this study (a modified version of the TIMSS HER index). For example, the possession of a computer in the 1990s may carry a very different meaning and value to the possession of a computer in the 2010s with respect to what it says about a student’s SES* background. We are uncertain which items see a drift in meaning, how much drift there is over time, and how such drift manifests itself in different countries. In spite of these concerns, we believe that for our study this was not a critical problem because we compared students in the highest and the lowest SES* quartile for each country and in each cycle separately. In other words, the meaning of the items or even the scale may change slightly, but this should not have had a strong influence on the comparison of the achievement gaps that were calculated based on the distribution of students’ SES* in a particular educational system and cycle. Nevertheless, future research should analyze measurement invariance of the SES* index itself or even reconstruct an item response theory scaled version of the HER index for years prior to 2011 so that analyses with that index would be possible across all TIMSS administrations.

Third, it is important to recognize that the meaning of high- and low-SES* differs by societies. We decided to use educational system specific cut-offs to define SES* groups because the current study focused on the trend in educational inequality within a society. Therefore, in interpreting comparisons between societies, it should be recognized and understood that high-SES* students in one country can be very different from high SES* students (in an absolute sense) in another country.

Finally, our analyses showed relatively distinct patterns of change in educational inequality in the first and second decades of TIMSS across countries (see Table  4.1). Future research should especially focus on the second decade of TIMSS, namely the period 2003–2015, as many significant changes in SES* achievement gaps occurred in this decade. As the number of countries participating in TIMSS has expanded since its inception, this would have the added value of allowing more educational systems to be included in the analyses. Moreover, there would be a greater availability of comparable country level macro-economic indicators if 2003 is taken as the base year instead of 1995.

5.4 What Have We Learned from Twenty Years of TIMSS Data?

Over the 20 years of TIMSS, we found that only a few educational systems were able to significantly reduce the achievement gap between high- and low-SES* students, to improve the performance of their most disadvantaged students, and to increase their national average score. Most of the education systems that we studied did not observe such a promising three-fold trend. Among the 13 education systems studied, only Slovenia observed such a trend in mathematics and only the United States for science. This further reflects the difficulty of fostering positive change in academic performance, or maintaining high performance, for all students over time, while also counteracting a general rise in inequality through policies in the education system that would enable a closing of the SES* achievement gap and effectively address the needs of disadvantaged students.

By contrast, some educational systems observed some significant changes in one of the two decades of TIMSS (1995–2003 and 2003–2015), but not the other. For example, Australia experienced a significant decrease in the SES* achievement gap for science between 1995 and 2003, followed by a significant increase between 2003 and 2015. This resulted in an overall non-significant trend over the 20-year period. There are many other examples where a more detailed study of trends broken down into the 1995–2003 and 2003–2015 time periods would be of interest. Researchers with a deep understanding of local contexts should take a closer look at such countervailing trends.

For the second decade of TIMSS (2003–2015), three tentative patterns emerged when contrasting changes in country level indicators over time and changes in the SES* achievement gaps. First, there was an inverse relationship between the changes in SES* achievement gaps and the changes in TIMSS national averages for both mathematics and science. Second, almost all the education systems with an increase in their SES* achievement gaps were categorized as “decentralized” education systems in the study. Third, the education systems with a reduction of investment in education happened to observe a significant increase in the SES* mathematics achievement gap. Although these patterns are preliminary, we encourage further investigation into the country-level changes with additional countries being included in the analyses.

Footnotes

  1. 1.

    The SES measure used in this study is a modified version of the TIMSS home educational resources (HER) index and does not represent the full SES construct, as usually defined by parental education, family income, and parental occupation. In this report, we therefore term our measure SES* to denote the conceptual difference (Please refer to Chap.  1 for more details).

References

  1. Freeman, R. B., Machin, S., & Viarengo, M. (2010). Variation in educational outcomes and policies across countries and of schools within countries. NBER Working paper series no. 16293. Cambridge, MA: National Bureau of Economic Research. Retrieved from http://www.nber.org/papers/w16293
  2. Mullis, I.V.S., Martin, M.O., & Loveless, T. (2016). 20 Years of TIMSS: International Trends in Mathematics and Science Achievement, Curriculum, and Instruction. Chestnut Hill, MA: TIMSS & PIRLS International Study Center, Boston College. Retrieved from: http://timss2015.org/timss2015/wp-content/uploads/2016/T15-20-years-of-TIMSS.pdf

Copyright information

© International Association for the Evaluation of Educational Achievement (IEA) 2019

Open Access This chapter is licensed under the terms of the Creative Commons Attribution-NonCommercial 4.0 International License (http://creativecommons.org/licenses/by-nc/4.0/), which permits any noncommercial use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

Authors and Affiliations

  • Markus Broer
    • 1
  • Yifan Bai
    • 1
  • Frank Fonseca
    • 1
  1. 1.American Institutes for ReserachWashingtonUSA

Personalised recommendations