Results: Evidence for the Globalization of Science Curricula from TIMSS

Abstract

In TIMSS, participating education systems are asked in the curriculum questionnaire which TIMSS science topics are included in their intended science curricula. By coding the changes in countries’ responses, some measure of the changes in national science curricula between 1999 and 2015 (for Grade 8) and between 2003 and 2015 (for Grade 4) may be determined. Countries can also be categorized by how stable or changeable their curriculum is on the basis of the number of changes they have made to their curriculum, the average number of curriculum changes at Grades 4 and 8 for each TIMSS cycle and the topics common to the vast majority of participating countries’ intended science curricula. The analysis found there was considerable variation among countries in terms of the number and nature of changes they made to their science curriculum between 1999 (2003 for Grade 4) and 2015. Cluster analysis and discriminant analysis revealed which science topics were most discriminating in terms of predicting country clusters. For both grades, and for each TIMSS cycle, the analyses produced two groups of countries based on responses to the curriculum questionnaire. The cluster and discriminant analyses provide stronger evidence for convergence in science curricula over time at Grade 8 than at Grade 4. The implemented science curriculum of 15 countries was examined in more depth using the TIMSS teacher questionnaire responses; due to difficulties in obtaining robust comparable data for countries across the different TIMSS cycles and the significant caveats that would have applied to any conclusions drawn from these analyses, these data are not considered.

4.1 Coding of Curriculum Questionnaire Data Results

4.1.1 The Extent of Change in Countries’ Science Curricula Over Time

By coding the changes in countries’ responses to which TIMSS science topics are included in their intended science curricula, it was possible to identify the extent to which countries’ intended science curricula had changed between 1999 and 2015 (for Grade 8) and between 2003 and 2015 (for Grade 4). We found considerable variation in the extent to which countries’ intended science curricula had changed. At both Grade 4 and Grade 8, a number of distinct groups of countries could be identified from the data. These ranged from highly stable (fewer than three changes to TIMSS science topics included in countries’ intended science curricula over the period of comparison) through to highly changeable (more than ten changes to TIMSS science topics included in countries’ intended science curricula over the period of comparison).

In this coding exercise, a change to a science topic could either be its removal from the intended curriculum, its addition to the intended curriculum, or a change in emphasis for the science topic in the intended curriculum. A change in emphasis could be an increase in emphasis; for example, previously it may have been taught only to the more able students but is now intended to be taught to all or most students. Alternatively, it could be a reduction in emphasis; for example, previously intended to be taught to all or most students but now only intended to be taught to the more able students.

We coded these changes for all the countries that participated in both the 2003 and 2015 TIMSS cycles at Grade 4 (Table 4.1) and for all the countries that participated in both the 1999 and 2015 TIMSS cycles at Grade 8 (Table 4.2). For Grade 4, there were 21 TIMSS science topics included in this coding exercise whilst for Grade 8 there were 20 TIMSS science topics. The same comparison was made between 2003–2007 and 2007–2015 for Grade 4 and between 1999–2007 and 2007–2015 for Grade 8; in each case they yielded similar results.

Table 4.1 Country categorizations for Grade 4 between 2003 and 2015

Table 4.2 Country categorizations for Grade 8 between 1999 and 2015

The Singaporean curriculum was noticeably more stable than that of other countries at Grade 4 (Table 4.1). There was only one change in the TIMSS topics intended to be taught in Singapore at Grade 4 between 2003 and 2015, and the next most stable country (Lithuania) made only four changes to its intended science curriculum in the same period. By contrast, in Belgium (the country with the most changeable curriculum), there were 15 changes in the TIMSS topics intended to be taught. Given Singapore’s high performance on TIMSS across the years it is unsurprising that the number of changes to the intended science topics taught was minimal between 2003 and 2015, as there was likely less incentive to revise the curriculum. By contrast, Morocco is one of the lowest performing countries in TIMSS science at Grade 4. It is likely that the high number of changes to its intended science curriculum have been instigated to improve future science performance generally, not just on TIMSS.

Similar patterns emerge from the Grade 8 data in terms of distinct groups of countries demonstrating varying number of changes to their intended science curriculum (Table 4.2). However, the countries in each group do not necessarily match the Grade 4 groupings. For example, although Singapore shows remarkable stability in its science curriculum at Grade 4, changes in the intended science curriculum at Grade 8 have been far more extensive. It is important here to note that, for Grade 4, the comparison is being made between 2003 and 2015, whereas for Grade 8 the comparison is between 1999 and 2015 and so covers a longer time period. Additionally, the countries taking part in TIMSS at Grade 4 and Grade 8 are not exactly the same.

4.1.2 The Nature of Change in Countries’ Science Curricula

This analysis enabled the nature of changes to the science curriculum of each country to be identified in addition to the number of changes made. Whilst countries could be grouped into categories based upon the extent to which their curricula had changed or remained stable, the profile of countries within each grouping could be quite different. The analysis identified some countries as “cutters” and some as “adders”. Cutters are countries where curricular changes resulted in fewer TIMSS science topics appearing in the intended science curricula, whereas adders are countries where curricular changes tended to result in additional TIMSS science topics appearing within the intended science curricula. A third group of countries was identified, which we decribe as “balancers”. These are countries where curricular changes resulted in roughly equal numbers of TIMSS topics being added and removed from the intended science curricula. We identified countries for each of these categories for Grade 4 (Table 4.3, comparison between 2003 and 2015 TIMSS data) and for Grade 8 (Table 4.4, comparison between 1999 and 2015 TIMSS data). Similar patterns emerged when comparing the data for 2003–2007 and 2007–2015 at Grade 4 and the data for 1999–2007 and 2007–2015 at Grade 8 so, in the interests of space, only the 2003 and 1999–2015 results are presented here.

Table 4.3 Country categorization for Grade 4 (2003–2015)

Table 4.4 Country categorization for Grade 8 (1999–2015)

Whilst the same categories exist for both Grade 4 and Grade 8, the proportion of countries in each category varies for the two grades (Tables 4.3 and 4.4). Additionally, countries can have different profiles for Grade 4 and Grade 8. For example, at Grade 4, Morocco is categorized as an adder, as between 2003 and 2015 it added four science topics to its curriculum and removed one. By contrast, at Grade 8, Morocco is classed as a balancer, as between 1999 and 2015 it added four science topics and removed four science topics from its curriculum.

4.1.3 Changes to Science Curricula Between TIMSS Cycles

The analysis enabled the average number of curricular changes between TIMSS cycles to be calculated (Tables 4.5 and 4.6). As the number of countries participating in each TIMSS cycle varies, the number of countries within each comparison differs (Tables 4.5 and 4.6).

Table 4.5 Average number of changes to science curriculum at Grade 4

Table 4.6 Average number of changes to science curriculum at Grade 8

For each comparison, at both grades, the average number of science topics added is greater than the average number of topics removed (Tables 4.5 and 4.6). This suggests that, across countries as a whole, changes in a country’s science curriculum are more likely to lead to an increase rather than a decrease in the number of TIMSS science topics included in the curriculum.

It is also more common for topics to be completely removed or added to the science curriculum than it is for there to be a change in emphasis to a science topic in the curriculum. Additionally, the results suggest that the science curriculum at Grade 4 is more subject to change than the science curriculum at Grade 8, as the average number of topics added or removed in each comparison is higher for Grade 4 than Grade 8. This finding is consistent with our earlier observation that, at Grade 8, a higher proportion of countries are categorized as having highly stable or moderately stable curricula than at Grade 4 (Tables 4.1 and 4.2).

4.1.4 TIMSS Science Topics Included in Countries’ Curricula

This analysis enabled us to identify the extent to which TIMSS science topics were included in the intended curricula of participating countries, and whether there were particular TIMSS science topics that were core to the majority of participating countries’ science curricula.

As the science topics included in each TIMSS cycle changed slightly each cycle, the science topics from the earlier TIMSS cycles were mapped to the 2015 TIMSS science topics to identify those that were common across all three cycles.

The number of TIMSS science topics included in the majority (80% or more) and minority (50% or fewer) of countries’ intended curricula for Grade 4 and Grade 8 change over time (Tables 4.7 and 4.8).

Table 4.7 Science topics taught in the majority and minority of participating countries at Grade 4

Table 4.8 Science topics taught in the majority and minority of participating countries at Grade 8

For Grade 4, between 2003 and 2015, the number of TIMSS science topics taught in at least 80% of participating countries increased, whilst the number of topics taught to fewer than half of participating countries decreased (Table 4.7). This suggests that there is a growing number of core science topics at Grade 4 that are included in the majority of participating countries’ intended science curricula. However, caution needs to be exercised in interpreting these results, as the number and range of countries (in terms of their geographic distribution and level of economic development) participating in TIMSS at Grade 4 has increased between 2003 and 2015 and the TIMSS science framework itself changes slightly between cycles.

At Grade 8 (Table 4.8), the number of TIMSS topics included in at least 80% of participating countries’ curricula was much higher than for Grade 4. There was only a modest increase of two topics between 1999 and 2015 in the number of TIMSS topics included in at least 80% of participating countries’ curricula. Additionally, the number of TIMSS science topics covered by fewer than 50% of participating countries at Grade 8 was less than at Grade 4, and remained at zero in all three TIMSS cycles. The same caveats apply for Grade 8 as Grade 4 when interpreting these results and making comparisons over time.

4.1.5 Core and Non-core TIMSS Science Topics Included in Science Curricula

In addition to identifying the number of TIMSS science topics that were included in a country’s science curriculum, the coding enabled the identification of core and minority (non-core) science topics. We identified the core TIMSS science topics as those that were included in the intended science curricula of 80% or more of participating countries at Grade 4 in 2003, 2007 and 2015, as well as the topics that appeared less frequently in the intended science curriculum of countries (Table 4.9). We undertook the same exercise at Grade 8 for the 1999, 2007 and 2015 TIMSS cycles; at Grade 8 all science topics appeared in the intended curricula of more than 50% of the participating countries (Table 4.10).

Table 4.9 Core and non-core TIMSS science topics at Grade 4

Table 4.10 Core and non-core TIMSS science topics at Grade 8

The growth in the number of Grade 4 TIMSS science topics taught in the majority of participating countries increased most in life sciences (from one to four). Earth sciences also saw a modest increase from zero to two topics in 2015. There was no change in the total number of physical science topics that were taught in over 80% of countries. Only one topic, “characteristics of living things and the major groups of living things”, was taught in over 80% of countries across all three TIMSS cycles considered. This topic is fundamental to biology and so its presence as a core topic across years is perhaps unsurprising.

There was only a small growth in the number of Grade 8 TIMSS science topics taught in the majority of participating countries’ curricula (Table 4.8). The variation in the change across the three TIMSS cycles for each of the four content domains is therefore not unexpected: biology and earth science saw both an increase and decrease in the number of science topics taught in more than 80% of participating countries, while chemistry and physics saw an increase and no change respectively in the number of topics taught in more than 80% of countries. No particular content domain is becoming increasingly more common across all countries.

Six science topics were taught in over 80% of participating countries across all three TIMSS cycles considered. “Cell structure and function” is one of these core topics and is again a topic that has been identified as one of the fundamental concepts in school science (Harlen 2010). It is thus unsurprising that it features as a core topic throughout.

4.1.6 Comparison of TIMSS Topics Taught Across Countries

We have identified science topics that are considered core and non-core in the science curriculum of countries participating in TIMSS (Sect. 4.1.5). However, as the number of countries that has participated in TIMSS has varied between cycles, comparisons are challenging. To overcome this, we focused on science topics included in the science curriculum of countries that have participated in all three TIMSS cycles under consideration for Grade 4 (Table 4.11) and for Grade 8 (Table 4.12).

Table 4.11 Number of countries teaching Grade 4 TIMSS science topics in all three cycles

Table 4.12 Number of countries teaching Grade 8 TIMSS science topics in all three cycles

At Grade 8 a far higher number of the TIMSS science topics were included in the curriculum of most participating countries than at Grade 4. At both grades there were a number of topics that were taught in the majority of cases across all TIMSS cycles, for example, “states of matter” at Grade 4 (Table 4.11) and “life cycles, sexual reproduction and heredity” at Grade 8 (Table 4.12). However, at Grade 4 in particular, there are some topics that are not commonly included in the curriculum in any of the three TIMSS cycles considered. One such topic is “understanding what fossils are and what they can tell us about past conditions on Earth”.

We found that restricting the analysis to countries participating in all three TIMSS cycles under consideration nonetheless revealed very similar patterns to those we earlier identified in the wider pool of countries that had participated in at least one of the TIMSS cycles under investigation (see Sect. 4.1.4).

4.2 Cluster Analysis and Discriminant Analysis

We applied cluster and discriminant analyses of the curriculum questionnaire data to determine potential convergence of curricula; countries may be clustered into groups on the basis of the topics included or not included in their intended science curricula at Grade 4 and Grade 8.

4.2.1 Grade 4

The cluster analysis performed on the 2003 TIMSS data for Grade 4 resulted in countries being aggregated in two groups. As mentioned previously, the optimal number of groups is assessed as that with the lowest Bayesian information criterion (BIC).¹ The cluster quality could be described as fair according to the silhouette measure of cohesion and separation.² Cluster analysis identified the first five most discriminating items for TIMSS 2003 at Grade 4 (Fig. 4.1).

Fig. 4.1

Most discriminating topics in 2003 cluster analysis: Grade 4

The most discriminating science topic was “Understanding that some characteristics are inherited and some are the result of the environment”, indicating that this specific science topic is the one that is the most important in defining whether a country may be classified as belonging to Group 1 or Group 2 (Fig. 4.2). For each of the five most discriminating science topics, we mapped the number of countries in Group 1 and Group 2 that were teaching the science topic to all or most students, only to the more able students, or not including the topic in the curriculum at that grade.

Fig. 4.2

Answers by group for the five most discriminating topics identified by cluster analysis of TIMSS 2003 data for Grade 4

While for the most discriminating item (Fig. 4.2a), there is an almost perfect separation between answers given by countries in Group 1 and countries in Group 2, the answers given by countries in the two groups tend to become more similar as the item becomes less discriminating (Fig. 4.2b–e).

Having identified two groups using cluster analysis, we conducted a discriminant analysis in order to estimate a classification model for countries in the successive cycles of TIMSS. As already mentioned, the topics that we found to be the most discriminating in the cluster analysis were assigned higher coefficients in the regression. These topics contribute most towards identifying the countries belonging to one of the two groups in the second and third TIMSS cycles considered (Tables 4.13 and 4.14).³

Table 4.13 Classification of countries at Grade 4 based on discriminant analysis using cluster groupings from TIMSS 2003 data

Table 4.14 Classification of countries appearing in consecutive TIMSS surveys at Grade 4

For each TIMSS cycle, we ascertained the number and percentage of countries in Group 1 and Group 2 (Table 4.13). The proportion of countries in Group 1 increased slightly over time at the expense of Group 2. However, the gain was modest (+14.64% points) and the possibility that this increase was driven by newly entered countries that did not participate in TIMSS 2003 cannot be ruled out.

Countries moved between groups in subsequent cycles of the survey (Table 4.14).⁴ For instance, we can see that out of the 11 countries that were classified as belonging to Group 1 in 2003, ten were still classified as belonging to the same group in 2007, while the remaining country was classified as belonging to Group 2. Hence, between 2003 and 2007 countries in Group 1 were mostly stable. Conversely, a non-negligible fraction of countries belonging to Group 2 (45.45%) changed their classification. Between 2007 and 2015 few countries moved from Group 1 to Group 2 (23.81%) while, once again, a considerable fraction of countries that were classified as belonging to Group 2 (55.56%) moved to Group 1. This provides some evidence in support of the hypothesis that science curricula are converging and becoming more similar over time at Grade 4.

The critical point is that, while in the first table the increase in the percentage of countries belonging to Group 1 might be driven by new countries that were not included in previous cycles (see Table 4.13), the same set of countries are in each of the tables (see Table 4.14). Hence, movements (and as a consequence variations in the percentage of countries in each group) occur only when a country that was originally classified in one group modifies its science curriculum to such an extent that it becomes more similar to countries in the other group.

To conclude the investigation of the results obtained for Grade 4, consideration needs to be given to how newly entered countries⁵ are classified in 2007 and 2015. The majority of countries that entered TIMSS in 2007 are classified as belonging to Group 1 and the same applies to countries that entered the study in 2015 (Table 4.15). The confluence of newly entered countries into one of the two groups does not indicate convergence by itself.⁶ Nevertheless, if a core science curriculum is emerging we would expect the majority of countries to adhere to it.

Table 4.15 Cluster analysis classification of newly entered countries at Grade 4

The increase in the proportion of countries belonging to Group 1 (Tables 4.13 and 4.15) suggests that this is partially because new countries tend to be classified more often than not in this group. However, even considering only countries that have been in two consecutive TIMSS cycles, Group 1 is growing at the expense of Group 2, suggesting a certain degree of convergence (Table 4.14). The finding that newly entered countries are more often classified as belonging to Group 1 is consistent with the emergence of a core science curriculum to which countries tend to adhere.

For the five most discriminating items (see Fig. 4.1), we further analyzed the answers given by the 18 countries that participated in the 2003, 2007 and 2015 TIMSS cycles (see Fig. 4.3a–e) to establish whether there were indications that science curricula at Grade 4 were converging. Only the 18 countries that participated in all the TIMSS cycles under consideration were included, to ensure statistical validity.

Fig. 4.3

Evolution of the distribution of the five most discriminating Grade 4 science topics (identified in 2003) between 2003 and 2015

Three science topics demonstrated a clear tendency to converge⁷ (Fig. 4.3a–c), while one remained stable (Fig. 4.3e) and the other underwent an inversion in tendency (the topic “forces that cause objects to move” was predominantly not taught in 2003, but the majority of countries had included it in the curriculum by 2015; see Fig. 4.3d). This is important because it confirms that the model we have presented identifies a general tendency to convergence in science curricula. The finding of overall convergence does not rule out the possibility that single topics are actually diverging. As long as the increase in similarities is enough to offset the emergence of dissimilarities, the model will identify overall convergence.

The model obtained by means of the discriminant analysis puts more emphasis on topics that were particularly important in defining the clusters in the first cycle and less on topics that were not discriminating at that point. We therefore repeated the same process in reverse. In practical terms, starting from the groups identified by means of a cluster analysis of the 2015 TIMSS cycle, we estimated a model that predicted countries’ membership based on the responses they gave to the 2015 curriculum questionnaire. This model was then used to group countries in earlier cycles. We used this approach to verify whether aspects that were similar in 2003 (i.e. science topics that were widely taught or not taught) have become increasingly different over time. Our previous model was not able to identify this phenomenon, as topics that were not discriminating in the first cycle give only a very minor contribution in predicting future membership. The model once again identified two clusters⁸ and the classification quality was fair according to the silhouette measure of cohesion and separation.

Among the five most discriminating items in 2015, there was only one topic (“understanding that some characteristics are inherited and some are the result of the environment”) that was shared with the 2003 analysis (Fig. 4.4). This supported the hypothesis of convergence. The topics that were very discriminating between Group 1 and Group 2 in 2003 had converged and dissimilarities with respect to other topics now defined the clustering. These dissimilarities might be pre-existing (meaning that the topics were already dissimilar in 2003, but they were overshadowed by other aspects that were even more relevant in defining the clustering of countries), or they might have emerged over time (topics that were widely taught, or not taught, by the majority of countries have been removed, or introduced, by some of them). The occurrence of this second event, which would indicate divergence along some dimensions, was why we repeated the analysis in reverse.

Fig. 4.4

Most discriminating items in 2015 cluster analysis: Grade 4

Applying this reverse approach, we found that the percentage of countries in Group 1⁹ had decreased between 2003 and 2007 and remained stable in the following cycle (Table 4.16). This contradicts the hypothesis of convergence of science curricula, as it suggests that the treatment of topics that distinguished countries in 2015 was more homogeneous in 2003. Nevertheless, the results are based on different sets of countries in each cycle and hence the evidence might be driven by new entrants. To address this issue, we replicated our investigation into the movement of countries between consecutive cycles (Table 4.17). Once again, the general tendency was for countries to move from Group 2 to Group 1. However, there was still a non-negligible fraction of countries that followed the reverse course, particularly between 2007 and 2015. Given that there appeared to be considerable movement across groups, the results provided only weak evidence in support of the hypothesis of convergence in science curricula (Table 4.17).

Table 4.16 Classification of countries at Grade 4 based on discriminant analysis using cluster groupings from TIMSS 2015 data

Table 4.17 Classification of countries appearing in consecutive TIMSS surveys at Grade 4

We also studied the classification of newly entered countries in 2007 and 2015 (Table 4.18).¹⁰ The majority of countries that entered the survey in 2015 were classified as belonging to Group 1, while new entrants in 2007 were distributed almost equally between Group 1 and Group 2. The results are not conclusive for the reasons mentioned before, and there are no strong indications that a core science curriculum is emerging.

Table 4.18 Cluster analysis classification of newly entered countries at Grade 4

We interpreted these results (Tables 4.16, 4.17 and 4.18) as suggesting that topics that distinguished countries as belonging to Group 1 and Group 2 in 2015 were not more similar in 2003 (and thus there was no divergence).

We also examined how the distribution of the most discriminating items had evolved over time (Fig. 4.5). As in the previous case (Fig. 4.3), the behavior of topics was heterogeneous. Some were converging (e.g. see Fig. 4.5a, c), while for the others it was hard to identify a clear pattern.

Fig. 4.5

Evolution of the distribution of the five most discriminating science topics (identified in 2015) between 2003 and 2015

Taking into account the outcomes presented so far, we conclude that there is only weak evidence to support the hypothesis that science curricula at Grade 4 have become increasingly similar over time.

4.2.2 Grade 8

We repeated the analysis we have presented for Grade 4 in order to assess convergence of science curricula at Grade 8.

As for Grade 4, the cluster analysis performed on the 1999 TIMSS data for Grade 8 resulted in countries being classified in two groups.¹¹ The cluster quality is fair according to the silhouette measure of cohesion and separation. Cluster analysis predicted the five most discriminating items for TIMSS 1999 (Fig. 4.6).

Fig. 4.6

Most discriminating science topics in TIMSS 1999 cluster analysis: Grade 8

The most discriminating topic is “chemical change (transformation of reactants, evidence of chemical change, conservation of matter, common oxidation reactions–combustion, rusting, tarnishing)”.

For each of the five most discriminating items (Fig. 4.7), we mapped the number of countries in Group 1 and Group 2 that were teaching the topic to all or most students, only the most able students, or not including the topic in the curriculum at that grade.

Fig. 4.7

Answers by group for the five most discriminating topics at Grade 8 identified by TIMSS 1999 cluster analysis

The more discriminating the item (from Fig. 4.7a–e), the sharper the distinction between Group 1 and Group 2.

We used the model that results from applying discriminant analysis on the clusters identified in 1999 to predict the probability of belonging to Group 1 and Group 2 in the later TIMSS cycles (2007 and 2015).

For each cycle, we ascertained the number and percentage of countries in Group 1 and Group 2 (Table 4.19), revealing a strong tendency for countries to concentrate in a single group over time. The two groups were of similar size in 1999, but the overwhelming majority of countries were classified in Group 2 by 2015. As highlighted previously, even given the apparent strength of this result, it has to be interpreted with some caution because the increase of one group at the expense of the other could be driven by new countries entering TIMSS in 2007 and 2015.

Table 4.19 Classification of countries at Grade 8 based on discriminant analysis using cluster groupings from TIMSS 1999 data

Countries moved between groups in subsequent cycles of the survey (Table 4.20). Between 1999 and 2007, the majority of countries that were classified as belonging to Group 1 moved to Group 2 (75%) while no country followed the reverse path (from Group 2 to Group 1). Similarly, between 2007 and 2015, all the countries remaining in Group 1 moved to Group 2, while only a small number of countries moved from Group 2 to Group 1 (9.68% of countries classified as belonging to Group 2 in 2007).

Table 4.20 Classification of countries appearing in consecutive TIMSS surveys at Grade 8

In both 2007 and 2015, the great majority of new entrant countries (all new entrants in 2015) were classified as belonging to Group 2 (Table 4.21). The results (Tables 4.19 and 4.20) provide strong evidence in support of the hypothesis of convergence in science curricula at Grade 8.

Table 4.21 Cluster analysis classification of newly entered countries at Grade 8

We repeated the exercise we performed for Grade 4 for the five most discriminating items identified by the 1999 cluster analysis, further analyzing the answers given by the 21 countries that participated in 1999, 2007 and 2015 (Fig. 4.8) to clarify why countries have been increasingly classified as belonging to Group 2 over time. Only data for the 21 countries that participated in all three TIMSS cycles under consideration were included, to ensure statistical validity.

Fig. 4.8

Evolution of the distribution of the five most discriminating topics at Grade 8 (identified in 1999) between 1999 and 2015

In contrast to Grade 4, among the five most discriminating topics, all items but one (“properties and uses of common acids and bases”) demonstrated a clear tendency to converge.

Repeating the cluster and discriminant analyses in reverse starting from the 2015 data to verify the findings once again identified two groups of countries,¹² with “fair” cluster quality according to the silhouette measure of cohesion and separation.

Fig. 4.9

Most discriminating topics in 2015 cluster analysis: Grade 8

Among the five most discriminating topics (Fig 4.9), there was only one item (“Earth’s processes, cycles, and history”) that was common to the 1999 and 2015 analyses. This confirms the convergence we already identified; the topics that were very discriminating in 1999 converged and other topics were driving the clustering.

Applying this reverse approach, we identified the number and proportion of countries classified in Group 1 and Group 2 over the three cycles of TIMSS at Grade 8 (Table 4.22). The percentage of countries in both groups was stable over time.

Table 4.22 Classification of countries at Grade 8 based on discriminant analysis using cluster groupings from TIMSS 2015 data

We also studied the movement of countries that participated in consecutive TIMSS cycles in order to rule out the possibility that the results (Table 4.22) were driven by countries that entered later cycles (Table 4.23). While there was some evidence that countries classified as belonging to Group 1 had moved to Group 2 in the ensuing cycle, movements in the opposite direction (from Group 2 to Group 1) were rare, evidencing a tendency for countries to concentrate towards Group 2. As a result, considering only the countries that have been in subsequent TIMSS cycles, not only did we find no sign of divergence, if anything, we found the opposite. Topics that were highly discriminating in the 2015 TIMSS cycle were treated in an even more unequal way in previous years. The fact that these items did not emerge as highly discriminating in the 1999 cluster analysis is due to the fact that other topics were even more important in defining group belonging, hence their effect was overshadowed.

Table 4.23 Classification of countries appearing in consecutive TIMSS surveys

Although confirming that the majority of countries entering TIMSS in 2007 and 2015 may be classified as belonging to Group 2 (Table 4.24) is not by itself conclusive, this does support the hypothesis that an internationally understood core science curriculum is developing over time.

Table 4.24 Cluster analysis classification of newly entered countries

Altogether, the results suggest that topics that differentiate countries belonging to Group 1 and Group 2 in 2015 were treated in an even more unequal way in previous cycles, indicating convergence (Tables 4.22, 4.23 and 4.24).

We also examined how the distribution of the most discriminating items (identified in 2015) evolved over time (Fig. 4.10). One topic (“Earth’s processes, cycles, and history”) displayed clear signs of convergence, while the remainder showed only minor or no evidence of convergence. This is mainly because there was already little variability in the 1999 TIMSS cycle, as science topics were treated in a similar way by all countries (the tendency was to teach these topics to all students in 1999). As a result, there is little capacity for further convergence. That said, we were mainly interested in confirming that no divergence occurred in topics that were not highly discriminating in the first cycle.

Fig. 4.10

Evolution of the distribution of the five most discriminating topics (identified in 2015) between 1999 and 2015

Taking into account the outcomes presented, we found strong evidence supporting the hypothesis of convergence in science curricula at Grade 8 over the last 20 years.

4.3 Further Analysis Using TIMSS Teacher Questionnaire and Encyclopedia Data

The results outlined in Sects. 4.1 and 4.2 are based on data from the TIMSS curriculum questionnaires. We also used the TIMSS encyclopedias, data from the TIMSS teacher questionnaire and additional data from the TIMSS curriculum questionnaire to build up a more detailed picture of other aspects of science curricula for a select number of countries (listed in Table 3.3). The objective of this analysis was to obtain information on aspects of countries’ implemented science curricula to complement the data gathered on the intended curriculum (from the coding exercise and cluster and discriminant analyses). The aim was to enable changes in some aspects of the implemented science curricula to be tracked over time.

Curriculum features investigated included the instructional time spent teaching science each year and the percentage of students who have been taught the TIMSS science topics. Although we intended to explore a number of other curriculum features, the data sources did not support further additional analyses. Features that interested us included the emphasis teachers placed on science investigation in their teaching, the emphasis on different cognitive processes in the science curriculum, and the percentage of instructional time devoted to teaching science.

We encountered a number of difficulties in collecting information on the implemented science curricula. Firstly, for the 1999 (Grade 8) and 2003 (Grade 4) TIMSS cycles, there were no published TIMSS encyclopedias. The encyclopedias were only produced from the 2007 cycle onwards. As a result, much of the information on participating countries’ science curricula that is available for later cycles was not available for the first TIMSS surveys considered in this report, thereby limiting the comparisons that can be made.

A second challenge was that, in many cases, the relevant items in the teacher and curriculum questionnaires changed between TIMSS cycles. This makes it impossible to directly compare responses between cycles. For example, in each TIMSS cycle there is a question that asks about teacher emphasis on science investigation; however, the wording of the question varies. In 2015, the question asked teachers to rate how often they emphasized science investigation in their lessons. They could select their response from “about half the lessons or more” or “in less than half the lessons”. By contrast, in the 2007 teacher questionnaire, respondents were asked, for seven different science investigation activities, to rate whether the students they taught had the opportunity to do the activity in about half the lessons or more. These activities included practices such as watching the teacher demonstrate an investigation or relating what they learned about science to their daily lives. As the questions are framed in different ways, it is difficult to make direct comparisons. We encountered this issue for a large number of the curriculum features that we were originally intending to compare.

We also found that there was a considerable amount of missing data. We had selected countries for inclusion in this sample on the basis that they had taken part in at least two TIMSS cycles but, in a number of cases, countries had not provided this additional information on the implemented curriculum. For example, in 2007, there was no information provided by New Zealand on the percentage of students taught the TIMSS science topics for Grade 4 or Grade 8. Again, this made comparisons challenging, both for a single country over time and between countries in the same cycle. As a consequence, and because of the significant caveats that would have applied to any conclusions drawn from these analyses, we decided not to pursue further in-depth analyses based on the teacher and curriculum questionnaires.