Talking About Education: How Topics Vary Between International Organizations

Abstract

In this chapter, Raphael Heiberger and Michael Windzio examine which topics are important for major education international organization (IOs). IOs in the field of education follow different ideological paradigms in the global education discourse. Yet, it is an open question as to whether different types of IOs focus on different topics and thereby support different paradigms of education. Based on more than 1000 documents with over 40 million words published by the World Bank, UNESCO, the ILO, the OECD, ISESCO, and SEAMEO, they explore education issues addressed in this sample. Using standardized methods of quantitative text analysis and topic modeling, Heiberger and Windzio reveal that major topics found in these documents do indeed differ between the different types of organizations.

Introduction

This chapter is a product of the research conducted in the Collaborative Research Center “Global Dynamics of Social Policy” at the University of Bremen. The center is funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation)—project number 374666841—SFB 1342.

Policymakers usually regard education as a domestic policy field, where responsibility rests either with the national government or, as in most federal states, is delegated to subnational units. That education policy is heavily influenced by transnational entities, and by international organizations (IOs) in particular, is a fairly new insight within the realm of academic research and also a new experience for some policymakers (Martens et al. 2010, 2014). Tracing the influences of these IOs in detail is challenging for several reasons, one being the great variety of IOs—currently around 30—that are active in the policy field of education (Niemann and Martens 2021). While many industrialized countries have responded to the results of the Organisation for Economic Co-operation and Development’s (OECD) Programme for International Student Assessment (PISA) study over the years, it remains an open question whether and how other organizations influence domestic education policies, and, if so, which countries are affected.

This study takes one step back from such a causal analysis and instead aims at analyzing the programs and recommendations communicated by IOs in their major documents in a descriptive manner. If we assume that IOs are more or less powerful actors in the global world system and that they try to influence domestic education policies (Martens et al. 2007), then we should first attempt to understand their missions and preferred goals in the field of education. In line with the studies by Niemann (Chap. 5) and Krogmann (Chaps. 7 and 8), we analyze the goals and ideas of IOs where education is concerned but apply a different methodological approach. We created a comprehensive text corpus (over 40 million words) from documents published by the six IOs that were selected to be analyzed in this book (Chap. 1) and applied structural topic models (STMs) to analyze the basic topics these organizations are dealing with. We selected the IOs according to a two-dimensional classification that distinguishes between derivativeanddedicated education IOs, on the one hand, and IOs with openorrestrictedmembership criteria, on the other hand.

According to our results, the six IOs do indeed focus on different issues. As we will see, this is basically due to the outstanding degree of homogeneity of topics communicated in the documents of those IOs that are dedicated to the issue of education, but at the same time restricted in their membership. These IOs are the Islamic Word Educational, Scientific and Cultural Organization (ICESCO) and the Southeast Asia Ministers of Education Organization (SEAMEO).

Theoretical Background

How can we explain why IOs communicate about education in different ways? This is an important question given that we know IOs can considerably influence domestic education policies. When the OECD published the results of its first PISA study in 2000, policymakers and researchers controversially discussed the comparisons of educational performance and their policy implications. Many countries took the opportunity to implement reforms after being disappointed with their poor performance. The OECD thus had a strong influence on their domestic education policies, even though the organization itself does not have any formal authority. Switzerland, for example, was receptive to the study because education reforms had been considered overdue for decades; however, they were difficult to implement in a federal system with many veto players (Bieber and Martens 2011). In Germany, education policy changed so dramatically after the PISA study that scholars even used metaphors such as “Turn of the Tide” (Niemann 2010) or “After the Big Bang” (Niemann 2014).

Not all countries in the world are members of the OECD, but its concepts and suggestions for economic development are rather general and might also be applicable for non-members. Similarly, the United Nations Educational, Scientific and Cultural Organization (UNESCO) is well known for its focus on the role of education for economic and social development in the Global South but is not at all limited to these issues. Other organizations focus on particular world regions or on countries where a particular culture or religion prevails.

Communication theories often regard communication as a system embedded in surrounding social systems (Luhmann 2000). Mass communication operates in specific ecologies where different media compete for niches to survive, maintain their communication, and get attention. Niche theory argues that the overall semantic space of communication at a given moment results from variation and selection within this competitive discourse environment. If a given ecology is unfavorable toward a particular communication, actors often seek to establish their own niche or try to fill existing, but alternative, niches (Riedlinger and Rea 2015). For instance, a new niche was established when communication on the Internet started, and what we observe today is an increasing differentiation of niches that becomes apparent in various different online platforms that engage in tight competition for attention. Despite this fine-grained differentiation, the proliferation of certain niches such as YouTube is a challenge for traditional media. Not only the form but also the content of a newly emerged discourse niche might even displace previously dominant niches in the future (Ha and Fang 2012).

According to this nichecompetition argument, instead of there being just one discourse on education led by one dominant and universalistic IO, there might be different discourses in distinct niches. Such niche differentiation might be brought about by perception that the dominant and universalistic IO does not put sufficient weight on a particular issue, or the perception that the discourse of the dominant IO is somehow biased by a hegemonic ideology. This argument might be particularly applicable to the field of education because education is closely related to the intergenerational reproduction of culture. In the long run, the persistence of different “cultural spheres” (Chap. 1) also depends on how countries organize the form and content of their education systems.

We thus expect there to be considerable differences in how these IOs “talk” about education. To date, however, there is no standardized analysis of the most prevalent topics being discussed by these IOs. Education meets various functional requirements in a society such as generating literacy in the population, supporting ideas of legitimacy appropriate to the respective institutional order, laying the foundation of human capital investments, integrating migrants and their descendants into society, and preventing juveniles from becoming delinquent (Windzio 2013). It is likely that these and other functions of the education system are weighted differently in the discourses of these IOs. According to the “cultural spheres” argument (Chap. 1), which highlights the importance of education for the intergenerational reproduction of particular cultures, we expect that the most prevalent topics differ between the IOs depending on whether they are derivative or dedicated, open or restrictive. We thus classify active IOs in the field of education along the two dimensions derivative–dedicated and open–restricted (Chap. 1). An IO is derivative with respect to education if it mainly deals with more general issues, such as economic development, while covering education as just one issue among many. A dedicatedIO is one whose major topic is in fact education. Furthermore, the nature of IOs’ membership rules can be open to any state or be restricted according to specific criteria such as geographical region or religion. Table 9.1 shows the categorization of the IOs and the number of documents considered in the analysis.

Table 9.1 International organizations (IOs) and number of documents by category

The International Labour Organization (ILO), the World Bank, and the OECD are organizations that are not primarily focused on education, whereby the latter IO is also restricted in its membership rules and essentially only includes economically developed countries. In contrast, UNESCO, ICESCO, and SEAMEO all have education as their main focus, but only UNESCO membership is open to any state. ICESCO only includes countries with a high share of Muslims and thus restricts membership to a particular religion, while SEAMEO membership consists of 11 Southeast Asian countries and is thus restricted to a particular world region. Given that there already existed an IO that supposedly handles the educational concerns of all countries in the world (UNESCO), there must be a reason why these member-restrictive organizations were formed and continue to be active today. We assume that this is because these IOs focus their communication specifically on their region, as in the case of SEAMEO, or on their particular culture and religion, as is the case with ICESCO.

In the following sections, we apply structural topic models to a large text corpus of IO documents in order to trace the major issues these organizations are talking about.

Data

Preprocessing

As a first step in the creation of the text corpus, we defined a search string in order to restrict the selected documents to the issue of education by using the keyword “education.” After the exclusion of clearly inappropriate documents, these keywords yielded a set of N = 1603 documents (see Table 9.1). The clearing of the text corpus included the removal of numbers, punctuation, and special characters. Subsequently, we identified English stop words and extended the set of stop words by 21 corpus-specific words (e.g., “chapter,” “table,” “director”). Words with less than three letters were deleted.

We then lemmatized the corpus by using the function lemmatizestring from the R package textstem. Finally, we removed tokens that appeared in less than three documents.

Structural Topic Models

To reduce the semanticspace of the documents published by the six IOs to meaningful dimensions, we applied a recent variety of topic models known as Structural Topic Modelling (Roberts et al. 2014, 2016). Topics are directly derived from “bag-of-words” (meaning that word order and grammar are ignored) by probabilistic algorithms. The main idea behind all topic models is that words co-occurring in and across documents describe meaningful themes (or topics). All topic models are generative models of word counts using a two-step method of clustering. A topic is defined as a mixture of words, where each word has a probability of belonging to a topic (1st cluster step), and a document is a mixture of topics, meaning that each document is composed of multiple topics (2nd clustering). As such, the sum of the topic proportions across all topics for a document is one (also known as θ, the topic load of a document). In addition, the sum of the topic probabilities for a word across all topics is also one (also known as β, a word’s contribution to a topic). Thus, the more often words co-occur in documents, the higher the probability that they constitute a topic.

Compared to other topic models (Jordan and Mitchell 2015), STM allows for the improvement of the estimation of topics by using document meta-data as covariates. STM does not assume that the distribution of words is the same for all documents, but words in documents with the same covariates (e.g., year, source) have a higher likelihood of being clustered together and forming a topic. It has been shown that the inclusion of covariates improves the quality of topic selection substantially (Roberts et al. 2014, 2016), especially for documents covering longer time periods (Farrell 2016).

Validation

Although the proposed STM solves other technical issues, such as finding the optimal starting parameters and providing consistent results by a “spectral initialization” (Roberts et al. 2016), it does not solve the issue of selecting an appropriate number of topics, which is crucial for any further analysis. Comparable to efforts in cluster analysis to determine the optimal number of clusters, there is no “right” answer to the question of how many topics are appropriate for a given corpus (Grimmer and Stewart 2013); though there do exist better and worse choices. Given the importance of the question, a careful examination and consideration of different topic solutions is key to every scientific dimensional reduction technique.

Internal Validation

The idea of internal validation is to identify a model with topics that best reflect weighted bags of words that are used by interviewees. In other words, we test which topics (respectively the most defining words of a topic) best predict interviewees’ responses (aka the text). For that general purpose, semantic coherence and exclusivity are widely used measures (Mimno et al. 2011; Roberts et al. 2016).

The coherence of a semantic space addresses whether a topic is internally consistent by calculating the frequency with which high-probability topic words tend to co-occur in documents. Coherence grows with the likelihood of a topic’s most probable words co-occurring together. The authors who introduced this measure, Mimno and Blei (2011), validated it for academic writing. They showed that it had high correspondence with the judgments of National Institutes of Health (NIH) officials on a set of NIH grants (Mimno et al. 2011).

However, semantic coherence alone can be misleading since high values can simply be a result of very common words of a topic that occur together in most documents. To account for the desired statistical discrimination between topics, we may consider a second metric proposed by Roberts et al. (2014) and measure theexclusivity of a topic. Exclusivity provides us with the extent to which the words of a topic are distinct from it. Both exclusivity and coherence complement each other and are examined in concert to give us a first impression of where topics represent word distributions in documents and at the same time provide distinct dimensions.

The developers of STM recommend that researchers look for the “semantic coherence-exclusivity frontier”—namely, the specification after which allowing for more topics fails to produce models that dominate others in terms of semantic coherence and exclusivity (Roberts et al. 2014, 7). Thus, what we are looking for is when both indicators build a plateau, that is, they do not improve (much) with higher Ks. This gives us an upper limit for a reasonable K.

Figure 9.1 reveals a “jump” in coherence at K = 70. Before and after that jump it decreases with higher values of K. Looking at exclusivity, we see that it increases only marginally after values of K are greater than 50.

Fig. 9.1

Exclusivity and semantic coherence

Figure 9.1 displays the distribution of exclusivity and semantic coherence. In general, we see that exclusivity gets better with a rising number of topics, but coherence gets worse.

In addition, we look at the held-out likelihood. This is “a measure of predictive power to evaluate comparative performance,” in this case among models that allow for different numbers of topics (Roberts et al. 2016, 22). To obtain the held-out likelihood of an STM, we first subset 10% of the documents in the corpus and hold out half of the words in them. We then evaluate the likelihood of the held-out words. Higher likelihoods indicate a more predictive model. In general, the log-likelihood of K is falling (more general topics, more predictive power). Thus, we are looking for some “breaks” in a straight line. Figure 9.2 shows a corresponding “plateau” to 1 around K = 70. That means the LL is not improving much between steps 60 and 70, where we would expect to see a linear falling curve.

Fig. 9.2

Held-out log-likelihood

Robustness

In addition to internal validation, we may want to include a second pillar by checking robustness across K. It is probably the most important way to show that our choice of K is not changing the whole semantic space and hence our (descriptive) analysis.

For this purpose, we use an indicator developed for comparing clustering solutions called “Fowlkes-Mallows Index” (FMI). It basically tests the relation of true and false positives/negatives across cluster assignments, or in our case, the assignment of a document to a topic. We used the max-approach to assign topics to documents, that is, max(theta) (maximum topic load) of a document defines its topic. The FMI may give us a straightforward way of measuring consistency by investigating the rate of change with regard to topic assignments across different Ks.

K on the x-axis represents the similarity of topic assignments for all documents between two consecutive Ks, that is, an STM with k-topics is compared to the next smallest STM with k = 10 (e.g. k = 10 and k = 20 are compared at the first x-tick).

Figure 9.3 shows the results for the FMI. We see consistent values above 0.7 for K = 70. This means that more than 70% of documents’ maximum topic assignments at K = 60 are the same at the next highest K. Thus, K = 70 again marks a local peak, this time in terms of “robust” topic assignments. In general, we note that the finer the semantic grid with rising K, the lesser the changes necessary to classify documents sufficiently.

Fig. 9.3

Fowlkes-Mallows Index

Therefore, all validation results point to K = 70 as the most valid and consistent choice for the number of topics. However, we see in Fig. 9.3 that the choice of K is not changing the semantic dimensions (i.e., topics) in a drastic way.

Results

According to the diagnostics presented in the previous section, we opted for a solution with 70 topics. Each of these topics has a specific prevalence, that is, a specific share in the overall set of these topics. Taken together, all 70 topics make up 100% of the semantic space. Figure 9.4 shows the ten most important topics according to their topic prevalence and the five most important words that best represent the respective topic. For an understanding of what these topics mean, it is necessary to inspect the topic-word list in the Appendix. Topic 57 refers to the level of education of students and the OECD, whereas topic 64 refers to national policies related to countries’ development. Topic 49 appears similar to topic 64 but focuses more on communities. Topic 13 focuses on the OECD, students, and policies in conjunction with schools. In contrast, topic 18 is more focused on teachers and their profession, while topic 53 is clearly centered on vocational training and the OECD. Topic 60 is also focused on teachers, but it additionally relates to recommendations and committees. Topic 26 is clearly related to labor training and the ILO, while topic 39 is centered on tertiary educational institutions (particularly universities) and topic 40 is again more focused on skills, training, and development. At first glance, topics 57 and 13 seem to be similar, but closer inspection reveals that topic 13 is limited to secondary education, whereas topic 57 also deals with tertiary education. As this example shows, it can sometimes be quite difficult to clearly discern the content of all 70 topics, but aside from such instances, the top ten topics of our analysis are actually quite easy to distinguish between since they are dealing with distinct topics such as performance levels of secondary education, teacher-related issues, tertiary education, vocational training, or labor-related issues.

Fig. 9.4

Topic proportions of the ten most important topics, solution with 70 topics

More important for our research question is the relative importance of the 70 topics for the different IOs. Figure 9.5 shows pie plots with the three most important topics for each of our six IOs. The higher the proportion covered by the three topics, the more focused the documents of the respective IO are on fewer issues. Contrariwise, if an organization discusses a variety of different issues in equal proportion, then the topic heterogeneity and overall share of the three most important topics are relatively low. It is thus striking in Fig. 9.5 that for ICESCO and SEAMEO, at least three quarters of their communication are concentrated among the three most important topics. As argued in the theoretical section of this chapter, these dedicated organizations are focused on education. Even though UNESCO is a dedicated IO whose membership is open to all countries in the world, ICESCO and SEAMEO create and maintain their own distinct thematic niches, which implies that, in their view, UNESCO does not appropriately address these niches (see Appendix for an overview of all 70 topics).

Fig. 9.5

Three most prevalent topics by organization

A closer inspection of the content of these topics reveals that this does in fact seem to be the case. ICESCO devotes 54.7% of its communication to topic 63, which is about “Islam, culture, and the state.” The remaining two most important topics make up only a small proportion of around 4–5%. Accordingly, the main focus of ICESCO is clearly on Islam, the social world, the state, and rights. The concentration of topics is even more pronounced forSEAMEO. Here, 78.6% of its communication is concentrated on topic 7, which is about “development, SE Asia” as well as on the countries Thailand and Malaysia. The remaining two most important topics make up around 3% each and focus on “early childhood and care” (topic 62), and “teacher and digital technology” (topic 33).

In contrast to ICESCO and SEAMEO, the three most important topics of theOECD make up only one-third of its overall communication combined. At the same time, the distribution of the topic proportions is more equal: The most important issue has a share of 12.1% and is focused on “earnings, tertiary and non-tertiary” (topic 57). In the overall text corpus, this is the most prevalent topic (Fig. 9.4). In addition, topic 53, which makes up 7.2% of the OECD’s communication, ranks sixth place among the most important topics in the overall corpus and is clearly focused on “vocational training and apprenticeship,” whereas topic 13 (“assessment, average, and Latvia”) makes up 8.3% and relates to schools, secondary education, and national averages. Even though the OECD is an organization with restricted membership, it is nevertheless rather broad and heterogeneous in its communication about education.

Much like the OECD, the three most important topics for UNESCOalso make up around one-third of its communication. Topic 64 is its most important topic, accounting for 13.1%, and rather broadly addresses issues on “support national development” and training. In total, 10.3% of UNESCO’s communication is again devoted to issues relevant to development, such as “problems of adult learning.” Interestingly, UNESCO also communicates considerably (7.1%) on topic 63 (“Islam, culture, and the state”). As we have seen already, this is by far the most important topic for ICESCO (54.7%), and despite being remarkably less relevant for UNESCO, it nevertheless accounts for one of its three most important topics. This probably has to do with UNESCO’s activities in fields such as the preservation of cultural heritage and culture in general.

The World Bank shows the highest heterogeneity in its topics compared to the other IOs, since only around 15% is concentrated on the three most important topics, which are topic 49 (“development projects and community”), topic 48 (“South Africa, development, and telecommunication”), and topic 40 (“vocational training and East Asia”). The ILO shows a concentration of topics: “labor, train, development, and worker” (topic 26, 15.1%), “selection of teachers” (topic 60, 14.2%), and “war, employment, and wages” (topic 55, 10.6%).

To sum up, what we can learn from this standardized analysis of the comprehensive text corpus is that our six IOs do indeed talk differently about education. These differences are best described by the combination of two characteristics, in the lower right cell of our 2 × 2 table (Table 9.1). If IOs dedicated to the issue of education are at the same time restricted in their membership (e.g., through their focus on either a particular cultural sphere or world region), then they will tend to focus their communication on specific issues much more than either derivative IOs or UNESCO, which is a dedicated IO but not restricted in membership. Interestingly, the case of the OECD underscores that restrictions on membership do not necessarily imply that the communication is limited to just a few topics (Niemann and Martens 2021). Despite its membership restrictions, the OECD seems to communicate on a huge variety of issues rather than focus on just a few specific interests.

Conclusion

We began our study by describing the classification of our six IOs into a 2 × 2 table, defined by the dimensions derivative–dedicated and open–restricted (Chap. 1). If comprehensive, universalistic IOs such as the World Bank or UNESCO claim to represent the global world system, the mere existence of dedicated IOs with restricted membership is a puzzling occurrence in and of itself. Why do regional or culturally specific IOs exist? Universalistic IOs such as UNESCO and the World Bank do not restrict membership. Nevertheless, it seems that some groups of countries do not feel represented by them. UNESCO is not regarded as an appropriate representation of certain countries when it comes to the cultural aspects of education, while the World Bank’s inaccessibility stems from its emphasis on the role education plays for the economy.

Our analysis of a huge text corpus of IO documents showed that the major topics communicated by the respective organizations clearly differ. The most important distinction is between ICESCO and SEAMEO, on the one hand, that is, the two IOs that fall in both the restricted and dedicated categories, and UNESCO, the OECD, and the World Bank, on the other hand. The former IOs show a rather homogeneous, highly focused content when it comes to their communication; there is basically one “big issue” they are dealing with. ICESCO’s “big issue” is clearly the topic “Islam, culture, and the state,” regardless of the fact that this topic is also somewhat important to UNESCO. Nevertheless, UNESCO deals with a variety of different issues and pays only minor attention to this particular topic. Perhaps this is the reason why ICESCO exists: religion and culture might be so important in the cultural sphere of Muslim countries (Chap. 2) that they consider it necessary to have a specific IO for this purpose. As Krogmann (Chap. 7) has shown, preserving the Islamic religion and culture while at the same time strengthening the economic and political power of Muslim countries is the major goal of ICESCO. Unsurprisingly, as a universalistic IO, it is not part of UNESCO’s major goals to “allow the Islamic world to regain its leading role in building human civilization” (ISESCO 2017, 7), even though “Islam, culture, and the state” do play a role for UNESCO as well. Our results are thus in line with the idea that cultural spheres are important in global education policies. However, it remains an open question how important and effective ICESCO actually is in shaping education policies as well as systems in its member countries.

The goals of SEAMEO are less clearly defined. It is focused on its member countries and thereby on the particular region of Southeast Asia. Education policy is considered important for the development of this region, but it is again an open question why countries in this region do not consider themselves to be well represented by the World Bank or UNESCO. In contrast to these two IOs, the other four show much more diversity in their major topics. Even though the ILO is focused on labor issues, its communication seems to be spread more evenly over different topics, and these topics address all countries in the world.

Education is a special policy field since it is related to the reproduction of national or regional cultures. At least with respect to ICESCO, it is quite reasonable to assume that the reason this IO exists is because of the importance of Islamic civilization in the eyes of its member countries. An interesting question is whether we would detect regional or culturally specific IOs in other policy areas as well. If so, we could ask similar questions: What motivates countries to create and maintain activities in dedicated IOs and to maintain and occupy specific thematic niches? Why do these countries not consider themselves well represented by the major universalistic IOs? Is education the only policy area in which, for example, culture is so important that specialized IOs become active? This might be an interesting research agenda for studies on IOs in other areas of social policy.

Appendix

No.	Topic	Coherence	Exclusivity
1	Assessment	Program tool policy question use assessment goal analysis identify module	Module yes please tool feed questionnaire saber analyze dimension interview
2	Privatization	Public private government sector fund pay finance country cost service	Private public voucher subsidy fee loan for-profit revenue charge pay
3	Latin America	Chile Mexico Colombia educacin state del Para los nacional quality	Salud anuales alianza financiamiento calidad gasto seguro poblacin estatal evaluacin
4	Neo-liberal growth	Growth world country service bank trade economy percent export access	Freshwater liberalization avg. debt barter export pop high technology dioxide wine
5	Costs in Africa	Education percent expenditure level high school primary cost sector secondary	Pbet drc francophone ababa Addis tte postbasic Madagascar recurrent Cameroon
6	Inclusion	Education student school child support immigrant need disability language special	Inclusive resilience disability impairment refugee mainstream adversity disable inclusion multicultural
7	Development, SE Asia	Education seameo Thailand country school Centre programme Malaysia development learn	Seameo tropmed seamolec oecdunesco recsam southeast searca thailands Brunei biotrop
8	ILO convention French	Des les dans que par qui convention travail pour Sur	Avait larticle avons ouvriers repos fabriques serait dautre dernire avaient
9	Region: Former SU	Education school student teacher review OECD republic high secondary national	Kazakhstan moes Lithuania Estonian Lithuanian canary Estonia gymnasium mone unt
10	Women in Arabic countries	Woman gender female labor country rate region MENA man girl	MENA Yemen Egypt Morocco womens Algeria Lebanon Saudi Oman Arabia
11	Latin America, non-tertiary	School education primary secondary percent teacher country student rate level	Wei Honduras Salvador Nicaragua Guatemala Ethiopia quintile enrollment Sierra Leone
12	PISA assessment China	Student performance Pisa country read OECD level school score science	Tableau kongchina gradient donnes shanghaichina bold enjoyment annualise slope macaochina
13	Assessment, average and Latvia	Education OECD student policy school secondary high system average national	Latvia eag outlook latvian httpdxdoiorgen spotlight talis latvias yearolds Riga
14	Methods of data collection	Datum indicator statistic use survey information analysis source system statistical	Datum statistic indicator statistical handbook classification UIS dataset collection internationally
15	Assessments dimensions	Problem student solve Pisa item read assessment text question literacy	Noncontinuous problem funke mathematical solver knowledge reading science problemsolving framework
16	Occupational training, informal work, global south	Train skill sector education percent informal labor worker market development	Oecs nurse caricom Yunnan informal kur trainee Lanka sri ghana
17	Quality of lifelong learning	Education learn quality qualification system country policy assurance national lifelong	Crossborder ria assurance lifelong qualification IBRD the recognition GAT NQFS NQF
18	Teachers’ careers	Teacher teach school education professional policy student development system work	Teacher profession induction teach professional career reward bonus high performing retain
19	Training and European labor market	Skill OECD train adult learn labor need education market work	Diagnostic adult learning Slovenias activate Flanders PES low skilled Italys SMEs activation
20	Maths-science performance	Student school Pisa country OECD performance mathematics science index report	Self-beliefs country economy ESCS IIIA BSJG Macao self-reports truancy IIIB nonimmigrant
21	Development and World Bank	Bank world country development research policy work poverty Washington economic	Balkans hci ravallion mdgs weve ida deininger ington cpia worldbankorg
22	Tertiary and Bologna	Education tertiary student high institution OECD program university research review	Teis internationalization tertiary IBRD the tei cruch bologna undergraduate doctoral postgraduate
23	Development Bank and norm violation	World development right law medium bank human people legal state	Customary journalist insult pluralism news justice newspaper criminal rights based shareholder
24	Local level	Education school fund resource level local budget government system governance	Subcentral bec budget earmark territorial allocation district formula governance BOSDA
25	Latin America higher education	Education high university research international institution science country development student	Internationalization Antioquia Antioquias UNITWIN interuniversity Medellin drain Cuban scientist Cuba
26	Labour, train, development and worker	Labour train ILO development program country employment activity project worker	Seminar tripartite biennium ILOs ILO Turin advisory rehabilitation symposium fellowship
27	Skills foreign born adults	Adult skill immigrant country OECD level proficiency literacy language difference	Foreign born PIAAC technology rich native native born proficiency immigrant first generation numeracy second generation
28	Employment Israel Palestine conflict	Worker Israeli territory Israel Arab occupy Palestinian work Gaza employment	Samaria Judea Palestinian Gaza Israeli occupy strip Histadrut Israel territory
29	State and trade law	Law country may legal service professional bank provide state agreement	Nancial supra benets trustee qualications judicial ecommerce rst specic efciency
30	Workers’ skills and cognition	Skill job worker education employer cognitive market work occupation need	Saa anticipation noncognitive mismatch transversal skilling emotional personality cognitive conscientiousness
31	Youth and health	Youth young people program school health work country world child	Condom soul SRH Buddyz HIV AIDS sexually lovelife sexual parenthood gang
32	Evaluation and inspection	Assessment evaluation education student appraisal national teacher review system learn	Appraisal self-evaluation summative evaluator formative inspectorate inspection evaluation wwwoecdorgeduevaluationpolicy ero
33	Teacher and digital technology	Technology computer ICT learn digital student teacher information education school	OER informatics digital multimedia computer software hardware computational nit ICTCFT
34	ILO, green development	Social country work develop employment need development policy ILO labor	Industrialize ILOs environmental pollution green industrialization tripartite greening constituent ILO
35	Children’s health Central Asia	Child health school social education program improve intervention learn development	Kyrgyz Tajikistan Bishkek MOH nutrition deworming malnutrition immunization Kyrgyzstan street
36	Trade unions	Worker labor union organization trade country work social relation problem	Nonmanual labor management undertaking union collective conciliation dispute organization industrial arbitration
37	Problems of adult learning	Literacy learn program community adult skill education learner people train	Facilitator literate ESD UIL Tostan low literate prison literacy multilingual illiterate
38	Convention and workers insurance	Work act convention state office worker may labor employment person	Seaman seamens workmens vesselSsunday insure Rumania stoker furnace workman
39	Autonomy and quality of universities	University high education institution student quality research system academic college	HEIs HEC faculty self-perception Irish HEA autonomy university accreditation affiliate
40	Vocational training and East Asia	Train education system development skill vocational policy government institution program	Xinjiang Moe Korean saber country report Singapores systems approach for better education results Singapore TVE instructor workforce
41	Antisemitism and holocaust	Holocaust education textbook history curriculum genocide teach war conflict study	Holocaust antisemitism genocide Shoah Jew Nazi atrocity national sozialismus perpetrator Hitler
42	Innovation	Innovation education change student OECD practice teacher country level point	Innovation systemic innovate Thuringia innovative PIRSL math HEGESCO scratch crossroad
43	Leadership	School OECD teacher education student learn system review also leadership	Wale Flemish leadership leader talis Welsh Nusche municipality SLO school
44	East Asia, reforms, globalization and demography	Education OECD country student high unite good state Japan school	Reformer Shanghai Japanese globalization MEXT demography Japans performer Japan CSE
45	Primary and secondary	Education school secondary country student percent teacher primary learn system	Jse sse Seia lowersecondary ondary secondaire lewin ssa agepa pacic
46	ILO, conference	Conference ILO programme work session country committee international concern body	Seafarer lLO maritime delegate director general ship resolution conference ILO session
47	Study inequality	School student Pisa country OECD cent mean index datum sample	Spss sas stq scq wfstr hisei icq syntax cnt grp
48	South Africa, development and telecommunication	Africa ICT south service development country information access sector communication	Fixed line outgoing egovernment Pretoria telecommunication broadband SETAs incoming ZAR estrategies
49	Development projects and community	Project development knowledge community process support activity good change staff	Wbi cda sdv ieg ppd. toolkit coalition dialogue subprojects wbis
50	Brazil, adult education	Education adult Brazil learn educao state train federal literacy UNESCO	Ramaa adultos educao ale sra confintea eja ministrio belm formao
51	Child care, parents and kindergarten	Child care early service education Roma parent family childhood country	Roma nonroma ag kindergarten preschool parental pedagogue care ECEC Romania
52	Entrepreneurship	Program business train entrepreneurship entrepreneur evaluation participant entrepreneurial student group	EET entrepreneur venture entrepreneurial entrepreneurship acceleration startups EPAG startup mentor
53	Vocational training and apprenticeship	Train education OECD vocational vet skill review program student apprenticeship	VET apprentice apprenticeship Kuczera work based WBL PET Fachhochschulen postsecondary Musset
54	Returns and overeducation	Education country high percent labor return level school wage firm	Quantile return unobserved TFP OLS overeducation overeducated premium equation payoff
55	War, employment and wages	Country economic worker work may increase social problem wage employment	Postwar depression prewar underdeveloped PRP wartime automation war manpower coal
56	Russia and Bulgaria, curriculum	Education school educational system train new curriculum change problem information	Russian Moscow Russia Bulgarian ill votec fhe of the Sofia Russias
57	Earnings, tertiary and non-tertiary	Education country student OECD level secondary educational tertiary program institution	Glance tertiary type nontertiary graduation ISCED earnings USD upper descend fulltime
58	Agriculture, mobile information	Mobile information service farmer market access technology system agricultural use	Dlrs farmer mlab mlabs traceability mobile smallholder app phone sms
59	Banks and money	Financial literacy education impact survey money group behavior bank saving	Saving Lusardi financial lottery simulator money save debit behavior takeup
60	Selection of teachers	Teacher education recommendation committee teach country government school ILO organization	Ceart ilounesco allegation highereducation ece ssr ceartrsectoendocx vicechairperson zenkyo dialogue
61	School, training and university	Education course school secondary study train university work student vocational	Pronatec dominican uasd danish polytechnic fic wyszego szkolnictwa oraz amks
62	Early childhood and care	Child early childhood education ECEC quality care development staff curriculum	Ecce staffchild toolbox ecec samuelsson melqo childhood whriki kindergarten pramling
63	Islam, culture and the state	Right state organization social world international action Islamic cultural human	Islamic Muslim ISESCO Islam declaration OIC dignity religion covenant globalization
64	Support national development	Education development country national policy plan UNESCO Support Programme train	Efa capefa Dakar postconflict caped sectorwide jomtien fpe unescos nfe
65	Methods of data analysis	Item student school Pisa sample country scale test datum use	Booklet coder NPMS parameter stratum item nonresponse verifier marker reliability
66	Language and linguistics	Language read student text write word teacher teach learn grade	Syllable phonological phoneme alphabet decipher grapheme verlan ARED CEB blackboard
67	Emotion, neuroscience and cognition	Learn research knowledge education study practice student vol environment work	Emotion neuroscience service learning ILE music inspire correlational experimentation art cognition
68	State, primary, district and teacher	Education state school primary percent teacher district study student level	DPEP Catarina Bengal Kerala Pradesh Santa Orissa Assam Karnataka ICDS
69	University, innovation, development	Education university high development regional research innovation institution OECD student	Wroclaw Andalusia Sonora lombardy basque Arizona catalonia hei paso Penang
70	Monitoring primary education	Education country school global child learn low development monitor primary	Sdg sdgs gem parity uis gpi oda gpe aymara efa