1 Introduction

International Large-scale Assessments (ILSAs) have become the quality checklist for education systems globally, especially since the adoption of the Sustainable Development Goals (SDGs) in 2015. With streamlining, converging education policies in favour of sustainable economic growth, ILSAs are becoming a major criterion for measuring the sustainability of education systems. A critique often highlighted in the existing research considers ILSAs as a part of a neo-liberal agenda which has pushed education systems (including Adult Education) towards crossroads. Yet despite resistance from various stakeholders, ILSAs have gained ground across countries as a predominant tool for the measurement of learning outcomes.

ILSAs not only claim to measure the quality of education systems but also compare them against one another. They hence have the potential to stimulate various stakeholders engaged directly or indirectly with education systems, due to the fact that ILSAs may indeed identify weak and strong aspects linked to specific stakeholders.

With the increasing complexity of adult education systems, information provided by ILSAs may influence stakeholders and thereby influence the flow of resources. Irrespective of whether states participate or choose to abstain from participating, ILSAs influence state policies directly or indirectly. ILSAs might open windows of opportunity for introducing changes in education systems and may play a crucial role in deciding their future pathways. This paper discusses the role of ILSAs in shaping education systems in general and adult education systems in particular, with a global perspective.

The research paper explores the question: How have ILSAs shaped adult education systems globally? In terms of methodology, the research is based on document reviews comparing four ILSAs and analysing their respective linkages with the Global Alliance to Monitor Learning (GAML), SDGs and global policy changes in the adult education systems. The four selected ILSAs include the Programme for the International Assessment of Adult Competencies (PIAAC), used by the OECD, the Skills Towards Employability and Productivity (STEP) Survey of the World Bank Group (WBG), School-to-Work Transition Survey (STWS) used by the ILO, and the Literacy Assessment and Monitoring Program (LAMP) of the UNESCO Institute of Statistics (UIS). The conceptual framework chosen for comparison and analysis is the “Box Model” developed by Ehlers (2019).

The paper argues that ILSAs have facilitated major changes in adult education systems, enabled alignment of such changes with the global policy framework favouring sustainable economic growth, and helped expand the space for international organisations and transnational policies. Furthermore, ILSAs have contributed to weakening the authority of nation states over education systems in general, and of civil society over adult education systems in particular. While their influence has given more freedom to the learner, it has curtailed the freedom of adult education systems.

The paper views the changes in adult education systems as part of a global policy shift led by the OECD, with ILSAs having served as some effective catalysts.

2 State of the art

Adult Education systems have undergone at least three major changes in the period since World War II.

First, socio-cultural priorities of adult education systems aiming at nation-building or/and social change have been replaced with economic priorities (Ehlers 2013). Reflecting the human capital approach and economic competition, education policies nowadays predominantly consider education an investment rather than a cost (Ehlers 2013).

Second, a transnational policy (which implies the policy of education for adults resulting from the integration of labour market policies and education policies and not the philosophical idea of lifelong learning which implies learning from cradle to grave and across different disciplines) on lifelong learning emerged in Europe between 1966 and 1996 (Ehlers 2019). Originating from national policies in Norway, lifelong learning policies developed into a transnational policy backed by various stakeholders, including people in key decision-making positions, international organisations like the Nordic Council of Ministers, as well as market actors like Nokia and the European Roundtable of Industrialists (Ehlers 2019). This development also brought about a marked change from top-down teaching to bottom-up learning approaches in adult education systems (Ehlers 2019). A sector-specific concept of education was replaced by an approach that aimed to integrate all sectors, including adult education, into one education system, which was to be further integrated with the labour market (Ehlers 2019). Objectives changed in favour of reducing skills gaps and ensuring optimum utilisation of resources spent on education (Ehlers 2019).

Third, employability developed as a global norm for the international policy on education (specifically adult education) between 1994 and 2016 (Singh and Ehlers 2020). This shift notably followed the OECD recommendations for major economic reforms towards member countries which were struggling with unemployment (Singh and Ehlers 2020). Employability became a norm for OECD countries between 1994 and 2006 (Singh and Ehlers 2020). In the wake of the economic crisis post-2008, and at the initiative of the OECD and G20 countries, that same norm was pushed by the ILO to non-OECD, low- and middle-income countries, with the support of the WBG and the UN agencies (Singh and Ehlers 2020). Finally, from 2016 onwards, employability was recognised as the sustainable aspect of education by the OECD, the ILO, the WGB, as well as the UN agencies (Singh and Ehlers 2020).

These developments have been part of the OECD-led change and the shift in the global policy framework for development in favour of sustainable economic growth (Singh 2020). Even though sustainable economic growth had been on the OECD agenda since its inception (1961), its successful push onto the international policy agenda occurred only in 2015 (Singh 2020). The key question here is: To what extent and in what way have ILSAs influenced these changes?

Despite access to education having expanded between 1990 and 2015 through the Education For All (EFA) policy and a strive towards the achievement of the Millennium Development Goals, quality concerns in terms of learning outcomes have been grave (Rogers and Demas 2013). Global competitiveness forces countries even more to assess the avialbility and quality of skills (Schleicher 2008). ILSAs can thus serve as useful policy tools, so to say thermometers, that measure the quality of education systems and estimate the availability of skills (Lockheed and Wagemaker 2013). However, they can also be used as means for putting pressure (“whips”) to induce policy changes in education systems (Lockheed and Wagemaker 2013). Standardisation and similarity in design and approach of ILSAs make their data internationally comparable and thus more insightful in understanding education systems, as compared to the data provided by national assessments (limited in scope by their contextual relevance) (Cresswell et al. 2015). ILSAs may therefore provide useful insights beyond findings which national systems might otherwise generate, and consequently, widen the scope for the improvement of education systems (Cresswell et al. 2015).

The use of evidence in policy formulation is, however, a matter of hot debate. Some researchers (often called “constructivists” who are perceived as belonging to the “Reject school of thought”) outrightly reject the role of evidence in developing policies, arguing that they are meant for defining political standpoints and that excessive dependence on data might lead to conditions where policies go beyond human control (Newman 2017). Others (also designated “rationalists” and aligned to the “Reinforce, Reform or Reinvent schools of thought”) explore the extent to which evidence might influence policies in different ways (Newman 2017; French 2019). The lack of transparency of the policy development process, as well as the use of evidence throughout, make it difficult to identify the concrete role of evidence. Therefore, policies based on an analysis of evidence might be best characterised as evidence-informed rather than evidence-based (Parkhust 2017).

The debate regarding the use of evidence is not limited to its influence on policies but rather extends to its nature, the way it influences policies and related consequences. Research shows that evidence-informed policies might become exclusionary (i.e. exclude stakeholders who fail to produce relevant evidence and ensure its deliberation), or might get influenced by power differences, complicated political bargains, faulty analysis, contextual constraints, or even methodological factors like information gaps, secrecy, data protection and faulty assumptions (Banks 2009; Cairney 2016; Saltelli and Giampietro 2017; Singh 2017; Muller 2018; French 2019). For instance, the evidence-informed EFA policy (based on return on investment in education) had serious methodological flaws, which only became evident years later (Heyneman 2012). Research also shows that arguments around an inappropriate use of evidence cannot be ruled out—e.g. the possibility that evidence is of only limited authority, has credibility issues, reflects vested interests, tends to be used for the simplification of knowledge and promote unfair competition among non-equals, or to deliberately influence public opinion, make unscientific claims, and generally comes with ample possibilities for manipulation, inaccuracy and selectiveness (Pawson 2006; Kreiner 2011; Lawn and Grek 2012; Grek 2013; Hegarty 2014; Nordin and Sundberg 2014; Johansson 2016; Singh 2017; Gustafsson 2018; Keslair 2018; Rutkowski and Rutkowski 2018). Despite this, alternatives to ILSAs seem scarce and their data too important to be ignored (Johanson 2016; Singh 2017; Gustafsson 2018; Rutkowski and Rutkowski 2018). Claims regarding constant improvement in their methodologies, media attention followed by political upheaval, and research regarding their scientific use (Johanson 2016; Singh 2017; Gustafsson 2018; Rutkowski and Rutkowski 2018) make ILSAs quite convincing for all kinds of stakeholders.

3 Conceptual framework

The conceptual framework used in this paper is Ehlers’ Box Model (2006, 2019)—a model which highlights the relevance of sources in research, and provides insights about the production and use of evidence by different stakeholders. The four different types of stakeholders engaged with policy, research, profession and practice of adult learning have different stakes and thereby different rationalities, and hence, any sources they produce should be clearly differentiated (Fig. 1).

Fig. 1
figure 1

Conceptual Framework: Ehlers’ Box Model (2006, 2019). Source: Ehlers (2019, p. 21)

Full size image

An application of the Ehlers’ Box Model for the purpose of this paper suggests that evidence produced by international organisations (policy stakeholders) through ILSAs has a political rationality of “compromise and power”. This means that outcomes of ILSAs should not be treated as research results or scientific evidence (Singh 2017). They should be analysed instead as policy instruments, rather than psychometric tools for measurement of learning outcomes. In the same vein, their outcomes should be considered politically motivated rather than objective data, and their objectives should be perceived as politically guided aims rather than quests for objective knowledge.

4 Comparative analysis of ILSAs and their linkages with GAML and SDGs

The OECD, the WBG, the UIS and the ILO all claim to provide useful data and global policy recommendations necessary to improve education systems (UIS 2016; ILO n.d.; OECD n.d.; The WBG n.d.). In relation to adult education systems, their ILSAs are particularly important because these organisations provide global policy recommendations, globally comparable policy evaluation statistics, and are thereby likely to influence resource allocation.

The OECD, using PIAAC, claims to provide data regarding the development of basic cognitive and generic work skills of the adult population relevant for employability for over 40 countries (OECD 2019, n.d.). Based on OECD’s model but with an idealistic approach (considering education as a right) towards adult education, LAMP banks upon inclusiveness of populations from linguistically, culturally and ethnically diverse contexts, especially from the non-OECD (primarily low and middle income) countries, by devising and translating tests that specifically reflect diversity (UNESCO and UIS 2009, n.d.). LAMP relies on states (in fact, governments) for infrastructure, resources, timing and inclusion (UNESCO and UIS 2009, n.d.). As a consequence, the formulation as well as implementation of LAMP is a costly affair and requires capacity-building of participant countries for mobilizing resources and conducting ILSAs independent of aid (UNSCO and UIS 2009). The WBG’s STEP primarily assesses employability among populations and maps specific factors related to urban centres and the formal economy (i.e. regulated by government laws) in low- and middle-income countries (OECD 2013; Pierre et al. 2014; The WBG n.d.). The quality standards of LAMP and STEP are lower as compared to PIAAC or other OECD surveys for adults on which they are based (OECD 2013), probably due to contextual considerations which indicate huge differences among adult education systems of OECD and non-OECD member countries. In fact, STEP has serious methodological drawbacks since a large amount of the population in low- and middle-income countries dwells outside urban centres and works in the informal sector (i.e. not regulated much by government laws); this sector could be huge and accounts for as much as around 93% in countries like India (Government of India 2020). The STWS by ILO aims at identifying effective policy choices which may facilitate a smooth transition of individuals from education to work (Schleicher 2008; Elder 2009; The WBG 2011). Unlike the educational perspectives underlying PIAAC, LAMP and STEP, the policy perspective underlying ILO’s STWS is employment-oriented (ILO n.d.). Similar to UIS, the ILO follows a rights-based approach but in favour of decent employment characterised by stability in jobs and/or job satisfaction (ILO n.d.).

Table 1 shows that international organisations conduct ILSAs to collect skill-related data of working age population. While PIAAC, STEP and STWS focus directly on employability, the needs and characteristics of the labour market and the adequacy of education systems to support both, LAMP explores socio-economic linkages regarding the same. Both LAMP and STWS follow a rights-based approach while PIAAC and STEP apply a service approach (education as a service to be sold for gaining employability). The inclusive approach of LAMP is guided by the OECD’s and the WBG’s objective of engaging all possible resources (in this case, human resources) to ensure optimum sustainable economic growth (The WBG 2011; OECD et al. 2016), since it explores the linkages between inclusion and sustainable economic growth (UNESCO & UIS 2009, n.d.). Surveys of employers by PIAAC, STEP and STWS specifically show that ILSAs consider the engagement of the market as a positive input, even though it might undermine state control.

Table 1 Comparison of Selected ILSAs
Full size table

ILSAs that appear to measure similar elements do not compete but rather complement one another. PIAAC focuses on high- and upper-middle income countries, specifically its member countries, while STEP, STWS and LAMP concentrate on low- and middle-income countries. STEP generates data regarding skills of the entire working age population, STWS focuses specifically on young adults, whereas LAMP collects data about all adults. STWS intends to collect information from the informal sector, remote areas and about vulnerable populations (in principle) that STEP does not include, probably due to questions of feasibility. Thus, these ILSAs focus on different target groups and aspects of adult education systems and complement one another to create a comprehensive picture about adult education systems and adult populations that engage with them. ILSAs do not only serve to collect and generate data but also encourage countries to develop policy evaluation capabilities to save resources and ensure sustainability in the long run (UNSCO and UIS 2009, n.d.; LMTF 2013; UNESCO 2012; The WBG 2011 in Cresswell et al. 2015). ILSAs thus provide comparative evidence regarding the functioning and adequacy of education systems and highlight the scope for their improvement (LMTF 2013; UNESCO 2012; The WBG 2011 in Cresswell et al. 2015). International organisations conducting ILSAs formulate policy recommendations for states and alter their own policies based on results they generate. Since ILSAs are done cyclically, they may serve to maintain pressure on countries to keep aligning their education systems progressively with international standards. That being the case, it seems relevant to ask here: what do the international standards for learning outcomes aim at? How are they created, who creates them, in whose interests are they developed, and what do they lead to?

5 Linkages amid ILSAs, GAML and SDG4

GAML is a global alliance of a wide variety of education stakeholders, formed in 2016, and led by the UIS (UIS 2016). Through standardised indicators and measurement tools, it claims to ensure transparent and optimum use of resources for improved learning outcomes; promote capacity build-up and thereby sustainability of national assessment systems for reducing aid dependency and develop relevant local expertise; advise on accountability of stakeholders; ensure effective interpretation and integration of ILSA data with national policies to measure and accelerate progress towards the achievement of SDG 4 (for education); control and channel development aid accordingly and thereby align education systems along the SDGs (Montoya 2015; UNESCO and UIS 2017a, 2017b, 2017c, n.d.).

GAML indicators for adult education systems include indicators such as: participation rate of youth and adults in education and training by gender; measurement of technical and vocational skills for employment, jobs and entrepreneurship; literacy standards by gender and vulnerable populations; participation in tertiary education; education spending per person; and relevant skills for employment, “decent” work and employability (UNESCO and UIS 2017c). However, since the SDG4 targets represent an integrated education system, GAML harmonises sector-specific ILSA data and evaluates the contributions of each sector in the achievement of a common SDG4 (UNESCO and UIS 2017c).

GAML criticises education systems for wasting resources, following unsustainable practices, providing sub-standard education and being short-termed (UNESCO and UIS 2017a) if they fail to align with its standards. It asserts fairness and inclusiveness of its approach and indicators designed to address contextual bias (UNESCO and UIS n.d.). Apart from economic indicators aiming at employability and labour market, GAML includes social indicators (which cannot be directly measured in economic terms) such as sustainability and sustainable education, human rights education, education about sexuality, about HIV and about citizenship (UNESCO and UIS 2017c).

GAML’s methodology seems commendable but not completely free from anomalies. For instance, relevant skills for employment, decent work and employability are measured only through Information and Communication Technology (ICT) skills; literacy benchmarks are contextual rather than standardised, and identification of vulnerable populations is left to nation states represented by governments (who are in many cases themselves responsible for the vulnerability of such populations) (UNESCO and UIS 2017c). GAML relies primarily on ILSA data, even though some data, not available through ILSAs, is procured from governments or government-authorised agencies (UNESCO and UIS 2017c). Thus, even though certain assessments remain non-standardised, the data is presented as comparable and standardised (UNESCO and UIS 2017c). Reporting against all GAML indicators is also not obligatory for states, which implies that states can choose to refrain from providing information about indicators that might feel uncomfortable for reasons best known to them (UIS 2016).

GAML sets internationally comparable quality standards for education systems defined in terms of SDG 4. Since ILSAs, to a very large extent, provide this data for GAML, ILSA inputs are decisive for determining the quality and thereby resources for adult education systems.

6 Discussion

ILSAs have facilitated major changes in adult education systems.

The first change marks a shift in priorities and approach of education systems from idealistic (socio-cultural) to materialistic (economic). ILSAs measure employability and explore its socio-economic linkages with key indicators for economic growth. Through ILSA, the collective socio-cultural concerns around nation-building and/or social change have been effectively integrated with individualistic concerns for inclusion of all (as human resources for optimum growth) and sustainability (with economic self-sustainability at the core and calculation of externalities defined in terms of environment and social factors as other relevant factors). Both inclusion and sustainable development are economically oriented (Singh and Ehlers 2020; Singh 2020).

The second change has been marked by the development of a transnational lifelong learning policy. ILSAs complement one another and provide information from the perspectives of different stakeholders, applying a bottom-up approach. Despite receiving sector-specific data from ILSAs, GAML works to harmonise such outcomes through indicators woven together to assess progress against SDG4. Based on this, GAML also identifies weak and strong aspects in education systems that may promote or hinder the integration of sectors within the education systems and the integration of education and labour market policies. The correlation amid resources spent in specific sectors and the returns received either in terms of employability (in case of PIAAC, STEP and STWS) or in terms of conditions influencing economic growth determine the quality of sectors in education systems, decisive for their share in resource allocation. Sectors which do not yield effective economic outcomes struggle for resources. As the sectors align themselves against the sustainability framework, follow international standards and harmonise with other sectors within education systems (for instance, adult education systems harmonising with secondary education or higher education) and with labour markets, their possibilities for a larger share of resources increase and vice-versa. ILSAs therefore promote an approach towards education as an investment and help quantify its returns.

The third change is characterised by the development of employability as a global norm (Singh and Ehlers 2020). Aspects of education under SDGs can be categorised as materialistic (economic) and idealistic (humanistic). Since ILSAs prioritise the measurement of materialistic aspects, idealistic aspects are automatically de-prioritised or ignored in the agenda of many states and stakeholders with limited resources, for reasons of political (un)willingness or other handicaps. When ILSA data is lacking, non-standardised data might be provided by states projected as standardised and comparable. This offers a kind of license to states for excluding vulnerable populations, manipulating or overlooking data, or simply not providing key information for a variety of reasons.

Similar outcomes can also result due to methodological flaws in ILSAs which are insensitive to idealistic concerns. For instance, information provided by STEP does not represent the concrete reality of vast populations living in rural areas, working in the informal sector or struggling through socio-economic exclusion in remote areas. Trends in ILSAs prioritising the measurement of materialistic aspects might therefore be dangerous and could negatively influence sustainable economic growth in the long run.

Why are materialistic outcomes prioritised over idealistic outcomes in ILSAs, GAML and even SDGs? Fig. 2 shows the positioning of ILSAs in relation to the global policy framework.

Fig. 2
figure 2

Positioning of ILSAs in the global policy framework. Source: Author’s Illustration

Full size image

As shown in Fig. 2, ILSAs are embedded in the sustainable economic growth policy framework led by the OECD, facilitated by the WBG and the UN agencies. The seventeen SDGs adopted in 2015 represent different policy areas integrated under the sustainable economic growth policy framework, with education represented by SDG 4. Along with other contextual variables, SDG4 objectives and evaluations of previous policies may influence national policy agendas, especially in what are termed as “aid-dependent” (based on global policy framework) countries. In line with this agenda, any policy evaluation using indicators (developed by GAML) measures the quality of education systems. The indicators may measure inputs, processes, outcomes and/or correlations among them.

Based on the data about outcomes, ILSAs provide information regarding the robustness of education systems, including inter-linkages among constituent parts, required to achieve sustainable economic growth in the long run. ILSAs can therefore serve as an integrated part of the global policy framework for international development.

The concrete (but not necessarily credible) ILSA data might influence national and global policies, form the basis for policy interventions, open windows of opportunity to induce policy changes, or provide strong arguments in favour of certain policies and stakeholders.

ILSAs have expanded the space for international organizations, sometimes against the interest or willingness of nation states. The role of international organizations is not limited to providing incentives or pressure (“thermometers” and “whips”). They deliver data which inform global policy knowledge bases and facilitate deeper understanding of education systems. Their data is used globally to engage a wide variety of stakeholders with education and labour markets, and to resolve concrete problems like mitigating skills gaps or providing education for livelihoods. In the worst case, they provide room for exclusion too. Internationally comparable data generated through ILSAs is beyond the management of nationstates and provides leverage for international organizations equipped with adequate expertise, knowledge base and other relevant resources.

ILSAs are thus powerful tools for facilitating change. Notwithstanding questions around their credibility, alternatives to ILSAs are scarce.

7 Conclusion

Adult education systems are experiencing radical changes in the face of globally available ILSAs, with their respective performances evaluated and compared against one another. In the past century, education for adults often consisted of idealistic, non-profit initiatives by the civil society, which were relatively independent from state regulations and were usually not included in mainstream national education systems and/or labour market systems.

Today, however, most adult education systems are funded on the basis of their performance and ILSAs play a vital role in generating such data. Irrespective of how education is organised (formally, non-formally or informally), who are the providers, who comprise the target groups and what objectives are met or forgone, performance of adult education systems is measured along the criteria of what is learnt and can be sold in the labour market. The monopoly of the teacher and regulatory systems over the learner has fallen apart; instead, internationally comparable data has brought the learner (as a customer) to the centre of these systems, thereby instituting a perspective that can be considered more “bottom-up.”

States (accountable for spending public resources) and employers (driven by profit) contribute a larger share of resources for adult education systems rather than civil society today. Thus, materialistic data from ILSAs about returns on investment and concerns of employers seem relevant for resource allocation.

This has rather put the efficacy of adult education systems as autonomous sectors under question. SDG4 and GAML both indicate that even though adult education sectors might serve as relevant abstractions for research, in reality they are losing significance. Integration of adult education systems with lifelong learning has no doubt curtailed the autonomy of adult education and reduced them to dependent sub-systems ruled by the economic logic of state and market. The continuous focus of global policies on sustainable economic growth implies that adult education systems are bound to follow the materialistic norm of employability in order to survive in a competing world.