This paper deploys a mixed-method analysis of SLM documentation while situating this planned change within wider policy rhetorics, seeking out the influences, continuities, and ruptures in and between texts from macro, meso and micro policy levels. This approach is considered apposite since it is widely recognised that global discourses, shared by international agencies and INGOs, influence decisions and practices at the national and the grassroot levels of educational change efforts. Analysis begins with selected macro level policies and subsequently identifies the extent to which these global rhetorics are reproduced, modified, amended, or transformed in documents (re-)produced at national (meso) and grassroots (micro) levels.
For this analysis, we selected a range of documents published by the main international actors that cooperate closely with the Sierra Leonean government and support the educational sector financially (macro), by the government of Sierra Leone itself (meso) as well as the documentation of the SLM model (micro). Macro policies analysed included those most recently published by the main donors in the educational sector (WB, UN, UK, USA, Ireland and GPE) as well as the recent educational strategy published by the African Union. The meso level documents comprise the most recent educational sector plan of the Sierra Leonean government along with the professional standards for teachers. Both document sets express mainstream views on the problems and solutions in education, reflecting global rhetoric on education reform (Fuller & Stevenson, 2019; Sahlberg, 2016). At the micro level, SLM documents include manuals, guides and curricula for clubs and conversation groups. Table 2 provides a summary of the documents used in our analysis.
This mixed-methods analysis has two main components: initial quantitative textual analysis followed by qualitative interrogation employing elements of critical discourse analysis and grounded theory. Both quantitative and qualitative analysis employed abductive reasoning (Alvesson & Skoldberg, 2000). Initially, an exploratory analysis utilising the keywords attributed to each paradigm enabled us to identify the prevalence of change discourses across the three levels of documentation (Fig. 1). A quantitative approach was preferred in this exploratory phase for a systematic examination of the 18 documents. Each document was converted into a corpus and the relative frequency of key terms compared, identifying the prevalence of these terms across these documents. As part of this analysis, we have also measured the degree of lexical similarity between documents (Benoit et al, 2018) which provide empirical support for the grouping of documents into three categories: macro, meso and micro. As an initial foray into the three document sets, Fig. 1 indicates the prevalence of the five paradigms in each group of documents.
Initial quantitative textual analysis identified the prevalence of keywords in the documents, qualitative analysis focuses on their meanings. Qualitative textual analysis began with close reading of all 18 documents and initial inductive coding of the texts. Following the rules of circular deconstruction based on the methods of grounded theory, data were separated into meaningful segments, coded in open, axial and selection coding systems and then categorised to formulate the main conceptions related to the topics foregrounded in the quantitative analysis (Glaser, 1992). The codes that emerged from this process were approached from the perspective of how the dominance of the scientific management paradigm, evident in Fig. 1, was used to frame ‘the problem’. This metamorphosis of the problem and its solution is reported on in the empirical section of the paper.
Analysis: anatomy of reform discourses–from crisis to intervention
The dominant presence of scientific management paradigm language is particularly striking in macro documents,Footnote 1 with relative absence of other paradigms also indicated. Consequently, at the risk of being reductionist, we conflated the keywords (Table 1) in the alternative paradigms to scientific management and repeated the analysis (Fig. 2).
The evidence above indicates the dominance of scientific management at macro level, while arguably this is softened somewhat by the increased presence of other paradigms at meso and micro levels. Beyond this relative paradigm ‘head count’, and mindful of Datnow’s evidence above regarding co-construction, Fig. 3 indicates the presence of scientific management terms in each of the micro documents, conscious of a bottom-up as well as top-down approach to bringing about change.
Even though these micro documents were written by actors other than those responsible for the macro and meso texts, the scientific management paradigm continued to have a strong presence, particularly though not surprisingly in relation to literacy. Consequently, we sought to combine quantitative and qualitative methods in our efforts to shed light on the framing and creation of the intervention. Given the dominance of the language of scientific management, it appropriates the terrain, framing the problem in its own image. Our immersion into macro and meso texts identified several central discourses deployed as tropes to frame the problem. Prevalent among them was crisis, measurement, assessment and tests. Dominating the discourse, not only provides the language in which the problem is framed, but the manner of its solution.
The concept of crisis has become fundamental to modern society (Cordero, 2017). Literature on development and education policy describes various kinds of crises: food, environment, migration, COVID 19 pandemic, economic, poverty and politics. As Gamble (2009) argues, this ‘crisis’ rhetoric justifies urgent programmes and quick decisions that provide interventions and strategies for action. Education is no exception; the language of crisis has been highly prominent in international and national documents for decades, while understandings of the nature of this crisis have changed during that time (Smith, 2016).
Analysis of macro and meso documents reveals that the concept of crisis is widely used in these texts. For example, the education policy strategies of both UK DFID (2018) and USAID (2018), as well as the World Development Report (WB, 2018), are prominent in this regard. The term crisis is mentioned 28 times in the USAID document, 45 times by DFID, and 54 times in the WB report. Both the UKAID and WB documents contain sections entitled ‘learning crisis’. The UN’s Incheon declaration uses the term crisis 14 times, whereas the African Union document uses the term once. The prevalence of this term is lower among the national (meso) documents where ‘crisis’ is cited 11 times in the Government's Education Sector Plan and not mentioned in the Teaching and Service Commission policy. Finally, the term is absent from all the SLM documents (see Fig. 4).
Although the word is frequently used in the international documents, closer examination of texts indicates that this word may have different connotations. Figure 4 shows the most frequent words preceding the word ‘crisis’ which are learning, conflict, violence, education, and conflict-affected. Qualitative analysis reveals two prominent uses of crisis: (1) a broad variety of political, economic, and social failures and (2) failure of educational systems. The latter is often labelled as a ‘learning crisis’ in the documents. These connotations are not mutually exclusive. Rather, the learning crisis is frequently placed in the broader context of other crises such as poverty, gender inequality, and conflicts. Interestingly, development agencies tend to discuss the educational problems within the broader context of their developmental work on prevention and mitigation of other crises, while international agencies are more likely to focus explicitly on the learning crisis. For example, in its declaration of work priorities, DFID claims that it will ‘deliver for children whose education is disrupted by conflict, protracted crises and natural disasters’' (2018, p. 26). Similarly, although the WB acknowledges the problems of conflicts and other crises, it argues that they are only some of the issues that contribute to the overall ‘learning crisis’ resulting in low educational outcomes that should be addressed in the first instance.
While the language of crisis has been present for decades (Smith, 2016), the concept of the ‘learning crisis’ is relatively new. It appeared initially in a UNESCO (2013) report, ‘Global Learning Crisis’, claiming that, despite growing enrolment rates, ‘250 million children cannot read, write and count well, whether they have been to school or not’ (2013, p. 2). This report is significant for two reasons. First, it was the first sign of a shift from a more traditional focus on school enrolment to the quality education agenda. Second, it explicitly links the concept of quality education with learning outcomes in literacy and numeracy. Subsequently, this focus on the outcomes as quality markers becomes foundational to a significant shift, a new focus in education, resonant with a scientific management paradigm of educational change, its preoccupation with achievement expressed in numbers.
Our analysis indicates that the discourse of learning crisis as low educational outcomes is present in all international documents, though only indirectly in some. WB and DFID identify the current situation in education as a learning crisis, while other actors address the same set of problems and solutions without explicitly referencing this term. DFID (201, p.9) identifies the problem as ‘ensuring children learn the basics of literacy and numeracy’ as the means of ‘tackling the learning crisis at its root’.
Similarly, USAID (2018, p.7) wants to ‘improve learning outcomes’ by increasing ‘access to quality basic education for all’ especially ‘marginalized and vulnerable populations’. Through the ‘algorithms’ of these macro documents, there is consensus that learning outcomes need to be prioritised as quality markers of education. The consensus extends also to agreement on the perceived causes of such a crisis of learning outcomes. They include low school readiness, lack of skills among teachers, ineffective school management, lack of inputs (such as textbooks and school infrastructure) and bad governance practices including the lack of cooperation and unproductive spending (WB, 2018).
Preoccupation with learning outcomes as markers of quality in education has their genesis in neoliberal ideologies of education as human capital formation for the purposes of economic growth (McMahon & Okatch, 2013). Such ideological underpinnings are clearly visible in almost all international documents. USAID discusses the benefits of education thus:
Human capital development through education contributes to addressing poverty reduction, health and nutrition, economic growth, and labor market opportunities, as well as peacebuilding, social cohesion, and the promotion of democratic institutions. (2018, p.13).
The accounts of other actors echo this idea, linking quality education (as good learning outcomes) with increase in ‘productive employment’ (DFID, 2018, p. 9) and ‘growth and development’ (WB 2018, p. 4).
Only two international documents (Incheon declaration (2016) and a Government of Ireland 2019) in part have a different rhetoric. Both documents regard education as a human right. This implies the existence of duty bearers who are responsible for vindication of this right, while right holders are entitled to claim their right through a range of individual and collective practices. This rights-based approach to education seeks to tackle the root causes of inequalities in education.
Interestingly, regional and national documents frame their statements within a neo-liberal perspective of economic growth and human capital development rather than human rights. The Sierra Leonean Ministry of Education claims that its primary goal is: ‘to provide opportunities for children and adults to acquire knowledge and skills, as well as nurture attitudes and values that help the nation grow and prosper’ (MEST, 2018, p. i). This logic assumes that improving the country’s international competitiveness will reduce poverty.
The focus on skills acquisition to overcome poverty rather than address structural issues such as inequalities and violence embedded in the society is also reflected in the framing of an international education change agenda. Following the postulates of a scientific management paradigm, those interventions that aim to improve (measurable) learning achievement of students are prioritised, while more serious structural issues such as access to knowledge are often neglected or ignored. Although the concept of ‘safe learning environment’ understood as the absence of violence in school and gender equality is present in all documents, it is considerably less prevalent at the macro or strategic planning level. To illustrate this, we established the level of frequency of the terms ‘safe’ and ‘violence’ in the three document sets. Their frequency is greater in the micro texts (230th and 47th, respectively), followed by the meso (351st and 330th) and macro (746th and 773rd), clearly manifesting a declining presence from micro to macro.
The WB’s (2018) definition of the learning crisis does not address these issues at all, focusing instead on the various aspects of learning outcomes. Other international documents (DFID, 2018; USAID, 2018) and Sierra Leonean national educational plan (2018) address such concerns more explicitly, yet when it comes to defining desirable outcomes of education, they continue to focus on literacy and numeracy skills as the most suitable way to understand the efficiency of the educational system, another neo-liberal trope, whereby efficiency trumps efficacy (Gross & Stein, 2001). Thus, despite recognition of structural disadvantages and violence in school being problematic, positioned outside the parameters of the learning crisis there is no necessity for them to appear as part of a solution. Reframing education as acquisition of certain skills in literacy and numeracy constructs the notion of ‘quantitative effectiveness [of education] which relies on test results to verify effectiveness and quality’ (Bivens et al., 2009, p. 98), thus enabling measurement and standardised tests to dominate the education landscape.
Reframing the learning crisis: measurement in education
Since low learning outcomes are cast as the central educational problem, their measurement becomes a central task for both international and national policy-makers. This is not a new idea and has been present in the literature even before the conceptualisation of the learning crisis. For example, a WB (Pritchett, 2001, p. 379) publication argued that ‘the quality of schooling across countries is impossible to measure without internationally comparable examination’ and declared that the WB would support the measurement of learning achievements worldwide. Measurement reinforces the conception of education as acquisition of certain skills that can (and should) be measured through national examinations, international assessments like the OECD Programme for International Student Assessment (PISA) and local-level standardised tests such as Early Grade Reading Assessment (EGRA) and Early Grade Mathematics Assessment (EGMA). This discourse of measurement was evident in all macro and meso documents, even those documents that were framed or oriented within a rights-based tradition are intent on supporting: ‘national monitoring and evaluation systems in order to generate sound evidence for policy formulation and the management of education systems as well as to ensure accountability’ (UNECSO, 2016, p. 11).
Organisations that frame their educational change work as human capital development insist that measurement is an integral part of the intervention process. USAID’s (2018, p. 6) aim ‘is to achieve sustained, measurable improvements in learning outcomes and skills development’. Similarly, the WB calls for the creation of ‘learning metrics’ that measure learning outcomes nationally and internationally, creating a ‘political space for innovation’ (2018, p. 24) and providing support for interventions that address the learning crisis. The GPE also follows this logic prioritising assessment as a major focus of its work in Sierra Leone:
The ESP [Education Sector Plan] is therefore focused on: improving the teaching and learning situation, monitoring learning through learning assessment tests and analysis of scores, improving performance in the WAEC conducted National Primary School Examination (NPSE), Basic Education Certificate Examination (BECE), and West African Senior School Certificate Examination (WASSCE). (GPE, 2018, p. 8).
National (meso) documents analysed repeat this international rhetoric, framing low scores in international standardised tests as the main evidence of low learning outcomes of students in the country, thus declaring national and regional examination results as indicators of success. Consequently, ‘measurable’ becomes synonymous with ‘valuable’ and only those skills that can be easily quantified become the fulcrum on which improved educational quality depends (Unterhalter, 2019).
Policy rhetorics: melding macro, meso and micro
These globalised ideologies of education as acquisition of measurable skills play a significant role in the development of national educational policies especially in the context of high aid dependency in the Global South. Lingard and Ranotte (2011) argue that globalisation of educational agendas has led to the relocation of political authority beyond the nation state. This process of incorporation of global discourses in the national context is highly visible in Sierra Leone. The GPE and UNICEF (2018) developed a plan for the improvement of education there that reflects global ideas of measurement as an integral part of the reform process. This plan does not use the term ‘learning crisis’ but clearly works within this paradigm as the problem of education in Sierra Leone is framed as remarkably low learning outcomes, measured through such standardised tests as EGRA and EGMA. The plan argues that measurement of learning outcomes in numeracy and literacy is ‘central to the learning process’ (Ibid. p. 72) and sets regular measures of learning outcomes as one of the priority areas for the whole intervention plan.
National documents too are framed within the general discourse of measurement of literacy and numeracy skills as the most appropriate means of addressing the problem and monitoring progress. While international documents such as WB Report (2018), DFID (2018) and USAID (2018) discuss the problems of measurement in education in more general terms, the Education Plan developed by the Ministry of Education is more specific, repeating the statements of GPE that low EGRA and EGMA scores along with high dropout rates are the most serious problems of education in Sierra Leone. To improve this situation, measurable interventions are needed and, to capture the progress of these interventions, the Ministry suggests the use of the results of national and regional examinations as an indicator of the quality of education. Towards this end, it develops a set of numerical indicators such as ‘percentage of WASSCE candidates passing English and Maths’ and ‘percentage of learners meeting assessment criteria for their levels’ (2018, p. 57) arguing that:
The performance of candidates in school level examinations conducted by the West African Examinations Council (WAEC) provides an idea of the state and quality of teaching and learning in the schools. (2018, p. 49).
In turn, the professional standards for teachers developed by the Sierra Leone Teaching and Service Commission (2017) focus extensively on assessment practices in classrooms and the importance of exams. Arguably, in contexts where teacher education is limited, when such a climate is created, concern for and preoccupation with testing has potential to be severe, in the absence of resources and professional support. This preoccupation with testing and achievement reflects the main postulates of the scientific management paradigm and thus limits opportunities for building teachers’ pedagogical repertoires, imbued with more progressive or critical orientations. Rather, such is the pervasive presence of a testing-measurement culture in Sierra Leone ‘that quality schools and teachers are those with the best examination results’ (Wright, 2018, p. 55).
The language of SLM: ‘grassroots’ or scientific management Chameleon?
Figure 3 has already indicated a strong presence of scientific management language in the SLM documents, particularly the literacy manual, and more surprisingly in the TLC (Teaching Learning Circles) document. While undoubtedly, these documents retain ‘bottom-up’ features, scientific management reinvents itself in different hues. For example, although assessment does not appear among the terms most associated with learning, it is prevalent nonetheless in documents pertaining to the teaching dimension of the SLM intervention. However, when the frequency of the scientific management chameleon terms (assessment, testing, measurement-EGRA) are searched for in the SLM documents, their frequency reveals a more pervasive presence.
Figure 5 indicates that in the three school-related documents (TLC Workbook, Literacy manual and Workbook for Coaches), the language of scientific management is significantly prevalent; much less so in relation to the community focused documents. Awareness of this degree of influence, in a context where conditions and resources are Spartan, and the knowledge base of teachers is thin or non-existent; conditions for pedagogical capacity building are far from optimal, placing a premium on coaches’ expertise, an under researched area even in much more favourable circumstances. For Concern, education is ‘one of the best routes out of poverty’ that gives ‘extremely poor children more opportunities in life and to support their overall well-being’ (Concern 2019, p. 30). Education ‘as a route’ has identifiable neo-liberal credentials; an instrumentalist education as a means of poverty alleviation and economic growth, prevalent also in the macro and meso documents. This perspective promotes a mindset or orientation that focuses on the process of skills acquisition required for poverty reduction and thus strongly influences Concern’s understanding of ‘quality education’ as good learning outcomes, free access to education along with improved school facilities (Concern 2019, p. 30), indicating considerable ‘alignment’ with quality education as articulated within the global framework of the learning crisis.
Being framed within the discourses of education as learning outcomes and their measurement, the SLM planned change is to tackle poor educational performance in Sierra Leone through addressing literacy of primary school children. However, the concept of literacy in this intervention is narrowly framed as ‘the ability to read and write’ only (Concern, 2017a), which allows Concern to measure this literacy using international standardised tests. This narrow focus on early literacy is resonant with the GPE-UNICEF (2018) assertions that literacy in first grades of primary school should be given priority in Sierra Leone due to remarkably low student EGRA scores.
Reframing education as learning outcomes that should be measured has considerable implications for teachers in their classrooms. There is widespread recognition in national and international documents that ‘business as usual’ is unacceptable, and current teaching practices need improvement. However, there is a surprising lack of discussion about the pedagogy in the classroom (see, Sayed & Moriarty, 2020). Nevertheless, an instrumentalist reframing of quality education into improvement of measurable learning outcomes funnels efforts into specific techniques that teachers can deploy to improve test scores. At that point, scientific management needs the assistance of a teacher professionalism paradigm if teacher learning through enhanced understanding and expanded pedagogical repertoires are to build their expertise.
The perception that teachers’ skills are crucial for the improvement of learning outcomes is not new and is widely accepted across the educational sector. As we have seen above, it is also present within the discourse of the learning crisis and major international actors in the educational landscape of Sierra Leone call for improvement of teaching practices in schools. The WB (2018, p. 22) advocates provision of targeted teacher training (‘often around a specific pedagogical technique’) to facilitate ‘effective teaching’. Other international documents, including Irish Aid (2018) and USAID (2018), describe their aims in terms of improvement of efficiency of teaching using specific training packages, training, and ongoing coaching, with little reference to broader pedagogical concepts.
National documents follow the same logic, focusing on specific techniques and tools intended to improve learning outcomes. Among these techniques is the development and dissemination of highly structured and detailed lesson plans to schools, arguably a version of ‘teacher proof’ curricula, anathema to teacher agency (Priestly et al., 2015). Other targeted techniques include use of instructional aids during lessons, questioning techniques and time management. Assessment techniques both formative and summative (often in a form of standardised tests such as EGRA) are presented as being integral to improving learning outcomes. It is stressed that teachers’ ability to assess students is a most important professional skill, melding measurement discourse with discourses of effective teaching. However, since the majority of teachers in the Tonkolili district are unqualified, and with modest educational attainment at best, the potential of the intervention to improve teaching and learning is hampered considerably due to the realisation that ‘differences in teacher cognitive skills can explain significant portions of the international differences in student performance’ (Hanushek et al., 2019, p. 859). In the absence of an appropriate knowledge base among teachers, teaching to the test may be the most likely outcome.
The SLM intervention is framed around the necessity to improve teachers’ skills through the focus on ‘activities [that] aim to develop the competencies related to early grade literacy of teachers’ (Concern Worldwide 2017a, p. 4). This rather narrow focus on competencies related to literacies reflects the general framing of the intervention as the improvement of the ability to read and write. In this regard, it follows the discourse of the learning crisis that focuses on the development of certain skills rather than recognising the significance of teacher agency, or building their professional capacities in an expansive manner, reflective of more progressive or transformative paradigms. Rather, SLM documents present a tool kit with basic teaching techniques. These include the use of lesson plans, letter tracing, visual aids and regular formative and summative assessment of students to improve the literacy skills of students measured through an EGRA assessment Concern Worldwide 2017a). School-level interventions are focused on teacher training and development through regular in-classroom coaching, teachers’ workshops, and support of teacher learning circles where teachers can share their experiences, along with support of school management and provision of teaching materials (Concern Worldwide 2017a; Concern Worldwide 2017b). However, in practice, the lockstep lessons plans mentioned above appear to be the pivot on which support revolves; perhaps foreshadowing the GPE assertion that ‘coaching only works if it is conceptualized, designed and implemented well’ (emphasis original) (Burns, 2020). Such designs are constrained since ‘very little [is known] about what makes for an effective coach or what a system for selecting and training an effective corps of teacher coaches should look like’, challenges particularly problematic in a context where many teachers are untrained or under-trained (Kraft & Blazar, 2017, p. 1058).
Although aspects of the SLM indicated above are evidently narrow in scope and transactional, there is evidence also of more holistic elements in the following:
Children’s educational progress will be enhanced when they live in communities where there is more support for gender equality and children’s wellbeing. (Concern Worldwide 2017a, p.2).
This assertion claims that well-being of children and gender equality in the classroom and the community may positively affect student learning outcomes. The model tries to improve learning outcomes directly through the improvement of teaching practices at the school level and indirectly through work on gender equality both in schools and in communities and supports improvement of food security and access to health services at the community level. Concern explicitly recognises community as being central to the discourse of a safe learning environment and quality education.
However, when it comes to evaluation of the SLM, literacy skills measured by standardised tests are foregrounded. This measurability of literacy skills accords with the macro discourse of measurement of learning, a perspective articulated by Concern Worldwide 2019, p. 37) when it states that it uses ‘results-based management’ in its work with the intention of finding measurable changes in the society. This core focus on measurement is also evident in the evaluation of the SLM being conducted by the authors on behalf of Concern. Though the evaluation includes a mixed-methods approach, with interviews and field observations with teachers, learners, headteachers, parents, grandparents, teenagers, and community leaders, central to the evaluation is a randomised control trial (RCT) based on EGRA scores and numeric well-being measures. As authors and researchers, we recognise that in contracting to undertake the research, we contribute to the policy process where multiple crises become a learning crisis, that is, reframed to be measurement of learning outcomes in literacy and well-being.