Keywords

1 Development Challenge

More than 617 million children and adolescents worldwide do not possess the basic reading and mathematics skills required to live a healthy and productive life and contribute towards economic growth (UNESCO, 2017; see Text Box 11.1). This global learning crisis is particularly acute in sub-Saharan countries, where minimal proficiency levels are not being met by 88% of children and adolescents in reading and 84% in mathematics. Across the region, education reform is required to address significant disparities in access, quality and equity. Children living in challenging contexts, such as remote rural locations, urban slums and border cities, are particularly vulnerable, and certain groups of children are disproportionately affected – especially girls, children living in extreme poverty and children at risk of, or diagnosed with, special educational needs and disabilities.

How the Global Learning Crisis Affects Capacity

  • UNESCO ( 2018 ) report that the global learning crisis is costing $129 billion a year.

‘Ten per cent of global spending on primary education is being lost on poor quality education that is failing to ensure that children learn. This situation leaves one in four young people in poor countries unable to read a single sentence and perform basic mathematics’.

  • UNESCO (2019) warn that the world is off track in achieving the global education goal, SDG 4.

‘Even though more and more children are starting school, one in six aged 6–17 will still be excluded by 2030. And of those who are enrolled, many are not learning or will drop out early. By 2030, when all children should be in school, four in ten young people will still not complete secondary education. The European Commission highlighted that by the end of the Agenda for Sustainable Development, about 800 million young people, half of them girls, will not possess basic skills. This is a critical global issue, as many adolescents and young people will not move into the job market with the right skills needed for relevant employment that responds to the challenges faced by their countries. Exacerbated by complex global threats like climate change, the situation is most acute in low-income countries and holds back economic growth and political stability, which would help bring people out of poverty and foster economic growth and social wellbeing’.

The United Nations has recognised the significance of this global crisis in education in the 2030 Sustainable Development Goals (SDGs). All member states have agreed to work towards addressing SDG 4: Quality Education, which aims to ensure inclusive and equitable quality education and promote lifelong learning opportunities for all.

Radical solutions are required to eliminate existing barriers to quality education for all children, anywhere in the world. Whilst the vast majority of children are now in primary school globally, their learning attainment is highly uneven and often persistently low (Hubber et al., 2016). This is not simply a matter of efficiency but one of human rights and social justice (Barrett et al., 2015).

SDG4 has focused attention on the quality of learning outcomes. Evidence suggests that technology-enhanced learning materials could be utilised by developing countries to promote significant and cost-effective improvements in learning (e.g. Mitra et al., 2005). However, realising these benefits across different school contexts remains a major challenge. For example, the One Laptop Per Child project had limited success in the United States, Peru, Rwanda and Tanzania, mainly due to poor implementation (e.g. limited Internet access in schools, poor support for repairs, non-child-directed delivery) and lack of teacher training (see Hubber et al., 2016). Evidence from the Second Information Technology in Education Study (SITES) indicates that significant investment has been made in technology-enhanced learning initiatives across the developing world, but this investment has not yet resulted in learning gains (Law et al., 2008). One reason may be that teachers and schools implement such initiatives in ways that reinforce rather than transform existing practice (Ruthven, 2009). This suggests there may be considerable potential in initiatives that improve learning but which have a close match to existing pedagogical practice.

Unlocking Talent

The initiative described in this case study, Unlocking Talent, was established in 2013 to address the global learning crisis by harnessing technology to deliver high-quality education. It takes an inclusive system approach to embedding evidence-based e-Learning platforms within schools, with the aim of improving early-grade learning outcomes in reading and mathematics. Unlocking Talent is a growing, global initiative made up of an alliance of partners that focus on putting children and their educational needs first. At its core, the project uses e-Learning to help overcome the challenges that hold learners back. These include, but are not limited to, lack of trained teachers, overcrowded classrooms, lack of learning resources and high dropout rates. For context, in Malawi, where Unlocking Talent began, the pupil to teacher ratio is 77:1, and the dropout rate at primary age is over 50% (UNICEF Malawi, 2019).

Whilst e-Learning platforms alone will not solve these problems, good educational technology is key to raising learning standards and could complement other anti-poverty interventions, such as cash transfers. For example, a study by Kilburn et al. (2017) showed that financial constraints are a major barrier to school attendance in Malawi, because the cost of uniforms and books is beyond what many families can afford. Cash transfers can increase attendance and reduce dropout, but this does not change learning outcomes. High-quality e-Learning platforms can increase learning outcomes, although children need to attend school on a regular basis to benefit from this technology. Hence, to raise learning outcomes in low-income contexts, such as Malawi, multiple synergistic interventions are required.

Why Malawi?

In 2013, the Unlocking Talent alliance started to introduce a customised e-Learning platform within state primary schools in Malawi, to explore if technology could provide a means of delivering high-quality education to marginalised children and raising learning outcomes. Malawi was chosen because of the extreme challenges facing education within the country. Whilst most innovators pilot their education technology in small private schools, the Unlocking Talent alliance focused on state schools from the onset, so as to reach the most marginalised children.

Malawi is one of the poorest countries in Africa (UNESCO-IBE, 2010) and has some of the largest educational challenges globally, ranking second lowest in the Southern and Eastern Africa Consortium for Monitoring Educational Quality (SACMEQ) III report (Hungi et al., 2010). Whilst primary education is free for all children aged 6–13 years, the education system is in poor condition, fraught with chronic overcrowding, limited resources and inadequately trained teachers (Milner et al., 2011). As a consequence, many of Malawi’s children do not attend or fail to complete primary education.

Allied to very poor retention rates, very few pupils meet minimum learning standards. Less than 10% of poor rural girls meet such standards, and attainment of basic reading and mathematics is among the lowest in the region (World Bank, 2010). As a result, adult (15+) rates of reading and mathematics proficiency are just 66% (UNESCO Institute for Statistics, 2016).

Additionally, structural inequalities exist that disadvantage certain groups (Yates, 2008; see Text Box 11.2). This has significant consequences for Malawi’s potential for long-term economic growth. Poor reading and mathematics skills mean many people in Malawi struggle with basic day-to-day tasks, such as being able to read the newspaper, understand forms and buy and sell goods at the local market, resulting in long-term dependency and inability to secure and sustain employment and actively participate in society. As discussed by our research team (Pitchford et al., 2019), levels of reading attainment are linked to a country’s economic growth and to an individual’s health, nutrition, rate of fertility and mortality and income potential (Verner, 2005); however, mathematics attainment has been shown to have a stronger link to an individual’s income potential (Crawford & Cribb, 2013; Dickerson et al., 2015; Geary, 2004). Findings from several African countries indicate an increase in mathematics test scores of only 0.1 standard deviations is associated with an increase in income of between 2% and 6.5% (Dickerson et al., 2015).

How Structural Inequalities Within Malawi Education Affect Gender and Other Vulnerable Groups

There has been strong international focus on disparities in learning attainment within countries. To illustrate how particular groups of children are disproportionately affected by education in Malawi, data on numeracy levels are summarised below from the SACMEQ-III report (Spaull, 2012).

  • Gender: 64% of grade 6 girls were classified as functionally innumerate compared to 56% of boys.

  • Wealth: 61% of pupils from the poorest 25% of society were classified as functionally innumerate compared to 58% of pupils from the richest 25% of society.

  • Location: Higher functional innumeracy rates were reported for grade 6 pupils in rural (63%) than urban (51%) regions.

  • Disability: 98% of children with special education needs and disabilities in Malawi are excluded from quality education (MacDonnell Chilemba, 2013).

Wealth and location interact as, in general, poverty is higher in rural than urban regions, but within urban regions, there are pockets of extreme poverty. Rapid urbanisation has resulted in large informal settlements in major cities, often with very poor education facilities, which can overturn the general urban advantage (Chimombo, 2009).

In addition, children with home languages other than Chichewa, such as those residing in the north of Malawi, can be further disadvantaged by receiving instruction in school in a language (Chichewa) other than their home language (Essien et al., 2016).

2 Implementation Context

The context in which we have implemented a customised e-Learning platform in Malawi is extremely challenging, due to limitations in key components of the country’s integrated system of learning attainment. These include challenges with school infrastructure (e.g. space, grid supply of electricity, Internet connectivity), capacity and capability of teachers (most of whom have limited digital literacy skills and little time for learning new skills), short school days (restricting available time to introduce a new intervention without missing out on other key areas of the curriculum) and overcrowded and mixed-ability classes. With an average of 77+ children in a class and often more, all of different abilities, it is difficult to ensure all children have sufficient access to the technology at the required dosage to be effective.

In addition, there are concerns about theft of technology and the greater impact this has on school security and community beliefs about the value of technology for learning. Malawi has a seriously underfunded education sector that struggles to provide even the most basic learning materials, let alone expensive technology. This all adds to the complexity of implementing an e-Learning platform in a low-income country such as Malawi.

With these challenges, it was necessary to work with many stakeholders to introduce an e-Learning platform within the Malawi primary education system, with the view to scale nationally (see Fig. 11.1). Every country and context are different, so having partners with the right skills, processes and support needed to implement education technology is vital for success. From the outset, the Unlocking Talent alliance has been partnering with the Government of Malawi to ensure the project is fully embedded within the Ministry of Education, Science and Technology (MoEST) and aligns with the national primary education strategy. Voluntary Service Overseas (VSO), an international development organisation, leads the Unlocking Talent initiative and is responsible for implementing the project in Malawian primary schools. VSO has developed a close working relationship with the MoEST in Malawi over the last 50 years, which has enabled them to have an active project management unit working with all relevant departments and the senior management team in the MoEST. The other core partners of the Unlocking Talent project are the software developer – onebillion – and our research team at the University of Nottingham.

Fig. 11.1
figure 1

Partnerships for scaling a customised e-Learning platform in Malawi adopted by the Unlocking Talent alliance

Full size image

Innovative, interactive, child-centred apps, designed and developed by onebillion, are the centre of this e-Learning platform, which is delivered through handheld tablets. This case study describes nearly a decade of research evaluating the effectiveness of the onebillion apps – and VSO’s implementation of the platform – within Malawian primary schools, providing the critical evidence base for scaling the initiative.

To ensure the research is culturally sensitive and to draw on specialist in-country knowledge of education practice and policy, the research team has partnered with academics at the University of Malawi. The in-country academic partners operationalise the research by assembling, training and managing teams of local evaluators who carry out assessments with children and schools in the local language. This partnership also builds research capacity through supervision of doctoral and masters students and early career researchers at the University of Malawi. To promote scaling of the project, in line with the research evidence, the alliance has engaged funding partners including the Norwegian Embassy; Kreditanstalt für Wiederaufbau (KFW); Comic Relief; the UK Foreign, Commonwealth and Development Office; UNICEF; Cisco; and Airtel.

Other critical stakeholders are end users of the technology, including teachers, school leaders, primary education advisors, community leaders, parents and children themselves. In developing their customised e-Learning apps, onebillion has gone to lengths to ensure that they meet the needs of children, through rigorous piloting of their software and hardware with children in different settings. When implementing the technology, VSO engages with parents, community leaders, teachers and school leaders, to promote cultural acceptance and sensitisation and to address any concerns they might have, by holding sensitisation meetings at local schools. In addition, there is a steering committee for the Unlocking Talent project in Malawi, comprising the national secretary for education, district-level education officers, representatives from school management (head teachers) and class teachers, to ensure their voices are heard and inform contextually sensitive shaping of the project as it scales.

3 Innovate, Implement, Evaluate and Adapt

3.1 Innovation: Customised E-Learning Platforms

Customised e-Learning platforms have potential to address some of the major challenges related to the global learning crisis facing many low-income countries in sub-Saharan Africa and elsewhere around the world, especially when based on the seven principles of universal design: (1) equitable use, (2) flexibility in use, (3) simple and intuitive use, (4) perceptible information, (5) tolerance for error, (6) low physical effort and (7) size and space for approach and use (The Center for Universal Design, 1997). Key software features include a wide array of multisensory representations of information (such as pictures, sound, video and animation), interactive tasks of varying degrees of difficulty, clear goals and rules, learner control, response feedback and repetition. These features promote an individualised learning environment that places the child in active control of their learning, with little or no adult support (Condie & Munro, 2007; Rose et al., 2005). The apps are delivered through touch-screen tablets, which are mobile and lightweight, eliminate the need for extra dexterity-reliant devices (e.g. keyboard and mouse) and have the capacity to store multiple child-friendly apps (Kucirkova, 2014).

We have found that customised e-Learning platforms have potential for wide-reaching acceptance and uptake by primary school teachers. The platforms can address the broad range of individual abilities typically represented within a large class and provide high-quality tailored instruction that is consistent for all children. However, certain alignments at a country level, such as language of instruction, are required for e-Learning platforms to be optimally effective. For example, Outhwaite et al. (2020) have shown that proficiency in language of instruction correlates positively with progress through the onebillion software. In addition, teachers are more likely to adopt e-Learning platforms that align closely with existing curriculum and pedagogical practice, as they supplement what is being taught in class and therefore can be used as a remedial resource for children that are struggling to learn with standard classroom instruction (Pitchford et al., 2018).

E-Learning Platform Used in the Unlocking Talent Project

As noted earlier, the software adopted by the Unlocking Talent initiative has been developed by onebillion – joint winners of the Global Learning XPRIZE. The software provides a comprehensive course of hundreds of learning units designed to support the acquisition of basic reading and mathematics from scratch, with little or no adult support. The course content is grounded in the national primary curriculum of Malawi and covers topics that are essential for the development of basic reading and mathematics, such as identification of letter sounds, syllable and phonological awareness, reading aloud, comprehending single words and sentences, counting to 100, adding, subtracting and telling the time (see Fig. 11.2).

Fig. 11.2
figure 2

Examples of activities within the interactive apps developed by onebillion to support acquisition of basic reading and mathematics

Full size image

The course content is modular, guiding children through a series of learning units, and is delivered in their local language, Chichewa, by a friendly virtual teacher. The software offers child-centred tuition through interactive picture, audio and animation formats with clear objectives, instructions and immediate formative feedback, consistent for all users. These app features align with the principles of active, engaged, meaningful and socially interactive learning (Hirsh-Pasek et al., 2015) and are characteristic of direct instruction and retrieval-based learning, within a self-paced, individualised and inclusive learning environment, all of which are known to be important for raising learning outcomes (Grimaldi & Karpicke, 2014; Kirschner et al., 2006; Slavin & Lake, 2008). For example, the onebillion software enables individualised reward and feedback on each interaction with the software. Children are rewarded with a bright yellow tick and a high-pitched sound when completing a task successfully, but if they perform an inappropriate action, the software alerts this incorrect response by giving a low-pitched sound. Individualised immediate feedback is known to motivate children to stay on task, especially children with attention difficulties (e.g. Marx et al., 2018), yet this is scarcely available from teachers in a crowded classroom (see also Hirshleifer (2017) for another example of how educational technology can increase student rewards).

When used in formal education settings, such as primary schools, a remote monitoring feature within the software records when children are using the apps and logs their progress through the app content. This information is fed back to teachers, which enables teachers to direct attention to children that become halted on a particular topic and assist children who are making slow progress. Other stakeholders can access this information at a school level, so they can monitor how frequently schools are using the technology and how fast children are progressing.

The latest version of the software includes a customisation algorithm which is designed to rapidly assess a child’s skill level each time they open the app. The software then places the child at an appropriate skill level within the app content. This adaptive learning feature has been designed especially for non-formal education contexts, such as out-of-school outreach programmes, refugee settings and direct-to-community approaches for children living in remote contexts where access to formal education settings is scarce.

Whilst onebillion designs and develops the software content and features, VSO and the research team provide feedback on usability within schools, which can be used to enhance the software through an ongoing iterative redesign process. Likewise, class observations by the research team of the intervention’s delivery by teachers in schools are fed back to VSO to improve ease and fidelity of implementation.

Fig. 11.3
figure 3

Illustration of different implementation approaches of the onebillion e-Learning platform in Malawian primary schools: (a) and (b) show the main implementation approach for individualised learning within a learning centre, whereas (c) shows a whole class implementation approach using a solar-powered projector facilitated by the class teacher

Full size image

3.2 Implementation Approach

The main approach for implementing the e-Learning platform that is currently being used in Malawian primary schools involves small groups of children (typically 30–60 children depending on the size of the school) attending a 40-minute session at a learning centre, a specially designed classroom equipped with solar power to enable use throughout the day, even in remote rural regions that are off-grid (see Fig. 11.3a). Individual children use a handheld tablet connected to a set of headphones to access the interactive, child-centred apps developed by onebillion (see Fig. 11.3b). Teachers help, guide and praise children as they learn with the app. They also use a dedicated teacher tablet to run the session. One teacher, who is selected as the learning centre coordinator, is responsible for the proper functioning and effective use of the learning centre. VSO members train teachers in how to use the technology and how to lead sessions using the software. This in-service training involves two 4-hour sessions which are delivered by VSO to teachers in the school. This implementation approach has been subjected to formal evaluation and has been shown to be effective at raising learning outcomes (Pitchford, 2015; Pitchford et al., 2019). It does, however, pose some challenges for schools, as it involves small groups of children leaving their class at any one time throughout the school day to attend a session in the learning centre, which can be disruptive. In addition, with large class sizes and short school days, it is often difficult to provide all children with access to the learning centre on a regular basis.

To address these issues, VSO is currently exploring other implementation methods, such as paired learning using headphone splitters, to increase regular usage of the technology within primary schools. An alternative implementation approach, which could have potential to reach all children within a targeted ability group, involves connecting a tablet to a solar-powered projector which beams the onebillion app content to a large screen (e.g. a white sheet) attached to the classroom wall (see Fig. 11.3c). In this dynamic classroom environment, the teacher is able to guide learning of the whole class simultaneously, actively engaging children with the visual and audio content in the software. The teacher actively scaffolds learning by navigating the digital content, moving around the class and selecting individual children to complete in-app tasks, either by responding directly on the tablet or by indicating their response on the large screen. Class teachers are equipped with a rich source of high-quality learning materials and a large suite of interactive activities that they can utilise to support their pedagogical practice. However, these alternative approaches do not capitalise on the customised learning features that are key to the original e-Learning platform and, to date, have not been subjected to formal evaluation, so it is currently unknown if they will be effective at accelerating learning.

3.3 Evaluation: Research Approach

Since 2013, a research team from the University of Nottingham has been evaluating the effectiveness of the Unlocking Talent project at raising basic reading and mathematics of primary school children in Malawi and other countries, including the United Kingdom, Brazil, South Africa, Tanzania, Kenya and Ethiopia. Other independent evaluations have also taken place by Imagine Worldwide in Malawi and the Education Endowment Foundation in the United Kingdom.

This case study describes a staged and systematic approach to evaluating the effectiveness of the Unlocking Talent project in different countries, as illustrated in Fig. 11.4. This starts with piloting the technology within a particular context to assess its feasibility at raising learning outcomes and then builds evidence across increasingly large-scale randomised control trials (RCTs) which compare groups of children either learning with the customised e-Learning platform (intervention group) or learning through standard classroom instruction (control group). Performance of individual children is measured at baseline, prior to the delivery of the intervention, then again at endline, after the intervention has ceased, and differences in attainment over time are compared between groups. This measures the impact of the intervention in terms of learning gains. In addition, process evaluations are conducted to assess the fidelity of the intervention’s implementation against intended use. A process evaluation involves conducting class observations with a structured checklist, interviewing teachers and surveying school and district education managers on the drivers thought to influence the fidelity of the intervention. Drivers of implementation fidelity in this context include child-level features (e.g. engagement with task), class-level features (e.g. trained teaching staff, dedicated space, calm environment), facilitator features (e.g. technical support, pedagogical support, behaviour management) and school leadership features (e.g. support for the intervention, provision of time within a daily regime to deliver the intervention). This combination of quantitative and qualitative research methodologies provides rich contextual information about the extent to which the intervention is effective at accelerating learning and how modifications might improve its effectiveness. In scaling the project, the strength of evidence accumulates, and the scope for modifying the intervention reduces, such that by the end of a large-scale effectiveness trial, the final model of the intervention – one that will be scaled nationally – should be identified.

Fig. 11.4
figure 4

Staged and systematic research approach to evaluating the effectiveness of the customised e-Learning platform adopted by the Unlocking Talent project. A series of successfully larger RCTs helps to de-risk the intervention, effectively creating ‘checkpoints’ as the project scales

Full size image

Given the multisector approach to implementing a customised e-Learning platform within primary schools across Malawi, a holistic research approach is warranted that investigates different components of an integrated system that influences educational attainment. Key components of this integrated system include the technology (e.g. software and hardware), schools (e.g. teachers’ capacity and capabilities, school leadership, availability of time within the school day, class size), communities (e.g. parent and community attitudes and support towards the introduction of technology in primary schools) and government commendation and advocacy for the project, as well as what the child brings to the learning task (e.g. prior ability in reading and mathematics, attendance at school, development of associated cognitive and motor skills and any associated health issues that may impact propensity to learn). All of these components have potential to influence how well children engage, interact and learn with this technology-based intervention to require investigation.

This holistic approach to evaluating the Unlocking Talent project is captured in the Theory, Research, Assessment, Comparison, Evidence (TRACE) research framework, summarised in Fig. 11.5. It is grounded within the theoretical framework of developmental psychology, especially theories of reading and mathematics acquisition. It involves examining how children acquire these foundational skills with the assistance of this customised e-Learning platform, establishing which children benefit from this intervention and identifying factors that prevent or facilitate learning. The breadth of the research questions necessitates mixed-method research, involving both quantitative and qualitative methodologies, including RCTs, case-control ability-matched designs, predictor studies, questionnaires, focus groups, interviews and observations.

Fig. 11.5
figure 5

Summary of the TRACE research framework for evaluating technology-based education interventions

Full size image

Reliable and valid assessment of children’s abilities is a critical aspect of this research. Yet typically, assessment is inadequate in low-income countries, such as Malawi, due to (1) a paucity of sensitive tools, normed for a particular country, that can measure different aspects of child development and scholastic attainment; (2) a scarcity of trained evaluators that speak the local language; and (3) the poor quality of national- or school-level tests, which renders them unsuitable as outcome variables. Governments and funders often require programmes to utilise tests of attainment that allow for international comparisons to be made, such as the Early Grade Reading Assessment (EGRA; USAID, 2010) and the Early Grade Mathematics Assessment (EGMA; USAID, 2011), yet these are often crude measures of attainment as they typically span a large age range. Measures of core cognitive and motor skills that are associated with reading and mathematics attainment, which are suitable for use in the early primary years, are sparse in Malawi, to say the least.

To address this lack of suitable measures, a digital assessment was developed by the Nottingham research team that can be administered to groups of children, thus increasing efficiency in the assessment process, and has been shown to be a reliable and valid cross-cultural measure of core abilities (Pitchford & Outhwaite, 2016b). Paucity of suitable measurement tools is particularly problematic when assessing the extent of functionality in children with special educational needs and disabilities across domains that may impact their ability to interact with digital technologies. Again, to address this need, a teacher-rating scale was developed by the Nottingham research team for measuring teacher perceptions of extent of difficulty in individual children across five key areas of functioning: mobility, hearing, vision, language and learning. This enabled a quantitative measure of extent of disability; this was shown to be predictive of children’s progress in mathematics with the onebillion app (Pitchford et al., 2018).

To gain a comprehensive understanding of the potential reach of the Unlocking Talent project, different comparisons must be conducted. For example, the onebillion software covers content in reading and mathematics, so comparing learning gains (i.e. difference in performance at baseline and endline on measures of attainment) across these domains highlights weaknesses in the software at supporting acquisition of particular skills. These can then be addressed by the developers. Similarly, comparisons of learning gains between low-achieving and high-achieving children at baseline indicate the scope of the software at supporting learning for all children. In addition, profiling different groups of learners in terms of their cognitive and motor skills can provide insight into why some children struggle to learn whilst others excel at learning with this technology. This can inform whether the intervention should be targeted towards particular groups of children.

Furthermore, comparisons of learning reading and mathematics with this technology across primary school children in low-income and high-income countries, such as Malawi and the United Kingdom, enhance understanding of the universality of this customised e-Learning platform, which is important when considering where and how to scale to other countries.

Synthesising evidence from empirical research and systematic literature reviews of other technology-based learning programmes produces an evolving evidence base, which can be augmented over time. This generates new theoretical and implementation frameworks on which informed practice and policy decisions can be based.

3.4 Adaptation: Scaling Approach

Within Malawi, the scaling approach for the Unlocking Talent project currently focuses on key areas of activity.

Teacher Training

In preparation for scaling the Unlocking Talent project nationally across Malawi and fully embedding it within the primary education sector, VSO is working alongside the MoEST to incorporate use of digital education technology into the training courses delivered at government-owned teacher training colleges. This builds capacity within the teaching profession to maintain effective use of technology-based education interventions as they are introduced within primary schools and supplements the in-service teacher training that VSO delivers as part of their current implementation process.

Direct-to-Community Approaches

To increase the reach of the Unlocking Talent project to the many children in Malawi that do not attend primary schools, direct-to-community approaches are being piloted. To address the learning needs of out-of-school children, onebillion has developed ‘onetab’ – a robust, solar-charged, low-cost tablet pre-installed with their adaptive reading and mathematics software. The onetab device has been designed to work in challenging environments, with a protective bumper, bright screen, loudspeaker and strong solar-charging connector to ensure it can stand up to daily use by children in the home. The adaptive learning algorithm that onebillion has incorporated into their latest version of the software, which is installed on the onetab device, enables a direct-to-community approach, as children can access reading and mathematics instruction at an appropriate level for their ability, independently of a class teacher. This innovative adaptation extends the current implementation approach of the Unlocking Talent project within formal education settings to non-formal education settings, such as out-of-school outreach programmes and refugee camps in Malawi. It can also provide uninterrupted education during unexpected school closures, for example, in response to global pandemics such as COVID-19, as learning can continue to take place within the home.

Implementation Toolkits

As the Unlocking Talent project scales within Malawi and to other countries, it is critical to maintain a high level of implementation fidelity, to ensure that the customised e-Learning platform used in the project continues to be implemented as intended. A common consequence of scaling innovative education initiatives is ‘pollution’ of implementation, as different organisations take on delivery in other contexts and make alterations to the programme, which can lead to dilution of previously observed intervention effects (Mihalic, 2009; Samara & Clements, 2017). In an attempt to prevent wash-out and maintain high implementation fidelity of the Unlocking Project as it scales, VSO, the University of Nottingham and Imagine Worldwide have recently produced and published a series of toolkits that provide guidance to organisations on designing, planning, launching and monitoring an education technology-based project. The toolkits are designed for non-governmental organisations (NGOs) and governments that are thinking about designing and implementing tablet-based learning programmes. They are targeted at programme directors, project managers, software developers, site leaders, staff and facilitators, including teachers, parents and community leaders. The toolkits are modular, so organisations can pick and choose sections depending on their needs. To date, the toolkits have not been rigorously tested, but were designed to reinforce fidelity across implementation contexts (see also Outhwaite et al., 2019).

Localisation

The Unlocking Talent alliance is also being scaled by introducing this customised e-Learning platform to other countries that are in need of education reform. Localising the software is an important step in this process, and onebillion is responsible for this. For each new language they introduce, onebillion works with universities, linguists, authors, book editors, teachers, parents, children and pedagogy experts to ensure the software is localised appropriately to the needs of the child. Further information on how onebillion localise their software is summarised in Text Box 11.3.

Process of Localising the Onebillion Software to Different Languages

When introducing the onebillion software to different communities, it is important to localise the software to ensure it meets the language needs of children. The process adopted by onebillion to localise their software to different languages involves the following:

  1. 1.

    Partners. onebillion works with partners to discuss when to localise their software into target language(s). This is a complex task that needs to be adopted comprehensively, so it is important that partners understand what is involved.

  2. 2.

    Scoping. onebillion works with local linguistic experts to (i) build a description of the culture and language, including phonetic structure and high-frequency words; (ii) determine the order that letters should be introduced to the child in order to meet syllables and words as soon as possible; (iii) identify, create and test any new learning units needed; (iv) check the bank of words and images against the language and culture, working with graphic artists, to ensure they further the child’s understanding; and (v) creatively translate all stories, keeping them relevant and engaging for the child.

  3. 3.

    Recording. onebillion works with voice-over actors who are native speakers in the new language area, to ensure a familiar accent for the child. They record everything in-house to ensure their high standard is maintained – this involves approximately 160 stories; 185,000 words; and 32,000 audio files.

  4. 4.

    Sequencing. Course designers at onebillion sequence the course, combining existing curricula and their own experience and carefully selecting a variety of activities for the child. This is an iterative process based on feedback from children in the new context that the software is being localised for.

  5. 5.

    Quality assurance. Onebillion fully test all 4000+ units, multiple times, for functionality and language. This happens both in-house and with native speakers, in the target countries. Only then is the course ready for the child to start learning.

  6. 6.

    Tools. The localisation process is supported by onebillion’s in-house software and the structure of their software, which enable them to work with experts all over the world and to tailor the course for each new language.

Funding Model

Scaling requires funding from a diverse portfolio of donors. The funding approach is agile, with the Unlocking Talent alliance responding to opportunities as they arise, as well as being proactive in seeking funding from donors operating in countries where the alliance wishes to expand. As VSO is committed to evidence-based practice, evaluation research is cost into their implementation proposals to donors. Dedicated funding is also available for global challenges and education research from some organisations. For example, the Education Endowment Foundation funded a large-scale efficacy trial of the onebillion software with young children in the United Kingdom who were struggling to learn basic mathematics.

4 Results/Lessons Learned

Evaluation Findings

Using a mixed-method approach, as outlined above, our research team has demonstrated that children and teachers enjoy using the customised e-Learning platform with onebillion software adopted by the Unlocking Talent project and that this technology significantly raises learning outcomes compared to standard classroom practice (Pitchford, 2015). The randomised evaluation incorporated a placebo group that used the handheld tablets in the learning centre to engage with a simple design software that required similar drag and drop movements as the onebillion maths software. This group did not show significantly enhanced learning in maths above the level shown by the control group. As a result, the observed learning gains can be attributed to the software’s implementation, rather than novelty effects arising from interacting with a new technology (see Table 11.1).

Table 11.1 From Pitchford (2015): group performance (mean (SD), min–max) at pretest and post-test and percentage gain for the tests of mathematical concepts (MC; maximum score 48) and maths curriculum knowledge (CK; maximum score 50)
Full size table

Furthermore, the benefits of learning with this customised e-Learning platform are sustained even as the programme scales (Pitchford et al., 2019). When learning with the onebillion software, gains in ability have been shown to be consistent over time, across different cohorts of children, between different countries, and assessed by different groups of researchers. This demonstrates that consistent and reliable learning gains can be achieved with this education technology regardless of context. In general, this amounts to a 3+ month advantage for children learning basic mathematics and a 4+ month advantage for children learning basic reading with the onebillion software compared to standard classroom instruction (Education Endowment Foundation, 2019; Imagine Worldwide, 2020; Outhwaite et al., 2020; Pitchford et al., 2019).

Our research in Malawi, in particular, shows that after interacting with the onebillion software on a daily basis at primary school for eight consecutive weeks, additional benefits are observed: children’s attentional skills improved, enabling them to concentrate better in class (Pitchford & Outhwaite, 2019). Importantly, as shown in Table 11.2, girls learn just as well as boys with this technology, and when implemented at the start of primary education, the Unlocking Talent intervention can prevent gender-based attainment gaps in mathematics that typically emerge over the first year of schooling in Malawi through standard classroom instruction (Pitchford et al., 2019).

Table 11.2 From Pitchford et al. (2019): learning gains (mean %) and effect sizes (Cohen’s d) for boys and girls in mathematics (EGMA) and reading (EGRA) after intervention with the customised onebillion e-Learning platform compared to standard practice. For mathematics, standard practice promotes a gender advantage for boys over girls, whereas girls learn just as well as boys with the technology-based intervention
Full size table

The ease of use and flexible structure of this customised e-Learning platform make it suitable for all children, including those with special educational needs and disabilities. We have shown that a group of children attending a special needs unit attached to two Malawian state primary schools, who presented with a diverse range of difficulties across the group, could interact and engage with the onebillion software and made progress through the app, albeit at a slower pace than mainstream peers (Pitchford et al., 2018). However, extent of disability, as measured by teacher ratings, correlated negatively with progress through the app, indicating that children with profound difficulties struggled to learn with this technology. Our research suggests that whilst customised e-Learning platforms might be suitable for some children with mild-to-moderate special educational needs and disabilities, children with profound difficulties require a different type of educational support.

Similar learning gains with this customised e-Learning platform have also been shown in the United Kingdom, demonstrating this technology is equally effective across countries differing vastly in income status and primary education provision (Education Endowment Foundation, 2019; Outhwaite et al., 2017; Outhwaite et al., 2018; Pitchford & Outhwaite, 2016a). Furthermore, our research in UK primary schools has highlighted the importance of implementation in securing improved learning outcomes for basic mathematics with the onebillion app (Outhwaite et al., 2019). Qualitative research conducted through a process evaluation of using the onebillion software showed that when schools established a consistent routine for embedding the apps into their daily classroom schedule, children learnt more basic mathematics than when schools varied the implementation routine. Results showed that ‘established routine’ predicted 41% of the variance in children’s learning outcomes with the apps. Established routine involved schools implementing the intervention at a consistent time each day, having a dedicated member of staff whose responsibility it was to implement the intervention, having well-organised equipment (e.g. colour coding the tablet devices so that they were easily identifiable by children) and having a dedicated space within the classroom and having a seating plan where children used the onebillion apps. In another study, the effectiveness of the onebillion software in supporting acquisition of basic mathematics by early-grade children attending a bilingual immersion school in Brazil was evaluated, and proficiency in language of instruction was shown to be positively associated with progress through the apps (Outhwaite et al., 2020). These findings are important as they show that factors related to the implementation of this intervention in primary schools are critical for securing improved learning outcomes – seemingly, the customised e-Learning platform and interactive apps alone are not the only factors driving success.

Overall, this emerging evidence base, built upon a robust scientific methodology and psychological theories of reading and mathematics development, has generated confidence that the Unlocking Talent project can be implemented successfully at scale in Malawi. The solution significantly raises attainment of core foundational skills, and it can address some of the structural inequalities that are inherent to the country’s education system.

Challenges and benefits of working with partners.

Global challenges, such as providing quality education for all, require genuine cooperation and collaboration from multiple sectors to enable policy-makers to make informed decisions on the best available evidence. This is a difficult landscape to navigate and one that a traditional academic career does not necessarily prepare you for. Working with partners across a range of sectors has its challenges, but these are often offset by the benefits it affords. External partners can bring insights into the research design and process, based on their wealth of knowledge from working on the ground. This adds value to the quality of the research and provides a direct pathway to apply the research findings to people that matter.

When initiating global challenges research, it is useful to map who the partner organisations are at the outset of the design process (e.g. Fig. 11.1) as this will be critical to the sustained success of the project. Partnerships take time to build and require constant nurturing to maintain engagement and foster trust. This is easier to achieve when partners share a mutual goal and appreciation of the varying demands that different organisations may have, which at times might clash with what you are trying to achieve. If difficulties arise, it is best to focus on the overarching goal you and your partners are trying to achieve and move towards a way of working that will facilitate you reaching this.

Working in partnership with other sectors also requires a flexible and fast way of operating which is not aligned to the academic year. This is especially relevant to technology-based fields, where adaptations and innovations are rapid and frequently precede the evidence base. There is often an urgency in addressing global challenges. Yet, despite the time it takes to accrue a strong evidence base, we need to know that what we are doing is going to be effective and of benefit to the end users – without incurring harm. Forging ahead with adaptations and innovations to a product that has preliminary evidence of being effective, before developing a full understanding of factors that determine its effectiveness, can result in product developers altering the components that make it a success.

In addition, the language used by different groups of partners can often hold distinct meanings, so clarity of expression is key to successful communication. An example is the term ‘significance’. When researchers use this term, they are referring to statistical significance – the probability that a given result occurs by chance – however, partners from other sectors frequently use this term to imply importance. This can change the interpretation of research results when partners are communicating research findings, so it is important to engender a mutual understanding of shared terminology.

Addressing global challenges also requires large-scale investment. Academics that are new to this field may underestimate the resources needed to be able to conduct high-quality research at a level that will have global impact. It is also a competitive field in which the funding structures supporting this work are often short-lived, requiring a quick turnaround of activities, which does not promote long-term understanding of the impact the work can have. Funding opportunities that are supported through the government can change rapidly in a volatile political environment. This makes planning for long-term projects difficult. Whilst agile funding streams can address global challenges as they happen, such as in response to the COVID-19 pandemic, it can make it difficult to sustain funding for large-scale projects that require long-term impact evaluations, such as the effectiveness of technology-based education programmes on improving learning outcomes.

Despite the challenges of working in this field, the rewards can substantially outweigh the difficulties. Working collaboratively with the Unlocking Talent alliance towards a shared goal and working with humility are what makes for long-term sustainable change.

Implications for practice and policy. To support future work by practitioners, education software developers and policy-makers, here, we provide several key implications of the implementation and evaluations of the Unlocking Talent project in Malawi and other countries.

Practitioners

  • Schools should look for customised e-Learning platforms that are localised to the language of instruction, are easy to use and have a flexible structure, as these are accessible to learners of varying abilities, even children with special educational needs and disabilities.

  • For children with special educational needs and disabilities, teachers and parents should assess the extent of disability before using interactive educational apps, as this type of technology might not be suitable for children with profound difficulties.

  • When implementing customised e-Learning platforms within formal education settings, teachers should maintain a well-established classroom routine to optimise effectiveness.

  • School leadership should monitor implementation fidelity of customised e-Learning platforms within classrooms on a regular basis, to ensure teachers are using the technology as intended.

  • Teachers and parents wishing to use interactive apps with their children should consider children’s proficiency in the languages spoken within the software, as children need to have a sufficient language proficiency to access curriculum content and to respond to instruction given in educational apps. Look for apps that have been carefully localised to the language of instruction.

Software Developers

  • Software developers should consider accessibility of their apps for different groups of children, especially those with special educational needs and disabilities, and build in adaptations to maximise accessibility for all learners.

  • Software developers need to consider the role of children’s language development in their app design features and content and ensure that the language used in their apps is appropriate for the age of the intended users.

  • Software developers should consider adapting their technology on the basis of user feedback and research findings to increase uptake and improve quality of their products.

  • Building assessment features into software could facilitate monitoring of children’s progress by teachers and researchers. This might involve in-app assessment of skills to be taught in a learning activity prior to app-based instruction, followed by an end-of-activity in-app quiz which requires children to apply the knowledge learnt to new materials. Progress charts built into software can display this graphically over time, providing information on the rate of learning by individual children.

  • Monitoring of time on task, number of correct and incorrect responses per learning activity, sequence of responses per learning activity and time to complete a learning activity can also be valuable information to teachers and researchers; developers can consider how to build these metrics into their software. This instrumentation can provide rich information as to how children are interacting with the software content.

Policy-Makers

  • Governments should consider introducing customised e-Learning platforms that encompass features of universal design into their primary education sector, as this technology has been shown to significantly raise learning outcomes compared to standard classroom practice across a range of low-, medium-, and high-income contexts.

  • Governments considering introducing customised e-Learning platforms within their education system should assess the country’s readiness to implement this technology effectively, including scoping the infrastructure, teacher capacity and capabilities and availability of time within the school day required to support successful implementation.

  • When introducing e-Learning platforms within an education system with the view to scale, key partners (especially within government) need to be engaged from the outset to ensure sustainability, cultural sensitivity and accountability. Maintaining budget lines for e-Learning within the education budget will also foster commitment to long-term sustainability.

  • Interactive apps that have been shown to mitigate gender differences should be implemented at the start of formal education to enable girls and boys to learn at a similar rate before gender disparities become entrenched.

  • Governments could consider introducing customised e-Learning platforms direct to communities, to reach children who cannot attend school and to provide continued education during unexpected periods of school closures, such as in response to the COVID-19 pandemic.

  • Governments, schools and parents should seek professional guidance in choosing customised e-Learning platforms to improve children’s learning outcomes, to ensure selected technologies are supported by a rigorous scientific evidence base demonstrating effectiveness.

Discussion Questions

  1. 1.

    Within limited funds, which children should be prioritised to receive the customised e-Learning platform currently being used by the Unlocking Talent project in Malawi? Provide a rationale for your answer.

  2. 2.

    How might the e-Learning platform currently used by the Unlocking Talent project in Malawi be adapted to enable more children to access the technology on a regular basis?

  3. 3.

    What adaptations could be made to e-Learning platforms to increase accessibility and learning for children with special educational needs and disabilities?

  4. 4.

    Which factors might influence learning outcomes with customised e-Learning platforms when implemented outside the school system, such as through direct-to-community approaches? How might these be addressed through policy, practice or technological advances?

  5. 5.

    How might other educational technologies, such as radio or TV programmes, be used in low-income contexts to provide high-quality education for children not attending school? What features would need to be incorporated into these types of technologies to maximise learning in non-formal education settings?