Over the last decade, I have approached the claimed potential of technology as a way of improving access to quality education through the lenses of both an international development professional and researcher. Throughout, I have observed an odd contrast between the continuous multiplication of EdTech projects, which are presented as a necessary step in a process that will eventually lead to scaled solutions, and the lack of solutions that actually scale. Intrigued, in 2017, I decided to focus on this issue in one of the studies conducted as part of my PhD research, conducting 19 interviews with EdTech stakeholders. That study shed light on a range of issues already observed by critics of the project-based approach and confirmed that multiple dysfunctions prevent the scale-up of EdTech solutions and, hence, any structural improvement to education systems in developing countries (von Lautz-Cauzanet, 2018).

Three years later, when the global pandemic hit all education systems so hard, the failure was highlighted: 15 years of “promising” EdTech projects may have been locally useful but had not transformed systems. Still intrigued—but this time by a Covid-19 debate that seemed to sweep this inconvenient truth under the carpet— I turned to my network, to stakeholders who fund, design, implement, evaluate EdTech or work within profit and non-profit EdTech companies in developing countries. In total, I conducted another 17 unstructured interviews. While I had asked them in 2017 to share their perspective on the EdTech approach, the 2020 interviews were guided by one overarching question: “Do you see any signs that the project-based approach is finally changing?”

Unfortunately, they don't.

In this viewpoint article, I highlight three long-standing issues perceived as most pressing by the actors involved in project-based EdTech initiatives. Their voices allow us to grasp the magnitude of the projects’ dysfunctions, their origins, and their consequences. Most importantly, they confirm that we need to reform the project-based EdTech approach if we want it to contribute to what education systems need now more than ever: resilience.

Issue 1: The project-based approach is not scalable

EdTech pilots are presented as a nec plus ultra step toward scaled solutions while also making it impossible for those projects to fulfill that very mission.

Short time frames generate incoherent indicators and reduce ownership…

If stakeholders could pick just one issue, it would probably be short time frames. Project durations hardly exceed 36 months and are systematically perceived as far too short. The induced pressure makes collaboration difficult among all involved partners and, most importantly, has a negative impact on the quality of implemented activities. Stakeholders describe how they end up struggling to make the project work, and not only spend money and justify budgets; they feel they are evaluated more for their capacity to respect disbursement timelines than for the outcomes of their work. Because of the perversity of the system, they experience significant frustration and disbelief in the effectiveness of their work. The perversity is that an EdTech project approach incarnates a fundamental contradiction: the attempt to observe systemic educational outcomes over a short-term period—an impossible endeavor, given that education is a long-term project. Stakeholders are ultimately forced to focus on criteria related to what can be done during the life of the project and openly question the usefulness of their (success) criteria; for many, these criteria are chosen because they are achievable and can be presented as success once the project is over.

Indeed, projects are set up from the beginning in a way that makes it difficult to adopt a future-oriented perspective—a perspective that is necessary for the design of scaled solutions. Ex-post-facto studies are rarely organized because they are rarely envisaged by project leaders (and their donors). This lack of follow-up is also the result of a lack of ownership among local partners, an often-observed characteristic of short-term projects. As WHO indicated in 2007, “scaling up is an institution building process, it takes time” (Simmons et al., 2007). Unfortunately, this time is not made available. During the project period, local partners are, like everyone else, under pressure to hand in deliverables and quarterly reports and meet deadlines and tight time frames. All interviewees agreed that they are part of an environment that is not set up to nurture local ownership.

As a result, the aftermath of a project is often terra incognita: Despite being crucial for scaling, information on sustainable and unexpected education outcomes of projects remains mostly unknown.

What can be observed, however, is a sort of never-ending dead-end process: the multiplication of “one-shot” projects that never leads to the ideal of a scaled solution capable of improving systemic access to quality education.

. . . making it impossible to design and test scalable EdTech solutions

When criticizing the short time frames of EdTech project approaches, stakeholders systematically describe what they call the scalability discourse and how it stands in full contradiction to the way that projects are actually set up. They typically state that it’s not about the potential for doing something at scale, not about putting in place the mechanism for doing something at large scale but about what you can you do in a year, or two years, or maybe three if you are fortunate and have a patient funder.

They find themselves in a dilemma, forced to adopt scalability as a stated project goal despite being aware that it is impossible to achieve. This is particularly true for EdTech companies, as they are the project partner perceived as the innovator. More than any other players in the EdTech project sphere, such companies need to create a narrative for funders that shows how their product could possibly scale—which they describe as almost unachievable.

Indeed, it’s important to recall that the development of an EdTech solution is accompanied by a process composed of multiple components, including a process of user adoption and appropriation. Those steps take time to develop and be evaluated—and that time cannot be compressed. As the following statement shows, stakeholders seem to have time to ask a question but not to find an answer: “We had mixed forms of adoption . . . and we need time to get to [the] stage where teachers trust and use the resource enough [to see] the impact, the results: Do we actually see student results going up? Do we see improved conceptual understanding? We had no time to observe this”.

The combination of small sample sizes and short time frames during which practices can be analyzed further increases the risk of generating poor data. Stakeholders end up with suggestive indications that it is working but without a robust data set to make predictions.

Consequently, when projects terminate, although short-term criteria might well have been met, the generated data sets do not allow predictions to be made about sustainable practices, let alone scaled scenarios. A stakeholder described this stage of their solution as an insufficient middle ground that allows for little more than a cynical “Hey, it looks like it’s working!”—not exactly the starting point for evidence-based policy-making.

. . . and then Covid made things worse.

The 2017 interviews highlighted discrepancies between the discourse of alleged goals and actual setups of EdTech projects. The 2020 interviews revealed that those discrepancies have even worsened since the outbreak of the pandemic: Stakeholders reject the discourse that the crisis provides a possible opportunity for large-scale EdTech solutions, and they disagree with its presentation as a fertile ground for innovation. More than ever, this conflicts with their experience on the ground since the outbreak of the pandemic. For them, the pandemic has made it even more difficult to get to that stage where coherent large-scale scenarios can be designed. As one stakeholder put it, “Everyone was like ‘Oh, EdTech, you must be doing so well now’. Yes, in theory. [but covid prevented us from] generating the impact, the indicators we needed, those indicators we would normally have had . . . ”

Indeed, because of school closures and lockdowns, most projects and EdTechs had to stop or adapt their activities, which delayed implementation and hence also evaluation activities, and this in turn further affected the quantity and quality of data sets.

As the crisis seems to have accentuated the overall short-term approach, EdTechs have found themselves penalized, unable to deliver valid data on scalable solutions. As a result, the same EdTechs that for years had been presented as promising the large-scale solutions of the future were suddenly de-prioritized. Indeed, the crisis triggered interest in technologies such as radio or even WhatsApp and—which appears noteworthy—in technologies that are already scaled. For new EdTech solutions this was, however, bad news. A stakeholder remembers those days: “[When school closed] people were like: ‘Well let’s do something, let’s do anything, what can we do?!’ It wasn’t any more about testing. It was about doing. It was about utilizing what’s already there”.

As in every crisis, the need for ad hoc solutions to contain the damage is an understandable reaction.

In this case, however, it came with a worrying side-effect: the strong focus on short-term solutions and the decision to focus on technologies that have already been tested or widely adopted seem to have led to a more conservative attitude toward innovation in general. And most importantly, it has reduced even more the willingness to set up an EdTech project-based approach capable of design-testing new, large-scale, Ed Tech solutions. As one interviewee noted, “in practice, however, they [donors, governmental actors] all pulled back in their openness to new things. It’s a horrible paradox”.

You can’t just scale project resources and behavior

When stakeholders criticized EdTech approaches for creating non-scalable projects, they were referring to the obligatory counterparts of short time frames and the allocation of extraordinary technical, financial, and human resources. These factors ensure that short-term criteria can be met—which can certainly contribute to useful results for the target group in the short term—but they are rarely valid for large-scale context. In 2017, a donor representative described a situation that doesn’t seem to have changed ever since:

A pilot is always [a] success, let’s be honest . . . because you make sure you do the necessary to succeed: actors will be mobilized, content will be ready on time, tools are delivered, teachers take part in the training, drop-out is low, and at the end, the evaluations as well as the external evaluations will be positive—all this . . . happened [in our case]. You put the necessary money on the table. Sure, money doesn’t explain all, but still; and if you have issues, you’ll make the experts come . . . all this, you won’t have in real life . . .

Overall, the project-based approach also comes with its very own kind of incentives and motivations that contribute to the short-term success of a project but disappear once the project ends because you can’t easily clone people or people incentives, motivations, and capacities. In addition, the impacts generated by projects are representative of a specific context and depend on socio-economic and technical factors that would differ in scale scenarios. As a result, the data sets generated in such contexts are of limited use when it comes to predicting those scaled scenarios.

Furthermore, it is important to emphasize that systemically scaling a technology-based solution means transforming the very nature and output of that technology. Stakeholders pointed out that projects do not just involve the technology solution and noted that there were two elephants in the room. First, the cost structures of EdTech projects are entirely different in scaled scenarios. In the latter, expenses are often much higher than local partners or donors have so far (been willing to?) invest—capex and opex costs (capital and operational expenses) are the most evident issues. Stakeholders criticized a form of denial, especially when it comes to technology brought from Western markets and post-colonialist attitudes driven by technical capabilities (and not by regional market realities). They stressed the need to adopt throughout a new approach, combining participatory, bottom-up, and locally led approaches. As a senior stakeholder summarized it precisely, “Instead of going to Africa and saying: ‘I think I have the one solution that is going to change dramatically the life of millions of students and teachers’, we have to ask them: ‘What kind of project do you need and how can we support this here, now and with whom?”’

Secondly, the very assumption that educational behavior can be scaled through EdTech was perceived as utopian. Stakeholders insisted that making generalizable predictions and conclusions from behavior observed in EdTech project environments is risky as oversimplifies a highly complex process. They felt uncomfortable with the idea of scaling because it implies a one-size-fits- all approach. For some, this has even led to the conviction that the goal of scalability should not be pursued at all by EdTech projects. A stakeholder said:

Ten or 15 years ago, I was convinced that scalability should be part of the [EdTech] conversation [ . . . ]. Now, looking back, I realize that scalability should be banned from our conversation. Because in order to have something to scale, you have to make sure you have the proof that it’s a solution that works. But in the case of teaching, for example, it’s made of hundreds of little activities and operations. It’s very difficult to know and then to scale all this, very difficult to scale it to one solution.

Indeed, the current setup of project-based approaches does not sufficiently take into consideration how difficult it is to distinguish the impact of the technology factor on outcomes such as students’ results. For many interviewees, this has directly contributed to the void of systemic solutions that education systems needed to face the crisis.

Issue 2: The project-based approach is a political instrument

Despite the well-known organizational and conceptual problems, why do projects continue to multiply—paradoxically, even more since scalability has become a trend? The answer can be found by looking at projects as political instruments rather than as tools to enhance system resilience. Indeed, while the project-based approach doesn’t contribute to scalability as a goal, it instrumentalizes it as attractive branding. As Pablo Yanguas put it, “What you see is not what you get” (Yanguas, 2018).

Scalability is a buzzword—and a branding that leads to project multiplication

The actual power of the scalability branding becomes evident when stakeholders explain that projects labeled as pilots get a lot of attention because of the implication that the project may lead to a scaled solution. A stakeholder admits openly: “Calling something merely a project suggests a lack of ambition. People won’t get as excited about it as they would if you say it’s a pilot”.

Given their importance in the aid eco-system, donor organizations certainly contribute to the perpetuation of the scalability discourse. However, scalability branding is attractive for all involved parties, especially when it comes to EdTech. In addition to the allocation of extraordinary resources, which ensure that those projects will never fail, scalability branding adds a progressive and innovative image to a project. This type of instrumentalization comes with multiple benefits, such as facilitated access to funding. The cherry on top? The use of scalability as branding requires no significant political commitment. Because the project-based approach focuses on short-term outcomes, it will not be accountable for undesirable outcomes—or for no outcomes--in the long term. Concretely, stakeholders’ individual careers are not at stake. As one put it, “Heads roll when people make a commitment to something and they get it wrong—but you don’t have that in a project”.

The attractive risk-benefit ratio of scalable EdTech projects makes it quite clear why projects continue to multiply with little chance of ever being followed up. Everyone seems to “have an interest in multiplying pilots, because everybody gets a piece of the cake”, commented one stakeholder already in 2017. “It generates per diems. [They] know to make sure that everybody gets a bit. The projects pile up on each other. . . . Will this form a substrate on which ideas will grow, or will it be a cesspool that sticks on the shoes . . . I don’t have an answer”.

For some interviewees, scaling is simply not in the interests of an eco-system whose actors benefit from the scalability branding. The following statement from one interviewee describes this paradox neatly: “It’s cynical but . . . they don’t have any interest that one particular approach [tested in a pilot] ends up being imposed [scaled], because that would limit the funding possibilities, the missions. And that, that’s an obstacle. We have seen that. It’s a strategy”.

The power of the scalability discourse and the difficulty of questioning is palpable in stakeholders’ responses both in 2017 and 2020: Obviously torn between scalability as an overall goal in the development policy sphere and their own experience, many apologize that they are sorry for being so critical and qualify their opinion as “unpopular” when criticizing the project approach as an instrument for achieving systemic resilience.

. . . and creates barriers to systemic solutions.

To grasp the extent to which the instrumentalization of EdTech projects and the project-based approach is an issue, a look at its consequences is helpful. All stakeholders describe an ecosystem where there is insufficient communication. One stakeholder said, “Everybody seems to work on their own and regret how this lack of consultation creates projects that are not complementary in terms of content and organization: ‘The content isn’t the same! The modalities aren’t the same! There is no synergy at all…’”

Although one might imagine a complementary network of projects and solutions that, combined, might generate scaled solutions, projects in fact hinder each other during implementation and even afterward—and thus create an obstacle to the design of large-scale solutions. Indeed, the multiplication of projects comes with a multiplication of data sets that are often inaccessible, not shared, and most of all, not combinable. In parallel, each project mobilizes financial and human resources that would be needed for the design of large-scale scenarios.

In turn, this lack of alignment in terms of design, implementation, and evaluation fails to create the evidence that education systems need to develop sustainable large-scale solutions that would allow them to increase their resilience. From a development policy perspective, those issues indicate that although the project-based approach can certainly be locally very effective, it is an obstacle to the efficient design of large-scale, long-term solutions.

Finally, by raising awareness of how projects are used for political instrumentalization, interviewees also raise an ethical issue: By facilitating instrumentalization, the project-based approach contributes to the instrumentalization of the project beneficiaries. While those beneficiaries certainly benefit in many cases from projects in the short term, stakeholders insist that we have to question our perspective when it comes to the role of target groups: Projects often still correspond to our own needs or expectations; rather than helping the target group to develop themselves, we are using them to reach our own objectives.

Concretely, this instrumentalization further enhances the risk of introducing technologies as a response to no one’s problems.

Issue 3: International structural impediments weaken the project-based approach

In addition to the issues on the micro level, i.e., pertaining to the setup of projects and the negative outcomes of their instrumentalization, there are also problems on the macro level that explain why EdTech projects struggle to contribute to systemic solution design. Those problems are the structural impediments of the global aid environment in which projects are embedded. In this environment, the multiplication of donor organizations involved in the setup of EdTech projects has certainly led to the multiplication of information, resources, and, most importantly, funding. However, this multiplication has come with the price of complexity, bureaucracy, and lack of transparency. As one stakeholder described it, it is an environment composed of “so many systems, so many donors”, which is even more destabilizing as that environment determines their very existence.

Stakeholders described how EdTechs go out of business because people do not know how to navigate the systems and noted how important it is to be part of international networks. Even for well-established EdTechs, this dependency is an issue. One interviewee reported that “we have an international founder and a leadership board deeply embedded in the international system . . . so we have these connections . . . but it’s really complicated.”

Interviewee statements such as “We always need to ask what are the demands of our funding?” or “We have to see if we have the capacity now . . . if partners need a lot of reporting, and managing and things like that” . . . are typical of many of the interviews conducted in 2017 and 2020. They shed light once again on a paradoxical situation: Projects depend on a system that uses scalability as an alleged goal but whose processes actually impede it.

. . . and the instability of local partners lowers the chances of EdTech projects to scale even further.

Within this ecosystem, national governments and their ministries are perceived by all stakeholders as indispensable for the implementation of scaled EdTech solutions. In the meantime, they are also perceived as key obstacles to the latter as local partners are not guided by a coherent governmental EdTech strategy. For EdTechs already affected by the dependency on external organizations, the situation becomes almost Kafkaesque when it comes to working with the governments—because there is not one government, as this CEO neatly describes:

A quite high-ranking official said in a meeting that I was once in: “You appear to speak about government as though it is one thing”. That line reveals the underlying problem of why, despite all the promise, we don’t see governments picking up the projects—because you are not dealing with a single organization. You are dealing with multiple offices. Each office has multiple individuals, each individual has their own agenda. They are all trying to balance their own agenda with the office’s agenda, with the department’s agenda and it becomes this very disconnected process . . . . You can bounce from one office to the other for years, convincing, genuinely convincing people again and again, but nothing happens . . .

Even when a partnership for a local project is concluded, this lack of stable human resources within the ministry is systematically reported as an issue—not only for EdTechs but for all actors involved in the setup of a project. All regret that there are rarely governmental actors who can overview the project and its outcomes for a longer period. This weakness and instability of human resources affects the likelihood that a project will develop further and scale, or at least inform, other projects. As summarized by one interviewee, “The problem with a project that relies on only one person within the institution is that all disappears when the person leaves”’.

As with international funding, engaging in partnerships with governments comes with processes that are considered as highly resource-intensive and, hence, risky:

You need to be set up to compete under the government procurement processes and for small nonprofits or companies it is oftentimes hard to do that. You’re not like Microsoft, or Google or HP . . . If you are a small Nigerian NGO and you’re trying to get funding and you want to compete for a big government contract and you don’t win that—what do you do? Like you’re in trouble, maybe you’re out of business. Because you have to bend so much to get things in place to get people to meet the requirement of that procurement process . . . while Microsoft can just go on . . . they can go to Senegal. They have structures and teams in place to pursue the type of financing they would need to do something at scale.

Still, EdTechs are dependent on governments because there are hardly any alternatives. In Sub-Saharan Africa, the tech scene is perceived as just emerging, and—unlike in India, for example—there is no possibility of partnering with a large private school network or tapping solely into initial demand as a source of organic growth. This lack of alternatives forced one stakeholder to conclude that when it comes to education, if you are a nonprofit or a company and you want to scale and “sell” your product in Africa, at some point, you will have to sell it to one buyer: the Ministry of Education. Ultimately, this combination of financial dependency, reinforced by a poor EdTech sphere and low private demand, affects the impact of the multiple small EdTech projects in the short term, while making it difficult to picture a large-scale solution.

To conclude, it appears that the EdTech project approach favors the setup of EdTech projects and solutions that are per design unscalable, driven by a utopian perception of scalability, and instrumentalized in the name of a goal that is de facto only a branding. Despite the mobilization of tremendous human, technical, and financial resources, the current project-based approach supports the use of technologies that can—at their best—be very useful but which are neither cost-efficient nor system-transformative.

Made fragile by dependency on complex and resource-intensive aid ecosystems and having to deal with unstructured, nonpredictable governmental partners, current EdTech projects have little chance of contributing to or becoming part of systemic solutions that are needed now more than ever in order to allow education systems to “build back better”.

Just as it is crucial to be aware of this reality, it is crucial to be aware of existing solutions. Indeed, the same voices pointing out current issues also indicate that there are approaches we can learn from to design pathways towards scaled EdTech solutions. I shall outline those approaches in a subsequent article.