Introduction

The increasingly widespread use of education technology (EdTech) is forcing education to confront a range of sociotechnical issues. These include increasing inequalities, monopolistic business models, human–machine uncertainties through the use of artificial intelligence, climate impacts, exploitative practices relating to labour and natural resources, and searches for EdTech alternatives (Selwyn et al. 2020). Relatedly, emerging modes and applications of datafication and artificial intelligence across society are testing existing limit points of traditional approaches to research, highlighting the need for critical approaches to researching sociotechnical issues that bring to the surface ‘new problems for interrogation’ (Williamson et al. 2019: 87). The argument of this paper is that the complexity of socio-technical inquiry cannot be addressed by disciplines, sectors, or communities working in isolation from one another. Considering emergent challenges and research possibilities, this paper responds to the challenge of designing opportunities for ‘practitioners, researchers, and others impacted by sociotechnical systems … to design futures and think about how to design futures that matter to them’ (Macgilchrist et al. 2023: Para. 2). As such, this paper is located as part of postdigital inquiry invites generative discovery that ‘creates ideas, ways of thinking, new methods, conversations, conclusions, and connections (between groups of people; areas of thought; technologies, people, and wider systems and ecosystems)’ (Fawns et al. 2023: Para. 25). These rapidly changing and uncertain conditions of both education and research inform our paper’s key ambition: to think through how diverse education stakeholders might work within emergent challenges and their problematics to explore possibilities for democratisation.

To take up this call, we draw upon the design notion of ‘prototypes’. Prototypes are commonly viewed as central to the innovation process, with early phases focused upon either pushing a specific goal forward or investigating new possibilities (Yu et al. 2017). Prototyping is increasingly utilised across the early stages of new product development to better understand how materials, techniques, and tools enable testing and decision-making (Mathias et al. 2018). While prototyping has often been deployed for a range of design, engineering, and economic purposes, they can serve as ‘creative tools’ for problematising accepted ways of thinking and acting beyond disciplinary boundaries through inspiring ideas, demonstrating problems, and testing solutions (Gengnagel et al. 2016: 1). An example of a technological prototype that has sparked worldwide controversy is OpenAI’s ‘prototype AI chatbot’, which was released in late 2022 while still in its ‘incubation phase’ (Johnson 2022). ChatGPT is a general-purpose AI model that exemplifies a paradigm shift across ‘hyper-industrial society’ (Stiegler 2016): a globally networked technical system producing and exploiting digital traces across everyday life. The debates about ChatGPT and its implications for the future of student learning, teacher autonomy, and institutional change are often driven by a concern about the untested assumptions and impacts of these types of prototypes. Our ambition is to consider how ‘prototyping’ might be applied otherwise, outside of the big tech processes. We are mindful of Lyotard’s (1988) and Ranciere’s (2011) views that to co-produce spaces of democratisation is to create spaces where dissensus is sustained through the collaborative approaches, insights, and orientations applied to messy problems.

Our interest in prototyping is based on research into the need for new collaborative methods to engage with multi-faceted problems in education. We have been undertaking this work through experiments testing the limits and opportunities of technical democracy (Callon et al. 2009), a way of responding to ‘sociotechnical controversies’ that are increasingly evident in technological societies. These ‘controversies that unfold… are fostered not only by scientific and technical uncertainties but also by social uncertainties’ (Callon et al. 2009: 26). Methodologically, socio-technical controversies are ‘powerful and original apparatuses’ (Callon et al. 2009: 35) for collective learning and experimentation. Due to the complexity and uncertainty of such controversies, that exceed the capacity of one knowledge tradition to adequately encompass, socio-technical controversies require methods open to ‘new knowledge and points of view’ (Callon et al. 2009: 191). Prototyping presents one new method through which to advance this shared uncertainty (Thompson et al. 2023), and it offers opportunities to generate moments of fragile democratisation that radically shift beyond commercially driven product development.

The relationship between prototyping and socio-technical controversies in education informs the structure of this paper. First, we outline co-evolving technologies and methodologies informed by synergies between postdigital and technical democracy approaches. We then describe an emergent thought collective, a group of researchers and affiliates, united by a common interest in technical democracy and diverse modes of inquiry. This is followed by a design experiment that outlines the way education prototyping is a methodological device, which we test in the context of a research project exploring automated essay scoring in Australian schools.

Co-evolving Technologies and Methodologies

Artificial intelligence (AI) systems and associated technologies are growing in use across primary, secondary, and higher education sectors which both imbue and re-shape the boundaries of existing methodologies. While the definition of AI is broad and often heatedly debated, ‘the central technical concept in AI is that of an agent—an entity that perceives and acts’ (Russell 2021: 509) and can impact the environment in which it is used. In education, AI commonly includes autonomous or semi-autonomous computer systems that employ algorithmic processes. AI technologies are developed specifically for education, such as focusing on interacting and responding to learners (these approaches have been developed as part of the field of AIEd). AI is also developed outside of education, but applied within education, such as business intelligence systems. Much use of AI in education involves proprietary education technology (EdTech) products that are being rapidly introduced into schools, universities, and education systems (Gulson et al. 2022a, b, c).

All of this poses significant challenges notably: how can we collectively take stock of what is at stake with increasingly powerful and commercialised technologies in education? We propose that there are synergies between postdigital and technical democracy approaches that can articulate and coordinate joint knowledge work that is uncertain, transgressive, co-produced, and inventive. For instance, Jandrić et al. (2018a: 895) define the postdigital as follows: ‘messy; unpredictable; digital and analogue; technological and non-technological; biological and informational. The postdigital is both a rupture in our existing theories and their continuation’. It is this uncertainty, multifacetedness, and emergence that calls for new methods to engage in collective learning to ‘expand the scope of educational research, beyond a tendency to understand technology in terms of tangible devices and gadgets, towards a broader understanding of the socio-technical systems within which the project of education is constituted’ (Knox 2019: 368). Technical democracy can enable this, with its focus upon socio-technical controversies that ‘help to reveal events that were initially isolated and difficult to see, because they bring forward groups that consider themselves involved by the overflows that they help to identify’ (Callon et al. 2009: 28). In doing so, actors, problems, and solutions are exposed as part of a ‘triple inventory’ encouraging the ‘enrichment and transformation of the initial projects and stakes’ via reformulating problems, discussing technical options, and redefining objectives (Callon et al. 2009:32). In the following, we list the key characteristics of a methodology for investigating evolving technologies in education. This underpins our technical democracy approach that utilises prototyping as a means to co-produce collective learning and experimentation.

A Multiscalar Lens to Explore Issues and Concerns Such as Socio-Technical Controversies

The first concerns the scope or scale of the inquiry. Sociotechnical controversies are necessarily multiscalar, insofar as their boundaries are diffuse and their constituent issues cannot be contained to a single zone of inquiry. This multiscalar lens can enhance postdigital research, which stipulates the necessity of an ‘on-yet-around’ focus:

[R]researchers do not need a fixed focus on a specific technology or practice but a moveable gaze of inquiry, which must start somewhere, but then move around as they trace relations, and zoom in and out, from fine-grained features and micro-level activity, to broader assemblages and contexts. This movement involves navigating a tension between focusing on a technology to appreciate its uses and effects and focusing on relations within and across wider systems or eco- systems or assemblages, of which technologies are only a part. (Fawns et al. 2023: Para. 43)

For example, with the use of automated decision-making systems in education on the rise and the expertise and perspectives of those impacted often still marginalised, a technical democracy approach in education has the potential to broaden expertise and action about a range of socio-technical controversies in education, such as automated decision-making and scoring systems (Thompson et al. 2023; Gulson et al. 2022b), which have individual, institutional, social, and political impacts.

A Transdisciplinary Approach Encompassing Empirical and Hypothetical Work Aligned with Higher-Level Goals or Values

As outlined above, the complexity of sociotechnical controversies means that one research and/or disciplinary approach is insufficient. Transdisciplinarity, characterised by systemic, higher-level goals or values that transgress, or transcend, the project objective or life cycle (Eichmann and Nagy 2016), allows multiple perspectives, insights, and experiences to be brought to bear and sit in tension. As Bacchi (2009) argues, problems and solutions in education are often framed within one field of expertise meaning that many subsequent problems are ignored/silenced/missed. Technical democracy, as a higher-level goal, emphasises that a pre-defined solution or model of action based on isolated acts and individual actors is never the goal. Rather, its process is framed by ongoing activities, diverse stakeholders, and ‘measured action’, which is continually ‘remaining open to new information or new formulations of what is at stake’ (Callon et al. 2009: 223). For instance, heterogeneous questions drive an open-ended process of ‘technical problematisation’ (Barthe et al. 2022: 14):

What kinds of technology do we want or need? What potentials, limits, effects, flexibilities and alternatives do technologies have? What kinds of expertise, design, maintenance, funding and infrastructures do they require? And, importantly, who is able to raise and frame these questions and have a say in these discussions?

Fostering transdisciplinarity has the potential to strengthen postdigital inquiry and thinking ‘through different views of the empirical as well as through consideration of the hypothetical’ (Fawns et al. 2023). It is important that this transdisciplinary is not motivated by a naïve attempt to foster consensus. With its focus upon measured action beyond project lifecycles and open-ended process, technical democracy offers a systemic focus that transcends disciplinary and project boundaries. Transdisciplinarity also opens the possibility of incorporating more diverse expertise into public policy areas like education.

Diverse Stakeholder Perspectives and Expertise to Inform Measured Action

A key technical democracy feature involves designing dialogic spaces with stakeholders who have diverse expertise. This includes generating cooperation between the ‘secluded research’ of experts and the ‘research in the wild’ of laypeople to respond to controversies (Callon et al. 2009). In contrast to definitive decision-making models, technical democracy proposes ‘measured action’ as joint representations that emit ‘a warning system, a deepening of knowledge, and temporary measures’ (Callon et al. 2009: 205). Such a focus accords with another desirable element of postdigital inquiry, which is the inclusion of diverse voices and perspectives to inform new ideas:

Postdigital inquiry involves a relentless effort to question or refuse boundaries: between technologies, people, environments, disciplines, epistemologies, and methodologies. It stretches and generates ideas through expansion and exploration. Bringing in researchers from various disciplines, geographical locations, cultures, etc. can help us see the world differently. (Fawns et al. 2023, Para. 46)

For example, a generative event involving specialists and laypeople, known as the ‘hybrid forum’ (Callon et al. 2009), is where this style of discovery, openness to uncertainties, and varied contributions can unfold. At present, the hybrid forum is the most recognised mechanism for technical democracy, even as it remains amenable to ongoing critique and improvement. There is a significant literature critiquing the conceptual underpinnings of hybrid forums, including identifying the limitations of dialogic methods and the risk of reinforcing existing power relations (e.g. Joly 2015; Saito and Park 2016). However, there is a paucity of work that engages with the empirical undertaking of running forums (e.g. Farías 2016). In work that has looked to ‘doing’ rather than describing technical democracy, there is an explicit focus on ensuring that expertise and science are unsettled, even provisionally (Landström et al. 2011). Our primary point is that all research has limitations, and that what we found was that the objections about reifying science and technology implicit in the conceptual critiques were not evidenced in what we found in the AES example outlined below.

Inventive, Collaborative Methods that are Situated and Responsive to Complexity

As yet, many of the potential co-research mechanisms aligned with technical democracy are underexplored, as ‘modalities of cooperation … [are] clearly very varied and to a large extent remain to be invented’ (Callon et al. 2009: 105). While the specific modalities may differ, there ideally remain common core questions that are ‘at the heart of technical democracy’:

In what circumstances, under what conditions, according to what modalities, and with what effectiveness is collaboration between laypersons and specialists conceivable? Is it not, perhaps, just a case of occasional and superficial exchanges? Alternatively, can we conceive of a lasting cooperation? (Callon et al. 2009: 36)

In accord with this inventive approach, postdigital inquiry invites ‘taking up methods and methodologies—and creating new ones—in ways that are sensitive to the realities they produce’ (Fawns et al. 2023: Para. 50), to work creatively, speculatively, and compositionally with the ambiguity and complexity of specific issues of concern. Thus far, technical democracy approaches have predominantly used hybrid forums to foster shared uncertainty and collaborative problematisations (Thompson et al. 2023). Even in this attempt to foster uncertainty, there are always risks that forms of co-production of knowledge will fail to either shift prevailing positions or take into account the conflictual dimensions of sociotechnical controversies (Joly 2015). However, this speaks precisely to the point of the hybrid forums and the spirit of prototyping—it is about allowing dissensus to flourish, even if it is messy because this is what gets closed down in liberal versions of democracy (Rancière 2003). We argue that prototyping is a method that can deliver on the promise of technical democracy and augment the more common approaches such as hybrid forums.

An Emergent Thought Collective

The Technical Democracy CollectiveFootnote 1 is an emerging collaboration between interdisciplinary researchers in Australia and overseas, with expertise spanning education, policy and governance, artificial intelligence and emerging technologies, assessment, learning analytics, teacher professional learning, digital technologies, ethics, media and communications, and law. The collective is both the formation of people and expertise and the collective work that is being done in various configurations. While the range of research expertise is diverse, the common transdisciplinary interest of the collective is exploring new ways to respond to socio-technical controversies, which ‘allows the design and testing of projects and solutions that integrate a plurality of points of view, demands, and expectations’ (Callon et al. 2009: 32). To do so, we integrate practice-based, design, and ‘studio’-style approaches that re-imagine policymaking and institutional practices by critically and creatively opening up the politics of participation (Kimbell 2019). This ‘thought collective’ is a group that exchanges ideas and nurtures a particular ‘thought style’ that directs the object and mode of inquiry (Fleck 1979). This process of emergent thinking together sparks ‘proto-ideas’, not only within a thought collective (intracollective communication) but also between respective collectives (intercollective communication) (Fleck 1979). Our thought collective therefore seeks to articulate not only intracollective communication about technical democracy in education, but also how it might delineate a particular thought style and mode of knowledge coordination for intercollective communication between stakeholders with diverse expertise. In doing so, we seek to manifest and mobilise the potential of a collective approach to, among other things, democratic discussion, community building, and networked learning (NLEC et al. 2021).

Since 2021, members of this collective have produced an array of activities, events, and artefacts to explore the potential of a technical democracy approach in education, with a focus on datafication and emerging technologies. These have included a retreat, two Australian Association of Academic Research symposiums, two workshops, several journal articles, a white paper and policy brief, plus a journal special issue to explore concepts, tools, and methods aligned with a technical democracy approach.Footnote 2 These activities and outputs operate as ‘boundary objects’ (Star and Griesemer 1989), which we see as a useful way to understand how knowledge representations between different collectives can be coherently coordinated not only for joint knowledge work, but also for different needs. Importantly, the Education Futures Studio, established at the University of Sydney is an infrastructure that aims to connect these boundary objects: ‘a space that manifests through shared ideas, practices, and tools … to make technical democracy a defining feature of education futures’.Footnote 3 The Education Futures Studio operates as a fragile ‘boundary infrastructure’ (Bowker and Star 1999) serving multiple, distributed communities of practice. We recognise that having an institutional location such as the Education Futures Studio is both necessary and dangerous: necessary as moments of fragile democratisation require support, but dangerous as institutionalisation risks becoming anti-democratic. We see the Education Futures Studio, therefore, as an infrastructure that encompasses a range of emerging boundary objects for local variation and provides sufficient structure presently for coordination, but that it may become insufficient for and counter to the needs and aims of technical democracy. Significantly, boundary objects within transdisciplinary research networks are ‘prototypes’ which can exceed and carry ideas beyond infrastructures, as they visualise mental ideas; support the comprehension of complexity; enable communication; are constrained by a specific question, test functionalities, and requirements; create a common understanding; and localise user interests and/or interactions with objects (Ängeslevä et al. 2016). Carried across thought collectives, these representations, ideas, and approaches for practical solutions ‘become manifest in prototypes’, which then ‘enable us to consider and test them as well as to communicate about them’. (Gengnagel et al. 2016: 1). We next outline an experiment to explore the potential of prototyping as a potential boundary object for this emergent thought collective.

A Prototyping Design Experiment

Our design experiment that provides the context for this paper intends to generate a boundary object to identify and coordinate technical democracy possibilities within, and beyond, an emergent thought collective. Our aim is to build upon affinities between technical democracy and postdigital inquiry, which draws attention to a multiscalar lens (with sociotechnical controversies and concerns), transdisciplinarity goals (systemic and transgressive), diverse stakeholders (expertise and perspectives), and inventive methods (responsive to ongoing cooperation and generative techniques). This process is inspired by experimental postdigital dialogue as a ‘practical and theoretical starting point for retooling our educational and research toolbox to adapt to and shape our postdigital reality’ (Jandrić et al. 2018b: 181). We therefore build upon the need that ‘design practices can be enacted otherwise and provide means and processes of questioning and transforming legacy (institutionalised) structures and categories through creativity and imagining’ (Macgilchrist et al. 2023: Para. 10).

To do so, the design concept provoking our experiment is the ‘prototype’ concept and process. Tracing the genealogy of the prototype from an anthropological perspective, Jiménez (2014) proposes it as a generative concept for collective thought, for ‘laying out an experimental field where tools, objects, spaces and forms of encounter are all devised anew’ (p. 387). Prototypes are therefore not passive objects in a product development process but are rather manifestations that influence social interactions and enable ‘new types of communication, learning, and decision-making’ (Lauff et al. 2018: 061102–9). In the fields of human–computer interaction and design, prototypes are generally understood as a key communicative object in the design process that filters and concretises ideas, in order to make possibilities and limitations visible (Lim et al. 2008). This prototyping process is characterised by two modes of thought, that of ‘suspension’ and ‘expectation’ (Jiménez 2014). To suspend thought requires a convergent field of focus (by necessity, partial) to pause, or slow down thinking, whereas expectant thought invites divergent trajectories to accelerate and expand new possibilities. Both suspension and expectant thought bring opportunities for what Farías and Blok (2016) describe as a core principle of technical democracy, the ability to live in and work with shared uncertainty around sociotechnical controversies. Prototypes are characterised by these filtering and manifestation dimensions, alongside different variables selected throughout the prototyping process (Lauff et al. 2018). The prototyping concept that imbues this design experiment aims to articulate alternative ways of communicating text and ideas about the potential of designing postdigital education futures with a technical democracy approach.

Education Prototyping: Introducing a Methodological Device

Prototype thinking invites new research rhythms and lines of inquiry: to not only suspend, slow down, and focus thought, but also to spark and accelerate trajectories of new and expectant thought. Therefore, education prototyping is a methodological device to mobilise matters of concern—plus expand the momentum of—critical postdigital educational research. Prototypes are a ‘means of generative and evaluative discovery’ (Lim et al. 2008: 7), enabling both filtering for evaluative and reflective purposes and manifesting for design ideas and possibilities. This tight interplay is a key strength and characteristic of prototypes, as it is ‘incompleteness that makes it possible to examine an idea’s qualities without building a final copy of the design’ (Lim et al. 2008: 7). It also enables the traversal of an idea’s dimensions, for instance: deferring decisions about dimensions until decisions about other aspects have been made. Our prototyping involves divergent, interrelated practices for articulating this nascent research field of technical democracy in education. This interplay of dynamics and practices serves critical and creative purposes: first, to slow down or suspend existing claims/orders about education, technology, and democracy and, second, to accelerate, or expand, new possibilities and configurations.

The key dynamics and practices of this conceptualisation are described (Fig. 1). Key aspects include the following: (i) prototyping dynamics: problematisation and prefiguration; and, (ii) prototyping practices: temporal, methodological, relational, material, and spatial.

Fig. 1
figure 1

Education prototyping

Full size image

As Callon et al. (2009: 35) note, ‘[b]y trial and error and progressive reconfigurations of problems and identities, socio-technical controversies tend to bring about a common world that is not just habitable but also liveable and living, not closed on itself but open to new explorations and learning processes’. In combination, this process offers a shared way to identify socio-technical issues and imaginative trajectories aligned with the politics of a technical democracy approach. The sections that follow introduce key dynamics and practices associated with education prototyping and discuss trialling this device to facilitate collective learning and experimentation about socio-technical controversies in education. This trialling aims to show the possibilities of this prototyping by highlighting how the dynamic and practices can inform and extend work that we did on the issue of automated essay scoring (AES) in Australian schools.

Dynamic 1: Problematisation

This dynamic encapsulates a common analytical aspect for our thought collective and education prototyping: the desire to identify and examine, or problematise, novel socio-technical issues of concern in education. Prototyping is reliant upon delimiting and traversing a particular design space to generate meaningful knowledge that ‘filters the qualities in which designers are interested, without distorting the understanding of the whole’ (Lim et al. 2008: 7). This prototype dimension offers a common, investigative aspect for our thought collective and education prototype: the desire to filter and scrutinise, or problematise, particular sociotechnical matters of concern in education. One view of problematisation is the process of defining what is a problem in a given situation, identifying the causes of this problem and proposing ways to resolve it (Barthe et al. 2022: 14) (emphasis original). A related idea is that all problematisations are transformative (Stengers 2021). Combined, the problematisation process guides an interruptive (and ultimately incomplete) mode of investigation, facilitating a shared way to slow down, or momentarily suspend, existing claims and orders. Applied to our thought collective, this prototype dimension enables us to frame our studies of sociotechnical matters of concern in education by filtering particular qualities to spark problematisation, yet always in relation to an openness to new understandings and possibilities. Moreover, Stengers (2021: 86) argues that the ‘power to problematise’ can also awaken the imagination, as this philosophical ethos ‘must involve an experimentation with possibility’ (Stengers 2021: 89).

Automated essay scoring (AES) has been controversial in education, particularly in regard to its use in evaluating student work that has high stakes associated with it. We selected this for an experiment in technical democracy due to the range of stakeholders involved and conflicting views about what expertise mattered in this issue. In what follows, we utilise elements of our provisional methodological device to frame our problematisation of this particular education prototyping case. To explore the sociotechnical controversy of AES in Australian schools, we deployed a multiscalar lens to explore issues and concerns. AES is multiscalar in practice, as controversy boundaries are diffuse, and their constituent issues are entangled across a range of learning, professional, institutional, policy, and industry contexts. We also deployed a transdisciplinary approach encompassing empirical and hypothetical work aligned with higher-level goals or values that guided our systemic focus and heterogeneous, open-ended process towards ‘the democratization of democracy’ (Callon et al. 2009: 119). Critically, diverse stakeholder perspectives and experiences to inform measured action co-produced a pluralistic space and dialogue ‘to not only expand understanding of problems and concerns arising from the use of AES, but also to identify possible strategies for influencing policy development in the area’. (Gulson et al. 2022b: 8). Moreover, we were open to inventive, collaborative methods which enabled us to compose a unique mode of collaborative inquiry that was situated and responsive to complexity (detailed in the practices further below).

Dynamic 2: Prefiguration

This prototyping dynamic encapsulates a common, imaginative aspect for our thought collective and education prototyping: the desire to accelerate and expand, or prefigure, novel socio-technical possibilities and configurations in education. Inherently provocative and provisional prototypes are ‘an incomplete portrayal of a design idea’ (Lim et al. 2008: 7) involving particular choices about a prototype’s level of detail and scope, with the ultimate aim not being to prove solutions, but instead to discover different problems, that is, to explore ‘new solution directions’ (Lim et al. 2008: 7). The concept of prefiguration is a powerful accompanying dimension to problematisation within our education prototyping. With roots in anarchist writings, feminist scholarship, and social movement literature, prefigurative politics is defined as:

… an inherently spatial and performative genre of political activism in which people enact a vision of change – through organisation, design, architecture, practices, bodies, or something as simple as a gesture or demeanour – and promote this as indicative of an imminent or more distant ‘future’ (Jeffrey and Dyson 2021: 643)

In contrast to ‘ends-guided’ prefiguration (which focuses on distant and specific ends), ‘ends-effacing’ prefigurative politics is an open-ended, non-prescriptive style of politics that entails ‘commitment to experimentation and resistance to pre-determined political goals’ (Swain 2019: 60). To this end, it simultaneously unites people who hold shared critiques of society but who may also hold diverse priorities and differing alternatives. Applied to the context of research and scholarly outputs, Asad (2019) proposes that prefigurative politics can reframe how social relationships and resources are distributed in research with communities, which can help to build ‘counter-institutions’. These can provide the means to address the needs of people affected by injustice by offering structures for such collaborations, which can enable their greater autonomy and agency over digital designs and systems within and beyond research timeframes. Amsler (2014) links prefigurative politics to three varying modes of politics and education: first, a focus on political recognition and struggles in education; second, education as a form of political action; and, third, a ‘politics of possibility’ that frames ‘everyday life itself as a pedagogical experience’ (Amsler 2014: 277) that can help to politicise educational practice.

Aligned with this broader ‘politics of possibility’ commitment, prefiguration is key to our education prototyping in terms of enacting a vision of change aligned with technical democracy, while also recognising there are multiple and diverse spaces—and experimental ways—to pursue that goal. For example, the purpose of the AES forum activities was not to convince stakeholders of the utility of an AES approach, nor was it to convince participants to arrive at a consensus on what should (or could) be done. Rather, it was to create a pedagogical moment that focused on emergence produced through collective experimentation, even if this was transitory or short-lived, to pre-figure possibilities for action. Prefigurations that emerged from the forum were represented in the form of a white paper and policy brief. In the white paper (Gulson et al. 2022b), prefigured recommendations aligned with the following identified issues: AES systems complexity and contexts, school infrastructure capacity to deploy AES, the impact of AES upon professional practice, cross-sectoral interests and values associated with AES, and policy uncertainty regarding AES and emerging EdTech. The policy brief (Gulson et al. 2022c) represents a range of prefigurative spaces and possibilities for collective policymaking (such as participatory procurement, education-specific frameworks, AES-focused audits, alongside new forms of experimentation, governance, and knowledge coordination).

Based on the problematisation and prefiguration dynamics outlined above, education prototyping offered us a methodological device to co-produce a pluralistic space that exposed challenges and tested possibilities for democratisation. How to enact such an approach is explored next, which we frame as prototyping practices.

Prototyping Practices

Prototyping practices offer an unfolding process inspired by technical democracy to interrupt the ingrained habits of researchers, both within our thought collective and beyond. Prototypes are filtered manifestations of ideas (such as text, sketches, images, and digital artefacts) that provide the form, shape, and appearance of prototypes, and which offer ‘a sense of eventual possibilities or limitations inherent in the idea’ (Lim et al. 2008: 7) that that can be developed further as an idea becomes more refined. Problematisation is a social activity that combines the processes of qualification and of setting an issue within a space of reflection and specific treatment’ (Barthe et al. 2022: 15); furthermore, ‘the method is crafted alongside the work, with the problems encountered in the work’ (Stengers 2021: 76). In the selection and use of such tools, Stengers (2021: 185) insists that ‘habit must be resisted’, especially as the most relevant ‘tools for thinking’ are ‘the ones that address and actualise this power of the situation, that makes it a matter of particular concern, in other words, make us think and not recognise’. There are several, interrelated categories of practices that illustrate the high-level convergent, and divergent, aspects of education prototyping. We show how these were operationalised in our AES case.

Multiple Temporal Practices

These temporal practices draw attention to unfolding events and encounters for collective learning generated across timeframes and durations. Our AES forum used a controversy from 2015 about the use of AES in a national testing programme as an entry point. This controversy helped spark discussion about present and future overflows of automated technologies across education more broadly. The temporality of overflows draws attention to not only what was discovered at the time of the controversy, but also the unexpected connections, uncertainties, and reflections arising from the dialogic space of the forum—alongside the speculative ‘new lines of research to be explored’ through the identification of constraints which can ‘open up the elaboration of new projects and new solutions’ (Callon et al. 2009: 28–32). As a space of prefiguration, the hybrid forum provided the possibility of pluralistic visions of an educational future to emerge.

Situated Methodological Practices

Such practices are not pre-determined or formulaic but are specifically crafted in relation to a particular socio-technical matter of concern in education, alongside a research group’s expertise and context. In the AES forum our aim was to experiment with a ‘hybrid forum’ (Callon et al. 2009): a dialogic space where heterogeneous perspectives are shared for collective learning and experimentation. We took the view that everyone in the forum was an expert in some dimension of automated essay scoring (e.g. programming, assessment), but no one person was an expert in all dimensions. This distributed expertise helped to ‘reveal events that were initially isolated and difficult to see, because they bring forward groups that consider themselves involved by the overflows that they help to identify’ (Callon et al. 2009: 29). While the 2015 AES controversy was seen as settled, we were interested in whether bringing together both stakeholders and multiple forms of expertise can provide alternative ways of both understanding the 2015 controversy and mapping potential controversies if automated essay scoring was to be re-introduced in Australia (following its global take up since 2015). We were not interested in merely reproducing the issues of the original controversy but expanding the possibilities of how we can understand and respond in concrete ways to the use of automated technologies in education.

Diverse Relational Practices

Attention to relational practices requires thinking about the range of perspectives and expertise involved. The AES forum brought together multiple antagonists in the original 2015 controversy, along with academics from multiple fields (education policy, assessment, data science, technology ethics), and practitioners. Participants ‘discussed intersecting issues that emerged from the collective learning, rather than speaking as isolated stakeholders or experts’ (Gulson et al. 2022b: 8). This space recognised existing expertise and positions on the issue of AES, such relationships allowed for collective learning plus connections beyond the forum event. Diversity did not mean that power relations were not recognised; indeed, there was extensive discussion about the role of large education technology companies in the provision of AES and the desires of a range of stakeholders for the possibility of either not using AES or using it in open-source form.

Generative Material Practices

Attention to material practices draws attention to the scope of technical democracy artefacts that can be generated for collective learning and experimentation. Prior to the forum, participants were emailed an information pack with links and resources about AES, machine learning, and news articles about the controversy which ‘deliberately amplified rather than assuaged diversity of opinion’ (Gulson et al. 2022b: 8). In the forum, we asked participants to grade essays using a rubric and then compared these with results from the prototype AES system. This allowed for collective learning about both human and machine marking and discussion of bias in all kinds of systems. We had rebuilt an online AES system that participants were to use to ‘play’ with different inputs to see how algorithms make different decisions. This artefact, in analogue form, was based on an existing repository of an AES system that used deep learning algorithms. We ended up needing to change our plans when our computer scientist experts ended up with COVID. Our online site became the analogue example noted above. This array of material practices composed for the AES forum opened opportunities for slow, thoughtful conversation about the problems and stakeholders, the limits of resolution, alongside the prefiguration of possibilities.

Everyday Spatial Practices

A focus upon spatial practices mobilises technical democracy across a range of organisational, community, and societal contexts. Our aim in the forum was to collectively explore the multiscalar spaces of AES which span a range of learning, professional, institutional, policy, industry, and geographic locations. The co-produced policy brief (Gulson et al. 2022c) highlighted the AES ‘sphere of impact’ and the identified need to mobilise collective policymaking across multiple spaces: classroom/schools, regional/state/territory, national/federal, and international/global. With growing concerns about the encroachment of commercialised EdTech across teaching, learning, and assessment practices (such as large language and general-purpose AI models), the ways in which ‘global technical systems’ (Stiegler 2016) produce and exploit data traces across everyday life demand closer attention.

Concluding with an Invitation

While always provisional and partial, we hope that this introduction to education prototyping can open up multiple lines of inquiry associated with the multiscalar lens, transdisciplinary goals, diverse expertise, and inventive modes of cooperation that this methodological device might carry forward. We aimed to show that this prototyping is made possible by the thought collective outlined earlier. In contrast to the rapidity of corporate-based prototyping, our approach was a slowing down to collectively produce prototypes through the combination of different skills, expertise, and disciplines that open up spaces of democratisation.

Inventing methods, as Lury and Wakeford (2012: 6) propose, permit a change in knowledge that ‘need not be limited to ascertaining what is going on now or predicting what will go on soon, but may rather be a matter of configuring what comes next’. Given this permission and active encouragement to invent, our paper responds to the need to critically recognise and creatively respond to how ‘power relations and tensions lie at the heart of assumptions about designing futures’ (Macgilchrist et al. 2023: Para. 1). Education prototyping seeks not to be prescriptive but provocative, in terms of offering a novel analytical, reflective, creative, and communicative tool for practice and research. Importantly, education prototyping is an emergent boundary object oriented towards a better understanding of what a technical democracy approach might offer emergent thought collectives. It is therefore as much to locate our own thinking and to provide us with a thinking tool, rather than an exercise to produce a formulaic rendering of technical democracy.

We invite readers to respond to and build upon the provisional methodological device introduced in this paper. This invitation is inspired by the concept of ‘writing as a method of inquiry’, which foregrounds writing differently—as a method of discovery and analysis (Richardson 2000; St Pierre 2007). Aligned with this ethos, prototypes are ‘a tool to try out proposals but also to evoke dilemmas’ (Hillgren et al. 2011: 181), a means not for reducing uncertainty but, rather, of enhancing uncertainty, at least insofar as this uncertainty provides the foundation to think differently about problems and solutions, based on new perspectives and new vantage points. Education prototyping therefore aims to articulate the common thinking and diverse practices of an emergent thought collective. Key questions we invite readers to respond to include:

  • Utilising a multi-scalar lens, what socio-technical controversy or concern in education does your transdisciplinary research problematise and prefigure?

  • What range of practices (temporal, methodological, relational, material, and spatial) has been composed and enacted?

  • What are the key enablers and constraints for involving diverse stakeholders in collaborative research for measured action?

  • What provisional ideas, objects, and infrastructures can further develop and test education prototyping to expand this emergent thought collective?

This invitation to write prototype ‘things’ (Bjorgvinsson et al. 2010) seeks to open up spaces of democratisation that expose challenges and test possibilities. Based on an understanding that subjectivity is historically and locally situated, this mode of writing locates language as ‘a site of exploration and struggle’ (Richardson 2000: 929) that is open to ‘competing ways of giving meaning and or organizing the world’. This mode of reflexive writing offers an intermittent promise and ‘field of play’ of continual movement and uncertainty, driven by an expectation that ‘we may produce knowledge that will change the world’ (St Pierre 2007: 5306). In doing so, education prototyping offers the potential to problematise challenges and prefigure possibilities, in pluralistic ways. We hope readers are inspired to develop, test, and distribute this methodological device towards co-producing spaces of democratisation.