Introduction

Maker education, also referred to as the ‘maker movement,’ is a cultural and educational movement that engages learners in playful and creative approaches to scientific and technical activities, often inspired by industrial design and engineering. With roots in the pedagogical theories of Dewey and Montessori (Blikstein, 2014), learning through ‘making’ involves the active development of knowledge and capacity through practical activity. While overlapping with the traditional design and technology curriculum, Maker activity has been distinctive in its innovative use of emerging digital technologies such as computer-aided design and 3D printing. This learning has typically been facilitated by the provision of a hybrid workshop known as a makerspace.

In the application of Making investigated in this study, the Australian educational technology company Makers Empire have pushed the hybrid learning space further into the ‘digital’ by creating a virtual makerspace. In doing so, they have radically increased access to making. This is important as while physical makerspaces are frequently presented as accessible, they are actually quite expensive to set up and require a high level of technical and pedagogical skill to implement and maintain (Leonard et al., 2022). This means that physical makerspaces are typically only present in places such as university campuses and larger public libraries, which are often quite inaccessible to children in low socio-economic or rural communities.

The potential of a virtual makerspace was highlighted by the Covid-19 pandemic. During the pandemic we have seen a significant increase in the use of online learning within traditional school settings (Dishon, 2021). This shift, though, showed that the pedagogical preparedness of schools to work in online settings was limited (Bozkurt et al., 2020). We refer here not to the myriad technical challenges that an unplanned pivot to online learning presented, nor even to difficulty of inventing new forms of ‘classroom’ management via video conference. Rather, the pandemic highlighted that the systems available for online learning in schools have tended to rely on rather mechanistic, transactional and behaviourist theories of teaching and learning (Leonard et al., 2022). While research in educational technology has long promoted the potential of new technologies to promote learning informed by constructivist and constructionist theories of learning (Papert, 1993), such pedagogies have not been widely realised in popular commercial learning management systems. In this context, the Makers Empire platform is particularly interesting because the company has gone beyond a proof-of-concept model within a research project and have rolled their solution out to a large international customer base.

As with the design of learning in physical makerspaces, the educational design that Makers Empire have deployed in their virtual makerspace is heavily informed by self-determination theory (SDT, Deci & Ryan, 1985; Ryan et al., 2021), which posits that educational interventions will be more highly motivating when they address the basic psychological needs of competence, relatedness and autonomy. It is in its capacity to provide for these needs together in authentic and meaningful ways that makes the maker movement stand out as an educational option, particularly in online or online-blended learning. Making begins by offering learners real and authentic choice, and hence autonomy, in their learning. It then provides ways to pursue these choices in connection with others, and it can scaffold experiences of competence and success.

These feelings of achievement are supported not only by the structure of the activity but also by the inherent capacities developed whilst engaging with Makerspaces. One such capacity, the ability to think spatially, has been identified by the literature as playing a critical role in enabling maker activities (Munoz-Rubke et al., 2021; Smith, 2018). Additionally, this capacity has been shown to be malleable to training (Hawes et al., 2022; Uttal et al., 2013) including when using technology (Fowler et al., 2022; Hawes et al., 2022) and within the context of spatially complex STEM (science, technology, engineering and mathematics) problem-solving contexts inherent to makerspaces (Zhu et al., 2023). The purpose of the present study therefore was to investigate the impact of this virtual approach to Maker learning on spatial thinking, and also on student anxiety and enjoyment towards learning. The study used a within-subjects pre-/post-test design making use of the previously validated Spatial Reasoning Instrument (SRI, Ramful et al., 2017) and the Schools Attitudes Survey, which was developed from the School Science Attitude Survey (Kennedy et al., 2016). Nine schools were purposefully selected for the study to ensure variation in school size, regional and metropolitan locale, and socio-economic status.

This study contributes to a small and nascent research literature on the impact of maker learning on spatial thinking (Ramey et al., 2020) and to a larger body of recent research that has investigated the role of digital technologies on spatial reasoning (see, for example, Di & Zheng, 2022; Dokic et al., 2021; Leonard & Fitzgerald, 2018; Sinclair et al., 2016). References to the spatial elements of thinking are diverse and vary across countries and disciplines due to the linguistic limitations we encounter when attempting to describe a phenomenon that is often indescribable (Harris, 2021). Terms such as spatial structuring (Battista et al., 2018), spatial ability (Tam & Chan, 2022), spatial intelligence (Hegarty, 2010), and spatial skills (Pittalis & Christou, 2010) have all been employed to characterise these mental processes. However, since this paper primarily focuses on the application of these processes, we have adopted Ramful et al.’s (2017) concept of spatial constructs that combine to form spatial reasoning. These three constructs of mental rotation, spatial visualization, and spatial orientation are derived from the requirements of the Australian Numeracy curriculum and have received considerable support (see, for example, Baumgartner et al., 2022; Fowler et al., 2022; Harris, 2021; Lowrie & Logan, 2018). Additionally, they have been identified to play a crucial role in the types of problem-solving that underlie STEM fields (Lowrie et al., 2017).

Spatial reasoning is a particularly important capacity to develop in students as, beyond its influence on achievement in STEM careers (Wai et al., 2009), it is becoming increasingly evident that higher spatial reasoning ability contributes to academic success in a wide range of specific STEM subjects such as chemistry (Small & Morton, 1983), physics (Miller & Halpern, 2013), engineering (Sorby, 2009) and mathematics (Geer et al., 2019; Mix, 2019; Verdine et al., 2017).

Interest in the technologically enabled development of this important capacity has led to multiple studies, mainly focused on the early years (e.g. Bower et al., 2022; Larkin & Lowrie, 2022) or senior secondary and beyond (e.g. Krüger et al., 2022). Recent scholarship has identified that spatial reasoning can be impacted by factors such as early language skills (Möhring et al., 2021) or gender differences in adolescence (Harris et al., 2021). However, little research has explored the important developmental stage of the middle years when the brain is primed for learning (Wolfe, 2010). In Australia, the middle years of school refers to children typically aged 11–14-years-of-age.

Anxiety and learning enjoyability are important constructs to consider from the perspective of control-value theory (Pekrun et al., 2007). Within this model, they are considered to be emotions that enable or inhibit students from engaging with activities or achieving outward looking outcomes. As such, innovative and creative educational design approaches, such as the use of makerspaces, regularly employ these constructs within the design rationale. The common approach to measuring student engagement in an innovative learning design using an on-the-day survey may lack validity as apparent improvements may be driven by novelty—anything is more interesting than another day of regular schooling—and so may not indicate a persisting change in attitudes towards learning within that subject area. Our novel approach to measuring growth rather than absolute value is more likely to provide a valid picture of the lasting impact of an intervention on these constructs.

The specific successes of this approach to learning that we will describe in this paper are particularly noteworthy in the context of SDT. In the study we report here we were particularly interested in the impact that engagement in the virtual maker space had on students’ spatial reasoning. This paper will address the research question, “to what extent can virtual makerspaces impact on the development of critical skills—such as spatial reasoning—and students’ attitudes to learning in a complex learning environment?”

An educational design to improve spatial reasoning

The Makers Empire virtual makerspace is a digital application accessed either on a tablet device or through a laptop. After an initial introduction of the program’s features, students are encouraged to attempt different challenges and share their results with their peers. Introduction to new content is in the form of videos and quizzes related to a curriculum-aligned theme, such as animal adaptations or biomimicry. Students access a range of set challenge courses and various timed competitions from a central ‘landing page’ reproduced in Fig. 1.

Fig. 1
figure 1

Screenshot of the central landing page of the Makers Empire program including spaces to go to training, mazes, competitions, and challenges

Full size image

Prior to commencing the program with their students, teachers attended either a face-to-face or an online briefing session about the Makers Empire program. This session provided some background information on spatial reasoning, outlined the administration of the SRI test, and allowed teachers to familiarise themselves with the Makers Empire software and learning program.

In all schools in this study, teachers followed the following structure. In week 1, students were guided in setting up their Makers Empire accounts and then undertook a series of basic training modules to understand how the program works. Between weeks 2 and 10, students had free access to the range of challenges, competitions, and other activities from the central landing page.

Spatial reasoning

Spatial Reasoning is an overarching term used to refer to a complex network of cognitive skills and capabilities. These skills and capabilities are identified and described in the form of spatial constructs, which are often characterised in similar ways between the fields of cognitive psychology and mathematics education. However, discipline-based methodological viewpoints mean there is often a lack of transdisciplinary alignment between perspectives (Bruce et al., 2017; Lowrie et al., 2020).

As we will describe in this section, however, there is solid research evidence of a relationship between several constructs within the spatial reasoning category and success in learning in the STEM disciplines. We propose that the Makers Empire virtual makerspace design holds the potential to influence the spatial constructs identified by Ramful et al. (2017) in their development of the SRI, which encompasses a majority of spatial tasks outlined in primary and secondary curricula. Below, we will provide examples illustrating Ramful et al.'s constructs of mental rotation, spatial orientation, and spatial visualisation.

Mental rotation

Mental Rotation is an extensively studied spatial construct (Harris, 2021). It involves use of the motor parts of the brain (Berneiser et al., 2018) to hypothesise about changes to 2D shapes or 3D objects as they are rotated by various degrees in different directions. Notably, some researchers (Harris et al., 2021; Lauer et al., 2019; Rahe & Quaiser-Pohl, 2021) have found that, of the spatial constructs discussed here, mental rotation is the most likely to be influenced by a student’s gender. This observation has been linked to the tendency of female participants to tackle spatial tasks using an analytical approach, as opposed to the active visualisation methods typically employed by male participants (Lauer et al., 2019).

Mental rotation is beneficial to the study of many STEM subjects, but it is of particular relevance to exploring geometric properties (Battista et al., 2018) and dynamic geometric environments (DGEs, see, for example, Crompton et al., 2018; Dokic et al., 2021; Venturini & Sinclair, 2017). Computer Aided Design (CAD) programs are, in essence, DGEs but their purpose has traditionally been more focused on the output product than the learning within the program. The Makers Empire virtual makerspace, in contrast, makes use of CAD software to create opportunities for the development of mental rotation. The Makers Empire activities utilise so called primitives—simple 3D shapes such as cubes, spheres and cones—to construct more complex objects. Moving, positioning and angling these primitives in 3D space makes heavy use of mental rotation as seen in the ring creation example in Fig. 2

Fig. 2
figure 2

Screenshot showing the use of primitives to create a ring in the Makers Empire program

Full size image

Spatial orientation

Spatial orientation, or perspective taking, is a particularly important, but understudied, predictor of mathematical performance (Harris et al., 2021). It refers to imagining the perspective of a place or space in the world, that is not your current perspective. It is key to the important mathematical understanding of proportion and ratio. Estimations of measurement, perspective and describing the spatial relationships between different shapes and objects also benefit from a well-developed understanding of spatial orientation.

Some tasks within the Makers Empire virtual makerspace such as the maze task reproduced in Fig. 3 explicitly explore this spatial construct by having students move an avatar through a 3D maze. This engages what Ramey et al., (2020, p. 473) describe as ‘intrinsic-static spatial skills.’ Other tasks within the virtual makerspace build on the affordances for CAD to design and create 3D structures within a defined virtual environment, therefore tapping into more dynamic spatial skills. These skills are further nurtured when creations are positioned within the students’ real-world environment, either through 3D printing or augmented reality (see Fowler, 2021 for a detailed example).

Fig. 3
figure 3

Screenshot of a maze task as an example of spatial orientation from within the Makers Empire program

Full size image

Spatial visualisation

In contrast to spatial orientation, where the object is fixed and the observer moves, spatial visualisation explores the multi-step manipulation and transformation of objects from an egocentric perspective (Sorby & Panther, 2020). Spatial visualisation, frequently considered one of the most challenging spatial constructs for students (Fowler et  al., 2022), is also among the most receptive to training (Hawes & Ansari, 2020). Makers Empire provides students with multiple situations in which they need to use the 3D geometric tools to rotate, split, delete, add and augment objects to achieve particular aims.

Figure 4 shows an example from within the game where students are asked to make a centaur through the resizing, manipulation and grouping of primitives. This complex task is guided by a ‘walkthrough’ to teach students skills before they make their own monsters. Students constantly need to consider how shapes will align from multiple viewpoints in order to build what they have been directed to create. The simplicity with which this can be done cannot be replicated in real life contexts so there is potential for great improvement in this area.

Fig. 4
figure 4

Screenshot showing the creation of a centaur in Makers Empire program as an example of spatial visualisation

Full size image

Methods and materials

Instruments

In order to explore the impact of the Makers Empire program on middle years students spatial reasoning, we make use of the validated Spatial Reasoning Instrument (SRI, Ramful et al., 2017). This instrument is a pen and paper measure of three spatial reasoning constructs: mental rotation, spatial orientation, and spatial visualisation. These three constructs are well established in the research literature and clearly align with the demands of the Australian Mathematics Curriculum (Ramful et al., 2017). The SRI consists of 30 multiple choice items spread evenly across the three constructs and results in four scores: a score for each of the three sub-scale constructs in the range 0 to 10 and an overall SRI score in the range 0 to 30.

To explore the impact of the Makers Empire program on students’ attitudes towards digital technologies we made use of the School Attitude Survey (SAS) which we have developed from the School Science Attitude Survey (Kennedy et al., 2016). The SAS is a self-report tool that measures a student’s attitudes towards their school subjects against nine Attitudinal Factors. These factors are:

  • Anxiety

  • Creativity

  • Difficulty

  • Enjoyability

  • Intentions

  • Relevance

  • Self-Efficacy

  • Career Usefulness

  • Personal Usefulness

Student attitude ratings are captured using a digital tool and reported on a visual analogue scale from − 50 to + 50 (Fig. 5). The Attitudinal Factors Anxiety and Difficulty are both reverse keyed.

A student’s mean attitude rating for each attitudinal factor across all of their subjects is calculated and is known as the student’s Composite Attitude Rating (CAR). A student’s CAR can be thought of as a measure of their average attitude to the academic aspects of school as a whole and acts as a comparison baseline.

Subject Attitude Ratings (SAR) are then calculated by subtracting a student’s CAR for a specific attitudinal factor from their raw attitude rating for the individual subject for that same attitudinal factor. A SAR could therefore theoretically fall in the range −100 to +100. A student’s SAR can be thought of as a measure of their attitudes towards a single subject area in comparison to their attitude towards school as a whole.

Attitude ratings for a grouping of students are processed into an Attitudinal Profile. In an Attitudinal Profile a high rating represents the desired outcome. Therefore, a positive rating for anxiety represents a low-level of anxiety (relaxed students are the desired outcome) and a positive rating for difficulty represents a low-level of difficulty (students who are not struggling is the desired outcome).

Participants and analysis

In this study, we collected data in nine Australian primary schools in which the Makers Empire program was being delivered resulting in a total of 340 participants ranging in ages from 10 to 13 years old. Ethical approval was provided both by the University and the Department for Education.

As summarised in Table 1, the schools were purposefully selected to ensure variation in school size, regional and metropolitan locale, and socio-economic status. Each class engaged with the virtual makerspace for one session of around an hour each week for ten weeks. While the schools were purposefully selected from our existing network of contacts, student sampling adopted a more convenience sampling approach as all students and their parents opted-in to the study. Note that, students who did not opt into the study group were still able to access and utilise the Makers Empire platform. As can be seen in Table 1 this has resulted in slightly unbalanced sample groups and so a degree of caution needs to be adopted when interpreting any results.

Table 1 Details of participating schools
Full size table

Teachers administering the SRI were provided with clear written and video instructions detailing the conduct and marking of the test onto a provided excel sheet. Students completed the SRI prior to and immediately after the term’s work on the program. Students were identified by anonymous codes to enable tracking from pre-test to post-test. Individual student growth (post score minus pre score) for each of the SRI subscales and overall SRI results in a further four data measures to complement the raw SRI scores. Student growth, both overall and for individual spatial constructs, was analysed using Welch’s t-test.

To facilitate analysis, each school’s ICSEA was recoded to a three-level categorical variable with cut-off points as follows: low < 950, 950 ≤ average < 1060, high: ≥ 1060. Descriptive statistics were calculated for various groupings within the data. SRI scores—both overall and sub-scales—were hypothesised to depend on student sex, school location, school size and ICSEA score. School location, size and ICSEA were also hypothesised to exhibit interaction effects on student SRI scores. Factorial ANOVA was carried out to explore these relationships, with a p-value cut-off of 0.05 being adopted for the determination of significance.

In order to examine the different growth trajectories of participants, a longitudinal k-means analysis was caried out using the R package kml (Genolini et al., 2015). This approach considers data not to be single points in time, but to represent data-series or trajectories. Whereas a k-means clustering algorithm groups single time-point data into similar, homogenous groups, the kml algorithm groups longitudinal data into homogenous data-trajectories. Multiple quality criteria are used to determine the most appropriate clustering solution and it is thus possible to identify students who show similar growth trajectories—meaning direction, magnitude, shape and timing— in their spatial reasoning performance over the duration of the Makers Empire program. All analyses were performed using R version 4.1.2 (R Core Team, 2021).

Results

Growth in spatial reasoning

The overall pre and post-test SRI results are shown in Table 2. Students at all schools demonstrated positive growth in overall spatial reasoning scores over the duration of the Makers Empire program. A Welch’s t-test showed that students demonstrated a growth of 1.82 points (SD = 3.67) in their spatial reasoning score and this was statistically significant with a moderate effect size, t(340) = 4.40, p < 0.0001, d = 0.50. Welch’s t-tests, performed on a per-school basis, showed that the within-school growth was statistically significant in three of the nine schools and these exhibited a moderate to strong effect size as measured by Cohen’s d.

Table 2 SRI Total scores at individual schools showing pre and post scores with growth and Welch's t-test statistics for growth
Full size table

Table 3 shows the pre and post test SRI results broken down by spatial reasoning construct. A statistically significant growth of 0.43 points to 7.55 out of 10 (SD = 2.70) can be seen in Spatial Orientation sub-scores for the participants overall but the effect size of this growth is small, t(678) = 2.70, p = 0.007, d = 0.24. Spatial Visualisation, which recorded the lowest average score prior to the Makers Empire program showed growth of 0.49 points to 4.36 out of 10 (SD = 1.69) which was statistically significant, t(673) = 3.30, p < 0.001, d = 0.29. The largest growth was seen in Mental Rotation. Between pre and post tests, the mean Mental Rotation score rose from 4.79 (SD = 2.42) to 5.67 (SD = 2.52) out of 10. This growth of 0.88 points was statistically significant and had a moderate effect size, t(677) = 4.62, p < 0.001, d = 0.43. Examining the growth in the spatial constructs on a per-school basis shows that it is the same schools—School 1, School 5 and School 7—that exhibit statistically significant growth in each case.

Table 3 Spatial reasoning component scores at individual schools showing pre and post scores with growth and Welch's t-test statistics for growth
Full size table

A factorial ANOVA was performed to examine the effect of growth in the individual spatial reasoning constructs, as well as the effect of their interactions, on overall growth in spatial reasoning total score. The main effects for all three constructs were significant at the 0.001 significance level but none of the interaction effects were statistically significant.

A second factorial ANOVA was performed to examine the effect of school SES, school location, school size and student gender, including the interaction terms of school SES, school location and school size, on growth in students’ overall spatial reasoning score. The main effect of student gender was not found to be a statistically significant predictor of overall spatial reasoning growth.

The analysis indicated that there was a statistically significant interaction between school SES and school size on students’ growth in overall spatial reasoning scores and this was statistically significant, F(1,331) = 5.81, p = 0.016, ω2 = 0.013. A Tukey honest significant differences test indicated that the overall SRI growth for students in small, low-SES schools was 2.37 points higher than for students in large, high-SES schools, padj = 0.003. Similarly, the overall SRI growth for students in small, low-SES schools was 2.86 points higher than for students in large, low-SES schools, padj = 0.033.

The analysis indicated that there was a statistically significant interaction between school SES and school location on students’ growth in overall spatial reasoning scores, F(1,331) = 3.93, p = 0.048, ω2 = 0.008. A Tukey HSD test showed that students in metropolitan, low-SES schools exhibited 1.72 points more growth in overall SRI score than students in metropolitan, high-SES schools, padj = 0.038. Likewise, students in metropolitan, medium-SES schools showed 1.52 points more growth in overall SRI score than students in metropolitan, high-SES schools, padj = 0.018.

The simple main effect of school size showed a significant impact on growth in student SRI scores, F(1,331) = 4.93, p = 0.027, ω2 = 0.011. Students in small schools showed on average 0.86 points more growth than students in large schools, padj = 0.036. The simple main effect of school SES also showed a significant impact of student SRI growth, F(1,331) = 7.209, p < 0.001, ω2 = 0.034. Students in low-SES schools showed 1.26 points more growth than students in high-SES schools (padj = 0.048) while students in medium-SES schools showed 1.55 points more growth than students in high-SES schools (padj = 0.002).

Spatial reasoning trajectories

Using a longitudinal k-means analysis, four distinct spatial reasoning development trajectories were identified. While the number of trajectories is often hard to define, both the Ray and Turi and the Davies and Bouldin criteria suggested that a four-trajectory solution was optimal for these data (see Genolini et al., 2015 for a full discussion.). This analysis (Fig. 6, Table 4) revealed that there are four distinct types of trajectory:

  • Trajectory A (31.8% of students)—these students begin with overall SRI scores below average and show the most rapid growth over time.

  • Trajectory B (30.9% of students)—these students begin with overall SRI scores above average and show slow growth over time.

  • Trajectory C (20.6% of students)—these students begin with overall SRI scores much below average and show very little growth over time.

  • Trajectory D (16.8% of students)—these students begin with SRI scores much above average and show slow growth over time.

Fig. 5
figure 5

Screenshot of the School Attitudes Survey

Full size image
Table 4 Allocation of students to trajectories by school
Full size table
Fig. 6
figure 6

Overall spatial reasoning trajectories

Full size image

These data clearly show that a student’s SRI Total score growth trajectory is dependent on their initial SRI score prior to the Makers Empire program and further examination of Table 4 shows that the distributions of students to trajectories is very dependent on school; i.e. there is no obvious pattern in trajectory distribution at each school. Multinominal logistic regression was used to explore the relationships between school SES, school size, student gender, or school location and which trajectory a student is allocated to. This analysis showed that students at a school in a metropolitan location are 3.52 times more likely than students at regional schools to be on trajectory A rather than on trajectory C. This is statistically significant p = 0.005. Likewise, a student at a low-SES school is 3.42 times less likely of being on trajectory D compared to trajectory A than students attending schools with high-SES. This is statistically significant p = 0.019. All other relationships were not statistically significant.

Changing attitudes

In addition to the SRI, 151 students from school 3, school 7 and school 9 also completed the SAS at the beginning of the Makers Empire program. These students were offered the SAS again at the end of the program and 105 students contributed data. In general, students were found to hold moderately positive attitudes towards school in general at the beginning of the program and these did not change in any statistically meaningful way over the time of the study.

However, considering the subject of Digital and Design Technology there are some small but interesting changes in student attitudes over the duration of the Makers Empire program as seen in Fig. 7. Prior to the program, students’ average SAR towards Digital and Design Technology across all attitudinal factors was 5.3 points lower than their average CAR—their average attitude rating towards school in general. Following the program, this average SAR had risen slightly to 3.2 points lower than their average CAR. In particular, students’ reported creativity SAR improved from a mean of −7.29 points (SD = 19.4) before the Makers Empire program to −2.8 points (SD = 16.0) afterwards. This improvement was statistically significant, t(260) = 1.98, p = 0.049 d = 0.25. While not quite significant at the 95% confidence level, an improvement in students’ anxiety towards the subject was also observed. Prior to the program, students’ anxiety SAR was 5.0 points (SD = 20.0) lower than their anxiety CAR. This decreased to 0.38 points (SD = 18.8) lower than their anxiety CAR by the end of the program, t(259) = 1.87, p = 0.062, d = 0.24.

Fig. 7
figure 7

Partial Attitude Profile for Digital and Design Technologies showing changes in student attitudes over the duration of the Makers Empire program. Note: positive SAR scores indicate the desired outcome

Full size image

Multinominal logistic regression was performed to explore whether or not there were any differences in initial SAR profiles for Digital and Design Technology between students allocated to the different SRI trajectories. While there are some differences between the groups, there were no statistically significant differences at the 90% confidence level or better.

Discussion

Students who undertook this Makers Empire program showed improved spatial reasoning abilities across all three spatial reasoning construct areas. Initial scores for spatial orientation were the highest of the three constructs which might be expected due to its egocentric development (Fernandez-Baizan et al., 2021). The growth in this construct is of even greater interest when the 14% of students who scored 10 out of 10 in this area on the pre-test—and hence cannot show growth—are considered. It is clear that a virtual makerspace environment can have positive impacts on developing students’ spatial orientation.

While the largest growth occurred in mental rotation, improvements in spatial visualisation were more modest. The very small growth in spatial visualisation, a construct where we also found relatively low pre-existing skills, was surprising given the frequency with which spatial visualisation skills should be needed to complete tasks in the virtual makerspace. Spatial visualisation, however, has been connected to mental rotation and spatial orientation by other researchers (see Hegarty & Waller, 2004; Tversky, 2011), and while early years patterning tasks appear to improve spatial visualization (Rittle-Johnson et al., 2019), more complex learning activities appear to have a waning effect on the construct (Harris et al., 2021). Lowrie et al. (2019) reached a similar conclusion when their study investigating the impact of digital technologies on spatial reasoning had similar findings with respect to spatial visualisation.

Improvement in spatial reasoning, however, was not even across the cohort and four distinct trajectories were identified. Students with very low initial spatial reasoning showed very little improvement over the course of the program (trajectory C). This suggests either a threshold effect or a continuing lack of student engagement with learning. Students with above average initial spatial skills (trajectories B and D) showed statistically significant but modest improvement. Students on trajectory A with below average but not very low spatial skills, however, showed the greatest gain.

As has been noted, students in metropolitan locations were 3.52 times more likely than their regional peers to be on the high growth trajectory A rather than the flat-line trajectory C. Similarly, students in low SES schools were 3.42 times less likely than their peers in in high-SES schools to be on the high initial spatial reasoning trajectory D rather than the lower initial spatial reasoning trajectory A. These observations suggest that the strength of the improvement seen in the students on trajectory A might be explained through the lens of disadvantage supporting previous empirical studies (Casey et al., 2011; Levine et al., 2005). Spatial orientation, for example, has been linked to executive function skills such as visuo-spatial working memory, which are often delayed in students coming from disadvantaged backgrounds (Brock et al., 2018). Students from disadvantaged schools in regional or low socio-economic contexts were over-represented on trajectory A, although the associations were weak and data on individual student disadvantage was not available. Further research, therefore, is warranted.

Interpreted from the perspective of self-determination theory, the improvement seen in students on trajectory A might also indicate that the improvement was due more to the autonomy within the learning design than the engagement with spatial learning activities per se. That is, if we treat the pre-test spatial reasoning scores as the outcome of standard schooling within our within-subjects study design, it is possible that standard schooling had better met the basic psychological needs of autonomy, relatedness and competence of students on trajectories B and D, than for students on trajectories A or C. Such an outcome might also be explained through disadvantage, as students from disadvantaged backgrounds are less likely to feel empowered within standard models of schooling (Ryan & Deci, 2020). Again, further research is needed to confidently confirm or reject this mechanism.

Interestingly, our results show that the null hypothesis “student gender, as a main effect variable, is not a statistically significant predictor of overall spatial reasoning growth” cannot be rejected in this study. Historically, some researchers (Hyde, 2014; Lauer et al., 2019; Levine et al., 2005) have suggested that student gender can be a predictor of spatial reasoning development particularly in regard to mental rotation. The data collected in this study do not support this position. There are numerous possible explanations for this lack of effect, including that the unbalanced nature of the design across all possible groupings in our ANOVA has led to a Type II error. The SRI may potentially mitigate this gender effect, as it imposes minimal time pressure, targets middle school students, and relies on 2D images—factors that Linn and Petersen (1985) identify as influential in spatial reasoning research results. Harris et al.'s study (2021) supports this notion, as they found little significance in gender differences while employing the same research instrument. Future research could explore the possibility of using the SRI as a tool to examine gender differences in spatial reasoning growth across a larger and more diverse population.

While of small effect size, our findings with regards to anxiety and creativity are methodologically important in this area of research. Research on learning innovation is too often reliant on affective measures collected immediately post-intervention. Such data presents a clear validity problem in that the positive reaction may be due to a novelty effect (Rodrigues et al., 2022) and may not represent a lasting attitudinal change in the students. Using the SAS, however, we were able to follow much longer-term impacts on attitude towards a particular school subject—in this case Digital and Design Technologies—and we are able to normalise that data against a student’s overall school experience.

In this study we found evidence at a 90% confidence interval that students experienced a reduction in subject anxiety and an increase in creativity in Digital and Design Technologies that is most likely due to engaging with the Makers Empire program. While we were unable to associate different SAR attitude profiles to each of the SRI trajectories with any confidence, we have shown that the allocation of students to trajectories is very dependent on school. Furthermore, the most significant changes in SRI scores were found in School 1, School 5 and School 7 while SAS data were contributed by School 3, School 7 and School 9. This disconnect in the data causes us to consider these attitudinal findings to be ‘statistically interesting’ rather than statistically ‘significant’ but in this complex context it is a strong enough signal to suggest value in further, perhaps qualitative, investigation.

This study adopted an in-situ observer design rather than a randomised control experiment. This approach was adopted primarily to fit the constraints and needs of our school partners, as well as overcome some of the additional challenges brought on as a response to the Covid-19 pandemic. It would therefore have been virtually impossible to recruit an appropriate number of students from similar schools to form control groups. Consequently, we cannot confidently determine a typical spatial reasoning growth trajectory, or the additional growth contributed by the Makers Empire program. However, further investigation of this relationship and the establishment of expected growth trajectories would form the basis for fruitful future research.

Conclusion

Makers spaces inhabit diverse contexts, each with their own affordances and constraints. Throughout this study, it has become apparent that the motivational features and expanded learning opportunities offered by digital makerspaces that make use of CAD have the potential to unlock essential capacities like spatial reasoning, even in the absence of tactility. As technological advancements continue to offer innovative ways of engaging with spatial design processes and students increasingly inhabit virtual spaces, it is crucial to further define and investigate how educators can leverage these processes to directly impact student learning. Nevertheless, we should not overlook the significance of learning within physical environments, as they form the basis of our interactions with space.

The Makers Empire program explicitly explored in this study clearly showed the positive impact of digital design on most students’ spatial reasoning development, particularly in relation to their skills in metal rotation and spatial orientation. While further research is certainly needed, the four distinct growth trajectories identified in this study point to the need for a complex and nuanced model to explain the way that different students engage in a novel learning environment such as the Makers Empire virtual makerspace. By far the most significant impact was with students who had initial spatial reasoning skills that were below average for the cohort. However, a threshold effect in the data was clear. Students with very low spatial reasoning prior to the intervention were not sufficiently in the zone of proximal development for this program to have much impact. A ceiling effect was also evident in the data, and this led to the top half of the cohort showing a much lower rate of growth in spatial reasoning skills than the lower group. It is not clear from the current research, however, if these improvements are attributable to opportunities the program offered to simply practice those skills; to the way that learning design enabled some students to better meet their basic psychological needs in comparison to standard schooling; to a combination of these factors; or to some other, unidentified factors.