1 Introduction

Many aspects of work and everyday life require an understanding of rational numbers (Handel, 2016; Reyna & Brainerd, 2007), and rational numbers are seen as a major obstacle to further progress in mathematics (NMAP, 2008). Rational number understanding has been linked to algebra readiness and algebra ability (Booth & Newton, 2012; Hurst & Cordes, 2018; Siegler et al., 2012). Moreover, rational number knowledge predicts arithmetic proficiency and later mathematics achievement in general (Bailey et al., 2012; Siegler et al., 2011). Hence, research findings have indicated the importance of good rational number skills, and at the same time, they show that many students and even educated adults, including (prospective) teachers, find rational numbers difficult. Many learners struggle with the transition from natural number reasoning to rational number reasoning (Depaepe et al., 2015; McMullen et al., 2015; Ni & Zhou, 2005; Van Hoof et al., 2018). Thus, there is a need to develop instructional interventions that aim to promote rational number knowledge. The purpose of this study was to examine the learning outcomes of a quasi-experimental classroom intervention that used a new digital game-based learning environment, the NanoRoboMath game. We used a pre- and post-test design with experimental and control conditions to investigate the effects of playing the game with respect to (a) rational number conceptual knowledge and (b) adaptive rational number knowledge.

2 Natural number bias and conceptual change

A long history of research has investigated various qualitative changes in the way children understand numbers, including transformations from multiplicative reasoning with whole numbers to rational numbers (Fuson, 1988; Gelman & Gallistel, 1978; Nunes & Bryant, 1996). Many difficulties learners face in understanding rational numbers can be explained by a phenomenon called natural number bias, which is a tendency to apply natural number features in rational number tasks in which natural number reasoning leads to an incorrect answer (Ni & Zhou, 2005; Vamvakoussi et al., 2018). Multiple theories of conceptual change argue that prior conceptions can constrain students’ ability to learn new content (Chi, 2008; diSessa, 2008; Vosniadou & Verschaffel, 2004). In mathematics, previous research has shown that conceptual change is needed to accommodate learners’ initial natural number biased ideas to new aspects of rational numbers that are incompatible with their prior knowledge. Typical examples of natural number biased errors relate to size, representations, operations, and density (Merenluoto & Lehtinen, 2004; Van Hoof et al., 2015b; Vosniadou et al., 2008).

First, the representation of a rational number is not related to its size in the same way it is with natural numbers. Students often mistakenly think that longer decimal representations correspond to larger numbers (Resnick et al., 1989; Smith et al., 2005). Moreover, learners wrongly assume that the magnitude of the fraction increases as its denominator or both the denominator and numerator increase without paying attention to the relationship between these two components (Clarke & Roche, 2009; Stafylidou & Vosniadou, 2004). Second, learners are faced with the difficulty that despite the differences in their notation and context of use, fractions, decimals, and percentages are not different types of numbers but are merely alternative ways to represent rational numbers. As they do not realize that the same number can be written in infinitely many different ways (for example, 50% = 0.50 = 0.500 = 2/4 = 3/6), they treat the different symbolic representations as different numbers (Vosniadou et al., 2008).

Third, the effects of arithmetic operations on the magnitude of the result differ from the natural number context. Learners may keep assuming that the rules “multiplication makes bigger” and “division makes smaller” are valid in the rational number context (Christou, 2015; Fischbein et al., 1985; Vamvakoussi et al., 2012; Van Hoof et al., 2015b). However, with rational numbers, the product may be smaller than the multiplicand if the magnitude of the multiplier is smaller than one (for example, 0.4 × 8 = 3.2). Similarly, if the magnitude of the divisor is less than one, then the quotient can be greater than the dividend (for example, 3 ÷ 0.5 = 6). Fourth, many learners struggle to understand the dense structure of rational numbers (Neumann, 1998; Vamvakoussi & Vosniadou, 2004; Vamvakoussi et al., 2011). Every natural number n has a unique successor n + 1, and there is a finite number of natural numbers between any two natural numbers. For any rational number, there is no next larger rational number, and between two different rational numbers, there are always infinitely many rational numbers.

Research findings have suggested that substantial conceptual change may be required to understand fully the inconsistencies between natural and rational number reasoning (McMullen et al., 2018; Van Hoof et al., 2018). However, aspects of rational numbers that are incongruent with natural numbers are rarely explicitly stated in mathematics instructional material (Van Dooren et al., 2019). This is problematic, because teachers themselves may have limited content knowledge and pedagogical content knowledge on rational numbers (Depaepe et al., 2015). Without explicit material to support instruction, the natural number biased aspects of rational numbers may be dealt with insufficiently in a comprehensive school context.

Developing students’ intuitive concepts of number towards rational number reasoning is challenging for learners and teachers (Merenluoto & Lehtinen, 2004; Vosniadou et al., 2008). Even instructional interventions that aim to elicit conceptual change appear to have limited effects on student learning, for example, in the case of density knowledge (Vamvakoussi & Vosniadou, 2012). On the one hand, approaches in which students are explicitly forced to confront their misconceptions have appeared successful. For example, refutational texts, which systematically attempt to engage, challenge, and remediate readers’ misconceptions, have been found to facilitate conceptual change in different areas of science and mathematics education (Christou & Prokopou, 2019; Mikkilä-Erdmann, 2001; Tippett, 2010). On the other hand, some researchers have suggested that we should work with — rather than against — students’ intuitions and in an empowering way shift students’ perceptual attention towards properties aligned with expertise (Shvarts & Abrahamson, 2019; Smith et al., 1993). Further research is needed to determine the teaching approaches that are effective in different contexts (Leonard et al., 2014). Vosniadou and Verschaffel argued that the role of instruction is to enable intentional learners to develop metacognitive skills and multiple new perspectives to overcome the barriers imposed by their initial explanatory frameworks (Vosniadou & Verschaffel, 2004). Some studies have indicated that game-based learning might be one approach that is helpful in supporting this process (Ketamo & Kiili, 2010; Koops & Hoevenaar, 2013). To build on this body of knowledge, we examined the effectiveness of a digital game-based learning environment in promoting rational number knowledge in this study.

3 Adaptive rational number knowledge

Many studies have concentrated on difficulties in rational number understanding (for example, Fuchs et al., 2013; Tian & Siegler, 2017), but there is also some work aimed at investigating and supporting the development of high-level skills and adaptive expertise with rational numbers (Baroody, 2003; Verschaffel et al., 2009). Previous examinations of high-level rational number knowledge have focused either on specific aspects of conceptual knowledge, such as density concepts (Vamvakoussi & Vosniadou, 2004), or more nebulous constructs of a rational number sense that involve a wide range of skills and knowledge unsuitable for large-scale, targeted investigations (for example, Moss & Case, 1999). Recently, McMullen and colleagues (2020) examined students’ capability to integrate multiple features of their procedural and conceptual knowledge of rational numbers in a novel task, as a means to examine a specific behavioural manifestation of adaptive expertise with rational numbers, which they called adaptive rational number knowledge. According to Baroody (2003), adaptive expertise depends on well-connected conceptual knowledge and its integration with procedural knowledge for use in novel contexts.

Adaptive expertise in arithmetic problem-solving involves switching between multiple solution strategies and choosing the most appropriate strategy for a given problem based on task characteristics, personal characteristics, and socio-cultural norms (Verschaffel et al., 2009). Thus, it has been theorized that adaptive expertise with arithmetic is supported by two key components, flexible switching between solution strategies in a problem-solving situation (Torbeyns et al., 2006) and adaptive number knowledge (McMullen et al., 2016). Adaptive number knowledge is defined as a rich network of knowledge of numerical characteristics and the arithmetic relations between numbers, which can be flexibly applied in solving novel tasks. This knowledge includes being aware of the nature of the number system and identifying numerical characteristics, such as being close to “nice” numbers (for example, 47 is close to 48, which has many factors) (Dowker, 1992) and recognizing or estimating factors and multiples.

Adaptive number knowledge was first considered with whole numbers (McMullen et al., 2016, 2017), but the concept has recently been extended to rational numbers (McMullen et al., 2020). The arithmetic and numerical skills and knowledge needed for strong adaptive rational number knowledge appear to align with features of conceptual knowledge. However, adaptive rational number knowledge was distinct from students’ conceptual and procedural knowledge of rational numbers (McMullen et al., 2020). Based on their analysis of students’ responses to the rational number version of the arithmetic sentence production task, McMullen and colleagues (2020) suggested that the integration of this knowledge across multiple concepts most clearly supports adaptive rational number knowledge, for instance, making a connection between the following: (a) knowledge of magnitude (for example, knowing that 0.25 is less than 1), (b) knowledge of the effects of arithmetic operations (for example, knowing that 0.25 × 4 is less than four), and (c) representations knowledge (for example, knowing that 0.25 and ¼ are interchangeable, so ¼ × 4 is equal to 0.25 × 4). Such rich cross-conceptual connections and their affordances in creating novel procedural solutions are in line with theoretical descriptions of adaptive expertise.

As with adaptive whole number knowledge (for example, McMullen et al., 2016), we expect that there are specific arithmetic and numerical skills and knowledge related to adaptive rational number knowledge. Numerical knowledge may revolve around the nature of the rational number system, for example, extending the base-ten structure of whole numbers to decimal numbers and linking this representation with the representation of fractions. In addition to the principles extended from the natural number context, like commutativity and associativity, well-developed arithmetical skills needed for strong adaptive rational number knowledge may include skills like simplifying and extending fractions and finding common denominators. For example, in the subtraction problem 2.25 − 3/4, crucial arithmetic and numerical knowledge may involve recognizing that 2.25 is two and one quarter but is also nine quarters, close to two, or one quarter less than two and a half. Similarly, 3/4 is 0.75 but is also one quarter less than one, close to one, or a half and one quarter. Both numbers are multiples of quarters, there are two whole numbers between them, both differ from the closest whole number by a quarter, 2.25 is a multiple of 3/4, and so on. Being aware of the characteristics of numbers and their relations influences the strategy chosen to calculate the difference (Threlfall, 2002).

An ability to use adaptive number knowledge when carrying out whole number arithmetic is a relevant component of students’ mathematical knowledge (McMullen et al., 2017, 2020). Substantial individual differences in adaptive number knowledge exist from primary to university levels (McMullen et al., 2016, 2020). These individual differences cannot be fully explained by procedural or conceptual knowledge. In a sample of middle school students, adaptive rational number knowledge substantially differed even among the top 45% of students, who all had strong conceptual knowledge of rational numbers’ size, representations, and operations as well as strong procedural knowledge of rational number arithmetic. Those students with high adaptive rational number knowledge were fluent in switching between decimal and fraction notation, and they could integrate different aspects of rational number procedural and conceptual knowledge flexibly. Adaptive number knowledge with both whole and rational numbers uniquely predicts later pre-algebra skills, even after taking into account conceptual knowledge and procedural fluency (McMullen et al., 2017, 2020).

4 Digital game-based learning environments for rational number learning

Playful learning is considered fundamental in early cognitive development (Pellegrini, 2009; Rogers & Sawyers, 1988). Mathematical knowledge can develop through exploratory play activities (for example, Ginsburg, 2006), and technology may enhance mathematical thinking by providing a new kind of tool for the discovery of mathematical ideas (Devlin, 2011). In addition, technology may have positive effects on students’ attitudes towards mathematics and may increase engagement with the subject matter as well as improving students’ mathematical achievement and conceptual understanding (Baker et al., 2015; Guerrero et al., 2004; Kiili et al., 2018; Li & Ma, 2010). Deeper engagement and deeper forms of learning are supported by immersion and flow in the imaginary world of play (Csikszentmihalyi, 2014). Thus, games, playful learning, and the use of technology are recommended in many policy documents and curricula (for example, the Finnish National Agency for Education, 2014; OECD, 2010, 2018).

However, technology is often used only as an alternative content-delivery method without providing novel ways to interact with educational content (Bray & Tangney, 2017). Hence, the potential affordances of technology for mathematics education have not yet been fully exploited. Most game-based learning environments focus on drill-and-practice exercises (Laato et al., 2020), which aim to enhance procedural fluency while neglecting other aspects of mathematical proficiency, such as conceptual understanding, strategic competence, and adaptive reasoning (Byun & Joung, 2018). In many cases, the relationship between game features and mathematical learning content is problematic. Integrating learning content into the design of educational digital games instead of just placing it as an additional extra has been seen as vital for successful educational game design (Devlin, 2011; Habgood & Ainsworth, 2011; Young et al., 2012). Such integrated game design directly links core game mechanisms with learning objectives. Achieving coherence between game features, learning content, and means of measuring intended learning outcomes requires an iterative process of design and testing (Brezovszky, 2019; Vanden Abeele et al., 2012).

In recent years, studies examining the effects of digital game-based learning environments on players’ rational number competencies have yielded positive results. For example, games have been shown to support students’ understanding of fraction equivalence and comparison, number line estimation, arithmetic skills, and attitudes towards fraction learning (Gresalfi et al., 2018; Masek et al., 2017; Nejem & Muhanna, 2013; Riconscente, 2013). Further, the affordances and efficiencies provided by properly designed digital games may improve students’ engagement and motivation and enhance their conceptual understanding of rational numbers (Zhang et al., 2020). In general, research has highlighted several criteria for well-designed learning environments. They should be engaging and challenging, provide feedback and accommodate individual needs, support sense-making with visual representations, grow curiosity and support exploration, stimulate interaction, and be intuitive to use (Pelton & Pelton, 2011; Zhang et al., 2020). A game design model recently developed for fraction learning emphasized also the role of immersion and the player’s identity (avatar), the need to increase complexity gradually, rewards for completing levels, a variety of ways to monitor players’ progress, and above all, the need to be instructional in a way that enables players to learn on their own by meeting the challenges of the game (Cyr et al., 2019).

One of the main ways games achieve positive learning effects is the use of visual models to support the cognition process (Zhang et al., 2020). Examples of such models include the concept of splitting and the use of area models to develop players’ understanding of the part–whole meaning of fractions as in the games Refraction (Martin et al., 2015), Pizza al Lancio (Gaggi et al., 2018), and Slice Fractions (Cyr et al., 2019). Another type of visual model that emphasizes the measurement interpretation of rational numbers is the number line, which was used, for example, in Semideus (Ninaus et al., 2017), Motion Math (Riconscente, 2013), and Catch-the-Monster (Braithwaite & Siegler, 2020) to support players’ understanding of numerical magnitudes. Notably, many studies advocate the use of number lines in fractions instruction over part–whole representations (Hamdan & Gunderson, 2017; Sidney et al., 2019; Siegler et al., 2010).

Evidence suggests that these game-based environments improve students’ conceptual understanding of rational numbers. There are two main limitations of these studies that the present study aims to address. First, they mostly deal with magnitude concepts and visual and symbolic fraction representations. Second, previous studies rarely utilized conceptual change perspectives in designing learning environments. Game-based learning environments may allow for the playful exploration of assumptions such as “multiplication makes bigger, division makes smaller”. Thus, the present study aims to address the above-mentioned limitations by determining whether the use of exploration learning and a conceptual change approach in game design can support students’ conceptual knowledge of fraction and decimal operations, representations, and density.

Game-based learning environments have also been proposed as one way to support adaptive expertise in mathematics (Lehtinen et al., 2017). The opportunities to explore numbers and arithmetic relations afforded by game-based learning environments may promote the rich network of knowledge needed for adaptive expertise in mathematics (Yu & Denham, 2021). For example, a digital math game, Wuzzit Trouble (Pope & Mangram, 2015), promotes a wide range of competences indicated by students’ number sense. Further, a large-scale randomized classroom trial showed that the Number Navigation Game promotes adaptive number knowledge with whole numbers by presenting players with opportunities to explore numerical characteristics and arithmetic relations (Brezovszky et al., 2019). Another large-scale intervention found that an online game-based learning environment, Hit the Target, improves players’ algebraic reasoning during gameplay (van den Heuvel-Panhuizen et al., 2013). However, we are not aware of any game-based learning environments that target adaptive expertise with rational numbers.

5 Present study

The aim of the present study was to explore the effects of playing in a digital game-based learning environment, NanoRoboMath, on fifth and sixth grade primary school students’ (a) conceptual and (b) adaptive rational number knowledge. NanoRoboMath integrates mathematical concepts and practices into the core mechanics of the game (Habgood & Ainsworth, 2011) to improve players’ conceptual rational number knowledge of size, representations, effects of operations, and density. Moreover, following the constructivist game design framework, NanoRoboMath aims to engage players in exploring and experimenting with domain-relevant representations emphasized by Holbert and Wilensky (2019) to enhance their adaptive rational number knowledge. The game-like environment is expected to support cognitive and motivational factors that lead to conceptual change, such as highlighting natural number prior conceptions, increasing sensitivity to novel mathematical aspects, and promoting tolerance for ambiguity (Merenluoto & Lehtinen, 2004). A detailed description of the game design has been provided in a recent study by Kärki et al. (2021).

In the game, players adopt the role of a superhero who carries out challenges by navigating a nanorobot along a number line using four basic arithmetic operations and rational numbers. In the prototype piloted in this study, the player’s challenge was to clean polluted water by finding and destroying bacteria or to destroy a virus and cure a pet. The player’s goal was to move the nanorobot from a starting position (the “10” in the upper screenshot of Fig. 1) to a target position (the “\(3\frac{1}{4}\)” in Fig. 1). These positions are indicated in a number line that re-scales (zooming in or out) based on the distance between the location of the nanorobot and the target (see the difference between the number lines in the two screenshots of Fig. 1). The upper part of the screen shows a calculation panel for moving the nanorobot. Players are challenged by a problem-solving situation that resembles equation solving and must create an arithmetic sentence to move the nanorobot towards the target. The current location of the nanorobot (the “10” and “5” in Fig. 1) is given in the panel. The player chooses one of the four basic arithmetic operations (for example, the division and minus symbols selected from the drop-down menu in Fig. 1) and tries to determine a favourable rational number as a second operand (see the “2” and “1.75” that the player inserted). After pressing the calculate button, the game calculates the result of the arithmetic operation entered by the player, and the nanorobot moves to this new value (moving from 10 to 5 between the upper and lower screenshots of Fig. 1).

Fig. 1
figure 1

Screenshots of NanoRoboMath showing a player moving from 10 to \(3\frac{1}{4}\) in two steps. Translations: valikko = menu, mene kohteen täsmälliseen sijaintiin = go to the exact location of the target, kohde = target, pisteet = score, timanttiraja = diamond limit, kultamitaliraja = gold medal limit

Full size image

The player may use several moves and ways to reach the target. Hence, the game is not just drill-and-practice of rational number arithmetic, but it enables the free exploration of rational numbers and their operations. In our previous study (Kärki et al., 2021), three fifth grade students and one seventh grade student tested a preliminary version of the game. The goal of the study was to investigate what kind of rational number arithmetic practices NanoRoboMath elicits and what kind of playing strategies the players use. We found that a prototype of NanoRoboMath engaged players with rational number arithmetic and induced them to use a large variety of number–operation combinations. Observed combinations varied between players and in individual games. In this way, the game should support the formation of well-connected knowledge about the relations between numbers and operations and therefore enhance the adaptive rational number knowledge of players. The present study aims to examine this possibility.

One of the instructional implications of the conceptual change approach is that it can be used to identify concepts in mathematics that will probably cause particular difficulties in students’ learning and predict common systematic errors (Vosniadou & Verschaffel, 2004). In our game design, these concepts were natural number biased features: size, representations, effects of operations, and the density of rational numbers. To this end, there are several basic design features of the game that aim to improve players’ conceptual understanding about rational numbers. First, the number line representation is used to support players’ understanding of rational number magnitude. The dynamic, scalable number line might also help students understand that one can always find an infinite number of rational numbers between any two rational numbers. Hence, it aims to trigger conceptual change concerning the concept of density of the rational number set. The numbers shown on the number line are represented using decimal notation, but in the calculation panel, the player uses either decimal numbers, fractions, or a mixture of these representations in one arithmetic sentence. Thus, recognizing connections between symbolic calculations with different rational number representations and decimal numbers positioned on the number line might enhance players’ representation knowledge.

The two playing modes in the game were designed to promote students’ adaptive and conceptual rational number knowledge. The power mode encourages players to explore the numerical characteristics and arithmetic relations of rational numbers. In these tasks, players should reach the target by consuming as little power as possible. Power is represented by the magnitude of the number entered as the second operand in the arithmetic calculation. Hence, successful game performance requires that players minimize the magnitudes of the numbers used in arithmetic operations, which often makes the use of multiplicative operations and multiplicative inverses beneficial. Players are challenged by rewards and hints to take this feature into account. Experimenting with multiplying and dividing by numbers less than one could challenge the misconception that multiplying makes bigger and dividing makes smaller. In time mode tasks, players should move to a given interval surrounding the target as quickly as possible. Players gain more points the closer to the exact target the nanorobot gets and the faster the given target interval is reached. Thus, there is a strategic trade-off between (a) quick approximate and (b) slower precise calculations. Quick approximate calculations might lead players to estimate the magnitudes of rational numbers involved in the calculations and pay more attention to the relationship between magnitudes and operations, something that is not necessarily automatic (Braithwaite & Siegler, 2020).

In this study, we used a prototype of the game, which consisted of three sessions each containing two intro levels, six ordinary levels, and two bonus levels. The sessions differed from each other in the way they represented rational numbers in the calculation panel. The first session used decimal representations, the second session used fraction representations, and the third session contained calculations with both representation types. Intro levels consisted of two tasks (targets), while ordinary levels had four tasks, and extra levels had eight tasks. Half the levels were power mode tasks and the other half were time mode tasks. Students had to play ordinary levels in a given order, and the next level could only be opened after the previous level was completed. Intro levels could be played at any time, and extra levels could only be completed after finishing the preceding ordinary levels.

To examine whether NanoRoboMath could be an ecologically valid instruction tool that supports the development of conceptual and adaptive rational number knowledge in primary education, we carried out a quasi-experimental classroom intervention in which a group of students played NanoRoboMath during three mathematics lessons. A control group attended normal (business-as-usual) math lessons, which did not contain rational number instruction. Both groups participated in pre- and post-tests that measured adaptive rational number knowledge and aspects of conceptual knowledge of rational numbers. We investigated the following research questions:

  • Does playing NanoRoboMath enhance students’ rational number conceptual knowledge, and in particular conceptual knowledge of:

    • Representations of rational numbers,

    • Effects of operations with rational numbers,

    • The density of rational numbers?

  • Does playing NanoRoboMath enhance students’ adaptive rational number knowledge?

  • Does students’ performance on NanoRoboMath predict the development of mathematical learning outcomes?

6 Methods

6.1 Participants

Participants were 195 fifth and sixth grade primary school students from two schools in two cities in the southwest of Finland. Overall, 10 classes took part in the study. Randomization to experimental and control conditions was done at the classroom level, because classrooms are considered ecologically valid units of measurement in the field of education (Hedges & Rhoads, 2010). One fifth grade and one sixth grade class from School A and two fifth grade and two sixth grade classes from School B formed the experimental group. One fifth grade and one sixth grade class in both Schools A and B formed the control group. Table 1 provides a description of the distribution of participants by grade in experimental and control groups.

Table 1 Number of participants by grade and experimental condition
Full size table

The ethical guidelines of the University of Turku were followed. Participation was voluntary, the informed consent of parents was obtained, and students gave verbal assent to participate in the study. Students could remove themselves from the study at any time. All students in experimental classes played NanoRoboMath during their normal mathematics lessons. Data were only gathered from students who had volunteered to participate in the study. Participants completed pre- and post-tests as part of their regular school work. For ethical reasons, the teachers of control classes were given the opportunity to use NanoRoboMath in their mathematics lessons after the post-test.

6.2 Procedure

Classrooms participated in the intervention either at the end of the autumn semester or at the beginning of the spring semester. For all classes, pre- and post-tests were within 2 weeks of each other. All measurements were carried out in a 7-week window. Overall, two thirds of experimental classes and half the control classes participated in the intervention during the autumn semester. Between tests, classes in the experimental group played NanoRoboMath in three mathematics lessons, whereas control classes continued with their regular school work. Before data gathering, teachers of classes in the experimental group received information about the main features of NanoRoboMath and written instructions on how to use the game during lessons. They also participated in a short training session, in which some technical aspects of playing the game and the basic game mechanics were demonstrated. Teachers could freely choose if they wanted their students to play the game in pairs or individually, and in case of pair play, teachers selected the pairs. The game was distributed to the experimental group on individual pen drives and was played on personal computers. Pen drives were also used to store players’ game log data. Gameplay replaced regular mathematics teaching; thus, the intervention group did not receive more mathematics training than the control group. Control group teachers did not receive any instructions concerning mathematics lessons during the intervention period. According to teachers’ notifications, mathematics teaching for the control group did not concentrate on rational number knowledge. Owing especially to the novel nature of tasks, the control group was used for controlling the testing effect caused by the repeated performance of a similar test.

In the beginning of each of the three playing sessions, teachers of experimental groups were instructed to play the two intro levels with the whole class. Whole-class discussion about the different choices of moves was encouraged. After the intro levels, students were supposed to play at least six additional levels independently. Based on log data, the average total playing time in the six experimental classes was 90 min (SD = 20 min). Average gameplay time varied from 81 to 110 min between classes, meaning that some classes spent almost half an hour more playing than other classes. We interpreted these differences in gameplay time as a natural variation that occurs when this type of method is introduced in a naturalistic school context. Our aim was to create an ecologically valid design, so we included all classes in comparisons between experimental and control groups despite these time differences.

6.3 Measures

Pre- and post-tests consisted of paper-and-pencil measures of rational number knowledge, including adaptive rational number knowledge and conceptual knowledge of (a) representations, (b) arithmetic operations, and (c) density. Due to limitations in testing time, we did not measure size concept separately but considered it relevant in the measures of adaptive rational number knowledge (McMullen et al., 2020) and operations (Siegler & Lortie-Forgues, 2015). These four measures appeared in this order in both tests and were calculated separately as their own sum variables. To create a balanced overall measure of rational number conceptual knowledge, we calculated the mean of the z-standardized scores of the three conceptual knowledge variables. The tests were administered by one of two testers, a researcher and an assistant, who used an automatic slide show, including a standardized timer and sound signals. The tests took around 20 min to complete. Both tests were coded to an electronic form by a research assistant who was unaware of the treatment applied to any given group.

Adaptive rational number knowledge was measured using the arithmetic sentence production task (McMullen et al., 2017), which aims to capture students’ ability to recognize and use different numerical characteristics and relations. In each item of the task, students were given five numbers and a target number and were asked to generate, in 90 s, as many mathematically correct arithmetic sentences as possible that combined some of these five numbers and produced the target number. The time limit was controlled by the tester, whose task was to ensure that all students turned their pages at the same time. No problems with following the time limit rules were observed. In arithmetic sentences, students could use four basic arithmetic operations, parentheses, and each of the given numbers repeatedly. The pre-test began with a practice item that asked students to produce arithmetic sentences with whole numbers 1, 2, 3, and 4 and a target number 6 in 60 s. After this item, students could ask clarifying questions to ensure that they had understood the task. In the post-test, there was no practice item. Table 2 shows the items for the pre- and post-test. Note that each item included two pairs of equivalent fractions and decimals (for example, ½ and 0.5, ¼ and 0.25). In terms of scoring, 1 point was given for each answer if it was mathematically correct, only used given numbers, and was not a literal repetition of previous answers. However, the correct use of parentheses was not required, and sentences with equivalent rational number representations were both counted as correct. For example, 2 − ¾ + ¾ = ½ was interpreted as 2 − (¾ + ¾) = ½, and ½ + ½ was counted as correct, even if the student previously answered 0.5 + 0.5. Cronbach’s α for arithmetic sentence production tasks was 0.89 for the pre-test and 0.85 for the post-test.

Table 2 Items of the arithmetic sentence productions task for pre- and post-tests
Full size table

Conceptual knowledge of representations was measured using an adapted version of the Number Sets Test (Geary et al., 2009). Students had 1 min to identify the symbolic and non-symbolic representations of a given rational number. There were two items in both tests. Each item had 15 alternatives, of which nine were correct. For each correct answer, 1 point was added, and for each incorrect answer, 1 point was subtracted. Hence, the maximum number of points per item was 9. The target numbers were ½ and 0.9 in the pre-test and ¼ and 0.1 in the post-test. Cronbach’s α for representation knowledge tasks was 0.75 for the pre-test and 0.76 for the post-test.

Knowledge of rational number operations was measured using six multiple choice items adapted from Van Hoof et al. (2015a). Items tested students’ knowledge of the effects of arithmetic operations with fractions and decimals using questions, including the following: “Is the outcome of 40 × 1/3 smaller or larger than 40?” and “What is half of 1/6?” This test measured students’ performance on items that would be answered incorrectly if natural number biased reasoning was used. One point was given for each correct answer. The pre- and post-tests contained the same items. Cronbach’s α for rational number operations knowledge tasks was 0.52 for the pre-test and 0.66 for the post-test.

Density knowledge was assessed using multiple choice items adapted from Vamvakoussi and Vosniadou (2010). Items asked students how many numbers there were in a given interval between two rational numbers, x and y. For each item, four options were given, which were scored as follows: totally sparse (“There are no numbers between”) was given 0 points, a limited set (“There are numbers x1, x2 … and xn between x and y”) was given 1 point, partial density (“There are a lot of numbers between”) was given 2 points, and full density (“There are an infinite number of numbers between”) was given 3 points. Pre- and post-tests contained exactly the same items; see Table 3. Cronbach’s α for rational number density knowledge tasks was 0.71 for the pre-test and 0.85 for the post-test.

Table 3 Intervals [x, y] and limited sets x1, x2 … and xn of density items
Full size table

Game performance was measured using log data from each player. The number of completed tasks, total score for power mode tasks, and total score for time mode tasks were computed from game log data. We argue that these measures of gameplay can be used as reliable estimates of students’ practice with different combinations of rational numbers and operations during the game. Together, they reflect the quantity and quality of participants’ gameplay. Thus, the three measures were independently z-standardized and summed to create a balanced measure of overall gameplay. We note that the total number of completed tasks does not mean that all these tasks were different; students could repeat the same task to achieve a better score. Similarly, the total score was the cumulative score obtained by a player every time they completed a task. On average, students completed M = 68.02 tasks (SD = 27.47) with a range of 11–150 tasks. The average total power mode task score was M = 2632.90 (SD = 1172.74) with a range of 517–6177, and the average total time mode task score was M = 2254.26 (SD = 1208.69) with a range of 154–5767. To ensure sufficient gameplay, we excluded from the analysis students whose number of completed tasks was lower than two standard deviations below the mean of all players. Moreover, to avoid including players who only repeated the first few levels, students who completed less than four ordinary levels (16 different tasks) were excluded. Based on these exclusion criteria, which were agreed in the pre-registration of our study (https://osf.io/gj23k), eight of the 118 students in the experimental group were excluded from the data analysis.

7 Results

First, we wanted to test the effects of the intervention on students’ rational number conceptual knowledge, both overall and separately as its three components. For each rational number conceptual knowledge measure, we conducted a repeated measures ANOVA with condition (experimental or control group) as a between-subject factor. Following the analysis plan set out in the pre-registration, we considered the interaction effect of condition and time as the main test for the overall effectiveness of the intervention. According to the pre-registration, we relied on one-sided tests and Holm–Bonferroni corrections of multiple tests to determine statistical significance. The results of this analysis have been presented in Table 4, which shows that the intervention had a small positive effect on students’ rational number conceptual knowledge. The percentage of correct answers increased from 37 to 39% in the experimental group and decreased from 46 to 44% in the control group. In addition, students’ conceptual knowledge concerning rational number operations showed a statistically significant positive effect, but there were no statistically significant interaction effects concerning representation and density knowledge. Note that the negative values in measures of rational number conceptual knowledge are due to standardization when forming the composite overall measure.

Table 4 Repeated measures ANOVAs showing the interaction effect of condition and time
Full size table

As a second research question, we investigated whether or not playing NanoRoboMath enhanced students’ adaptive rational number knowledge. Following the pre-registration, this was also tested with the interaction of condition and time in a repeated measures ANOVA. Table 4 shows that although performance improved in the experimental group more than in the control group, the interaction effect in the case of adaptive rational number knowledge was not statistically significant.

To investigate how participants’ gameplay predicted their learning outcomes, linear hierarchical regression analyses were conducted. We explored the impact of game performance on the experimental group’s learning outcomes considering their adaptive rational number knowledge and rational number conceptual knowledge of representations, the effects of operations, and density. As shown in Table 5, after taking into account grade-level and pre-test scores, game performance still explained part of the variance of adaptive rational number knowledge and knowledge of representations and operations. However, for density knowledge, the effect of game performance was not statistically significant.

Table 5 Hierarchical linear regression analysis showing the impact of game performance on mathematical learning outcomes in the experimental group (n = 110)
Full size table

8 Discussion

The main objective of this study was to determine whether a new game-based learning environment, NanoRoboMath, improved players’ rational number knowledge. Results showed small but significant effects of gameplay on rational number conceptual knowledge. Moreover, students’ game performance predicted their improvement in adaptive rational number knowledge and in conceptual knowledge of the representations of rational numbers and the effects of operations with rational numbers. However, we did not observe an effect of game performance on students’ understanding of the density of rational numbers.

In this study, rational number conceptual knowledge comprised three aspects, namely, representations, the effects of operations, and density. Interventions that aim to improve students’ understanding of these kinds of natural number biased aspects of rational numbers appear to have restricted learning effects (see, for example, Vamvakoussi & Vosniadou, 2012). Our results showed that playing NanoRoboMath seemed to have a small positive impact on students’ rational number conceptual knowledge in general, but in terms of its components, a positive effect was perceptible only in students’ understanding of operations. While the experimental group’s performance improved during the intervention, their answers did not indicate a firm understanding of the effects of operations. On average, only 27% of students’ answers concerning rational number operations were correct in the post-test, which was still less than the corresponding percentage in the control group (33%). It should be noted that the performance of the control group was lower in the post-test than in the pre-test, but this could have been at least partly due to the use of different items in the two measurements, even though items were similar overall. Moreover, reliability values concerning the scale of operations were low, so the positive effects of NanoRoboMath on students’ rational number conceptual knowledge should be interpreted cautiously.

The intervention had no significant effect on density scores. In both groups and for both measurements, the average proportion of correct solutions was 29–34%. Moreover, game performance had no significant impact on students’ density knowledge, even though it had a positive effect on representation and operation knowledge. While this is in line with earlier findings, which suggest that a correct understanding of rational number density may be difficult to accomplish (Vamvakoussi et al., 2011), the density property had a different role in the game than representation and operation knowledge. In the game, players had to mix the different representations of rational numbers, and they needed a good understanding of the effects of operations to achieve high scores in power mode tasks. In contrast, the density property was only indirectly manifested in the zooming feature of the number line, as when students moved closer to the target, it revealed more and more rational numbers between the nanorobot and the target. Moreover, the scores and completed tasks indicated the overall amount and quality of gameplay, but there was no specific measure of gameplay designed for measuring the understanding of the density property.

In addition to conceptual knowledge, our aim was to examine how playing NanoRoboMath affected students’ adaptive rational number knowledge. Free exploration with rational numbers, observed in an earlier investigation of NanoRoboMath (Kärki et al., 2021), should develop well-connected knowledge about the relations between numbers and operations and, therefore, enhance players’ adaptive number knowledge (Lehtinen et al., 2015). Our short intervention of three 45-min gaming lessons did not induce a significant difference in the development of adaptive rational number knowledge between the experimental and control groups. However, students in the experimental group improved their performance so that in the post-test, they created approximately 0.6 correct arithmetic sentences more than in the pre-test, while the average improvement was only 0.3 sentences in the control group. The increase in both groups might be partly explained by practice effects created by repeated testing, especially taking into account the novel nature of the sentence production task. Moreover, there was a small but statistically significant positive impact of game performance on adaptive rational number knowledge after taking into account participants’ grade-level and pre-test scores. This result encourages us to further develop the game, keeping in mind that the exploration of different number–operation combinations has a potential positive impact on adaptive number knowledge.

In future research, the implementation of the gaming intervention should be reconsidered to measure the learning outcomes of the game more reliably. NanoRoboMath should be considered a tool of instruction in a holistic classroom context and not an isolated learning method. Although the number line has been seen as a key representational tool for improving students’ conceptual understanding of rational numbers (Hamdan & Gunderson, 2017; Sidney et al., 2019; Siegler et al., 2010), it might be that the implicit feedback and hints in the game are insufficient for explaining the concepts of rational numbers and their operations, for example, the idea that multiplying can “make smaller”. Even though completing tasks in NanoRoboMath is fairly easy, good performance in the game requires either firm pre-knowledge of the meaning of rational numbers and their operations or appropriate pedagogical support. The supportive educational context and the role of the teacher are critical factors in the successful integration of digital technologies into mathematics classes (Drijvers, 2015; Gresalfi et al., 2018). Therefore, teachers were instructed to enable peer and whole-group discussion about different playing strategies. However, the timing of the present study prevented the full integration of the game into existing rational number instruction. Future studies should consider the role of the game in rational number instruction and whether it should be used as a supplement, extension, or elaboration of typical instruction. Future interventions using NanoRoboMath should also take into account in their study design the complexity and gradual development of the aspects of rational number knowledge that require conceptual change, for instance, by including a delayed post-test.

Following the iterative process of designing and testing game-based learning environments (Brezovszky, 2019; Vanden Abeele et al., 2012), this study suggests that the density concept should be more emphasized in future versions of NanoRoboMath. Game mechanisms should be more directly linked to learning objectives (Devlin, 2011; Habgood & Ainsworth, 2011; Young et al., 2012), and students should be explicitly guided to confront their misconceptions about density. For example, the game will contain tasks that require students to get closer and closer to a target without touching it, thereby illustrating the idea that there is no next larger or next smaller rational number. The intervention produced some improvements in students’ operation knowledge, but improvements were less evident in the case of representation knowledge and adaptive rational number knowledge. An understanding of the effects of operations is firmly integrated into the game’s design, as players are given points when they demonstrate a skillful use of operations. In future versions of the game, we could also consider integrating the flexible use of different rational number representations as well as the versatile use of number–operation combinations in scoring mechanisms.

One of the weaknesses of our study was the somewhat unsuccessful randomization at the classroom level. The control group obtained higher scores of adaptive and conceptual rational number knowledge in both tests, especially in the pre-test. Hence, original skill levels were different between the groups, so the effect of repeated testing might have differed between them. In addition, we did not control the content of the regular teaching of the control group. Thus, results cannot be interpreted as a comparison between a traditional learning method and a game-based method. Given the small number of classrooms included in the study, it was impossible to take into account possible classroom effects arising from the nested nature of our data. To confirm the effectiveness of the game, future studies would need to use an a priori power analysis that acknowledges the nested structure of the data.

There is earlier evidence that game-based learning environments could be used to support (a) adaptive number knowledge in the case of natural numbers (Brezovszky et al., 2019) and (b) conceptual rational number knowledge, especially in the case of the size concept (Ketamo & Kiili, 2010; Kiili et al., 2017; Zhang et al., 2020). Based on our textbook analysis, in a Finnish context, traditional mathematics teaching is not targeted to improve students’ adaptive rational number knowledge or systematically help students overcome challenges related to conceptual change in rational number learning. This limited the role of the business-as-usual group in our study design. Hence, our study did not enable conclusions to be drawn regarding the effectiveness of NanoRoboMath relative to other approaches to learning rational numbers.

Overall, the results of the pre-registered regression analysis that examined individual effects and the repeated measures analysis that examined group effects showed some indications of the positive effects of this intervention. Hence, we provided further evidence that digital learning environments could be helpful in rational number instruction. We note that some features of NanoRoboMath, such as the zooming of the number line and the flexible trial and error possibilities in exploring the effects of rational number operations, would have been difficult to implement non-digitally. Teaching and learning adaptive knowledge and the correct conceptual understanding of rational numbers are demanding. Many teachers lack the required content knowledge, and learning materials often ignore natural number biased aspects, so game-based learning environments are a valuable and recommended teaching tool that complements traditional methods. Our study showed that students’ adaptive rational number knowledge and conceptual knowledge of the representations and operations of rational numbers are, to some extent, sensitive to this type of short-term game-based interventions. However, developing a correct understanding of the density concept seems to require a different approach. In the future, large-scale testing of the effectiveness of NanoRoboMath with respect to learning goals is needed, and more wide-ranging tests, including also other aspects of rational numbers, such as procedural fluency in rational number arithmetic, could be used.