Keywords

Introduction

Nowadays, typical everyday learning and play situations – particularly at home – have changed. Not only but especially because of lockdowns and remote-schooling phases due to the pandemic, digital media and digital solutions have become a natural part of everyday life, even for young children. The EfEKt study evaluates different types of implementation of the app MaiKe (see below), which has been developed to support mathematical competencies of children aged 4–6 (Steinweg, 2016). The study is carried out in Germany, where special conditions in terms of media use have to be considered. The debate whether the use of digital media in early childhood education is appropriate or not is still vividly ongoing and divides educators as well as parents into two seemingly irreconcilable camps (for a detailed and reference-based description see Birklein & Steinweg, 2018). On the one hand, fears prevail that free media use will take over and displace real world play situations without stimulating any development in (mathematical) competencies. On the other hand, digital media and especially digital learning games are considered helpful for learning progress. As the project is conducted in Germany, this debate needs to be embraced in its research questions. The change in life in terms of digital transition processes may offer opportunities, but should be considered reflectively and be one focus of research in early mathematics.

Theoretical Framework

The EfEKt study is closely embedded in the theoretical discourse on digital learning, fundamental ideas in early mathematics, and last but not least learning supervision. Moreover, the theoretical framework is oriented towards relevant research. In the following three sections the main theoretical references are outlined.

Digital Learning

One way to support and encourage children’s mathematical thinking and learning might be to enrich existing play and learning environments with digital games. Despite persisting scepticism of parents and educators regarding the use of digital media in early education, there is a growing consensus that the potentials of multimedia resources cannot be simply ignored (Palme, 2007). At best, the broad availability of mobile devices (miniKIM, 2020) and technical innovations, such as the touch screen technology of smartphones and tablets, which is especially suitable for younger children, open up new learning opportunities and thereby also learning opportunities for mathematics.

More and more studies and meta-analyses (e.g., Knogler et al., 2017; Vogel et al., 2006) show effects of the use of digital media – usually compared to traditional learning activities. However, a closer look at the studies reveals that effects on learning processes depend on various context parameters. The potentials of digital learning environments can only be fully exploited under certain preconditions regarding content selection, adequacy of activities, and suitability of types of implementation of digital games into existing learning environments in kindergarten or families. Various research studies in the past few years addressed these questions (e.g., Lembrér & Meaney, 2016; Moyer-Packenham et al., 2018; Papadakis et al., 2018).

Fundamental Ideas in Early Mathematics Education

Everyday and play situations are considered promising opportunities to facilitate mathematical activities, mathematical discussions, and fruitful interactions, which, therefore, support the development of mathematical competencies (e.g., Gasteiger, 2010). Research emphasises fundamental or ‘big’ ideas of mathematics (Brownell et al., 2014; NAEYC & NCTM, 2010; Sarama & Clements, 2009). Furthermore, research in psychology identifies some of these ideas as key predictors of mathematical learning outcomes in school (e.g., Dornheim, 2008; Jordan et al., 2009; Lembke & Foegen, 2009). Various approaches on integrating these mathematical key ideas in early education have been proven to be effective in evidence-based research studies (e.g., Gasteiger, 2015; Gerlach et al., 2013; Krajewski et al., 2008). These predictive competencies are taken up in the presented project, as the MaiKe app contains tasks designed on the mathematical key ideas numbers and operations, space and shape, patterns and structure, quantities and measurement. A detailed and reference-based description of all other tasks offered by the app MaiKe is provided in Steinweg (2016). Also, the learning effects are tested with a standardized test based on the key ideas (LauBe, 2015).

For example – concerning the big idea of number – activities in the app invite children not only to count or to order numbers, but to compare structured and non-structured sets in order to stimulate the ability to perceive structures in sets and to use them to determine cardinality (Schöner & Benz, 2018). Sprenger and Benz (2020) identify two different but interrelated processes: perception of sets and determination of cardinality. The set can be perceived ‘in structures’, ‘as a whole’ or ‘as individual elements’ and the cardinality can be determined by ‘known facts’, ‘derived facts’, or ‘counting strategies’. Counting strategies and perception of sets as individual elements mark the least advanced responses. The use of the structure indicates more sophisticated competencies and is a precondition for subitizing strategies.

Learning Supervision and Support in Early Mathematics Education

The importance of educators’ professional learning support or supervision are beyond discussion (e.g., Gasteiger & Benz, 2018). This holds true for playing learning games together with educators and parents as well. For example, Schuler et al. (2019) could show significant differences between the influence of direct and indirect support on the percentages of shown verbal and non-verbal mathematical activities. Their findings give evidence that play situations with direct support stimulate children to verbalise their mathematical activities to a particular extent, and that these play situations are cognitively more challenging. The distinction made here between direct and indirect refers to differences in the adults’ behaviour. On the one hand, adults act as direct guides, and on the other hand, they supervise the play situation indirectly. The type of indirect supervision (in the sense of the presence of adults) often corresponds to the behaviour of parents in play situations at home. This holds especially true for digital (learning) games. The project, therefore, takes up the interesting question of whether the mere presence of an adult has an impact on play behaviour and especially learning outcomes.

Today’s everyday play situations include digital games. If children – as is customary in German kindergartens most of the time – are allowed to choose for themselves which game they want to engage with, tablet apps and digital options come into play. Again, these play opportunities can be supervised or guided by adults. There is evidence that children tend to benefit from different settings depending on their learning development:

There was some indication, however, that instruction by a teacher was more effective for children just beginning to recognize numerals, but the opposite was true for more able children. Children might best work with such programs once they have understood the concepts; then, practice may be of real benefit. (Clements, 2002, p. 162)

Further studies comparing the two approaches to learning support through digital learning games might be helpful.

Methodolgy

The EfEKt study exemplarily evaluates the implementation of one particular math app. The app MaiKe (Mathematik im Kindergarten entdecken [Discovering mathematics in kindergarten]) is designed as an enriched digital learning environment along the key ideas of mathematics (see above) to encourage children’s mathematical thinking and learning processes (e.g., Birklein & Steinweg, 2018; Steinweg, 2016). MaiKe offers six so called different worlds with 10 games each. Overall approximately 480 tasks are presented. Game access is given progressively depending on the progress the child makes. Throughout the worlds, complexity and difficulty of the games increase. MaiKe uses a digital form of feedback called AUC-feedback (answer-until-correct) (e.g., Narciss, 2012). Consequently, for all tasks the number of (unsuccessful) solution attempts is unlimited – which allows trial & error approaches. Each and every mathematical content in the realms of number, geometry, patterns, and measurement is revisited across worlds (spiral curriculum).

The EfEKt evaluation study uses a pre- and post-test design with different intervention groups (setting A and B) and a control group. Children in the control group receive no special support apart from the usual kindergarten’s daily activities.

In setting A, several tablets are provided on which only the app MaiKe can be played. The tablets are available to the children at any time at their own request for free play of the app (other apps are not enabled). A short introduction on how to use the tablets and how to open the app was given to everyone beforehand. In the further course of the entire project, the kindergarten educators are explicitly asked to assist only if children ask for help.

In contrast to setting A, regular play sessions organised by the researcher take place in setting B. The play time and duration is therefore fixed by the project design. Researcher and child meet in one-on-one situations. In principle, the interaction is structured by the individual progress of the child playing the app. The researcher acts as a supervisor and does not give any help in solving the tasks. The adult’s conversational impulses are limited to some interview questions placed in each case in the analogous situations and in selected game progressions to each participating child (guided interview). In this sense, the researcher interacts with the children whilst playing in order to get a deeper insight into the children’s thinking and learning processes.

Six kindergartens participate in the study and are randomly assigned to one of the settings described above in groups of equal size. The intervention takes place over a period of 1.5 years in total (Fig. 1). The whole sample of participating children (n = 66) is divided into two different age groups. The pre-school children, who constitute half of the sample (age group 1) are on average 6 years old at the beginning of the study. They participate in the intervention for 6 months before the post-test takes place shortly before they start school. The younger children (age group 2) are on average 1 year younger at the beginning of the intervention and take part over the whole period of the study (1.5 years). The age group 2 children completed an intermediate test half a year before their school entry.

Fig. 1
A set of 2 scales for age groups 1 and 2. Age group 1 has pre-test February 2016 and post-test July 2016. Age group 2 has pre-test February 2016, intermediate test February 2017, and post-test July 2017. They are grouped under setting A, setting B, and control group with descriptions for play sessions and measuring instruments.

Design of the study

Full size image

Methods of Measurement and Analysis

As shown in Fig. 1, different data sets are collected through different measures that can be used for further analysis and to answer quantitative and qualitative research questions (Birklein, 2020).

In both settings A and B, a specially designed research version of the app MaiKe provides automatically written log files of the individual use. The files make additional usage data within the intervention phase available for analysis. All participants have their own account for using the app. In this way, it is possible to back up the game score and to create individual log files. The log files document the start and end time of each game, the percentages of correct swiping actions as well as trial & error attempts, and the duration of the time played. The log file data is only available for the researcher and not for children, parents, or kindergarten educators.

In setting A, kindergarten teachers fill in observation sheets for each participating child, to document the playing behaviour and any unusual situations that may occur. These data provide additional information for the analysis of the log files.

The development of the children’s mathematical competencies are assessed by means of a standardized school entry test, which focusses on fundamental mathematical ideas (in particular numbers & operations, shape & space, measurement & seriation) with special consideration of predictive competencies (LauBe, 2015). The test is designed to be conducted one-on-one (see Fig. 1). The children answered mainly orally or with the help of material. The results are quantitatively analysed via construction of confidence intervals (CIs) with a confidence level of 95% (e.g., Hazra, 2017). The CIs include both, sample size and standard error, and, therefore, provide more information than a comparison of mean values. The CIs allow to compare the groups with each other (CIs for independent samples) and also enable a comparison within the groups over the different measurement times (CIs for paired samples).

The researcher-child-interaction in setting B is recorded by video and documented in transcripts to capture interactions and reactions during the play sessions. These data form the basis for qualitative analysis of individual learning processes, e.g., simultaneous number perception (e.g., Schöner & Benz, 2018) and are documented in detail in Birklein (2020). The qualitative analyses provide an insight into typical reactions when dealing with the MaiKe app and into individual learning and development processes. The case studies provide additional findings that are not represented by the quantitative data and that open up further interesting research perspectives.

In this research focus, special attention is paid to tasks on determining the cardinality of (structured) sets. Eight games throughout the MaiKe app contain different forms of representation of such quantities (finger pictures, dot-fields, etc.). In the available video data, verbal expressions and interactions of the children with the app are analysed in order to get some indication of whether the children perceive a structure in the representations and if they use the structure to determine the cardinality of the given sets. The chosen method combines quantitative with qualitative elements (e.g., Schmidt, 2015). First, a theory-based category system is developed to capture the children’s strategies for solving the tasks. The category system is based in particular on Benz’s theoretical model (Sprenger & Benz, 2020; Schöner & Benz, 2018). In contrast to Benz, the study presented here includes no eye-tracking. The assigned strategies, therefore, result from the qualitative analysis of children’s reaction and interaction (verbal utterances, gestures, and actions). The developed coding guideline includes descriptions and typical examples for each category, which are in particular ‘strategy use’, ‘gestalt matching’ (if applicable), ‘counting all’, and ‘trial & error’.

The categorization of children’s reactions offers the possibility to identify patterns in the data and to map development processes in order to find out whether there are changes in children’s strategies during the intervention period that indicate learning processes regarding the perception of sets and determination of cardinality.

Research Questions

The EfEKt project addresses various issues on digital learning game use and its effects that build on and complement each other. A complete overview of all questions and research results can be found in Birklein (2020). This paper addresses two of the research questions in more detail, and provides exemplary insights into a third question.

The first question takes up the current debate in Germany and addresses the concerns about extensive media use when it is not limited by adults (e.g., in terms of time). The question quantitatively analyses the use of an app by comparing two different types of implementation (with and without supervision by an adult):

  1. 1.

    Does the type of supervision in the implementation influence the children’s behaviour and their progressing through the game?

The findings of the first question may also provide a possible basis for interpreting results of the second research question, which investigates the quantitative effects on the development of mathematical competencies of children. Moreover, the question addresses the second (German) concern that digital games may have no or negative effects on learning progress:

  1. 2.

    Does the intervention affect the development of mathematical competencies of the participating children compared to a control group?

The third question focuses on individual learning processes while playing the MaiKe app that can be identified by qualitative analysis:

  1. 3.

    What qualitative indications of advancing (more sophisticated) thinking can be identified?

Results

The findings on each of the three research questions are presented in one of the following sections.

App Use in Comparison of the Different Settings

The log file data is analysed to answer the question, whether there are differences between the settings in terms of time of use and game play. Table 1 exemplarily shows the relevant data for the (younger) children of age group 2:

Table 1 Comparison of average usage data between setting A and setting B of age group 2
Full size table

The children’s progressing through the game shows that in the free play setting A, the children reach on average the seventh game of the so called fifth world (column #1), i.e. they successfully complete 47 of 60 available games in total. The children in the more guided setting B reach on average the eighth game in the sixth world, which means they complete 58 different games. Nevertheless, more games are played overall in the free play setting A (column #2). This indicates that children in setting A repeat games far more often than the ones in setting B.

Looking at the behaviour in terms of, for example, the total time children spend playing the games of the MaiKe app over the entire project period of 1.5 years, the values hardly differ (column #3). Although the children in setting A use the offer less frequently on their own initiative (16.5 times, column #4), they played a little longer than the children in setting B (on average 5 min more playing time, column #5). The findings are overall comparable to those for group 1. Due to the shorter duration of participation in the project (half a year), these children have not progressed as far in the app of course.

The Development of Mathematical Competencies in Comparison of the Different Setting

The mathematical competencies of all participating children from the two settings and the control group are assessed by a standardized test (conducted in one-on-one interviews in all groups), which is usually used at school entry to monitor basic mathematical knowledge acquired in kindergarten. The diagram (Fig. 2) exemplarily shows the results for the children of age group 1, who participated 6 months in the project.

Fig. 2
A multi-line graph, titled age group 1, of percentages versus pre-test and post-test. It plots supervised setting B, free play setting A, and control group with an increasing trend between 60% and 90% on the y-axis with C Is for independent and paired samples. All values are estimated.

Development of mathematical competencies of age group 1

Full size image

Although the average pre-test results differ between the groups (independent samples), the CIs rate these differences as random at this time. In contrast, the average results of the post-test indicate a statistically relevant difference in two cases of group comparison, i.e., between setting B and the control group as well as between setting A and the control group.

The changes within the groups (paired samples) over the different measurement times from pre- to post-test are statistically relevant for both intervention settings A and B. The development of mathematical competencies in the control group can be considered random.

Even though the mathematical competencies of children in setting B increase slightly more during the project period, this may – following the interpretation of the CIs – also be a random effect. In other words, there is no statistically relevant difference between the two implementation settings A and B.

On the Development of Thinking Processes

As an illustrative example of the qualitative analysis, a section of a progress chart for two games on determination of cardinality is presented in this paper. In the selected games analysed here, the required solution number needs to be assigned by swiping movements to the given dot-field on the right (Fig. 3). The remaining number figures in the middle have to be matched to the bin. The seventh game in the fourth world (game 4.7, Fig. 3, left side) is nearly identical to the third game in the sixth world (game 6.3, Fig. 3, right side). In the latter one an animation of a hand, which covers up the given dot-field after a short time (< 3 s) before the number figures appear on the screen, expands the challenge for the children.

Fig. 3
A set of 2 illustrations of a bin and 5 numbers with dark and light-colored circles in one and a hand in another. 2 connected tables below them, titled world 4 game 7 and world 6 game 3, respectively, have percentages for structure use, counting all, trial and error, and no assignment.

Section of a progress chart

Full size image

The frequencies for the strategies ‘structure use’, ‘counting all’ and for a ‘trial & error’ approach are recorded across all participating children as percentages. In some cases the analysis of the children’s interaction allows ‘no assignment’ to a specific solution strategy.

All children determine the number 8 in game 4.7; none of them uses a trial & error approach. The group of children identified as structure users (45%) is larger than the counting group (32%). In game 6.3 the major part of all children (86%) uses a structure to determine the amount of dots.

The arrows between the tables visualize the thinking development paths. The thickness indicates the occurring frequency. In these two games all children who perceive structures and use them to determine the number in game 4.7 stick to this strategy in game 6.3 as well. Almost all children who counted the elements one by one in game 4.7 perceive structures in game 6.3.

While the arrow diagram of frequencies provides a general overview of solving strategies and development across the whole group of participating children, individual case studies reveal aspects that are not captured in the category system. Thus, it becomes apparent that the children whose solution strategy is assigned to the category ‘structure use’ do not always see the identical structure in given representations and that they use different strategies – sometimes quite flexibly. For instance, Heiko explains his rapid determination of the number 8 in relation to the total number of the dots in the field: “"Because 2 are missing, it’s 8”. Emilia refers to a previous task with nine given dots: “that there were 9 and then I just counted 1 off again”. Lea first derives the 8 from the predecessor: “Because after 7 comes 8”. In a later stage of the game, she also uses the power of 5: “Because 3 points and then 5, then it equals 8”. Across all the case studies, a variety of variants of structures in the category ‘structure use’ and for some children an impressive flexibility in the use of their strategies can be documented. Overall, the decomposition of 8 into 5 (or ‘a row’) and 3 is used most frequently.

Discussion

Comparing the MaiKe app use in the implementation settings, the results are quite alike. In particular, the average time spent playing the app in each session is very moderate. The findings falsify the common assumption that no limitations and no instructions inevitably lead to excessive media consumption. Fears are unfounded that free play – at least in the case of this particular MaiKe app – will lead to extensive use of digital media. These findings are an important argument in discussions with parents and educators in Germany.

The only noticeable difference between the free and supervised play settings is recognisable in the two interdependent values of game progress (score) and repetition rate. The exercise, and thus the learning progress, takes place through repetition and thus replaces the guidance by an adult. These findings are consistent with those of Clements (2002).

The results strongly indicate that the MaiKe app use shows statistically relevant positive effects on the development of mathematical competencies, which are supportive for starting school. This becomes evident in both settings (compared to the control group). Again, the supervision of an adult, as implemented in this project, has no verifiable effects. The children taking their own time and choosing to play the app whenever they want (free play) are in no way inferior in their development to those who have played in presence of the researcher at set times. Both types of organisation, therefore, can be suggested as possible options to integrate an app in the daily routine of a kindergarten (or at home). They both offer opportunities to enhance mathematical learning and are thus more effective than the usual offers in German kindergartens (control groups).

Nevertheless, several studies and projects focus on the importance of learning support for children’s learning processes (e.g., Gasteiger, 2010; Klibanoff et al., 2006). The slightly higher development of mathematical competencies in supervised setting B in this study may be an indication in this regard. The difference is not statistically relevant in this study. This may be due to the fact of the very limited guidance in setting B; only in the context of prompting interview questions. Although, the interview impulses probably encouraged the children to deal with the mathematical contents in a deeper way, no further support was given. Therefore, there can be no talk of deliberate learning support – which research highlights as particularly effective – in setting B either. Further studies could explore types of attentive guidance versus free play situations.

The MaiKe app itself, which serves as means and material in this study, influences the test results in the basic competencies (matching the ‘big ideas’, see above). Due to the design of the app, children need to follow a certain way through the different games. If the solution rate is too low, the app first expects replays of the game before the next games are unlocked. In this sense the app itself acts as a learning guide.

Moreover, the game design influences the strategies used. The deeper insight in learning processes based on qualitative analyses indicates that the children use more efficient and sophisticated ways to determine quantities of sets over the course of the intervention period. Almost all children who were counting one by one in game 4.7 are actually able to use a structure for determinating the quantity in game 6.3 (Fig. 3). One possible reason for these developments may be that the games also become more challenging as they progress. For example, due to the hand animation in game 6.3 and the rapid coverage of the dot-field, less elaborated strategies (‘counting all’) are hardly possible any more. Children for whom such a strategy is not (yet) available therefore fall back on a trial & error approach.

Also, from this it can be drawn that free play (setting A) is not so free after all. At the same time, this also may explain why the development does not differ significantly in the different settings. The efficacy of playing with (learning) apps without further instructions necessarily depends on the app design. The tasks provided in the app must be mathematically and educationally sound – however, this condition might not be sufficient.

A promising research perspective may be the comparison or the combination of digital tasks with their physical material-based counterparts in terms of learning progress on mathematical key ideas and, where appropriate, understanding of mathematical concepts. Each game in MaiKe is inspired by common kindergarten material like building blocks, number cards, dice etc. The possibility of easily translating the digital into a material-based game was deliberately taken into account in the MaiKe app-design in order to generate ideas for real learning environments in kindergartens through the app. This kind of research is particularly worthwhile because there is a broad consensus that an app (like MaiKe) cannot replace a rich and stimulating play and learning environment, but it can complement it.

In further research, children’s prior knowledge may be taken into account. The experience gained in the context of this study as well as existing evidence suggest that children’s prior knowledge can have an impact on the way they deal with the mathematical content, on the reception of the digital feedback, on the willingness to repeat tasks and further rehearse their own knowledge, or on motivation in general.

In this study there is hardly any evidence of mathematical competence development in the children of the control group. This is at least remarkable and alarming from a mathematics educational perspective. In daily routine of kindergarten, too little attention is obviously paid to recognising mathematical learning opportunities and making them fruitful. German kindergarten educators are usually not trained in mathematics. This spin-off of the analysis of the results shows how important it would be to reorganise the training of kindergarten teachers.

When interpreting these and other results, it is important to note that they refer to this exemplary selected MaiKe app and a small sample. The specific concept as well as the design allow only a limited transfer to other digital learning environments. In order to allow generalizable conclusions, a study with a larger sample of apps and children has to be carried out to verify the indicated tendencies.