Keywords

1 Introduction

Many elementary school pupils (ages 12–15) struggle to learn chemistry concepts [1,2,3], including atomic models and molecule systems, as part of the mandatory curriculum. These difficulties might underpin their entire experience with chemistry [1,2,3], or even science, technology, engineering, and mathematics (STEM) related disciplines in general [4, 5]. Numerous previous studies have addressed some of the reasons why chemistry is a difficult subject for many pupils [1,2,3,4,5,6]. Scholars have especially emphasized the gap between abstract, difficult chemistry concepts and the world in which they live; this also includes language and syntax difficulties related to misunderstandings of the connections between models, symbols, and the microscopic and macroscopic levels [6, 7]. Given these difficulties and the imagination required to connect concepts to real-life situations [3], various attempts to use gamification and serious gaming to increase learning motivation in chemistry and to fill the gap between the abstract level and the real world have been presented [2, 3, 6, 8,9,10,11]. However, there is still a need to direct the serious gaming to address specific learning objectives in chemistry [3]. There also remains the challenge of finding the right match and balancing the interactions among various target groups (e.g., age, gender, motivation), contexts (e.g., cultural, schools, content/curriculum), and serious gaming to create actual learning using an engaging gaming approach. The background of using serious games in education (and, in this study, within chemistry), is based on the idea that games, because of their ability to engage end excite [2, 3, 6, 9–11, 32–35]; can provide a level of learning engagement among pupils. However, it is extremely challenging to outline evidence for improved learning using serious games. There are several reasons for this, such as the low number of participants, different contexts, short evaluation periods, a lack of longitudinal studies, a lack of baseline and control groups, and poorly defined evaluation criteria. This study does not solve all these challenges, but it provides another attempt to improve pupil’s difficulties in chemistry using a serious game. This study is based on the following research question: How can one design an engaging serious game that strengthens the understanding of the atomic model and molecular structure, as taught in chemistry in the Danish elementary schools to pupils ages 13–14?

2 Previous Research

The use of serious games for learning purposes is relatively small compared to that of games intended for fun and entertainment [12]. However, serious gaming for learning represent an important part of serious games [2, 3, 8,9,10,11,12, 19, 32,32,33,34], and is expected to increase. The worldwide five-year compound annual growth rate (CAGR) for serious games and services is estimated at 33.2%, and revenues will more than quadruple to exceed $24 billion by 2024. The expected growth is especially favorable within learning purposes and correlates to the generation of digital natives [13]; greater adaptability to technological change [14]; and ongoing innovations integrated into next-generation serious games including advances in psychometrics, neuroscience, augmented reality (AR), and artificial intelligence (AI) [12]. Exact predictions such as these are always difficult, as there is no consensus on the definition of serious games, and they are used in divergent ways, focusing on various perspectives depending on their purpose, players’ goals, and intended content [15, 16]. Previous definitions have emphasized that serious games are applications that are not designed exclusively for fun [17] or that are intended to be more than entertainment [15, 16]. However, there are still some unsolved categorical problems regarding what constitutes a game and what “more than entertainment” or “not exclusively for fun” actually means. Furthermore, there are often categorical problems within the terminology associated with serious games and gamification [18]. In spite of the diversity of definitions, there seems to be some general agreement on the growth of serious games, and a requirement for successful serious games for learning purposes is including complex reciprocities of engagement and motivation [3, 10, 15, 16, 19].

Engagement has been defined as a quality of user experience [20] and as an indicator of whether a player wants to continue playing [21]. The challenge is that engagement, particularly in the context of serious games, is a complex subject, as it encompasses various related concepts related to the user experience, including immersion, presence, flow, transportation, and absorption. Because of the interrelated nature of these various concepts, engagement is often used without a clear definition, leading to possible confusion in measuring how engaged a user is with, for instance, a serious game. Most often, engagement in serious gaming is a means for providing some kind of learning [17, 22]. Learning is also a multidimensional construct consisting of behavioral, affective, and cognitive engagement [23]. Furthermore, to design an engaging serious game for learning purposes, scholars have emphasized aspects of motivation (especially intrinsic motivation), such as curiosity, a desire for challenge, rewards, feedback, and involvement [13–15, 22–24]. However, the success of a serious game for learning purposes depends on both the teachers and pupils’ motivation to begin playing the game and to spend their time, effort, and energy on it. The experiences of flow [25] and enjoyment [26] are crucial to this process. When players have mastered specific challenges, they develop a greater level of skills that can be used and improved with more complex challenges in other levels or games [25]. This can have a positive influence on learning and intrinsic motivation in serious games [22].

Previous studies have presented some empirical short-term evidence that serious games can have a positive effect on student learning achievement and learning attitudes in chemistry [2, 3, 27]. However, as mentioned by other scholars [3], there is still a need for serious chemistry games that act as an interface for presenting specific chemistry concepts.

Previous studies have mainly used pre/post-test, surveys and questionnaires, observations, and interviews [2, 3, 6, 9,10,11, 32,33,34,35] when evaluating serious game with learning purposes targeting pupils and students. A part of the novelty in this study is to learn from this past work, and improve the methodology by including a substantial work in both the teacher involvement, pilot-testing, and evaluation. When evaluating a serious game for pupils it is important not to neglect the challenges finding the right match of both the participants’ cognitive abilities and a solid methodological approach.

3 Methods

3.1 Participatory Design and Pilot Testing

An important focus of this study was to involve the teachers who taught the pupils about atomic models and molecules. This was done by following a participatory design approach [31] in which the end-users included both teachers and pupils. The teachers served as gatekeepers who facilitated and controlled the process in areas such as the curriculum’s aims, focus, knowledge, skills, and analysis. Therefore, the teachers were involved as co-designers very early in the process. They were asked for input and feedback, but they also worked as partners in the design process regarding changes to aspects of the game’s development. Prior to the final game, efforts were made into various pilot testing. The pilot testing were performed in two stages. The first stage included a low fidelity paper prototype, and included 3 participants. The paper prototype was tested in order to make sure that the overall concept of the game could be understood. The second stage of pilot testing consisted of a usability test, and included 7 participants. Both stages of the prototype testing was set up in the school library at the same elementary school for the later evaluation, and it was made sure that the participants for the pilot testing were not included as later participants.

3.2 Participants

27 pupils were included in a formative evaluation. The participants were aged 13–14 with 16 boys and 11 girls. All participants were recruited from two chemistry classes from a Danish Elementary school in Copenhagen. All participants have had chemistry for one year.

All participants and parents gave informed consent and were informed that they could withdraw from the study at any time. We provided all participants with anonymized ID numbers, and all the data were labeled with these IDs. Furthermore, we applied special ethical considerations when recruiting children. Access, permission, and ethical approval were included from the State School. All participants were inform that they were respected on each individual’s speed and level, and there was no hurry or judgement based on speed or level.

3.3 Procedure and Analysis

The participants were sampled by the convenience sampling method, also due ethical reasons to make it voluntary to participate. The participants were asked to complete the same mission; to build hydrogen and oxygen and use these elements to construct H2O and O2. The participants were encouraged to play the game on their own, but could ask the researchers if they did not know how to proceed.

The data collection consisted of both a questionnaire, observations, and interviews.

The questionnaire was inspired by the User Engagement Scale (UES) short-form [27], and consisted of 12 items on a 3-point Likert scale. The scale was included with smiley faces for ease of conceptual understanding for the pupils.

The observations consisted of observation notes by two of the researchers. The observations included registrations of the participants chosen cards in an included card sorting. The participants were asked to pick up to five cards (out of 12) to describe their experience of the game. The cards had an equal distribution of positive and negative connotations. The wording on the cards were as follows: Pretty, fun, easy, cosy, pleasant, want to play more, confusing, boring, fast, irritation, difficult, and ugly. After picking the cards, the pupils were encouraged to share their reasons for picking these cards.

The questionnaires were analyzed by cumulative frequency. The expert interview were analyzed by traditional coding [29] following four steps: organizing, recognizing, coding, and interpretation.

4 Design and Implementation

The game was designed in Unity. It consisted of first-person perspectives in a 3D environment with the functionality of the atom and molecule builder presented as 2D assets. Before starting the game, the player was briefed on how to navigate and run the gameplay. The player could look around by using the mouse; walk using the W, A, S, and D keys; and jump by using the space bar, as is often the case in other games. When the player moved the mouse to look around, the vector of the camera view direction changed, so the camera followed the mouse, and the rotation of the player object changed in the game world coordinates. In this way, the movement directions were related to the player object. For example, forward was always the direction in which the camera pointed.

After the briefing, the player entered a 3D spaceship with three floors. In the spaceship, the player could move around and make their way to the two 2D laboratories: the atom builder lab and the molecule builder lab. Each lab was equipped with a desk. Once the player approached either of these desks, the game loaded a corresponding scene. The different scenes and switching between them was handled using Unity’s scene manager. The transitions contained a loading screen animation for effect (a black screen that persisted for 2.5 s, with the word “Loading” in the middle). The player had to complete three levels/learning objectives in the game (Fig. 1): 1) find out which element to create, 2) make the atoms for the element in the atom builder lab, and 3) create the molecule for the element in the molecule builder lab. Once finished, they could press the “Back” button to return to the spaceship.

Fig. 1.
figure 1

A flowchart over the game’s functionality

Full size image

The “Atom Builder Laboratory” scene contained buttons that generated protons, neutrons, and electrons. These could be placed on the concentric circles representing the atomic model, as well as in the atom’s nucleus. The player could push another button to display the periodic table and select the element for which they wanted to build an atom. This displayed the information necessary to build the atom (the numbers of protons, neutrons, and electrons). Once the player placed everything on the model, they could press the “Build” [Byg] button to check whether the solution was correct. If the atom was built correctly, it turned green on the periodic table and was available for use in the molecule builder laboratory. Mistakes could be reset by pressing the “Clear” [Tilbage] button. As a set learning objective, pupils were required to build a minimum hydrogen and oxygen in the atom builder lab to continue.

Fig. 2.
figure 2

Oxygen indicated as built correctly on the periodic table

Full size image

The “Molecule Builder Laboratory” scene (Fig. 3, left) included a list in which the previously built atoms appeared (Fig. 3, right). There was a list of molecules that needed to be built on the right side (Fig. 3, right). Once the player selected a molecule to build, a diagram of squares appeared in the middle of the builder window (Fig. 3, right). The player could then drag and drop atoms in these squares and use the right-side mouse button to trace links between them (single or double bonds). The solution could be checked by pressing the “Build” [Byg] button. If it was correct, the builder window turned green. Mistakes could be reset by pressing the “Clear” [Tilbage] button. If help was needed, the player could always press “Q” for further instructions.

Fig. 3.
figure 3

The Molecule Builder Laboratory (left). Diatomic oxygen built correctly in the molecule builder laboratory (right).

Full size image

5 Findings

5.1 Positive with Some Confusing and Difficult Elements

The pupils completed the requested mission (to build hydrogen and oxygen and use these elements to construct H2O and O2) in the serious game between 9 and 19 min, with an average time of 12.4 min. Most of the pupils (15), spent less than 13 min in the game, with only 4 pupils spending more than 17 min.

The card sorting revealed a clear pattern towards selections of positive cards, based on their immediate response after the gameplay (Fig. 4). 22 pupils picked pretty, 16 picked cozy, 15 picked fun, 13 picked want to play more, and 11 picked fast.

Fig. 4.
figure 4

Selected cards from the card sorting, each pupil could select 5 cards. n = 27.

Full size image

To contrast this, none of the pupils picked ugly, 2 picked annoying, and 4 picked boring. However, it is also worth of note, that 18 pupils picked confusing and 17 picked difficult. Alongside choosing the cards, the participants were encouraged to give reasons for their choices. Lots of the pupils expressed a long with the picked “confusing” and “difficult” cards, that they were unfamiliar with how to navigate in the game interface.

The majority of those pupils who picked “difficult” and “confusing” were mainly due the chemistry content, and expressed both that they were not good at chemistry, but also they would like better and more clear hints on how to fulfill the chemistry requirements/ the mission.

From the user engagement questionnaire the findings revealed that the pupils were engaged in the game (Table 1). Only 4% stated that they did not lost themselves in the game, 15% were not absorbed in the game, and only 4% were not interested in the game (Table 1).

Table 1. Questionnaire findings. n = 27.
Full size table

The most positive feedback were regarding the game’s aesthetics, with 70% of the pupils who found it pretty made. Also worth noticing that the pupils self-reported that the game taught them something, with only 4% answering not at all to this question. However, it should be questioned whether these questions from the user engagement scale actually are the right type of questions. These questions might not be aligned with the pupil’s capacity for being reflective (or not) in relation to his or her behavior, habits, and cognitive understanding.

5.2 Number of Errors and Tips Needed

During the gameplay, the observer noted the number of errors that the pupils made within three different categories: Input errors; chemistry errors, and the number of tips needed.

For the input errors, the findings revealed an evenly distributed amount of errors across pupils. The boxplot (Fig. 5) reveals that the participants had between 0 and 5 input errors, with a median of 2.4 and an interquartile range of 2 to 3. This indicates that the general usability for the game was good. Further, the findings revealed that those pupils being already familiar playing games (playing games several times per week or daily) had fewer input errors, compared to those pupils playing less than several times per week.

Fig. 5.
figure 5

Boxplot revealing the distributions of participant errors. n = 27.

Full size image

The number of chemistry errors were a bit surprising, as we anticipated that the majority of the pupils would be able to use previously learned knowledge from their chemistry classes. The findings for chemistry errors revealed a median of 3.4, and an interquartile range of 2 to 4, making it negatively skewed based on the outliers 8 and 9. This findings suggest that the pupils were able to recall certain aspects of the atomic and the molecular structures, but not all. It should also be noted that the number of errors in chemistry had a strong correlation to the number of tips needed. Several pupils did not attempt to perform the tasks without guidance. Only few pupils kept attempting to complete the task/mission without any guidance.

The findings also revealed that the number of tips needed varied wildly between 1 and 12, with a median of 4.9 and an interquartile range of 2 to 6.5 with a negatively skewed tendency. The number of tips could be interpreted as problematically high. However, it should be emphasized that the tips covered both input and chemistry errors, with most tips provided for chemistry errors. Further, the labelling “errors” is not the perfect terminology, or should at least be understood in a much more holistic learning context. The approach of asking, were a very familiar approach for the pupils, and by that this serious games also reflects the normal instructor approaches and practices within the class room setting. On the other hand, the structure of this serious game could be improved to give the pupils much further opportunities explore the game world on their own and create their own path of knowledge [30].

6 Conclusion and Future Work

Designing a serious game for pupils with the aim to increase the understanding of molecular structures, is not an easy task. We can conclude that the most important element in developing educational games may be that good games engage both pupils and teachers, and the interplay between game play, pupils and teachers can create some dynamic learning opportunities. However, a core foundation for making these learning opportunities possible, is to have the right balance of skills and challenges for the participants; both within specific learning objectives, but also for control of the gameplay.

This serious game was to some extend able to engage pupils within the subject of chemistry. The game itself was reported to be very engaging, but the specific learning outcomes remains uncertain. We can conclude that the serious game was visually appealing, the pupils were absorbed in the game, and wanted to continue playing. However, the designed serious game was also a bit confusing and made too difficult, resulting in frustrations for the pupils.

A majority of the pupils did not have the basic knowledge needed to engage with this serious game. In spite of a participatory design approach with the teachers in chemistry, the game was made too difficult in terms of specific chemistry tasks. The game should consider either to lowering the skill floor, add further instructions, provide a better tutorial, and make it as an adaptive game, with individualized adjustments for the learning elements.

Those participants not familiar with playing games, had major difficulties of the controls and the gameplay. A better introduction, gradually to the mechanics of the game, as well as the principles by which atoms and molecules are built, could possibly have helped mitigate the difficulties. Likewise, the confusion might also have risen from the serious game’s lack of story. The pupils simply start on a spaceship, and are given the objective of constructing hydrogen and oxygen, water and atmospheric oxygen without any clear context; neither from the game itself, nor from the teachers. However, this is a mistake by the game designers/researchers, not having the teachers been involved throughout all design stages, and possible also guided the teachers for how to introduce the game.

In research, there also remains much more attention towards how to evaluate serious game targeting children and adolescent. There are still some important challenges in how to increase the validity and reliability when evaluating serious games when children and adolescent are the users. Participants, including the teachers, should be motivated and want to participate – also in the evaluation part. Further, which method should be used, and how to ask the right type of questions, aligned with the child’s capacity for being reflective (or not) in relation to his or her behavior and habits. Future work is needed to generate significant evidence and insights regarding pupils’ learning of chemistry via serious gaming. First, a much higher number of participants is needed, and baseline and control groups should be included in the research design. Second, further details on the identification of the participants are needed (e.g., their confidence in serious gaming and game genre preferences, current knowledge, motivation, expectations, and technology acceptance). It is important to emphasize that there is no established taxonomy of serious gaming, and serious games are still diverse in their outcomes and certainly understudied as a means to provide knowledge about chemistry. It would also be interesting to create different options in the game design for target groups other than pupils aged 13–14, as well as to make the game more personalized with the inclusion of the participants’ own knowledge, motivation, and life stories.