A continuous growth in distance online learning has been observed in recent years and this method of learning is fast becoming a significant constituent of higher education (Allen & Seaman, 2013). However, compared to face-to-face learning settings, online courses are often faced with low student persistence and consequently low completion rates (Hall, 2011; Roper, 2007; Patterson & McFadden, 2012). The challenge, therefore, for instructors is to design learning environments which consider student online learning readiness and motivational factors so that students remain engaged with the material and their assignments.
A distance learning Bachelor degree programme in the educational sciences was selected in order to investigate predictors of successful learning in a Moodle-based gamified online learning environment employing the game elements of badges and progress bars. The aim of this case study was to find out how motivation and study-satisfaction are associated with different dimensions of student online learning readiness (OLR) as well as student’s attitudes towards online and computer games. In addition, the aim was to assess students’ online learning experience and their perceptions of the two game elements employed.
Online learning readiness
Several factors of student online learning readiness (OLR) are identified as being important predictors of online learning. Technological elements as well as computer skills are important success factors for social interaction, social communication and learning outcomes (Herrera & Mendoza, 2011; Wataluk, 2012; Yu & Richardson, 2015). Scales for measuring online learning readiness include at least four or five dimensions (Hung, Chou, Chen, & Own, 2010; Yu & Richardson, 2015) which capture the above-stated factors. Since online learning readiness is a relevant predictor of success in distance online learning (Demir Kaymak & Horzum, 2013; Keramati, Afshari-Mofrad, & Kamrani, 2011; Yimaz, 2017) it is expected to be also relevant within the context of gamified distance online learning environments. In addition, motivation is one of the key determinants for successful online learning (Lineham, Kirman, Lawson, & Chan, 2011). A well-recognized theory on motivation and motivational behavior is referred to in this case study and will be presented below.
The self-determination theory (SDT) of Deci and Ryan (1985, 1993) has been applied in many research fields as an approach to human motivation and personality (Ryan & Deci, 2017). As a part of online learning along with gamification, it has been used as a framework to investigate student’s motivation and engagement (Dichev, Dicheva, Angelova, & Agre, 2015; Lamprinou & Paraskeva, 2015; Sailer, Hense, Mayr, & Mandl, 2017; Shi & Cristea, 2016). SDT differentiates between types of motivation, gives explanations for motivational behavior and proposes predictors of high-quality learning. Intrinsic motivation is considered the most beneficial type of motivation as it is associated with activities that are done for fun and pleasure and satisfaction. In our case study, this regulation type is observed when students enjoy and have fun collecting badges. Identified motivation is observed when individuals engage in a behavior because they think it is important to them. Students collect badges since they believe collecting them is an important and valuable activity for them. Both intrinsic and identified regulations are self-determined and autonomous forms of motivation. Extrinsic motivation is exhibited when the reasons for performing a task are externally influenced, either because the task is an obligation or must be completed in order to avoid punishment. It represents a non-autonomous form of motivation. Students exhibit this regulation type when they collect badges because they are obliged to do so by the instructor. Finally, amotivation shows unwillingness or the state of lacking the interest to act (Ryan & Deci, 2000). Individuals do not care about a task and/or do not feel competent in accomplishing it. This regulation type is shown when students are not interested in collecting badges and/or do not feel competent in accomplishing the connected tasks.
Besides motivation, student satisfaction is another relevant factor that allows instructors to evaluate their students’ learning conditions and to gain information on how they cope with their course of studies. For researchers and practitioners, it is a major outcome variable in assessing teaching interventions, since it is associated with student persistence (Park & Choi, 2009; Lee & Choi, 2013). The concept of study-satisfaction includes content, conditions and coping as three important factors and uses self-reported measures to make statements about students’ satisfaction with their course of studies (Schiefele & Jacob-Ebbinghaus, 2006; Westermann, Heise, Spies, & Trautwein, 1996). Online learning readiness, motivation and study-satisfaction were explored in relationship to the gamification implementation, which will be presented next.
Gamification and games
The concept of gamification is still a rather new area of research and is defined as the use of game-like elements such as points, badges, leaderboards or progress bars in non-game contexts (Deterding, Dixon, Khaled, & Nacke, 2011; Werbach & Hunter, 2012). The use of game elements in online learning environments is said to foster students’ motivation and to achieve changes in students’ learning behavior (Deterding et al., 2011; Kapp, 2012; Werbach & Hunter, 2012). A recent meta-analysis concluded that more than 60% of the surveyed studies which investigated gamification in general were inconclusive and about 20% were positive. Ten per cent even showed a negative result (Dichev & Dicheva, 2017). The inconclusive findings either depend on the amount of game elements, used context, kind of media, or target group. Different results were also found in the application of the same type of game elements regarding motivation, engagement, or learning performance (Dichev & Dicheva, 2017; Nacke & Deterding, 2017). Some of the studies could show that acquiring badges had a significant positive effect on students’ learning behavior, motivation and encouragement (Hakulinen, Auvinen, & Korhonen, 2015; Hamari, 2015), their learning performance and engagement (Pechenka, Laurence, Oates, Eldridge, & Hunter, 2017) or at least tended to have a positive impact on students’ motivation and engagement in the context of automatic feedback with badges and progress bars (Utomo & Santoso, 2015). Despite the tendency towards positive results, less motivation, satisfaction and empowerment was observed in students when badges were combined with multiple game elements (e.g. leaderboards and virtual coins) compared with students in the non-gamified environment (Hanus & Fox, 2015).
Notably, studies investigating the effects of the use of both badges and progress bars in educational courses are limited. Since results on the benefits of gamification were mixed and only limited research has so far been undertaken on its use in educational subjects in comparison to science subjects, there is still insufficient evidence to support the benefits of gamification and a lack of knowledge on how to gamify motivational designs (Dichev & Dicheva, 2017).
As stated earlier, gamification adopts game-like elements, which are typically applied in online or computer games. Online games are played within a particular structure using rules giving clear instructions to the players. However, they also offer freedom of choice, randomness and promote problem-solving (Deterding et al., 2011). The fun in games arises from the mastery and comprehension of the player, resulting in satisfaction and promoting a sense of well-being (Koster, 2013; McGonigal, 2011; Seligman, 2012). Considering the fun associated with games, triggering happiness and well-being, players’ general attitudes towards online and computer games may be a successful predictor of study-satisfaction within the context of the study module under observation, which was gamified with the game elements of badges and progress bars. Therefore, it seemed meaningful to establish a connection between the students’ affinity towards playing online and computer games (Hainey, Westera, Connolly, Boyle, Baxter, et al., 2013) and reported study-satisfaction of the students in the gamified course. Attitudes towards gaming may also give insights into the acceptance of both game elements within the non-game context.
Gamifiying Moodle with digital badges and progress bars
A gamification concept using digital badges and progress bars was designed, developed and implemented in the learning management system Moodle for a distance learning Bachelor degree class in educational sciences (see Table 1).
Digital badges and progress bars were chosen as game elements because both features were expected to support progress, mastery and recognition and to thereby generate learner engagement, motivation and interaction (Hakulinen et al., 2015; Utomo & Santoso, 2015; Werbach & Hunter, 2012). In addition, badges and progress bars were expected to help foster own goal-setting and learning strategies (Werbach & Hunter, 2012) in the self-determined learning environment at hand.
In our gamification concept, the awarding of digital badges in Moodle was technically linked to the completion of specific online activities, tasks, or online group work. Badges were automatically awarded in the student’s profile once the associated and pre-defined activities were completed. Progress bars were technically matched to the activities and tasks in Moodle and were automatically unlocked when the user had carried out the pre-defined number of activities and actions. They thus indicated the individual learning progress and mastery of the course.
Five hundred and fifty-two students enrolled in the gamified course of a distance learning Bachelor’s class in educational science, 229 of which were registered for the end-of-term examination. As a preparation for the end-of-term paper, voluntary online group work in Moodle was offered for 3 weeks during the term in which 62 students continuously participated. They were automatically awarded a “Teamworker” badge once they completed the pre-defined tasks associated with the online collaboration. Further activities were linked to an “Expert” badge for the module as a reward for being active and for completing the pre-defined tasks throughout the term. Once awarded, the badge appeared in the students’ profile and recipients could decide whether to make the badge visible to their fellow students or to keep it to themselves.
The progress bars in Moodle were visible only in the student’s personal online learning environment. Visual signs in Moodle indicated where hidden progress bars were to be found and these were automatically unlocked as soon as the tasks and activities linked to them were completed. Thus progress bars provided individual feedback of the student’s mastery. Since data protection guidelines had to be observed, the processing of the associated tasks to the game elements were all optional for the students, only visible to the students themselves and did not affect their final grade.
A mixed methods approach was chosen and deemed to be suitable since the integration of both quantitative and qualitative data was expected to reveal a more comprehensive and complete picture in interpreting the problem posed (Creswell & Plano Clark, 2011). Given the inconclusive results of previous studies, such an approach was expected to be more appropriate in understanding how the students perceived the gamification concept.
Within the mixed methods approach, a convergent parallel design was used (Creswell & Plano Clark, 2011) to collect data. First, data of both quantitative and qualitative evaluation was simultaneously collected in a cross section within a period of 5 weeks during term. Quantitative data was surveyed by means of an online questionnaire while qualitative data was gathered in semi-structured interviews. Second, both methods were subsequently analyzed separately. Quantitative data was thereby evaluated with statistical analyses for non-parametric tests and qualitative data was assessed by content analysis. Finally, the results were compared using a side-by-side strategy and, where necessary, merged or presented separately (Creswell & Plano Clark, 2011).
The theoretical assumptions lead to the following considerations for this inquiry: Firstly, motivation is one of the key determinants for successful online learning and gamification is said to have a positive impact on students’ motivation and their learning behavior. However, findings of recent researches have been mixed so far and a deeper understanding about the associated kinds of motivation would be of great help explaining students’ learning behavior under gamified learning conditions. Secondly, as stated earlier, online learning readiness is a relevant predictor of a successful distance online learning in general and is therefore to be considered as a major factor within a gamified context as well. Students should not only be able to collaborate and to communicate online with teachers and their fellow students, but also be prepared to use online learning environments. If students feel competent to do so, this will have an impact on their motivation and online learning behavior. Finally, the word “game” in gamification throws the light directly on to the fun in games which triggers not only motivation and pleasure but also well-being through the game structure of choice, mastering and problem-solving. It is therefore of interest whether students’ general attitudes towards online and computer games is related to students’ satisfaction under gamified online learning.
The study question initially focused on the aspects of various types of motivation and how online learning readiness is related to these regulation types within gamified online learning in this specific context. However, in the study a subordinate question emerged in relation to games and their connection to well-being: (1) What kinds of motivations are associated with gamified online learning in this specific context? (2) How are Online Learning Readiness (OLR) and its dimensions associated with the different types of motivation in this specific gamified learning environment? (3) How are students’ attitudes towards online- and computer games related to their study-satisfaction with gamified online learning in this specific context?
Data collection and data analysis
In order to collect quantitative data, students were provided with a link to an online questionnaire in the online gamified course and invited to participate. The questionnaire comprised items of surveys previously used in other surveys and tested and adapted for this case study. These surveys had been confirmed in previous research to be valid tools in measuring the desired construct. Some of the items were translated with the forward translation method in order to be understood by the participating students (Beaton, Bombardier, Guillemin, & Ferraz, 2007).
The variable “gamification concept” was operationalized by the implementation of two game elements, badges and progress bars, in the learning management system Moodle. A four point Likert scale (1 = I completely disagree, 2 = I slightly disagree, 3 = I slightly agree, 4 = I completely agree) (Döring & Bortz, 2016) was then used to measure the variables ‘online learning readiness’ and “attitudes towards online and computer games”. The OLR scale included the four dimensions of technical competencies, social competencies with fellow students, social competencies with teachers (five items) and social competencies in general (four Items) (Yu & Richardson, 2015). The scale for measuring students’ attitudes towards online and computer games encompassed 10 items (Hainey et al., 2013). Finally, the variable “motivation” was measured by consulting the Situational Motivational Scale (SIMS) (Deci & Ryan, 1985; Guay, Vallerand, & Blanchard, 2000; Hartnett, St. George, & Dron, 2011). The scale assessed the four underlying dimensions of regulation types: intrinsic motivation, identified motivation, extrinsic motivation, and amotivation. Each dimension was comprised of 4 items. “study-satisfaction” with students’ academic studies was operationalized with the 3 dimensions of content, conditions and coping (each three items) (Schiefele & Jacob-Ebbinghaus, 2006; Westermann et al., 1996).
Thirty-two students of the gamified course voluntarily participated in the online survey, of these seven were male and 25 female. Their ages ranged between 25 and 55 years. An analysis of the items and scale reliability was made for the quantitative survey. The evaluation concluded that the dimension social competencies in general of OLR should be excluded, since the internal consistency α = .35 was not reliable enough for further evaluation. This is in contrast to previous results of Yu and Richardson (2015), who in their sample reached an internal consistency of α = .87 for this subscale. The exclusion was supported by the fact that the other subscales of social competencies were more specific than the excluded general dimension, and therefore, they were deemed to be of more relevant for the results.
In the study, the subscale amotivation also did not reach a sufficiently high Cronbach’s alpha (α = .49), similar to the findings of Guay, Vallerand, and Blanchard (2000). As a result, only two items (item one and three) were used to represent this regulation type. The omission was acceptable due to the fact that the two remaining questions were content-related to amotivation and typically represented this regulation type. Table 2 shows the scales and subscales with their statistical values, which were finally included in the evaluation.
The examination of correlations between the four regulation types showed a positive correlation between intrinsic and identified motivation. Negative correlations were found between both types of autonomous motivation (intrinsic and identified motivation) and the dimensions of extrinsic motivation and amotivation. These findings supported a construct validity of the scale for the four types of motivation (see Table 3). Since histograms, Q-Q-plots and a Shapiro-Wilk test of normality indicated that most of the variables were not normally distributed, Spearman’s rho for non-parametric tests was used to assess correlations.
Semi-structured interviews were conducted to answer the open-ended research questions, the guideline for which was prepared using a theory-based and rule-guided method (Gläser & Laudel, 2010). A notification was posted in the online learning environment of the gamified course seeking respondents for an interview who would answer a number of questions regarding the gamification concept. Eight students, 7 female and 1 male between the ages of 25 and 54 years of age volunteered. The introductory question “What do you think about the idea of a gamification concept in Moodle?” was open enough to give the interviewees the possibility of getting into the theme. Dichotomous questions were then used to filter and control the course of conversation (example: “Is a personal and direct conversation important for you?”). Supposing questions were employed to give orientation during the conversation and to gain the necessary information for the research (example: “Do you mean the entry of the group work?”). The interviews were finally anonymized, transcripted (Selting et al., 1998) and analyzed. Since the interest of the study was focused on content and theme, the qualitative content analysis was the applied method (Mayring, 2015).
First, the text material was analyzed by using the pre-defined anchor examples and encoding rules (see Table 4). All analysed text passages from the interviews were subsequently inserted into a table and allocated to the six pre-defined main categories of “participation”, “motivation”, “self-directed learning with gamification”, “study-satisfaction”, “future gamification in Moodle” and “design of the game elements”. Finally, the quotations were paraphrased and generalized for use in the results.
In the following section, the results from the quantitative and qualitative data are presented in a side-by-side strategy (Creswell & Plano Clark, 2011). Findings from the quantitative and/or qualitative data are used to answer the research questions and hypotheses of this study.
Gamified online learning and associated types of motivation
A descriptive evaluation of the quantitative data was made in order to answer research question (1) and in addition, data from the interviews were integrated in order to gain a more comprehensive insight about how the students matched a regulation type. Based on their self-reported answers, the students who participated in the online survey were assigned to one of the four regulation types in order to find out which regulation type was associated with gamified online learning (see Fig. 1). A new variable was thereby established that included values for 1 = amotivated, 2 = extrinsic, 3 = identified and 4 = intrinsic. The assignment was made on the basis of the student’s average highest approval of a given type of motivation (Ratelle, Baldwin, & Vallerand, 2005; Vallerand & Ratelle, 2002). The classification showed that 23% of the students were intrinsically motivated. They stated that collecting badges was interesting, fun and pleasurable. The largest group of 57% consisted of identified motivated students. The persons in this group found that collecting badges was a good and important task for themselves personally. However, 20% of the students indicated amotivation. These students believed that the reasons for collecting badges weren’t obvious to them and that they could not see any benefits in the game elements. None of the participants were assigned to the extrinsic regulation type. This is in line with expectations, as participation in the gamification program was voluntary and there were no extrinsic rewards associated with the gamification activities.
Based on their answers, the eight interviewees of the qualitative survey could be assigned to one of the three groups of regulation types considered in the quantitative survey. Two students were classified as intrinsically regulated on the basis of statements such as “This is interesting for me.”, or “I’m enjoying it.” Four interviewees were assigned to identified motivation. Answers such as “It is personally for me.”, or “I can identify with it” were indicative of this regulation type. Two participants were classified as amotivated on the basis of answers such as “It did not interest me at all.”, or “No, this is not for me.”. Table 5 shows three typical cases representing the regulation types intrinsic, identified, and amotivated, respectively. In line with the quantitative data, none of the interviewees was classified as extrinsically motivated.
Students’ online learning readiness and associated types of motivation
Results of the quantitative data for research question (2) showed high values on all three dimensions of OLR for intrinsic and identified motivated students (between 64 and 76%, see Fig. 2), as high scores of OLR were associated with high scores of autonomous motivation (intrinsic and identified motivation). Within the scale of OLR, the two dimensions of social competencies with teachers and technical competencies obtained the highest values of agreement from the students in the online questionnaire.
Low values in the dimension of OLR were associated with amotivation, whereas high values in the dimension of social competencies with teachers and fellow students were associated with high values of autonomous motivation (see Table 6). Statements made by the interviewees underscored the importance of communication and interaction with fellow students. Interviewees were organized into online study groups, worked together to complete the module tasks or exchanged their concerns about studying (e.g. “We discussed the tasks of the module in the study group.”)
High values in self-reported technical competencies were linked to high values of autonomous motivation. Almost half of the participants in the intrinsic group considered themselves competent in the use of computers and computer technology. The interviewees also displayed experience in working with computers and computer technology.
The dimension technical competencies of OLR showed a significant and positive correlation with intrinsic motivation (r = .44, p < .05) as well as with identified motivation (r = .43, p < .05). The size for both correlations is considered to be medium (Cohen, 2009). No significant correlation was found between technical competencies and extrinsic motivation or between social competencies with teachers or fellow students and all forms of motivation (see Table 6).
Students’ attitude towards playing online and computer games
The research question (3) revealed that only 25% of the students interviewed agreed to the items of online and computer games, whereas three quarter of the participants disagreed that online and computer games were enjoyable or interesting. All in all, students participating in the online survey reported a high to very high agreement of study-satisfaction in relation to content, conditions and coping for their study of course in the gamified learning environment (see Fig. 3). Even though there was a high disagreement for the items of online and computer games, meaning that students in this sample were generally not fond of online and computer games, high values in the dimensions of study-satisfaction in a gamified learning environment were reported. A high level of study-satisfaction within the gamified learning environment was also supported by statements made by the interviewees regarding the dimensions of content and coping (e.g. “I could focus more intensively on learning” or “I wanted to continue my studies.”). Some statements however expressed disagreement with online and computer games. Interviewees cited various reasons for this such as gaming itself, age or family members (“I don’t need this thrill.”, “I am too old for this.”, or “My sons like gaming, I don’t.”).
A significant and positive correlation (r = .38, p < .05) was displayed between the items of online and computer games and the dimension of coping (see Table 6). The effect size showed a medium effect (Cohen, 2009). There was no significant correlation between online and computer games and content or conditions.
Students’ overall assessment of gamified online learning
In order to gain insight into students’ perceptions of advantages and disadvantages of gamified online learning within this context, the text material from the interviews was analysed. Almost all the students interviewed indicated that the gamification concept in this specific module was an attractive offering (“It sounded very attractive to me.”, or “I liked collecting the badges.”), however not all of them participated in the concept (“The concept was fascinating, but I did not participate.”).
Students perceived a clear advantage of gamified online learning in the fact that it offered them the possibility to monitor and regulate their learning more effectively (“… great idea, for monitoring my mastery”). The direct feedback that was given by the game elements increased the motivation and satisfaction of most of the interviewed participants (“…for me, it was a direct feedback from the instructors” or “I was motivated by the badges and progress bars”). Badges represented a status symbol and a quality standard to them, showing that the owner had completed special activities and content (“The badge shows my fellow students what I have completed.”). In addition, the interviewees reported that badges promoted interaction between their fellow students (“I could easily get in touch with my fellow students.”).
With regard to the disadvantages of gamification, interviewees pointed to hidden performance pressure or to the fact that gamification could lead to demotivation (“I see the danger that someone could become demotivated.”). Reasons for not wishing to participate in the gamification concept were also given such as that no personal benefit could be perceived from the method or that the student had already developed their own learning strategy, making the collection of badges superfluous (“I have my own learning strategy. I don’t need those game elements.”). Nevertheless, almost all students stated that they appreciated the gamification concept and that it should be implemented into more modules of their course of study in the future.