Tailored gamification in education: A literature review and future agenda

Abstract

Gamification has been widely used to design better educational systems aiming to increase students’ concentration, motivation, engagement, flow experience, and others positive experiences. With advances in research on gamification in education, over the past few years, many studies have highlighted the need to tailor the gamification design properties to match individual students’ needs, characteristics and preferences. Thus, different studies have been conducted to personalize the gamification in education. However, the results are still contradictory and need to be better understood to advance this field. To provide a complete understanding of this research domain, we conducted a systematic literature review to summarize the results and discussions on studies that cover the field of tailored gamified education. Following a systematic process, we analysed 2108 studies and identified 19 studies to answer our research questions. The results indicate that most of the studies only consider students’ gamer types to tailor the systems, and most of the experiments do not provide sufficient statistical evidence, especially regarding learning performance using tailored gamified systems. Based on the results, we also provided an agenda with different challenges, opportunities, and research directions to improve the literature on tailored gamification in education. Our study contributes to the field of gamification design in education.

Introduction

To target the problems of students’ evasion, disengagement, and lack of motivation in educational environments, recent research has been using gamification¹) along with its activities (Battistella & von Wangenheim, 2016; Legaki et al., 2020; Oliveira et al., 2022). Gamification in education usually aim to improve students’ concentration, engagement, performance, and/or decrease students’ frustration and demotivation in educational systems (Cózar-Gutiérrez & Sáez-López, 2016; Shi & Cristea, 2016; Mostafa, 2019; Lopes et al., 2019; Metwally et al., 2020). Overall, these studies are implementing and evaluating the use of gamification in educational environments (Oliveira & Bittencourt 2019; Toda et al., 2019a, 2019b).

Recent studies demonstrated that these systems can offer different ways for students to perform desired educational activities associated with game elements (Majuri et al., 2018; Koivisto & Hamari, 2019; Bai et al., 2020a). In addition, gamified education may provide a number of benefits to students, e.g., increasing students’ motivation (Shi et al., 2014; Cózar-Gutiérrez & Sáez-López, 2016; Stuart et al., 2020), enhancing learning performance (Lo & Hew, 2020; Zainuddin et al., 2020), or improving training processes (Kapp, 2012; Larson, 2020).

However, some studies state that, in many cases, the use of gamification in an educational context (especially gamified educational systems) does not necessarily improve students’ outcomes (Toda et al., 2017; Koivisto & Hamari, 2019). These results have been drawing the attention of the community to better understand when and how the use of gamification effectively improves students’ experience and, hence, propose solutions to providing a better gamification design which might impact positively on learning performance (Hamari et al., 2016; Koivisto & Hamari, 2019; Rapp et al., 2019).

In educational systems (and general educational settings), it is of the utmost importance to consider students’ individuality (Qaffas et al., 2020; Azzi et al., 2020; Mustafa, 2020), as the recent literature has shown that depending on the characteristics of students (e.g., personal preferences (Kosztyán et al., 2020), learning styles (Sanjabi & Montazer, 2020), susceptibility to different pedagogical methods (Barth, 2020), knowledge structures (Tsai & Chu, 2019), and others characteristics), their experience and/or performance may change depending on the design of educational systems. Therefore, also in gamified education, depending on the gamification design (i.e., the game elements used), the students’ experience and learning outcomes may be different (Hanus & Fox, 2015; Toda et al., 2017; Koivisto & Hamari, 2019).

To solve this problem, over the last few years, some studies were conducted to understand how to tailor gamified educational environments to match students’ characteristics, needs and behaviors (e.g., Stuart et al., 2019; Oliveira et al., 2020; Santos et al., 2021). However, the results are still contradictory, and it is not possible to identify the learner traits used to personalize gamified educational settings or the impact of gamification personalization on students’ learning experience (e.g., learning outcomes and psychological states) (Orji, 2014; Hanus & Fox, 2015; Hamari et al., 2016; Monterrat et al., 2017; Toda et al., 2017). At the same time, despite the existence of some systematic studies on tailored gamification, including tailored gamification in education (Klock et al., 2020; Hallifax et al., 2019; Rodrigues et al., 2020), some research questions remain open.

Based on this, in this article, we aim to answer the following questions: i) what learner traits have been used as the basis of personalizing gamified education?; and ii) how has personalized gamification in education affected students’ learning outcomes and related psychological states?; Aiming to answer the aforementioned questions based on the state of the art on tailored gamified educational environments, we conducted a systematic literature review following the well known protocol proposed by Kitchenham (2004).

The main results demonstrate that a) most of the included studies use only the aspects related to students’ gamer type/user types to tailor the gamified educational systems; b) studies do not compare a personalized version with a non-personalized one, thus, its not possible to identify how personalized gamification affected students’ learning experience due to methodological issues in those studies; c) most studies do not consider aspects to personalize systems in real-time.

This context showcases the importance of considering other students’ characteristics and behaviors to tailor gamified educational environments (i.e., students’ gender, age, etc.), as well as providing automatic adaptations on the systems. Besides, the findings also highlight the importance of conducting new empirical/experimental studies to ground the impact of this kind of system on students’ learning outcomes, especially comparing tailored gamified educational environments with non-tailored gamified educational environments in terms of students’ learning outcomes.

This article is organized as follows: in Section 2, we present the study background, depicting an overview of tailored gamification. Section 3 presents the study protocol. In Section 4, we present our results. Section 5 presents a general discussion about our results. In Section 6, we present an agenda with a series of challenges, opportunities and research directions based on our results. Finally, Section 7 presents our concluding remarks.

Background

This section introduces the main topic related to our study, that is tailored gamification in education and present the main related works (i.e., some similar reviews recently conducted).

Tailored gamification in education

In recent years, several studies have been conducted to apply gamification in education (i.e., transforming educational systems to better afford similar motivational benefits as games often do) and investigating the effects of gamification on students’ experience and learning (Rocha Seixas et al., 2016; Oliveira et al., 2020). If for one side, using gamification in education, in general, increases student engagement and motivation (Koivisto and Hamari, 2019; Sailer & Homner, 2019; Bai et al., 2020b), on the other side, there are cases where gamification causes the opposite effects, discouraging or impairing the learning outcomes of some students (Hanus & Fox, 2015; Toda et al., 2017; Kwon & Özpolat, 2020).

At the same time, studies have shown that in educational settings, depending on different characteristics of students, the educational model (e.g., educational system or classroom) needs to be personalized to suit the characteristics of each student (Qaffas et al., 2020; Azzi et al., 2020; Mustafa, 2020). This situation led researchers to believe that one of the possible factors that can help improve the effects of gamification on the students’ experience is the personalization of the gamification design (Monterrat et al., 2017; Oliveira & Bittencourt, 2019; Stuart et al., 2020). Thus, in recent years, many studies have highlighted the challenges from tailoring gamification based on students’ individual characteristics (Vassileva, 2012; Orji et al., 2013; Monterrat et al., 2014a; Lavoué et al., 2018; Oliveira et al., 2020).

These studies propose different solutions in tailoring and also investigating the importance of personalizing those systems based on students’ characteristics (Klock et al., 2020). The idea of personalization in gamification comes from the concept that people have different personalities, behaviors, and needs (Sullivan et al., 2017; Bourdieu, 2017; Oliveira & Bittencourt, 2019), as well as from the fact that these differences alter the way people interact with each other, within computer systems, and the way they organize their study routine (Bartle, 1996; Bateman et al., 2011; Masthoff & Vassileva, 2015).

Considered as one of the first studies that address the personalization of gamification, Ferro et al. (2013) presented a theoretical background about the relationship among various personality types and traits. The authors also outlined player typologies and assumed that this relationship was a better way to inform designers on a deeper level of understanding about the type of users to whom the gamified systems are designed. In a more recent study, Orji et al. (2013) developed seven different models of healthy eating behaviors for the BrainHex gamer types exploring the differences between the seven models. She also proposed two different approaches to persuasive game design. These approaches were proposed to motivate the majority of the population, while avoiding the discouragement of any player, by proposing a personalized approach to better motivate a particular type of gamer.

More recently, Oliveira and Bittencourt (2019) published the first book on the subject, which addresses the history of tailored gamification in education, and presents some techniques for the personalization of gamification based on gamer types and gender. In summary, the studies related to tailored gamification concern identifying student’s individualities and relate them to their preferences regarding game elements (Orji et al., 2014; Monterrat et al., 2015; Lavoué et al., 2018). Considering how recent this field of study is, most studies do not present deep analyses related to the students’ learning outcomes on the tailored gamified educational systems.

Related works

To investigate the effects of personalized gamification on the user experience, in recent years, other researchers have also conducted some secondary studies. Stuart et al. (2019) addressed three research questions related to the kind of contributions in the field and the impact of personalized gamification. The results showed that the contributions were still incipient and that few aspects were evaluated (Stuart et al., 2019). Lopes et al. (2019) conducted a systematic literature review (SLR) in the field of personalized gamification in education aiming to understand how these adaptive features work, what its adapt and which strategies they adopt. They identified some strategies related to different learning topics, based on different factors to personalize the gamification (Lopes et al., 2019).

Rozi et al. (2019) conducted another SLR, however focusing in discovering the components, methods, and frameworks used to adapt gamification. The authors managed to discover that there are four components used, seven methods and three frameworks used to adapt the gamification (Rozi et al., 2019). In general, the results presented in the paper focus on more general and technical aspects, as well as, they are not focused on the area of education. Alomair and Hammami (2020) conducted a literature review to identify the methods used in adapting into gamification learning environments. Although the authors present and discuss some methods (Alomair & Hammami, 2020), this work also focuses on a unique technical aspect related to gamification adaptation and does not follow a systematic protocol.

Another recent SLR, Klock et al. (2020), concerns “tailored, personalized, adaptive and recommended” gamification. The authors selected 42 studies and found that the most considered characteristics of the user profile are the player preferences, gender and personality traits. The majority of these studies still consider methods of user modeling and that tailored gamification is still a trend (Klock et al., 2020). They recommended that future research should focus on dynamic modeling, exploring multiple characteristics simultaneously and understanding the effects of other aspects other than users’ profiles.

In summary, the secondary studies presented in this section that focus on the personalization of the general gamification (regardless of context), show that the majority of studies were conducted in the field of education and that it is necessary to conduct new secondary studies focused on this domain. At the same time, the studies that focus on the personalization of gamification in education, are not systematic or focus only on a technical aspect of personalization. As far as we know, our study is the first review to focus on tailored gamification for educational systems and answer questions related to the psychological and computational aspects used in this personalization, cross-referencing this information and summarizing empirical results related to the influences of personalized systems on students’ experience. In addition, we present a series of directions to meet the challenges of this area.

Methodology

Aiming to identify the state of the art on tailored gamification in education, we opted to conduct a SLR to identify, evaluate, and interpret available research findings related to our research questions, topics, or phenomenon (Kitchenham, 2004) (i.e., tailored gamification in education).

Protocol structure

The main purpose of conducting a SLR is to gather evidence from which some conclusions can be drawn (Kitchenham, 2004). According to Kitchenham (2004), a SLR is composed of three main phases: (i) Planning – where one needs to define the research questions, develop and validate the protocol; (ii) Conducting – where one identifies relevant research, selects primary studies, assesses study quality, extracts required data and synthesizes data; and (iii) Documenting – in which one writes and validates the report (Ampatzoglou and Stamelos, 2010). To perform the SLR, the guidelines proposed by Kitchenham (2004) were followed, as presented in the following sub-sections.

Study objectives

The focus of this literature review is to provide the state of the art on tailored gamification in education. In other words, we aim to identify topics that are not covered (or are scarce) in the literature such as the properties that are considered to develop tailored gamification in education (e.g., the user’s personality traits - psychological property - or the computational tools used to develop those tailored gamification in education - computational properties), the processes used to develop the tailored gamification in education and the evidence that has been provided until now. To achieve this goal, we defined three specific objectives:

• To identify the personalization approach (e.g., psychological aspects or design approach) used to personalize the tailored gamified environments;

• To identify how tailored gamified environments are developed (e.g., types of algorithm or systems used in the personalization); and

• To identify the empirical evidence related to the effects of tailored gamified environments in the student experience (e.g., effects on students’ learning).

By achieving these goals, we contribute to the current literature by providing an overview of tailored gamification in education. At the same time, based on our results, it will be possible improve the discussions about how to develop new studies to advance the literature on tailored gamification in education.

Research questions

Next, after defining the goals of our SLR, we developed our research questions. Each Research Question (RQ) was developed to address the specific objectives of our study as follows:

• RQ1: What learner traits have been used as the basis of personalizing gamified education?

  • RQ1.1: What psychological aspects have been used as the basis of personalizing gamified education?

  • RQ1.2: What personalization approach have been used as the basis of personalizing gamified education?

• RQ2: How has personalized gamification in education affected students’ learning outcomes and related psychological states?

  • RQ2.1: Do students using tailored gamified educational environments have different learning outcomes compared to students using non-tailored gamified educational environment?

Search string

Following the Kitchenham (2004) protocol, we defined our search string based on the recent studies on gamification and educational technologies (e.g., Bittencourt et al., 2016; Hamari et al., 2014, 2016; Santos et al., 2018; Koivisto and Hamari2019; Klock et al., 2020). It was validated by three experts in gamification and educational systems research. The validation was based on analyzing the groups of terms associated to the main term by including or removing them. To find studies on tailored gamification, the chosen terms were: “gamification” and “personalization”; “gamification” and “customization”; “gamification” and “adaptation”; “gamification” and “tailoring”. To define the terms related to educational technologies, we used the same sequence defined and validated by Santos et al. (2018).

Sources search (digital libraries)

After the definition of our search string, we established our sources. To ensure that we found as many studies as possible within the study scope, despite knowing that Scopus encompasses most studies included in other databases, inspired in different recent secondary studies in the field of education and gamification (e.g., Hamari et al., 2014; Bittencourt et al., 2016; Santos et al., 2018; Klock et al., 2020), as well as on the suggestions of experts in gamification and educational, we decided to use the main search digital libraries in the field of study. Then, we selected seven different digital libraries to conduct our research: i) ACM Digital Library; ii) Engineering Village; iii) IEEE Xplorer; iv) Science Direct; v) Scopus; vi) Springer Link; and vii) Web of Science.

Inclusion and exclusion criteria

Then, we defined the Inclusion and Exclusion criteria (IC and EC, respectively) for the selection of relevant studies, which are summarized in Table 1. We aimed at obtaining the maximum number of international studies, so we opted for primary studies (i.e., empirical/experimental studies) about personalization of gamification in education, that were written in English. Data where extracted by two experts.

Table 1 Exclusion and inclusion criteria

Data extraction and period

In the next step, we defined the data to be extracted from each selected study: (i) study information (reference; title; authors list; authors’ country; authors’ affiliations; source; source type (journal or conference); publishing year; and abstract (inspired on Santos et al., 2018)) (ii) publication date; (iii) application domain; (iv) approach used for tailoring the gamified educational environment; (v) psychological aspect used to tailor the gamified educational environment; (vi) computational aspect used to tailor gamified educational environment; (vii) tool; and (viii) empirical results related to the use of tailored gamified educational environment on students’ outcomes. The items (i), (ii) and (iii) are related to the general demographic characteristics of the study. Items iv, v, vi, and vii are required to answer RQ1, RQ1.1, and RQ1.2. Finally, item viii is needed to answer RQ2 and RQ2.1.

According to different secondary studies (Hamari et al., 2014; Koivisto and Hamari, 2019; Bai et al., 2020b), the first studies on gamification began to be published in 2011. Therefore, as it is a relatively recent area, we chose not to define an initial period for the search. This decision helps us to identify in which year the first study was published and how the area has evolved over the years.

Studies comprehensiveness assessment

In the following step, we defined the comprehensiveness assessment for the selected studies. We used the criteria presented in Table 2 for this assessment. The criteria one to eight evaluate the general comprehensiveness of the studies, following the recommendations of recent guidelines. The criteria nine to eleven were defined by us to evaluate specific points related to tailored gamified educational environments. To better classify the studies, the comprehensiveness assessment was conducted by three researchers with expertise in gamification and in the conduction of secondary studies (all authors of this study).

Table 2 Studies comprehensiveness assessment

Data collection and analysis

The data collection of the SLR was conducted from May to June (2020) by two experts in education and gamification, as well as in the conduction of secondary studies. These experts are authors of this article. Both experts read the title and abstract of all studies. Doubts regarding the inclusion of studies were discussed between the two experts. A software for management of secondary studies (Parsif.al²) was used to ease the review process. The software was responsible for managing which studies were already analyzed, whether the studies were rejected, performing quality assessment and data extraction. The software also allowed to automatically identify duplicate studies.

Results

In this section, we present the results of our SLR. At first, we present an overview of the demographic properties of our studies. Then, we answer each RQ in the following subsections.

Studies overview

After running our search string in the seven digital libraries (see Section 3.1.4), we found a total of 2108 studies. After removing the duplicated ones, we had 1962 studies left. Next, the two experts read the title, abstracts and keywords of the 1962 studies. In case of discrepant opinions, the experts discussed the study until they reached an agreement following the process previously mentioned. After this step, 29 studies were selected, which were thoroughly read by the two experts. Finally, 19 studies answered at least one RQ of this SLR. Figure 1 summarizes the filtering process and Table 3 presents the identifier (Id) and title of the 19 selected studies.

Fig. 1

Filtering of Studies

Table 3 List of selected studies

Authors and publication year

Before starting to answer the RQs, we present a series of important information related to the studies on tailored gamification in education. Our results indicate that a group of 37 authors conducted studies in the field of tailored gamification in education. Figure 2 presents an infographic with information regarding those. It shows the most productive authors in this field, which belong to the same research group, as well as demonstrates that over 80% of the authors has a single publication, indicating most scholars are entering / beginning to perform research in the field. In addition, as shown in Fig. 3, it is possible to perceive that this is a recent topic (with its first publication in 2014) and and was most popular in 2017. Our analysis identified only one article published in 2020, however, the selection of studies was last updated in 2020, June. At the end of 2019, the first book on tailored gamification to educational technologies (i.e., Oliveira and Bittencourt 2019) was released, however, the book does not come into our review (because is considered as grey literature).

Fig. 2

Authors

Fig. 3

Publication Year

Countries and universities

Researchers from 12 different countries contributed to the research on tailored gamification in education. The European continent is the one with the highest number of contributions (11 studies in total) from five different countries (Germany, France, Spain, Finland and Italy). North America and South America together contributed with seven studies, with participation from four different countries (United States of America and Canada in North America, and Brazil and Colombia in South America). In addition to these, the Asian continent had the participation of two different countries (Iran and Japan) that together collaborated with the production of three studies. Finally, the African continent collaborated in the production of one study (produced in Tunisia). Besides, it is possible to perceive that 21 universities from 11 different countries have researches that contributed to the field. Three French universities were the ones that appeared the most in the identified studies. Three other universities and one research institute contributed with two studies each. Other 14 universities contributed one study each. Table 4 present the relation of the countries, universities and publications.

Table 4 Universities and Publications

Studies comprehensiveness result

From the studies comprehensiveness assessment, it is possible to perceive that only three studies received the maximum score (11 points). The studies 2017, 2018and 2020 (Monterrat et al., 2017; Lavoué et al., 2018; Oliveira et al., 2020) received the highest mark in all the criteria evaluated in the studies comprehensiveness assessment. Two of the three studies with maximum score were published in journals. Next, tied in the second position with 9.5 points, we find the studies 2015and 2017 (Monterrat et al., 2015; Sailer et al., 2017), also published in journals. Tied in the third position with 9.0 points, we find the studies 2016 and 2017 (Roosta et al., 2016; Denden et al., 2017). (Roosta et al., 2016) and (Denden et al., 2017) were published in different conferences. Figure 4 presents an organization of the comprehensiveness assessment of each study.

Fig. 4

comprehensiveness assessment of studies

RQ1: What learner traits have been used as the basis of personalizing gamified education?

This section aims to identify which computational and psychological properties have been used to tailor gamified educational environments. To better present our results, we divided this question in two sub questions: the first subsection deals with psychological properties and the second deals with computational properties.

RQ1.1: What psychological aspects have been used as the basis of personalizing gamified education?

To answer this question, we identified which students’ traits (e.g., learning styles or gamer type), demographic factors (e.g., age or gender) or psychological states (e.g., motivational factors) were used with the purpose of personalizing the gamified educational environments in each study. From the 19 selected studies, 17 of those show the psychological or human aspect used to tailor the gamified educational environments. The results present 11 different approaches, organized according to the three aspects mentioned above. Figure 5 organizes the finds.

• Students’ traits: gamer type, learning style, goal orientation, personality trait.

  Fig. 5

  

  Psychological approach and tools

• Demographic factors: age, gender, place where live.

• Psychological states: user preference, psychological satisfaction, motivational stage.

• Other: instructor preference.

Most of the studies (10 studies) tailored the gamified educational environments using gamer type. Out of these, five studies used the BrainHex gamer type to tailor the systems (2015, 2014b, 2017, 2018, 2020), whereas 2017 and 2019used the Hexad player model and the study 2016 used the Bartle player types. Two studies (2014 and 2017) do not show which player model they used.

Three different studies used the students’ learning style to tailor the systems. 2017 used the Felder-Silverman Learning Style; 2014 and 2017used learning styles but did not identify which learning model was used. Other two studies used students’ personality traits: 2017 used the Big Five personality trait model; 2017 did not show which learning styles model was used.

2016 used students’ goals orientation as a theory to tailor the gamified educational environments, and it used the Elliot categorization as a specific tool to identify the students’ goal orientation. Two studies (2016 and 2014a) used the students’ age yet without specifying the tool they used. In the same way, 2014a also used the students’ gender without specifying their approach - for instance, which properties should be changed in gamification according to the age and gender of the students.

2017 used demographic data, whereas study 2017 used instructors’ preference. 2016 used users’ preferences. 2017 used psychological satisfaction. However, none of these studies reported the specific approach considering the human or psychological characteristic that was reported. Finally, 2016 and 2014 reported the use of motivational stage as human/ psychological property yet without reporting the approach they used. Only two studies (2017, 2017) did not report the use of a human/psychological aspect used to tailor the gamified educational environment.

Based on these findings, we can summarize the answer for RQ 1.1 by concluding that 11 psychological approaches have been used to tailor gamified educational environments, namely gamer type, learning style, trichotomous goals, personality trais, and personal data, whereas seven tool were adopted to implement these, namely BrainHex, Bartle model, Hexad, Felder-Silverman Learning Style, Elliot categorization, Big Five, and Mayers Briggs Type Indicator.

RQ1.2: What personalization approach have been used as the basis of personalizing gamified education?

To answer this question, we started by seeking and selecting computational approaches used to tailor the gamified educational environments, and specific types of study (e.g., an evaluation or an experiment). From the 19 selected studies, 13 of those present the computational approach used to tailor the gamified educational environments. The results shows six different approaches, including (i) tool (e.g., mobile app or high quality prototype), (ii) framework, (iii) process, (iv) architecture, (v) ontology, and (vi) computational model. Figure 6 presents the computational approach and type of each study, which demonstrates that some of those used more than one approach.

Fig. 6

Computational approach and type of study

Most of the selected studies (10 out of 17) proposed or used some tool. Four studies proposed a tool (2014b; 2016; 2017; 2014a) and three studies conducted an experiment using some of those (2017; 2017; 2020). Besides, 2016evaluated a tool, 2017 conducted a quasi-experiment, and 2017 only presented a specialist-based opinion. Other six studies presented frameworks to tailor gamified educational environment. 2017, 2017 and 2017 proposed frameworks, and 2016, 2017 and 2017 performed a validation of the frameworks. Based on that, it is possible to notice that 2017 and 2017 proposed and validated the frameworks.

Only two studies presented a process to tailor gamified educational environments. 2018 and 2019 proposed and validated this process. Also, only one study (2017) presented an architecture to tailor the gamified educational environment, although no evaluation was provided. 2014 validated an ontology (through a case study) to personalize the gamified educational environment, and finally, 2015 conducted an empirical experiment in a computational model to tailor the gamified educational environments.

We also classified the computational approaches in different types. Initially, we extracted whether the studies presented some aspect related to automating the tailoring process and whether the studies presented some tool. In the studies that offered an automatic approach, we analyzed whether the study presented the algorithm used to automatically tailor the gamified educational environment. In the studies that presented a tool, we also extracted what tool was used (Fig. 7) and mapped each one to the study using it in Table 5.

Fig. 7

Computational approach classification

Table 5 Tool used by study

Our results indicate that only three studies (2016, 2017 and 2014a) presented some automatic approach. However, in the studies 2017 and 2014a, the automatic approach is related only to the content, not to the gamification design. Thus, only study 2016, in fact, presents an approach to automating the gamification design. 2016 used a tutor module with artificial intelligence to tailor the gamification design, aiming to enhance the students’ engagement in an intelligent tutoring system. 2017 used Data Mining (via Naive Bayes) to personalize challenges in a gamified sustainable mobility scenario. 2014a also exploited Data Mining, using students’ data logs aiming to motivate learners through an adaptive gamified web-based learning environment.

Given these results, we can summarize the answer for RQ 1.2 by concluding that the computational properties that have been used to tailor gamified educational environments are, in frequency of use decreasing order, tools, frameworks, processes, architecture, ontology, and model; wherein most of those were proposed by the studies, whereas validating an existing approach was less frequent. Additionally, our results show over 80% of the studies do not use automatic adaptation, which were mostly based on Data Mining, and over 50% of those provide the tool they used (see Fig. 7).

RQ2: How has personalized gamification in education affected students’ learning outcomes and related psychological states?

To answer this question, at first, we identified the studies that conducted empirical experiments. Next, we extracted the empirical results reported in each one. In 11 studies, it was not possible to identify empirical results (2017, 2016, 2014, 2017, 2015, 2014b, 2016, 2017, 2017, 2014a, 2017), whereas for the other six it was, as shown in Fig. 8.

Fig. 8

Empirical Studies

Only two studies presented results indicating that the tailored gamified educational environments demonstrated positive learning outcomes. Only 2016identified that the tailored system improved the learners’ motivation and performance. 2018 identified that students interacting with adapted gaming features spent significantly more time in the learning environment (Lavoué et al., 2018). However, it also identified that learners with non-adapted features had a higher level of motivation.

2017 showed that the adaptation process did not improve learners’ engagement as expected by the authors, and the learners’ engagement was similar for the students in both the tailored and the non-tailored system. 2017 identified that badges, leaderboards, and progress positively affected competence and satisfaction, and also perceived task meaningfulness. At the same time, avatars, meaningful stories, and teammates affected experiences of social relatedness (Sailer et al., 2017). However, the study did not provide a relationship with other kind of player types.

2017 only confirmed that students had different preferences for gamification according to their personality, without providing a relation between these player types and their individual preferences in terms of gamification design. Finally, 2017tested different computational approach to better tailor gamified educational environments, identifying that Naive Bayes performed better than other methods and could be effectively used to improve the challenge selection process to different player types.

To provide a deeper discussion on this question, we also extracted the longitudinal experimentation from the selected studies. We considered longitudinal studies researches with a design that addressed repeated observations of the same variables (e.g., people) over a period of time (i.e., uses longitudinal data). That is often considered a type of observational study, although they can also be structured as longitudinal randomized experiments (Cook et al., 2002).

None of the selected studies presented a longitudinal study. Some studies (e.g., 2016, 2017, and 2014a) presented approaches that used data from a long-term course. However, these approaches are not based on repeated observations of the same variables, and, according to Cook et al. (2002), these are not longitudinal studies.

Overall, our results for this RQ show the lack of empirical and longitudinal studies in the domain of gamified education, corroborating the gamification literature (Hamari et al., 2014; Seaborn and Fels, 2015). Based on the small body of empirical evidence related to the use of tailored gamified educational environments available, we can conclude they are inconsistent, with some studies showing positive outcomes whereas others yielded inconclusive / negative results, as summarized in Table 6.

Table 6 Empirical evidence on tailored gamified education

RQ2.1.: Do students using tailored gamified educational environments have different learning outcomes from students using non-tailored gamified educational environments?

Previously, we analyzed the general results of personalized gamification in education, even when the analyzed study did not compare a personalized system with a non-personalized system. Now we will focus on analyzing only the results of studies that compared a personalized system with a non-personalized one in terms of students’ experience.

Only 2018 conducted an experiment that met such criterion. This study identified that students using adapted gaming features spent significantly more time in the learning environment (the learning outcome in the study’s context) than the students using non-adapted gamification features. Besides, students with features that were not adapted had a higher level of demotivation in comparison to the learners with adapted gaming features.

Hence, 2018 demonstrated that both the students’ learning outcomes and motivation can be impacted by the use of tailored gamified educational environments. Given this context, we can answer RQ 2.1 by hypothesizing that the students’ learning outcomes differed when comparing tailored to non-tailored gamified educational environments, although this claim is supported by a single study and can not be generalized.

Threats to validity

In this section, we present the recommendations for the improvement of future replications of the SLR quality and for the increment of the results’ generalization scope (Santos et al., 2018). We classified the threats to validity using the Internal, External, Construct, and Conclusion categories of Wohlin et al. (2012).

With regards to internal validity, we might have made some decisions subjectively, during studies analysis and data extraction. In some studies, the authors of our selected studies did not offer a clear presentation or objectives about their result, which made it difficulty for us to apply the exclusion or inclusion criteria, or an impartial data extraction (Santos et al., 2018). To minimize potential misleading, we adopted a peer-to-peer approach to conduct the selection process, i.e. two expert read all articles separately, and in case of conflict, they met and discussed the conflict until they reached an agreement. We also used the same approach to solving conflicts regarding exclusion or inclusion of some of our selected studies. As such, we managed to reduce the threats that might be caused by personal bias.

The external validation is concerned with how to generalize the SLR results. It is related to how the primary studies are illustrative for the overall reviewed studies (Vilela et al., 2016). To reduce the potential external threats, following (Santos et al., 2018), we defined our search string after several different trial searches and validation with the agreement of all expert authors. We also tested the retrieved studies coverage and relevance in the automatic database search. For some studies, we were not able to retrieve the document due to access restrictions. Nevertheless, to minimize this threat, we requested these studies by contacting the authors and we received all these studies from the authors.

About the construct validity, there were two key concepts in the main construct of this study, including “tailored gamification” and “educational technologies”. For the former, we used the term “gamification” and related terms to adaptation (i.e., customization and tailored) to ensure all selected studies were related to tailored gamification approaches. For the latter, several related terms to educational technologies were used. However, it is possible that some important terms have not been placed, and, consequently, that some studies have not been identified.

To mitigate this threat, the terms were defied according to the ones considered in some search string of recent secondary studies in this field (i.e. Dermeval et al.2014; Bittencourt et al., 2016; Santos et al., 2018) as well as based on researchers’ suggestions (researchers with expertise in the field). As there was not any conference or journal specifically concentrated on the joint use of the used concepts, we do not conducted any manual search in our study. In summary, we mitigated these threats through the inclusion of different terms that were related to the two main topics/terms in our study from the seven databases aforementioned.

Lastly, in regard to conclusion validity, it might be the case that we did not include all the studies that should have been included in the review process. Towards mitigating this specific threat, we carefully and together discussed the selection and the criteria of exclusion and inclusion. Additionally, there was a time-gap between the data collection process and the article writing-up - some recent studies published after June 2020 were not considered in our review process. Nevertheless, we believe this threat would not strongly affect the major results, as we considered a good amount of studies in our SLR.

Discussion

In recent years, several studies have begun to analyze the effects of personalizing gamification in educational settings. To understand the state of the art in tailored gamified education and to propose a research agenda in this field, we conducted an SLR. As a result, we identified that most studies consider only gamer types to personalize the systems, few studies propose the automatization of gamification personalization, and that the results do not make it clear whether the personalization of gamification is effective in students’ learning. Based on our results, it is possible to identify some topics covering the state of the art about tailored gamified education. We could perceive that this is a recent topic, with many studies conducted in the last few years, that is understood by the researchers as a topic with different open challenges.

Our findings show that most of the studies considered some human aspects to tailor the gamification (i.e., gamer type, learning style, goals orientation, personality traits, age, gender, demographics aspects, instructor preference, user preferences, psychological need satisfaction, and motivational stage). However, most studies addressed only gamer type, in detriment to other important human aspects. General cultural aspects were also not observed in the selected studies, nor gender differences on perceived gamification acceptance and learning outcomes. According to recent studies (e.g, Oyibo et al., 2016; Oyibo et al., 2017a; Oyibo et al., 2017b), cultural aspects can also change human preferences for the design of the system; also, students’ gamer types can change during their life experience. Hence, it is essential to highlight the importance of considering both other individual human aspects/characteristics as well as personal changes when tailoring educational systems towards smarter learning environments.

Besides, we identified that most studies did not conduct empirical and/or experimental researches. Instead, they sought support in theoretical studies or reported theoretical critics about penalization in gamified educational environments. Only two studies conducted experiments to compare tailored and non-tailored systems regarding students’ learning outcomes. However, the preliminary results indicated that tailored systems can be better in some cases (e.g., learners’ using an adapted system spend significantly more time in the learning environment), whereas the non-tailored system was more effective than the tailored system in others (e.g., better students’ motivation). These results support the relevance of adapting gaming features to enhance learners’ engagement and provide cues on ways to implement adaptation mechanisms (Lavoué et al., 2018). These results also corroborate the existent gamification literature, which express the need of tailoring game elements to the users’ needs.

In this sense, we consider the tailoring process must occur at two levels: (i) the content, that can be adapted to the users, when using intelligent systems such as Intelligent Tutoring Systems; and (ii) game elements, each of which can provide a different kind of experience to the user. As we can observe, this has not been explored in many of the gamification elements, but it suggests that those elements must tailor the user’s experience, so they can achieve a motivated state and, consequently, enhance learning outcomes. One example is to adapt the way the user wins points or badges, based on their interactions within the system, or based on their affective interactions that can be captured through a camera, while using the system.

Nevertheless, findings on the impact of tailored gamified educational environments are based on a single study, which highlights the importance of conducting new similar studies (comparing tailored gamified educational environments to non-tailored ones). This emphasizes the need for identifying whether tailored systems are better than non-tailored systems in terms of students’ learning outcomes, as well as learning how to design tailored systems to better improve the students’ learning outcomes, similar to other domains (e.g., behaviour change Orji 2014; shopping habits in e-commerce Adaji et al., 2018; health sciences Oyibo 2016; and others). We argue this type of comparison should be conducted in different perspectives, for instance, in terms of students’ psychological characteristics (e.g., students’ engagement, concentration and flow experience) or considering human-computer interaction (HCI) factors. Comparative studies are important to provide empirical support to tailored gamified educational environments as well as performing so in different perspectives improves findings’ generalization.

Agenda for future studies

According to the results obtained in this secondary study, it was possible to identify an overview about the studies on tailored gamified educational environments. This overview allowed us to identify some research challenges, opportunities and research directions in the field of tailored gamification in education. Thus, this section presents an agenda for future studies in this field.

Are tailored gamified educational environments better than non-tailored gamified educational environments in terms of students’ outcomes?

We have already presented that among the 19 primary studies included in our SLR, only two compared a tailored gamified educational environments with a non-tailored gamified educational environments, and neither compared the tailored environment with a non-gamified version of the environment. In general, the community is based on theoretical studies about different perceptions of people on gamification design (e.g. Bartle 1996; Nacke et al., 2014; Yee 2006). However, it is unknown whether tailored educational environments are better than non-tailored ones in terms of students’ experiences (e.g., engagement, motivation, flow experience, and others).

These results attest that there is no study demonstrating that tailored gamified educational environments are better than non-tailored gamified educational environments or better then non-gamified systems. In this context, one of the challenges lies in the importance to identity whether tailored gamified educational environments are better solutions than non-tailored gamified educational environments regarding students’ outcomes and some development aspects (e.g., financial and time costs). Based on this, future studies must identify whether and when tailored environments are better than non-tailored environments.

To tackle this challenge, the community must conduct new studies evaluating the students’ interactions in this kind of system to identify from different ways (e.g., user data logs, questionnaires, specialists evaluation) whether the students are presenting a better performance in the tailored gamified educational environments. Besides, recent studies on data mining (e.g., Hanna2004; Neeraj et al., 2017), and HCI techniques (e.g., Nacke 2017; Nacke and Deterding 2017) represent great opportunities to effectively evaluate students’ perceptions in this kind of system, providing a bridge between students’ perceptions on tailored and non-tailored gamified educational environments.

How to design effective tailored gamified educational environments in terms of students’ outcomes?

Most of the studies analysed in our SLR did not concern the tailored gamified educational environments regarding its design. This allows us to realize that the community does not know what are the best design solutions in tailoring gamified educational environments regarding students’ characteristics (e.g., gamer types, gender, age, and others). Thus, designing a tailored gamified educational environment is not an intuitive process. Our results also show a lack of studies that may guide on how to provide a good tailored gamified educational environment.

Based on our results, the community needs to study ways of designing tailored gamified educational environments capable of shaping users’ needs and preferences and to give subsidy so that designers can have a solid theoretical and empirical foundation for developing this kind of environment. For instance, when a gamification designer is planning a tailored gamified educational environment through a framework, the designer needs to know how to personalize the system according to its goals and the needs and preferences of its users.

In recent years, some frameworks to gamify educational systems have been proposed (Mora et al., 2015). Some of these frameworks have drawn attention to the importance of understanding the preferences of their users and even the importance of planning the system so that it will be adapted to its users, as cited in the secondary studies of (Mora et al., 2017). Thus, an important opportunity to solve this problem is related to the proposal of frameworks (new or adapted from other pre-existing ones) with a focus on supporting the gamification designer to tailor the gamified educational environments.

How to automate the process for tailoring gamified educational systems?

From our findings, it is notable that a large majority of the studies (82%) did not use any form of automation in the design of their tailoring approach. The small part that did use automatic approaches exploited the use of Data Mining techniques or a gamification engine. This context demonstrates that, within the field of tailored gamification in education, the researchers have not properly explored the potential of automation to improve the design process for tailoring gamified educational systems. Based on this, and corroborating the challenge of establishing how to design gamified educational systems effective on students’ learning outcomes, we highlight the challenge of how to automate this design process.

Automating the design process is relevant to improving designers/developers efforts when creating tailored gamified education systems. Without having to manually create personalized designs for gamified educational systems, the development team can focus on other tasks; besides, the automatic process would allow less experienced people to work on producing tailored gamified educational systems. Additionally, this automatic process would contribute to students by providing them, for instance, gamification designs specifically based on their characteristics (e.g., gamer type, age, or genre) that could be adapted throughout the use of the system, according to the specific outcome aimed by the learning task (e.g., automatically adapting the system to present a personalized design that leads students to cooperate).

Empirical and longitudinal studies on tailored gamification in education

Conducting empirical and longitudinal studies is a research challenge in many areas. Our study shows that this is also a challenge in studies related to tailored gamification in education. Among the selected studies in this SLR, only 35% (six studies) conducted empirical studies yet none of them was longitudinal. Based on this, future studies on tailored gamification in education must consider the possibility of conducting this kind of study. This type of study will be important for the community to have results based on experimental settings about the effects of personalized gamification on students’ learning outcomes, as well as how results perpetuate or change over time.

Despite the difficulty of conducting this kind of study, some resources are considered an opportunity to increase the possibility of conducting these studies. One of these possibilities is using data mining, which may allow tailored systems to get the students’ data log implicitly and continually. Thus, at the end of a large-scale course, researchers may have data on a large scale that can be statistically analyzed according to empirically defined metrics. Known studies with guidelines to conduct experimental and longitudinal research (e.g., Cook et al., 2002; Wohlin et al., 2012) can help researchers to conduct this kind of study. Table 7 present a summary for the proposed agenda.

Table 7 Agenda

Concluding remarks

This article presented a SLR for investigating how tailored gamified educational environments are designed, their effects on the students’ learning outcomes, and the research demanding on the design of tailored gamified educational environments. The main results indicate that the studies only use students’ gamer types/user types to tailor the educational systems, do not compare tailored gamification with non-tailored gamification, and there is no evidence on the effects of tailored gamification on students’ experience. As future studies, we suggest a deep analysis of such research challenges with inclusion of other studies that cover other issues not reported by the primary studies selected by this SLR, such as gender, gamer types and demographics differences in tailored gamified educational environments.