1 Introduction

The current dynamic environment and advancement of information communication technologies and the internet have altered education and learning processes in such a way that traditional passive learning methods are perceived as boring and ineffective (Dicheva et al., 2015; Ucar & Kumtepe, 2020). Instead, educators are urged to promote students’ creativity and help them expand their skills by active learning processes (Murillo-Zamorano et al., 2021). This is specifically true for distance education where space and time limitations have disappeared and it has become a more preferred education model. The emergence of the coronavirus and the crisis it created around the world had negative impacts on the social, material and moral lives of all individuals, however, as a result of this crisis, many institutions and organizations immediately switched to distance education in order to make education sustainable, and this proved once again that distance education applications are indispensable part of education. Motivating students to study and focus on the lesson is more difficult within distance education settings where individual work is prominent and peer support and collaborative learning is low. Chang et al. (2015) argued that learning and teaching practices in distance education are more successful and possible with the opportunities provided by new technology. Gamification-enriched programs can be one of the effective technological approaches since playing games is fun and a necessity in human nature.

Games cause strong emotional reactions such as curiosity, frustration, and joy (Deterding et al., 2011). Following relevant literature, gamification defined as “incorporating elements of game in a non-game context” (Deterding et al., 2011) is fairly a new trend and increasingly attracting more attention from educational researchers with its ability to decrease students’ boredom and increase student’ active learning, engagement and motivation(Hanus & Fox, 2015). Using gamification in education leads to more participation, collaboration, and fun in the learning process by means of positive feedback and aims to make students more motivated and interested in the lessons. Several reviews on gamification literature provide excessive support of its beneficial instrumental outcomes for students such as higher score, higher final grades, faster task performance, as well as psychological outcomes such as higher motivation, self-efficacy, joy, flow, perceived usefulness, and satisfaction (Dichev & Dicheva, 2017; Majuri et al., 2018).

However, despite, extensive amount of research focusing on different types of games and gamified projects, literature is still devoid of research appertaining to the challenge-based gamification applications (Koivisto & Hamari, 2019; Legaki et al., 2020). Challenge-based gamification in education is a new promising approach of gamified designs in which elements of achievement and intrinsic need satisfaction are integrated to increase students' interest in lessons, support the competitive spirit in the classroom, and ensure participation in the learning process by fulfilling individuals’ need for accomplishments and motivating students through use of tools like points, badges, levels and league tables (Deci et al., 1991; Dicheva et al., 2015; Xi & Hamari, 2019). Superior benefits of challenge-based gamification can be enlightened through constructivist learning theory which maintains the idea that socially constructed learning opportunities would enhance the active learning process. That is, learning is the result of learner’s interaction with the environment, the activity in which the learner is involved in, and other individuals in the learning environment who try to test the learner’s knowledge (Bada et al., 2015; Von Glasersfeld, 2012).

2 Research gap and contribution of the current study to the extant literature

The current study aims to contribute to the educational research in various ways. First, it seems that majority of research appertaining to gamification are conceptual in nature and current writings in education are bereft of research concerning the implementations of gamification specifically within higher education area (Murillo-Zamorano et al., 2021). The recent meta-analytic study by Sailer and Homner (2020) stressed that there is need for more experimental research centred on application of gamification and its relationship with students’ achievements and motivations. This echoes the review study of Dichev et al. (2015) which underscored that despite rapid increase in observed studies in gamification studies, majority of gamification related studies in education are descriptive only and the number of empirical studies that implemented the gamification in the learning context is still very rare.

Second, though there are studies which have tested the outcomes of gamification in education, evidence of its ability in enhancement of learning appears to be mixed. While some research demonstrated the positive outcomes of applying gamification such as better performance, more motivation and positive attitude (e.g. Araya et al., 2019; Legaki et al., 2020; Varannai et al., 2017), some other research had reported no serious impact or even adverse impact of gamification on students’ learning outcomes such as loss of performance and decreased motivation (e.g. Domínguez et al., 2013; Hanus & Fox, 2015; Toda et al., 2017). In fact the recent analytic studies have underscored the extant mixed results in regards to the impact of gamified learning process on students’ performance, motivation, and flow and called for further research in this respect (Bai et al., 2020; Majuri et al., 2018; Oliveira et al., 2021; Sailer et al., 2017). Thirdly, Huang et al. (2020) meta-analytic study calls for research concerning affective or behavioural learning outcomes of gamification since most of studies have explored cognitive learning outcomes of gamification (e.g. students’ academic performance). In current study we aim to fill this gap and explore students’ flow and motivation as two important affective outcomes of gamified learning environment.

Finally, challenge-based gamification is a new promising approach which has been rarely investigated. Legaki et al., (2020) draws attention towards challenge-based gamification and underlines that there is lack of empirical research in this area. In line with constructivist learning process, the challenge-based gamification emphasizes on active learning process through providing a learning context in which learners can communicate and collaborate with each other, get involved with what they learned and share their experience (Bada et al., 2015; Von Glasersfeld, 2012). Accordingly empirical result of the current research is likely to suggest important implications for academicians by developing and testing a novel gamification platform that can activate active learning process through challenge-based context.

3 Purpose of the study

The current research applies an experimental research design and aims to contribute to stream of empirical literature by developing and implementing a challenge-based gamified program and investigate whether the implementation of this gamification platform allows for increasing students' flow, motivation and academic achievement in higher education. The program that we designed is called Educhall which is a web-based gamified application.

More precisely the current study aims to assess the following research questions:

  • RQ1. After intervention of challenge-based gamification, is there a significant difference between students in the experimental group, and the control group, in terms of academic achievement?

  • RQ2: After intervention of challenge-based gamification, is there a significant difference between students in the experimental group, and the control group in terms of their overall motivation towards the course?

  • RQ3: After intervention of challenge-based gamification, is there a significant difference between students in the experimental group, and the control group in terms of the (ARCS) motivational subfactors of attention (A), relevance (R), confidence (C) and satisfaction (S)?

  • RQ4: After intervention of challenge-based gamification, is there a significant difference between students in the experimental group, and the control group, in terms of flow experience?

4 Background

4.1 Gamification

The idea of gamification is increasingly attracting scholars’ attention within distance education environment. Gamification is a term traditionally associated with games and is applied to a number of motivational triggers such as rewards and competition. Gamification which is “the use of game elements in non-game context”(Deterding et al., 2011) is increasingly applied within the educational context to enhance the learners’ experience. According to Bozkurt and Genç-Kumtepe (2014), gamification is defined as all of the actions that increase people's desire to be involved in the process, make the process more interesting and increase motivation. The concept of gamification emerges with components such as game-based interaction, game thinking, game mechanics, aesthetics, game dynamics, support and feedback, progress and limitations, relationships, cooperation and competition (Sailer & Homner, 2020). Unlike traditional classroom environments that have been used for centuries, gamification concept contributes to the creation of environments in which students will learn by having fun, increase their motivation and interest, and have a higher desire to learn.

Gamification not only makes the teaching process fun and increases motivation, but also creates a competitive environment.Challenge-based gamification is one of the most common mechanisms of gamification in which competition plays the key role; components of challenge-based gamification include points, achievements, levels, badges, user images known as avatars, collections, avatars that appear at the end of the game, competition, gift giving/sharing, tasks, virtual items such as virtual money, social graphics and scores(Koivisto & Hamari, 2019; Majuri et al., 2018; Pedreira et al., 2015). Competition is the desire to surpass others in pursuit of resources and rewards by comparing one's potential and achievements with others (Ruhl & Lordly, 2017; Sailer & Homner, 2020).Of course, certain conditions must be met in order to turn this competitive environment into a positive effect. For example, the success of competition as a motivational tool depends on whether the reward system is perceived as credible, transparent, compelling and fair (Buckley et al., 2017). When the leaderboard is shared at the end of any activity, unpleasant rivalries between students should be prevented (Ding, 2019).

Generally past and recent research indicate a positive association between gamification and desired learning outcomes such as increased motivation, self-efficacy, performance and creativity, however, the empirical studies within the education context indicate some mixed results associated with gamification as well which call to further research in to examining the potential outcomes of gamification in a real setting. For instance, in the research of Chiarelli et al. (2015), it was suggested that students gain more awareness in their own behaviours and are positively affected by the gamification technique. Legaki et al. (2020) examined the effect of a challenge-based gamification application called “Horses for course” on students and it was reported that students’ who used this gamification program had better performance than the traditional learning method. In their study, Hsu and Chen (2018) concluded that website administrators using gamification benefit their users and gamification can serve as a guide for research and development to create a competitive environment. Araya et al. (2019) examined the effect of an online gamification platform called Connect Ideas on the mathematics achievement of primary school students, it was revealed that the students who use the application are more successful than the students who do not use the application. Hamari and Koivisto (2014) conducted research by associating gamification with exercise; in their study the willingness of individuals to exercise, the effect on their social life and positive attitudes were examined. As a result, it was observed that gamification, combined with exercise, had a positive effect on individuals. In another study conducted by Varannai et al. (2017) with two groups of students in Hungary, an experimental study was conducted to investigate the behaviour of students while interacting with Kahoot. The results were analysed based on the technology acceptance model. In line with the findings, positive attitude, good experience and usability contributed to improving the student's performance and this situation increased the students' desire to use the application. Schöbel et al. (2019) showed students' motivation, commitment and problem-solving skills can be improved with a technology-supported education model and gamification. The online learning technique was preferred and gamification was used to improve students' problem-solving abilities.

On the other hand, Ding (2019) examined the effect of gamification to increase university students' commitment to an online discussion platform. It was revealed that the gamification approach is not very successful in supporting students' sense of community. Likewise, in their research, Hanus and Fox (2015) included a curriculum that contained gamification in the communication lesson of university students, put badges on the students, and took them for a semester. During the experiment, students' motivation, achievement, social behaviour, satisfaction, and academic performance were measured. As a result of the research, it was found that the students who participated in the lesson with the non-voting curriculum were more successful than the students who took the lesson with gamification. Domínguez et al. (2013) conducted a study by adding gamification to e-learning of university students. The opinions of the students were taken with the questionnaire method. As a result of the research, only 30% of the students stated that gamification was motivating and their participation rates were low. Goehle (2013) combined the mathematics lesson with the gamification technique. As a result of the 16-week application, it was determined that there was no serious indicator of success between the students' levels and the scores they received and they concluded that gamification is not very effective. As demonstrated, while the literature outlines the promising educational outcomes of gamification, there appears to be some mixed results as well.

4.2 Flow experience

According to flow theory, the flow experience is an optimal state which occurs when an individual feels deeply immersed and engaged in performing an activity in such a way that he/she would lose track of time and other external factors (Csikszentmihalyi, 1997). The individual who experiences the psychological condition of flow would be fully focused on their task throughout the entire activity, cannot relate with other peripheral issues in their surroundings, and would not notice the passage of time. The feeling of flow means that the person is both mentally and physically engaged in performing the task with joy. Flow theory contends that the feeling of flow would happen under certain conditions. One of them is that the difficulty level of activity should be matched with the person’s skills; the activity should be challenging, enjoyable and achievable at the same time. In other words, if the task is too easy it would be boring and if it’s too hard, it might create feelings of stress and anxiety. Furthermore, the individual should perceive a clear objective in performing the task and immediate feedback should be provided to reach such optimal experience (Csikszentmihalyi, 2014).

Previous research has underscored that flow experience of students would play important role in their learning process, cognitive absorption, creativity, and academic success specifically within online learning settings (Wang & Chen, 2010; Webster et al., 1993). In this regard, researchers pointed out that gamification and game-based learning environments can develop the flow experience of students and contribute to their motivation and engagement (Chan et al., 2021; Hamari et al., 2016; Oliveira et al., 2021; Özhan & Kocadere, 2020; Perttula et al., 2017), meanwhile a meta-analytic study by Oliveira et al. (2021) on gamification-related consequences on the flow experience, reported mixed results across the individual studies and called upon more research to examine the relationship between gamification and flow in educational contexts.

4.3 Motivation

Motivation is one of the prominent variables in predicting the human behaviour and success. In the context of education, motivation is strongly bundled with learning outcomes and is considered important in keeping students engaged and enhancing academic achievement (Deci et al., 1991; Schiefele, 1991). One of the well-known models that examines motivation within the online learning environment is the ARCS model of motivation developed by Keller (1987) which emphasizes that four elements of attention, relevance, confidence and satisfaction should be met to motivate the learners. ARCS model surmises that the learning environment (or course) should capture the learner’s attention constantly, learning environment should be interesting for the learner such that learner can understand its value to meet their needs, learners should become confident in their ability to complete the course related tasks, and subsequently learners should be satisfied with the learning setting. In fact, ARCS model had been widely used in different context both in face to face and distance education while its importance in online learning becomes undeniable and its application in gamified education is also generally accepted (Li & Keller, 2018; Su & Cheng, 2015). The attention (A) component in this model refers to the extent that student’s attention is captured in a sustainable way through the whole learning process. The relevance (R) element denotes the extent that students find the course meaningful and useful to meet their needs in real life. The confidence (C)component of the model represents the belief of students in regards to accomplishments and extent of confidence they have in their abilities to be successful. Lastly satisfaction (S) component signifies the general satisfaction of the outcomes which is critical for motivation. If someone is not satisfied, they cannot be motivated. According to Keller (1987), intrinsic rewards, providing positive feedback, recognition, equity in marking, and making the whole journey of learning enjoyable can contribute to students’ satisfaction.

It is surmised that employing games in the learning frameworks can positively affect the motivation of individuals (Dichev & Dicheva, 2017; Shernoff & Hoogstra, 2001). It has been well documented that gamification strategies such as use of e-learning media, providing interactive learning environment, providing claimable rearwards such as points for students, using leader boards which allows for student’s recognition, providing an online setting for students to experience sense of accomplishment in regards to the course they are taking and proving immediate feedback can boost the ARC motivational factors (Hamzah et al., 2015; Özhan & Kocadere, 2020; Su & Cheng, 2015). This aligns with self-determination theory by Ryan and Deci (2000) in which two types of intrinsic and extrinsic motivation are highlighted. As described by self-determination theory, an environment in which basic psychological needs of competence, autonomy and relatedness are nurtured, will intrinsically motivate individuals in well-functioning. The need for ‘competence’ denotes the individuals’ desire to be effective in their actions and interactions and their willingness to experience challenging opportunities that can enhance their competencies and skills. The need for ‘autonomy’ denotes the desire of individuals to behave and act according to their own perceived choices, interest and values. This need implies that people seek psychological freedom and tend to behave as a result of their own integrated self. The need for ‘relatedness’ implies the sense of belongingness to others and community. This need refers to the psychological need of human to be connected socially. When the studies on gamification are examined, it is understood that generally a gamified learning platform has the potential elements to satisfy these intrinsic needs of motivation (Mekler et al., 2017; Sailer et al., 2017), and this is particularly true for a challenge-based gamified learning process. In line with self-determination theory, the presence of elements of gamification that feature achievement and indicate immediate performance feedback is expected to contribute to the individuals’ level of intrinsic motivation.

5 Materials and method

5.1 Design process and development of Educhall platform

We developed Educhall which is a web-based application where teachers can prepare and direct questions, and where students can solve questions and challenge their friends. Educhall application aims to increase motivation and participation by providing a competitive environment. The challenge feature contributes to the competitive environment. In Educhall application, students use one-on-one challenge feature, encourage each other to solve questions and active participation. Educhall will introduce a whole new dimension to gamification as it aims to increase the positive effect of gamification on interaction (Çakıroğlu et al., 2017; Ding et al., 2017) motivation (Abramovich et al., 2013) and participation (Cronk, 2012) by strengthening the competitive environment through the challenge method. Therefore, it is important to test the application, of which pilot tests are already completed, in real teaching environments, so that students’ views can be determined and limitations and shortcomings that might be encountered in real life, are revealed.

Waterfall Model was used in the development of Educhall. Waterfall model is a software development model in which the software process is linear, that is, the previous phase must be completed in order to move on to the next process. Waterfall model consists of analysis, design, development, testing, implementation and maintenance steps, respectively (Balaji & Murugaiyan, 2012). Another distinctive quality of the Waterfall model is that the output of the previous step is always the beginning of the next step. Educhall has been developed using the waterfall model as its requirements are clear and easy to understand, and phases are separately processed and do not overlap. Waterfall model is not only understandable and easy to manage, but it is also suitable for small projects (Cusumano & Smith, 1995) (Fig. 1).

Fig. 1
figure 1

Waterfall Model

Full size image

5.1.1 Analysis stage

This stage helps system and business analysts to define both functional and non-functional requirements. This stage is usually where a complete and comprehensive description of the behaviour of the software is made (Bassil, 2012). There are many gamifications software prepared for use in educational processes. This software is unlike its counterparts in that it is developed to offer an environment to students where they can challenge one another. This will lead to a stronger competition atmosphere and thus stronger motivation levels. The application is student-cantered under the surveillance of the teacher and allows the students to participate in competitions to test themselves, to challenge their peers, and ultimately to allow students to encourage each other. There will be two types of user roles in the system; teacher and student. The teacher will be able to create lessons from the management panel and add quizzes. The teacher will determine the types of questions, the duration of the quizzes, scoring system, accessibility, and the maximum number of attendances. If accessibility is limited to a group or class, the target audience that the student can challenge will be also limited. The students are supposed to enter the competition, take the quiz, and after seeing the score, challenge other participants by email. This will help peers to encourage each other in solving questions and contribute to the competitive environment. A suitable algorithm has been created for the program in line with this goal.

It will be sufficient for the users to have a web browser and a device with an internet connection to participate in the Educhall application. Users must have basic computer skills in order to participate in activities after creating membership in the Educhall application.

5.1.2 Design Stage

The basic structure was developed to ensure an easy use experience and to meet the requirements in the analysis stage. Accordingly, colours, patterns, menus, sliders and footers were added in line with the established design principles. The locations of buttons to be used for sign-ins, sign-ups, score tables, about us and course lists were agreed upon.

5.1.3 Development stage

The system is a server-side dynamic Web application developed using the "Codeigniter 3" framework written in the php programming language. Codeigniter is based on MVC (model-view-controller) architectural design and it consists of 3 layers: “Model” that contains database operations, “View” that includes user interfaces, and “Controller”, which is the part where data is processed. It is aimed to develop into a safer, easier to code, easy to read and maintainable system by using MVC structure.

The system runs on an online server and keeps system records on a database located on an online server. The system database is a relational database designed considering the first, second, third, and Boyce-Codd normal forms (NF) (Elmasri et al., 2000). The database is hosted on a MariaDB database server and the phpMyAdmin web-tool was used to manage this server.

5.1.4 Testing stage

When Educhall was ready, three lecturers who were specialists in their fields, were asked to test the application. In addition, experiments were conducted in the classroom supervised by a group of 6 people and one of the researchers. The program was finalized in line with the feedback received.

5.1.5 Implementation stage

Students can easily register to Educhall with e-mail activation. Teacher accounts need admin approval to be active. Teachers can create classes in the system. The system determines a class ID and login key for the created class. Teachers can share this information with their students and ask them to register for the class, or they can add students who have an account in the system to their classes . The admin dashboard, teacher dashboard and student dashboard are shown in Figs. 2, 3, and 4.

Fig. 2
figure 2

Admin Dashboard of the Application

Full size image
Fig. 3
figure 3

Teacher Dashboard of the Application

Full size image
Fig. 4
figure 4

Student Dashboard of the Application

Full size image

Teachers can create quizzes by specifying the quiz title, description, category and difficulty level. They can add multiple choice, true–false or fill-in-the-blank questions to their quizzes. Questions can be added through the system or transferred from Excel. They can enrich their questions by adding pictures, audio and video files. The teacher determines the maximum time for each question and the points that the student will earn from 1 to 10.

The teacher can assign the created quiz to any student in the class. When assigning a quiz, it determines the start date, the latest answering time after the challenge, the quiz to be assigned and the number of questions that each student must answer. The system prompts each student with unique questions in as many equal points range as possible. The process begins with the e-mail sent to the assigned student. In the mail, the number of questions in the quiz, the points that can be earned in total, the duration and the last time to be answered are indicated as date and time. It alerts that a new quiz has arrived, with the notification sent to the browser and the different warnings shown in the quizzes section of their profile.

When the Student Quiz starts time, they can start solving the questions. The remaining seconds are displayed to answer each question. When the student answers, the next question is automatically displayed. At the end of each quiz, the students receive feedback on the total score and the questions they answered. Student can see the classmates, who have not yet participated in the event, with their pictures. Student continues the process by pressing the "Challenge" button just below the picture of a friend he wants. If the student does not participate in the quiz within the specified time, the system automatically forwards the quiz to another student.

Teachers can assign quizzes to their students as an out-of-class activity, or they can create activities for the classroom and create live events where students can answer questions at the same time. Students can earn points from these activities and increase their level. Within the scope of gamification applications, they reach levels such as novice, rookie, apprentice, journeyman, master, and senior master according to their total points. According to the challenge results, three people with the highest score are placed on the champion's podium on the home page. Also, in the winners tab the top three winners of day, week and month are shown (Fig. 5).

Fig. 5
figure 5

All Time Champions’s Podium

Full size image

5.1.6 Maintenance

Additional maintenance activities including adapting the software to the environment, meeting new user requirements, and increasing software reliability (Stellman & Greene, 2005) might be needed. In the light of the data obtained as a result of the research, necessary arrangements, new additions, improvements, and updates will be made.

5.2 Participants and procedure

The current research follows the experimental research design which includes a control group and experimental group and subjects are randomly assigned to the two groups, both are presented, with the course content, but only the experimental group is treated with challenge-based gamification. After the implementation and close observation, both groups are post-tested to measure the degree of change in study variables (flow, motivation, academic achievement) in each group.

The experiment took 8 weeks. The experiment was conducted on 60 bachelor students taking the “instructional technologies” course during the Spring 2020–2021 academic semester. Due to covid-19, classes of both groups were held online. None of the students had any previous exposure to the course content and they were all in third grade and taking the “instructional technologies” course for the first time and with the same faculty member. The students were randomly assigned to either control or experimental group. The control group consisted of 30 students (13 male and 17 female) who received traditional learning design in which there was no intervention of any gamified platform; students were taught by 2 h lecture per week where PowerPoint slides were the only the visual material used in the class. They had conventional quizzes after finishing each topic.

The experimental group consisted of 30 students (16 male and 14 female) who were taught by 2 h lecture per week. The materials used were PowerPoint slides as well as challenge-based gamified platform (Educhall application). In experimental group, after finishing each topic, the lecturer started the challenge process through Educhall platform by assigning the quizzes he had prepared to his students. In some lessons, active participation of students in the Educhall platform was ensured through in-class live activities as well.

To ensure about the homogeneity of the control and experimental sample in terms of their extant of knowledge, before implementing the experiment, we conducted pre-test for their academic achievement through an examination containing 30 multiple-choice questions. Conducting a pre-test is advised to confirm that the experimental and control group are equivalent in terms of their academic achievement. This procedure helps to controls the internal threats to validity of the study (McKenney & Reeves, 2018).

At the end of the 8 weeks, both groups were post-tested in terms of their academic achievement through an examination which consisted 35 multiple-choice questions. Data related to students’ motivation and flow experience were collected through online survey distributed to both groups of students. Conducting a pre-test for motivation and flow experience would not be reasonable because students should first undergo the course and then we can check their level of flow and motivation (attention, relevance, confidence, satisfaction) in respect to the course. Therefore, we did not perform a pre-test for flow and motivation. Students were not forced but were highly encouraged to take part in the study and there were ensured that their responses would be used for academic purpose only and their response would stay confidential. The first page of survey included information such as “There are no right or wrong answers in this questionnaire,” “Any sort of information collected during our research will be kept confidential,” “Participation is voluntary but encouraged”.

5.3 Instrumentation

At the end of the experiment, an online survey was used to measure students’ flow and motivation. Ten items with 7-point Likert scale type were used to assess the flow experience level of the learners in regards with the course. These items came from the scale developed by Rheinberg et al. (2003). The present study used the validated and reliable Turkish translated version of this scale by İşigüzel and Çam (2014).

For assessing the learners’ motivation level. Scale by Keller and Subhiyah (1993)was used. This scale consists of 34 items of 5-point Likert scale type ranging from 1 (not true) to 5 (very true). Items 4, 6, 7,8,11,17, 25, 26 and 31 in this scale were reverse coded. This scale was developed in line with the ARCS motivation model (Keller, 1987) and includes four subscales that measure the components of ARCS model; namely attention, relevance, confidence and satisfaction. Eight items of the scale measure attention, eight items measure confidence, nine items measure relevance, and nine items measure satisfaction. The present study used the validated and reliable Turkish translated version of this scale by Acar (2009). Both of the aforementioned scales were reviewed by three experts in the field and were piloted with five students who assured its readability and ease of understanding.

Academic achievement of the students was assessed via 35 multiple-choice questions related to the course content. Each question had one point. For measuring academic achievement before the experiment (pre-test), an examination consisting of 30 multiple choice question related to the course were conducted at the beginning of the semester with score of students in a range of 0 to 30.

6 Data analysis

The responses obtained were analysed using SPSS package for Windows version 25.0. With respect to the academic achievement, a one-way analysis of covariance (ANCOVA) was conducted. With respect to the motivation, and its subscales of attention, relevance, confidence and satisfaction, one-way multiple analysis of variance (MANOVA) was conducted. With respect to the flow level of students, independent sample t-test was conducted.

The data were checked in terms of normality by inspecting the skewness and kurtosis tests. If range for skewness and kurtosis lies between -2 and + 2, normal distribution can be considered for the data (George, 2011). Table 1 reports the mean, standard deviation, skewness and kurtosis test of the variables. As shown by Table 1. The skewness and kurtosis scores for all of the variables were between -1 and + 1. This highlights the evidence of normal distribution of our data.

Table 1 Variables’ Mean, Standard Deviations, Skewness and Kurtosis Statistics
Full size table

The validity and reliability of the data collection tools were addressed through the analysis. The scales used in the questionnaires contained validated measuring constructs that have been used extensively in the literature measuring motivation and flow experience (Li & Keller, 2018; Rheinberg et al., 2003; Su & Cheng, 2015). In order to determine the face and content validity of the translated scope of the questionnaire to see whether the scales reflected true meanings of the constructs, the judgment of three experts in the field were considered and items were revised in line with field experts’ suggestions, and the reliability of the questionnaire was tested through an internal consistency test. Cronbach’s alpha was used to assess the reliability of scales pertaining to motivation (attention, relevance, confidence, satisfaction) and flow. Scores of coefficient alphas for attention, relevance, confidence, satisfaction and flow were 0.70, 0.84, 0.76, 0.77 and 0.84 respectively which indicate satisfactory level of internal consistency (Nunnally, 1978). The Kuder Richardson (KR-20) test was used to evaluate the validity-reliability of the academic achievement tests used in the study. The KR-20 coefficient test for pre-test and post-test was calculated as 0.7 and 0.73 respectively which is considered a satisfactory level. For both pre-test and post-test examination of academic achievement, the content validity of the questions was determined by three experts in the field of instructional technologies and the similar difficulty level was ensured and amendments were made based on their feedback before determination of final version.

7 Results

In respect to the first research question, “RQ1: After intervention of challenge-based gamification, is there a significant difference in control group and the experimental group in terms of their academic achievement”, a one-way between-groups analysis of covariance (ANCOVA) was conducted to compare the effectiveness of challenge-based gamification interventions designed to increase participants’ academic achievement while controlling for their pre-test scores on this test. The independent variable was the type of group (control group, experimental group) and the dependent variable consisted of scores on the academic achievement administered after the intervention was completed (post-test academic achievement). Students’ score on academic achievement before administration of experiment (pre-test academic achievement) was used as the covariate in this analysis. Preliminary analysis was performed regarding assumptions of ANCOVA. The skewness and kurtosis scores of the variables were checked and there was no deviation of normality assumption. Concerning the assumption of homogeneity of regression slope, the significance value of the interaction term was inspected (academic achievement *group), the sig. level was 0.32, safely above the cut-off. Regarding the assumption of linearity between the dependent variable (post-test academic achievement) and covariate (pre-test academic achievement) for both groups, the scatter plot was examined and there was no indication of curvilinear relationship. For the assumption of reliability of covariate (pre-test academic achievement), reliability had been evidenced by Kuder Richardson test (above 0.7). The sig. level of the Leven’s test was 0.49 which indicated that data did not violate the assumption of homogeneity of variances.

The results of the ANCOVA analysis demonstrated that there was significant difference between the two control groups and experimental group on academic achievement test F (1, 57) = 24.70, p = 0.00, partial eta squared = 0.30. After controlling for pre-test academic achievement test, and comparing the adjusted mean score of the test, it is possible to assert that the experimental group which received challenge-based gamification had better academic achievement than the control group. Summary of the results of the ANCOVA analysis are shown in Tables 2 and 3.

Table 2 Test of Between Subject Effects, Dependent Variable: Post-test academic achievement
Full size table
Table 3 Adjusted Mean of Academic Achievements Test
Full size table

The second and third research questions of the current study were as follows:

  • RQ2: After intervention of challenge-based gamification, is there a significant difference between students in the experimental group, and the control group in terms of their overall motivation towards the course?

  • RQ3: After intervention of challenge-based gamification, is there a significant difference between students in the experimental group, and the control group in terms of the (ARCS) motivational subfactors of attention (A), relevance (R), confidence (C) and satisfaction (S)?

In other words, these research questions examine whether control and experimental group differ in terms of overall motivation? Are experimental group students better adjusted than control group in terms of their attention (A), relevance(R), confidence (C) and satisfaction (S)towards the course? To address the abovementioned research questions, multivariate analysis of variance (MANOVA) was conducted. MANOVA is used since our variables are different but related (motivational factors) and it’s preferred since it controls for the risk of type 1 error. Four dependent variables were used namely: attention, relevance, confidence and satisfaction. The independent variable was the group (control vs. experimental).

Before conducting MANOVA, a number of assumptions were examined. For normality assumption, skewedness and kurtosis values of the variables were between -2 and + 2 and provided evidence of normality. According to Tabachnick and Fidell (2007, p. 25), “a sample size of at least 20 in each cell should ensure robustness”, therefore our sample size is adequate. For the assumption of multivariate normality, the Mahalanobis distance score should be less than the critical value using a chi-square value with number of dependent variables (4) as degree of freedom and the alpha value of 0.001. For our data, the Mahalanobis distance score was 13.94 and the critical value was 18.47, so multivariate normality assumption was supported. For the assumption of a linear relationship between each pair of the dependent variables, we examined the scatter plots generated for each pair of variables separately for each group which yielded no serious indication of non-linearity and the assumption of linearity was supported. Regarding the multicollinearity concern, the correlation strength among the dependent variables (attention, relevance, confidence, satisfaction) was inspected and there was no evidence of multicollinearity since all the correlation coefficients were below 0.8. For the assumption of homogeneity of variance–covariance matrices, Box’s test of equality of covariance matrices was inspected, the sig. value was 0.71 indicating that the data did not violate this assumption. Prior to interpreting the MANOVA results, Leven’s test was inspected. Fortunately, none of the variables reported significant value for the Leven’s test of equality of variances which indicated that data did not violate the assumption of equality of variances.

As shown in Table 4. the results of MANOVA demonstrated that there was a significant difference among control and experimental group on the combined dependent variables (motivation), F(4,55) = 5.10 partial eta squared = 0.27, Pillas trace (p = 0.001), Wilks’s lambda (p = 0.001), Hotelling’s trace (p = 0.001) and Roy’s largest root (p = 0.001) all reached significant value.

Table 4 Multivariate Testsa
Full size table

For investigating the results of dependent variables separately, a Bonferroni adjusted alpha level was used in order to decrease the chance of type 1 error. Therefore, the alpha level of 0.05 was divided by 4 (number of independent variable), thus a new alpha level of 0.012 was considered. The results for the dependent variables using a Bonferroni adjusted alpha suggested that only two variables reach significant difference among the groups; confidence F(1,58) = 9.07, p = 0.004, partial eta squared = 0.135, and satisfaction F(1,58) = 16.25, p = 0.000, partial eta squared = 0.22.The results of the mean scores indicated that experimental group reported higher levels of confidence (M = 4.22, SD = 0.59) than control group (M = 3.76, SD = 0.60). Likewise experimental group reported higher level of satisfaction (M = 4.04, SD = 0.50) than the control group (M = 3.47, SD = 0.59). In line with Cohen’s (1988) guidelines the magnitude of the difference is large. Summary results of the MANOVA analysis are shown in Tables 4, 5, and 6.

Table 5 Test of Between Subject Effects
Full size table
Table 6 Estimated Marginal Means
Full size table

For the fourth research question of the current study: RQ3: “Does implementation of challenge-based gamification program affect the flow experience of students?”, the independent sample t-test was conducted to compare the scores of flow experience for experimental and control group. Prior to investigating the t-tests, Leven’s test was inspected to determine homogeneity of variances. The significance level of Leven’s test is p = 0.958 which indicates there is no violation of the assumption of equal variances. For interpretation of the magnitude of the difference, Eta squared scores were calculated using the formula (t2/t2 + (N1 + N2-2)), since p value alone informs us about the statistical significance but does not inform us about the magnitude of the difference. The results confirmed that there was no significant difference in score for experimental group (M = 5.57, SD = 1.03; t(30) = -0.54, p > 0.05) and control group(M = 5.43, SD = 0.96). This means that challenge-based gamification applied in our study did not significantly increase flow level of students, and in line with guideline proposed by Cohen (1988) magnitude of difference was very small (eta squared = 0.005). Result of the independent sample t-test are shown in Table 7.

Table 7 Independent Samples t-Test Result of Control and Experimental Groups
Full size table

8 Discussion

Our research shed new lights on educational research by developing and implementing a challenge-based gamification platform and successfully answered four research questions regarding the impact of challenge-based gamification on students’ academic achievement, flow and motivation within higher education context. This is important since, a careful search made within gamification literature in education, revealed that challenge-based gamification implementation is rare. This research also fills in the gap that most gamification studies have been conducted among school students and higher education context have remained underexamined (Murillo-Zamorano et al., 2021). Moreover, by assessing these linkages, our study responded to the calls for additional research regarding outcomes of gamification studies which have reported mixed results (Bai et al., 2020; Majuri et al., 2018; Oliveira et al., 2021; Sailer et al., 2017).The first finding of this study showed that the experimental group who were taught by the challenge-based gamified enriched program could earn significantly better academic achievement than the control group who received education without the intervention.While the magnitude of the difference was large, our findings are in line with past and recent studies (e.g., Legaki et al., 2020; López-Jiménez et al., 2022) which were conducted to investigate the effect of the gamified classroom models on students’ “academic performance”. This aligns with the general finding of the recent meta-analytic study by Bai et al. (2020) in which it was confirmed that learners who participated in a course involving gamification techniques had reported significantly higher academic achievement than learners who participated in a non-gamified course. Although some scholars (e.g. Hanus & Fox, 2015; Toda et al., 2017) had argued in their research that gamified learning adversely affects the students’ level of performance, the findings of the current study strengthen the knowledge that gamified learning can improve students’ performance and more importantly as predicted by constructivist learning processes, the results of current research underline that the learning within challenge-enriched gamification context is effective in boosting academic performance.

The second finding of the current study is that challenge-based gamified learning enhanced the overall motivation level of students towards the course. By making an assessment of this linkage, our paper responds to the call for future studies in gamification regarding affective learners’ outcomes (Huang et al., 2020) and provides a response to previous calls for research that have conveyed negative or mixed results in relationship between gamification and motivation(e.g. Hanus & Fox, 2015; Sailer et al., 2017; Toda et al., 2017).The experimental group which participated in the challenge-based gamified learning process, reported statistically significantly higher motivation levels towards the taken course than the control group. In respect to the subscales of motivation, learners who participated in the challenge based-gamification, had more confidence in becoming successful in the course, and were more satisfied with the course, while the magnitude of the difference was large. This aligns with self-determination theory applied to gamification setting, where the elements of challenge-based gamification that feature achievement, playfulness, immediate performance feedback, and sense of belongingness to a community, are expected to contribute to the individuals’ level of intrinsic motivation (Xi & Hamari, 2019). Learners feel intrinsically motivated when their efforts and achievements are seen and recognized by others, in other words, in a gamified learning environment, in which the virtual gamified elements, such as badges, points, and leaderboards are used, the students’ accomplishments would become visible and the learners’ need of signifying their accomplishments would be fulfilled and this will boost their motivation levels (Deci et al., 1991). Our finding also corresponds to the social comparison theory (Festinger, 1954) which states that individuals are inclined to evaluate their abilities by comparing themselves with the other people’s abilities. The gamified application which is based on the feature to challenge others, help the learners to gain this comparison and when the student challenges the other student and sees that their accomplishments are as good as or better than other competent learners, they will be more engaged and motivated. The conclusion made by the meta-analytic study also confirms that the gamification enriched learning environment leads to intensified motivation of the learners (Sailer & Homner, 2020).

Thirdly, the current study aimed to respond to the research call made by Oliveira et al. (2021) which reported mixed result appertaining to the association between gamified learning and flow experience and responds to the call for future studies in gamification regarding affective learners’ outcomes (Huang et al., 2020). The findings of our study showed that the experimental group’s level of flow experience did not significantly differ among the control and experimental group and the magnitude of the difference was very small. Our finding is not contrary to our expectation, as there was no negative impact. In other words, our research underscores that gamified learning environment did not harm the experience of being fully concentrated on the course, instead increased the flow experience though in small magnitude. In line with flow theory, gamified practices are helpful in boosting level of flow(Hamari & Koivisto, 2014). Although our findings did not find significant difference in flow experience of students of control and experimental group. Still, our data supported the idea that gamification improves the flow experience of students but the support is weak and further experiment is needed in this regard. Consistent with systematic review made by Oliveira et al. (2021), it appears that evidence appertaining to the association between gamified learning and flow variables is not clear-cut and this stream of research needs more conclusive data.

9 Practical implications

The results suggest important implications for academicians. While past research on gamification programs did not report consistent results on learners’ outcomes(Bai et al., 2020; Majuri et al., 2018; Oliveira et al., 2021; Sailer et al., 2017), our study showed that the challenge method in gamified program might add a new dimension to existing result. The challenge-based gamification increases student participation in educational processes, and increase their motivation levels by strengthening the competitive environment. With small improvements, the Educhall will introduce a novel understanding to learning process and to student–student interaction via its challenge feature. In distance education processes, it is more difficult for students to be motivated to work and learn compared to face-to-face teaching models. Such gamification practices should be adapted to teaching processes in order to enable students to be more willing to work while enhancing the collaborative learning spirit. According to constructivist learning theory, experimentations in which allow each individual student to test their knowledge and share their outcome with their peers is effective in constructivist learning (Bada et al., 2015). In effect, our proposed challenge-based gamification platform provides such opportunity for students to test what they learned and socially interact with each other, share their experience on the test, and challenge each other for gaining badges. Particularly, the fact that the students create a working network by challenging each other is sure to add a new dimension to both in and outside the classroom learning activities. In particular, challenge-based gamification methods can be used to increase the competition in the environment and to enable peers to encourage each other to work. In this way, students will work harder to get better points than their challenging friend and to earn more badges. Adapting such practices to higher education environments will move the stagnation of traditional methods away from interaction. Thus, in addition to ordinary study methods, higher education students will indirectly encourage each other to study. With the questions to be solved after the challenges, students will reinforce their knowledge, experience the use of information in different situations, and make their knowledge permanent. Such gamification methods strengthen peer support, as the challenged student actually directs his chosen friend to study. At the same time, it is possible for all students to answer questions simultaneously with classroom activities that teachers can organize. With this method, active participation of all students in the classroom can be ensured.

10 Limitations and future research directions

In this study, quantitative research methods were used. It is recommended that the study be carried out with qualitative studies and mixed research methods in which quantitative and qualitative research methods are used together. This research is limited to the web-based version of Educhall. The results of the study make it clear that it is important to develop a new Educhall version, to apply the improved version in real environments with wider groups and to report the results. Also, we believe that the fact that the results of this study are obtained through real environment experiences can help other researchers to develop educational software in the future. We believe that in the next step, the Educhall can be transferred to the mobile platform, the scoring system can be updated, it can be made possible to allow the users challenge more than one person, the design can be improved, the questions’ variety can be increased and question pools can be created for different topics and levels. Challenge features can be made possible through additional channels. Also, in the selection of the person to be challenged, a test can be applied to prevent challenging a student that already took the quiz. Likewise, a feedback module can be developed to increase educational benefits.

Furthermore, our study provided weak support in regards to the association of challenge-based gamified learning and flow experience which warrant further experimental research in future.

11 Conclusion

Gamification increases student participation, interest, and motivation levels in learning environments and contributes to the collaborative process of learning specifically in distance learning. We developed and successfully presented a gamified learning platform named Educhall in which the challenge method can increase student participation in educational processes and contributes to collaborative learning by strengthening the competitive environment through a gamification process. In line with constructivist learning processes, the results of the present research revealed that students in experimental group achieved significantly higher academic achievement. Consistent with self-determination theory, it was evidenced that application of challenge-based gamified learning method increased students’ level of confidence in succeeding in the course, and students’ level of satisfaction with the course and the whole motivation towards the course was increased. Likewise, grounded on flow theory, we explored whether application of challenge-based gamified learning influenced the flow experience level of students or not. Our results showed that gamified learning increased flow level of learners but not significantly which warrants further data collection and experimental research in future studies.